0% found this document useful (0 votes)

1K views1,930 pages

F Sharp For Fun and Profit

This document outlines the table of contents for an eBook on F# for Fun and Profit. It includes sections on getting started with F#, tutorials on key F# concepts like functions and types, examples and walkthroughs, and other miscellaneous topics. The getting started section provides information on installing and using F# as well as reasons for using F#. The tutorials section dives deeper into functional programming concepts through a series of posts on topics such as expressions and types. Later sections include worked examples and walkthroughs demonstrating F# in practice.

Uploaded by

mike vidal

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

1K views1,930 pages

F Sharp For Fun and Profit

Uploaded by

mike vidal

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 1930

Table

of Contents
Getting started
Introduction

1.1

Contents of the book

1.2

"Why use F#?" in one page

1.3

Installing and using F#

1.4

F# syntax in 60 seconds

1.5

Learning F#

1.6

Troubleshooting F#

1.7

Low-risk ways to use F# at work

1.8

Twenty six low-risk ways to use F# at work

1.8.1

Using F# for development and devops scripts

1.8.2

Using F# for testing

1.8.3

Using F# for database related tasks

1.8.4

Other interesting ways of using F# at work

1.8.5

Why use F#?

The "Why use F#?" Series

2.1

Introduction to the 'Why use F#' series

2.1.1

F# syntax in 60 seconds

2.1.2

Comparing F# with C#: A simple sum

2.1.3

Comparing F# with C#: Sorting

2.1.4

Comparing F# with C#: Downloading a web page

2.1.5

Four Key Concepts

2.1.6

Conciseness

2.1.7

Type inference

2.1.8

Low overhead type definitions

2.1.9

Using functions to extract boilerplate code

2.1.10

Using functions as building blocks

2.1.11

Pattern matching for conciseness

2.1.12

Convenience

2.1.13

Out-of-the-box behavior for types

2.1.14

Functions as interfaces

2.1.15

Partial Application

2.1.16

Active patterns

2.1.17

Correctness

2.1.18

Immutability

2.1.19

Exhaustive pattern matching

2.1.20

Using the type system to ensure correct code

2.1.21

Worked example: Designing for correctness

2.1.22

Concurrency

2.1.23

Asynchronous programming

2.1.24

Messages and Agents

2.1.25

Functional Reactive Programming

2.1.26

Completeness

2.1.27

Seamless interoperation with .NET libraries

2.1.28

Anything C# can do...

2.1.29

Why use F#: Conclusion

2.1.30

Thinking Functionally
The "Thinking Functionally" Series

3.1

Thinking Functionally: Introduction

3.1.1

Mathematical functions

3.1.2

Function Values and Simple Values

3.1.3

How types work with functions

3.1.4

Currying

3.1.5

Partial application

3.1.6

Function associativity and composition

3.1.7

Defining functions

3.1.8

Function signatures

3.1.9

Organizing functions

3.1.10

Attaching functions to types

3.1.11
2

Worked example: A stack based calculator

3.1.12

Understanding F#
The "Expressions and syntax" Series

4.1

Expressions and syntax: Introduction

4.1.1

Expressions vs. statements

4.1.2

Overview of F# expressions

4.1.3

Binding with let, use, and do

4.1.4

F# syntax: indentation and verbosity

4.1.5

Parameter and value naming conventions

4.1.6

Control flow expressions

4.1.7

Exceptions

4.1.8

Match expressions

4.1.9

Formatted text using printf

4.1.10

Worked example: Parsing command line arguments

4.1.11

Worked example: Roman numerals

4.1.12

The "Understanding F# types" Series

4.2

Understanding F# types: Introduction

4.2.1

Overview of types in F#

4.2.2

Type abbreviations

4.2.3

Tuples

4.2.4

Records

4.2.5

Discriminated Unions

4.2.6

The Option type

4.2.7

Enum types

4.2.8

Built-in .NET types

4.2.9

Units of measure

4.2.10

Understanding type inference

4.2.11

Choosing between collection functions

4.3

The "Object-oriented programming in F#" Series

4.4

Object-oriented programming in F#: Introduction

4.4.1

Classes

4.4.2

Inheritance and abstract classes

4.4.3
3

Interfaces

4.4.4

Object expressions

4.4.5

The "Computation Expressions" Series

4.5

Computation expressions: Introduction

4.5.1

Understanding continuations

4.5.2

Introducing 'bind'

4.5.3

Computation expressions and wrapper types

4.5.4

More on wrapper types

4.5.5

Implementing a builder: Zero and Yield

4.5.6

Implementing a builder: Combine

4.5.7

Implementing a builder: Delay and Run

4.5.8

Implementing a builder: Overloading

4.5.9

Implementing a builder: Adding laziness

4.5.10

Implementing a builder: The rest of the standard methods

4.5.11

Organizing modules in a project

4.6

The "Dependency cycles" Series

4.7

Cyclic dependencies are evil

4.7.1

Refactoring to remove cyclic dependencies

4.7.2

Cycles and modularity in the wild

4.7.3

The "Porting from C#" Series

4.8

Porting from C# to F#: Introduction

4.8.1

Getting started with direct porting

4.8.2

Functional Design
The "Designing with types" Series

5.1

Designing with types: Introduction

5.1.1

Single case union types

5.1.2

Making illegal states unrepresentable

5.1.3

Discovering new concepts

5.1.4

Making state explicit

5.1.5

Constrained strings

5.1.6

Non-string types

5.1.7

Designing with types: Conclusion

5.1.8
4

Algebraic type sizes and domain modelling

5.2

Thirteen ways of looking at a turtle

5.3

Thirteen ways of looking at a turtle (part 2)

5.3.1

Thirteen ways of looking at a turtle - addendum

5.3.2

Functional Patterns
How to design and code a complete program

6.1

A functional approach to error handling (Railway oriented programming)

6.2

Railway oriented programming: Carbonated edition

The "Understanding monoids" Series

6.2.1
6.3

Monoids without tears

6.3.1

Monoids in practice

6.3.2

Working with non-monoids

6.3.3

The "Understanding Parser Combinators" Series

6.4

Understanding Parser Combinators

6.4.1

Building a useful set of parser combinators

6.4.2

Improving the parser library

6.4.3

Writing a JSON parser from scratch

6.4.4

The "Handling State" Series

6.5

Dr Frankenfunctor and the Monadster

6.5.1

Completing the body of the Monadster

6.5.2

Refactoring the Monadster

6.5.3

The "Map and Bind and Apply, Oh my!" Series

6.6

Understanding map and apply

6.6.1

Understanding bind

6.6.2

Using the core functions in practice

6.6.3

Understanding traverse and sequence

6.6.4

Using map, apply, bind and sequence in practice

6.6.5

Reinventing the Reader monad

6.6.6

Map and Bind and Apply, a summary

6.6.7

The "Recursive types and folds" Series

6.7

Introduction to recursive types

6.7.1

Catamorphism examples

6.7.2
5

Introducing Folds

6.7.3

Understanding Folds

6.7.4

Generic recursive types

6.7.5

Trees in the real world

6.7.6

The "A functional approach to authorization" Series

6.8

A functional approach to authorization

6.8.1

Constraining capabilities based on identity and role

6.8.2

Using types as access tokens

6.8.3

Testing
An introduction to property-based testing

7.1

Choosing properties for property-based testing

7.2

Examples and Walkthroughs

Worked example: Designing for correctness

8.1

Worked example: A stack based calculator

8.2

Worked example: Parsing command line arguments

8.3

Worked example: Roman numerals

8.4

Commentary on 'Roman Numerals Kata with Commentary'

8.5

Calculator Walkthrough: Part 1

8.6

Calculator Walkthrough: Part 2

8.6.1

Calculator Walkthrough: Part 3

8.6.2

Calculator Walkthrough: Part 4

8.6.3

Enterprise Tic-Tac-Toe
Enterprise Tic-Tac-Toe, part 2
Writing a JSON parser from scratch

8.7
8.7.1
8.8

Other
Ten reasons not to use a statically typed functional programming language

9.1

Why I won't be writing a monad tutorial

9.2

Is your programming language unreasonable?

9.3

We don't need no stinking UML diagrams

9.4

Introvert and extrovert programming languages

9.5

Swapping type-safety for high performance using compiler directives

9.6

Introduction

F# for Fun and Profit eBook

This is an eBook version of my site fsharpforfunandprofit.com, created by popular request
for people who want to read it offline on their kindle or phone. Special thanks to Roman
Provaznik for the final nag.
The site (and this book) aims to introduce you to F# and show you ways that F# can help in
day-to-day development of mainstream commercial business software. On the way, I hope
to open your mind to the joys of functional programming - it really is fun!
Many of the posts were not really designed to be in a book, but I have arranged them so that
beginner posts come first, and more advanced posts later.

About F#
If you are completely new to F#, F# is a general purpose functional/hybrid programming
language which is great for tackling almost any kind of software challenge.
F# is free and open source, and runs on Linux, Mac, Windows and more. To download and
install F#, or to find out more, go to F# Foundation site at fsharp.org.

Getting started
Next, before randomly dipping into the posts, you should read the "why use F#?" page and
then the whole "why use F#" series. After that the "site contents" page provides suggestions
for further reading on functions, types and more.
There is a page with some advice on learning F#, and if you have problems trying to get
your code to compile, the troubleshooting F# page might be helpful.
I will assume that you do not need instruction in the basics of programming and that you are
familiar with C#, Java, or a similar C-like language. It will also be helpful if you are familiar
with the Mono/.NET library.
On the other hand, I will not assume that you have a mathematical or computer science
background. There will be no mathematical notation, and no mysterious concepts like
"functor", "category theory" and "anamorphism". If you are already familiar with Haskell or
ML, this is probably not the place for you!

Introduction

Also, I will not attempt to cover highly technical or mathematical applications. F# is an

excellent tool for these domains, but it requires an approach that is different from business
software.

Contents of the book

Getting started
Installing and using F# will get you started.
Why use F#? An interactive tour of F#.
Learning F# has tips to help you learn more effectively.
Troubleshooting F# for when you have problems getting your code to compile.
and then you can try...
Twenty six low-risk ways to use F# at work. You can start right now -- no permission
needed!

Tutorials
The following series are tutorials on the key concepts of F#.
Thinking functionally starts from basics and explains how and why functions work the
way they do.
Expressions and syntax covers the common expressions such as pattern matching, and
has a post on indentation.
Understanding F# types explains how to define and use the various types, including
tuples, records, unions, and options.
Designing with types explains how to use types as part of the design process, making
illegal states unrepresentable.
Choosing between collection functions. If you are coming to F# from C#, the large
number of list functions can be overwhelming, so I have written this post to help guide
you to the one you want.
Property-based testing: the lazy programmer's guide to writing 1000's of tests.
Understanding computation expressions demystifies them and shows how you can
create your own.

Functional patterns
These posts explain some core patterns in functional programming -- concepts such as
"map", "bind", monads and more.
Railway Oriented Programming: A functional approach to error handling
State Monad: An introduction to handling state using the tale of Dr Frankenfunctor and
the Monadster.

Contents of the book

Reader Monad: Reinventing the Reader monad.

Map, bind, apply, lift, sequence and traverse: A series describing some of the core
functions for dealing with generic data types.
Monoids without tears: A mostly mathless discussion of a common functional pattern.
Fold and recursive types: A look at recursive types, catamorphisms, tail recursion, the
difference between left and right folds, and more.
Understanding Parser Combinators: Creating a parser combinator library from scratch.
Thirteen ways of looking at a turtle: demonstrates many different techniques for
implementing a turtle graphics API, including state monads, agents, interpreters, and
more!

Worked examples
These posts provide detailed worked examples with lots of code!
Designing for correctness: How to make illegal states unrepresentable (a shopping cart
example).
Stack based calculator: Using a simple stack to demonstrate the power of combinators.
Parsing commmand lines: Using pattern matching in conjunction with custom types.
Roman numerals: Another pattern matching example.
Calculator Walkthrough: The type-first approach to designing a Calculator.
Enterprise Tic-Tac-Toe: A walkthrough of the design decisions in a purely functional
implementation
Writing a JSON Parser.

Specific topics in F#
General:
Four key concepts that differentiate F# from a standard imperative language.
Understanding F# indentation.
The downsides of using methods.
Functions:
Currying.
Partial Application.
Control Flow:
Match..with expressions and creating folds to hide the matching.
If-then-else and loops.

Contents of the book

Exceptions.
Types:
Option Types especially on why None is not the same as null.
Record Types.
Tuple Types.
Discriminated Unions.
Algebraic type sizes and domain modelling.

Controversial posts
Is your programming language unreasonable? or, why predictability is important.
Commentary on 'Roman Numerals Kata with Commentary'. My approach to the Roman
Numerals Kata.
Ten reasons not to use a statically typed functional programming language. A rant
against something I don't get.
We don't need no stinking UML diagrams or, why in many cases, using UML for class
diagrams is not necessary.

"Why use F#?" in one page

Although F# is great for specialist areas such as scientific or data analysis, it is also an
excellent choice for enterprise development. Here are five good reasons why you should
consider using F# for your next project.

Conciseness
F# is not cluttered up with coding "noise" such as curly brackets, semicolons and so on.
You almost never have to specify the type of an object, thanks to a powerful type inference
system.
And, compared with C#, it generally takes fewer lines of code to solve the same problem.
// one-liners
[1..100] |> List.sum |> printfn "sum=%d"
// no curly braces, semicolons or parentheses
let square x = x * x
let sq = square 42
// simple types in one line
type Person = {First:string; Last:string}
// complex types in a few lines
type Employee =
| Worker of Person
| Manager of Employee list
// type inference
let jdoe = {First="John";Last="Doe"}
let worker = Worker jdoe

Convenience
Many common programming tasks are much simpler in F#. This includes things like creating
and using complex type definitions, doing list processing, comparison and equality, state
machines, and much more.
And because functions are first class objects, it is very easy to create powerful and reusable
code by creating functions that have other functions as parameters, or that combine existing
functions to create new functionality.

"Why use F#?" in one page

// automatic equality and comparison

type Person = {First:string; Last:string}
let person1 = {First="john"; Last="Doe"}
let person2 = {First="john"; Last="Doe"}
printfn "Equal? %A" (person1 = person2)
// easy IDisposable logic with "use" keyword
use reader = new StreamReader(..)
// easy composition of functions
let add2times3 = (+) 2 >> (*) 3
let result = add2times3 5

Correctness
F# has a powerful type system which prevents many common errors such as null reference
exceptions.
Values are immutable by default, which prevents a large class of errors.
In addition, you can often encode business logic using the type system itself in such a way
that it is actually impossible to write incorrect code or mix up units of measure, greatly
reducing the need for unit tests.
// strict type checking
printfn "print string %s" 123 //compile error
// all values immutable by default
person1.First <- "new name" //assignment error
// never have to check for nulls
let makeNewString str =
//str can always be appended to safely
let newString = str + " new!"
newString
// embed business logic into types
emptyShoppingCart.remove // compile error!
// units of measure
let distance = 10<m> + 10<ft> // error!

Concurrency

"Why use F#?" in one page

F# has a number of built-in libraries to help when more than one thing at a time is
happening. Asynchronous programming is very easy, as is parallelism.
F# also has a built-in actor model, and excellent support for event handling and functional
reactive programming.
And of course, because data structures are immutable by default, sharing state and avoiding
locks is much easier.
// easy async logic with "async" keyword
let! result = async {something}
// easy parallelism
Async.Parallel [ for i in 0..40 ->
async { return fib(i) } ]
// message queues
MailboxProcessor.Start(fun inbox-> async{
let! msg = inbox.Receive()
printfn "message is: %s" msg
})

Completeness
Although it is a functional language at heart, F# does support other styles which are not
100% pure, which makes it much easier to interact with the non-pure world of web sites,
databases, other applications, and so on.
In particular, F# is designed as a hybrid functional/OO language, so it can do virtually
everything that C# can do.
Of course, F# is part of the .NET ecosystem, which gives you seamless access to all the
third party .NET libraries and tools. It runs on most platforms, including Linux and smart
phones (via Mono and the new .NET Core).
Finally, it is well integrated with Visual Studio (Windows) and Xamarin (Mac), which means
you get a great IDE with IntelliSense support, a debugger, and many plug-ins for unit tests,
source control, and other development tasks. Or on Linux, you can use the MonoDevelop
IDE instead.

"Why use F#?" in one page

// impure code when needed

let mutable counter = 0
// create C# compatible classes and interfaces
type IEnumerator<'a> =
abstract member Current : 'a
abstract MoveNext : unit -> bool
// extension methods
type System.Int32 with
member this.IsEven = this % 2 = 0
let i=20
if i.IsEven then printfn "'%i' is even" i
// UI code
open System.Windows.Forms
let form = new Form(Width= 400, Height = 300,
Visible = true, Text = "Hello World")
form.TopMost <- true
form.Click.Add (fun args-> printfn "clicked!")
form.Show()

The "Why Use F#?" series

The following series of posts demonstrates each of these F# benefits, using standalone
snippets of F# code (and often with C# code for comparison).
Introduction to the 'Why use F#' series. An overview of the benefits of F#
F# syntax in 60 seconds. A very quick overview on how to read F# code
Comparing F# with C#: A simple sum. In which we attempt to sum the squares from 1 to
N without using a loop
Comparing F# with C#: Sorting. In which we see that F# is more declarative than C#,
and we are introduced to pattern matching.
Comparing F# with C#: Downloading a web page. In which we see that F# excels at
callbacks, and we are introduced to the 'use' keyword
Four Key Concepts. The concepts that differentiate F# from a standard imperative
language
Conciseness. Why is conciseness important?
Type inference. How to avoid getting distracted by complex type syntax
Low overhead type definitions. No penalty for making new types
Using functions to extract boilerplate code. The functional approach to the DRY
principle
Using functions as building blocks. Function composition and mini-languages make

"Why use F#?" in one page

code more readable

Pattern matching for conciseness. Pattern matching can match and bind in a single step
Convenience. Features that reduce programming drudgery and boilerplate code
Out-of-the-box behavior for types. Immutability and built-in equality with no coding
Functions as interfaces. OO design patterns can be trivial when functions are used
Partial Application. How to fix some of a function's parameters
Active patterns. Dynamic patterns for powerful matching
Correctness. How to write 'compile time unit tests'
Immutability. Making your code predictable
Exhaustive pattern matching. A powerful technique to ensure correctness
Using the type system to ensure correct code. In F# the type system is your friend, not
your enemy
Worked example: Designing for correctness. How to make illegal states
unrepresentable
Concurrency. The next major revolution in how we write software?
Asynchronous programming. Encapsulating a background task with the Async class
Messages and Agents. Making it easier to think about concurrency
Functional Reactive Programming. Turning events into streams
Completeness. F# is part of the whole .NET ecosystem
Seamless interoperation with .NET libraries. Some convenient features for working with
.NET libraries
Anything C# can do.... A whirlwind tour of object-oriented code in F#
Why use F#: Conclusion.

Installing and using F#

The F# compiler is a free and open source tool which is available for Windows, Mac and
Linux (via Mono). Find out more about F# and how to install it at the F# Foundation.
You can use it with an IDE (Visual Studio, MonoDevelop), or with your favorite editor (VS
Code and Atom have especially good F# support using Ionide), or simply as a standalone
command line compiler.
If you don't want to install anything, you can try the .NET Fiddle site, which is an interactive
environment where you can explore F# in your web browser. You should be able to run most
of the code on this site there.

Working with the code examples

Once you have F# installed and running, you can follow along with the code samples.
The best way to run the code examples on this site is to type the code into an .FSX script
file, which you can then send to the F# interactive window for evaluation. Alternatively you
can type the examples directly into the F# interactive console window. I would recommend
the script file approach for anything other than one or two lines.
For the longer examples, the code is downloadable from this website -- the links will be in
the post.
Finally, I would encourage you to play with and modify the examples. If you then get
compiler errors, do check out the "troubleshooting F#" page, which explains the most
common problems, and how to fix them.

Projects and Solutions

F# uses exactly the same "projects" and "solutions" model that C# does, so if you are
familiar with that, you should be able to create an F# executable quite easily.
To make a file that will be compiled as part of the project, rather than a script file, use the
.fs extension. .fsx files will not be compiled.

An F# project does have some major differences from C# though:

The F# files are organized linearly, not in a hierarchy of folders and subfolders. In fact,
there is no "add new folder" option in an F# project! This is not generally a problem,
because, unlike C#, an F# file contains more than one class. What might be a whole
folder of classes in C# might easily be a single file in F#.
The order of the files in the project is very important: a "later" F# file can use the public
types defined in an "earlier" F# file, but not the other way around. Consequently, you
18

Installing and using F#

cannot have any circular dependencies between files.

You can change the order of the files by right-clicking and doing "Move Up" or "Move
Down". Similarly, when creating a new file, you can choose to "Add Above" or "Add
Below" an existing file.

Shell scripts in F#
You can also use F# as a scripting language, rather than having to compile code into an
EXE. This is done by using the FSI program, which is not only a console but can also be
used to run scripts in the same way that you might use Python or Powershell.
This is very convenient when you want to quickly create some code without compiling it into
a full blown application. The F# build automation system "FAKE" is an example of how
useful this can be.
To see how you can do this yourself, here is a little example script that downloads a web
page to a local file. First create an FSX script file -- call it " ShellScriptExample.fsx " -- and
paste in the following code.

Installing and using F#

// ================================
// Description:
// downloads the given url and stores it as a file with a timestamp
//
// Example command line:
// fsi ShellScriptExample.fsx http://google.com google
// ================================
// "open" brings a .NET namespace into visibility
open System.Net
open System
// download the contents of a web page
let downloadUriToFile url targetfile =
let req = WebRequest.Create(Uri(url))
use resp = req.GetResponse()
use stream = resp.GetResponseStream()
use reader = new IO.StreamReader(stream)
let timestamp = DateTime.UtcNow.ToString("yyyy-MM-ddTHH-mm")
let path = sprintf "%s.%s.html" targetfile timestamp
use writer = new IO.StreamWriter(path)
writer.Write(reader.ReadToEnd())
printfn "finished downloading %s to %s" url path
// Running from FSI, the script name is first, and other args after
match fsi.CommandLineArgs with
| [| scriptName; url; targetfile |] ->
printfn "running script: %s" scriptName
downloadUriToFile url targetfile
| _ ->
printfn "USAGE: [url] [targetfile]"

Don't worry about how the code works right now. It's pretty crude anyway, and a better
example would add error handling, and so on.
To run this script, open a command window in the same directory and type:
fsi ShellScriptExample.fsx http://google.com google_homepage

As you play with the code on this site, you might want to experiment with creating some
scripts at the same time.

F# syntax in 60 seconds

F# syntax in 60 seconds
Here is a very quick overview on how to read F# code for newcomers unfamiliar with the
syntax.
It is obviously not very detailed but should be enough so that you can read and get the gist
of the upcoming examples in this series. Don't worry if you don't understand all of it, as I will
give more detailed explanations when we get to the actual code examples.
The two major differences between F# syntax and a standard C-like syntax are:
Curly braces are not used to delimit blocks of code. Instead, indentation is used (Python
is similar this way).
Whitespace is used to separate parameters rather than commas.
Some people find the F# syntax off-putting. If you are one of them, consider this quote:
"Optimising your notation to not confuse people in the first 10 minutes of seeing it but to
hinder readability ever after is a really bad mistake." (David MacIver, via a post about
Scala syntax).
Personally, I think that the F# syntax is very clear and straightforward when you get used to
it. In many ways, it is simpler than the C# syntax, with fewer keywords and special cases.
The example code below is a simple F# script that demonstrates most of the concepts that
you need on a regular basis.
I would encourage you to test this code interactively and play with it a bit! Either:
Type this into a F# script file (with .fsx extension) and send it to the interactive window.
See the "installing and using F#" page for details.
Alternatively, try running this code in the interactive window. Remember to always use
;; at the end to tell the interpreter that you are done entering and ready to evaluate.

// single line comments use a double slash

(* multi line comments use (* . . . *) pair
-end of multi line comment- *)
// ======== "Variables" (but not really) ==========
// The "let" keyword defines an (immutable) value
let myInt = 5
let myFloat = 3.14
let myString = "hello" //note that no types needed

F# syntax in 60 seconds

// ======== Lists ============

let twoToFive = [2;3;4;5] // Square brackets create a list with
// semicolon delimiters.
let oneToFive = 1 :: twoToFive // :: creates list with new 1st element
// The result is [1;2;3;4;5]
let zeroToFive = [0;1] @ twoToFive // @ concats two lists
// IMPORTANT: commas are never used as delimiters, only semicolons!
// ======== Functions ========
// The "let" keyword also defines a named function.
let square x = x * x // Note that no parens are used.
square 3 // Now run the function. Again, no parens.
let add x y = x + y // don't use add (x,y)! It means something
// completely different.
add 2 3 // Now run the function.
// to define a multiline function, just use indents. No semicolons needed.
let evens list =
let isEven x = x%2 = 0 // Define "isEven" as an inner ("nested") function
List.filter isEven list // List.filter is a library function
// with two parameters: a boolean function
// and a list to work on
evens oneToFive // Now run the function
// You can use parens to clarify precedence. In this example,
// do "map" first, with two args, then do "sum" on the result.
// Without the parens, "List.map" would be passed as an arg to List.sum
let sumOfSquaresTo100 =
List.sum ( List.map square [1..100] )
// You can pipe the output of one operation to the next using "|>"
// Here is the same sumOfSquares function written using pipes
let sumOfSquaresTo100piped =
[1..100] |> List.map square |> List.sum // "square" was defined earlier
// you can define lambdas (anonymous functions) using the "fun" keyword
let sumOfSquaresTo100withFun =
[1..100] |> List.map (fun x->x*x) |> List.sum
// In F# returns are implicit -- no "return" needed. A function always
// returns the value of the last expression used.
// ======== Pattern Matching ========
// Match..with.. is a supercharged case/switch statement.
let simplePatternMatch =
let x = "a"
match x with
| "a" -> printfn "x is a"
| "b" -> printfn "x is b"
| _ -> printfn "x is something else" // underscore matches anything

F# syntax in 60 seconds

// Some(..) and None are roughly analogous to Nullable wrappers

F# syntax in 60 seconds

And with that, let's start by comparing some simple F# code with the equivalent C# code.

Learning F#

Functional languages are very different from standard imperative languages, and can be
quite tricky to get the hang of initially. This page offers some tips on how to learn F#
effectively.

Approach learning F# as a beginner

If you have experience in languages such as C# and Java, you have probably found that you
can get a pretty good understanding of source code written in other similar languages, even
if you aren't familiar with the keywords or the libraries. This is because all imperative
languages use the same way of thinking, and experience in one language can be easily
transferred to another.
If you are like many people, your standard approach to learning a new programming
language is to find out how to implement concepts you are already familiar with. You might
ask "how do I assign a variable?" or "how do I do a loop?", and with these answers be able
to do some basic programming quite quickly.
When learning F#, you should not try to bring your old imperative concepts with you.
In a pure functional language there are no variables, there are no loops, and there are no
objects!
Yes, F# is a hybrid language and does support these concepts. But you will learn much
faster if you start with a beginners mind.

Change the way you think

Learning F#

play with the examples, and get comfortable with the ideas before you try to start doing
serious coding. If you don't understand how functions and types work, then you will have a
hard time being productive.

Dos and Don'ts

Here is a list of dos and don'ts that will encourage you to think functionally. These will be
hard at first, but just like learning a foreign language, you have to dive in and force yourself
to speak like the locals.
Don't use the mutable keyword at all as a beginner. Coding complex functions without
the crutch of mutable state will really force you to understand the functional paradigm.
Don't use for loops or if-then-else . Use pattern matching for testing booleans and
recursing through lists.
Don't use "dot notation". Instead of "dotting into" objects, try to use functions for
everything. That is, write String.length "hello" rather than "hello".Length . It might
seem like extra work, but this way of working is essential when using pipes and higher
order functions like List.map . And don't write your own methods either! See this post
for details.
As a corollary, don't create classes. Use only the pure F# types such as tuples, records
and unions.
Don't use the debugger. If you have relied on the debugger to find and fix incorrect
code, you will get a nasty shock. In F#, you will probably not get that far, because the
compiler is so much stricter in many ways. And of course, there is no tool to "debug" the
compiler and step through its processing. The best tool for debugging compiler errors is
your brain, and F# forces you to use it!
On the other hand:
Do create lots of "little types", especially union types. They are lightweight and easy,
and their use will help document your domain model and ensure correctness.
Do understand the list and seq types and their associated library modules.
Functions like List.fold and List.map are very powerful. Once you understand how
to use them, you will be well on your way to understanding higher order functions in
general.
Once you understand the collection modules, try to avoid recursion. Recursion can be
error prone, and it can be hard to make sure that it is properly tail-recursive. When you
use List.fold , you can never have that problem.
Do use pipe ( |> ) and composition ( >> ) as much as you can. This style is much more
idiomatic than nested function calls like f(g(x))
Do understand how partial application works, and try to become comfortable with point-

Learning F#

free (tacit) style.

Do develop code incrementally, using the interactive window to test code fragments. If
you blindly create lots of code and then try to compile it all at once, you may end up with
many painful and hard-to-debug compilation errors.

Troubleshooting
There are a number of extremely common errors that beginners make, and if you are
frustrated about getting your code to compile, please read the "troubleshooting F#" page.

Troubleshooting F#

As the saying goes, "if it compiles, it's correct", but it can be extremely frustrating just trying
to get the code to compile at all! So this page is devoted to helping you troubleshoot your F#
code.
I will first present some general advice on troubleshooting and some of the most common
errors that beginners make. After that, I will describe each of the common error messages in
detail, and give examples of how they can occur and how to correct them.
(Jump to the error numbers)

General guidelines for troubleshooting

By far the most important thing you can do is to take the time and effort to understand
exactly how F# works, especially the core concepts involving functions and the type system.
So please read and reread the series "thinking functionally" and "understanding F# types",
play with the examples, and get comfortable with the ideas before you try to start doing
serious coding. If you don't understand how functions and types work, then the compiler
errors will not make any sense.
If you are coming from an imperative language such as C#, you may have developed some
bad habits by relying on the debugger to find and fix incorrect code. In F#, you will probably
not get that far, because the compiler is so much stricter in many ways. And of course, there
is no tool to "debug" the compiler and step through its processing. The best tool for
debugging compiler errors is your brain, and F# forces you to use it!
Nevertheless, there are a number of extremely common errors that beginners make, and I
will quickly go through them.

Don't use parentheses when calling a function

In F#, whitespace is the standard separator for function parameters. You will rarely need to
use parentheses, and in particular, do not use parentheses when calling a function.
let add x y = x + y
let result = add (1 2) //wrong
// error FS0003: This value is not a function and cannot be applied
let result = add 1 2 //correct

Don't mix up tuples with multiple parameters

If it has a comma, it is a tuple. And a tuple is one object not two. So you will get errors about
passing the wrong type of parameter, or too few parameters.

Troubleshooting F#

addTwoParams (1,2) // trying to pass a single tuple rather than two args
// error FS0001: This expression was expected to have type
// int but here has type 'a * 'b

The compiler treats (1,2) as a generic tuple, which it attempts to pass to " addTwoParams ".
Then it complains that the first parameter of addTwoParams is an int, and we're trying to pass
a tuple.
If you attempt to pass two arguments to a function expecting one tuple, you will get another
obscure error.
addTuple 1 2 // trying to pass two args rather than one tuple
// error FS0003: This value is not a function and cannot be applied

Watch out for too few or too many arguments

The F# compiler will not complain if you pass too few arguments to a function (in fact "partial
application" is an important feature), but if you don't understand what is going on, you will
often get strange "type mismatch" errors later.
Similarly the error for having too many arguments is typically "This value is not a function"
rather than a more straightforward error.
The "printf" family of functions is very strict in this respect. The argument count must be
exact.
This is a very important topic ? it is critical that you understand how partial application works.
See the series "thinking functionally" for a more detailed discussion.

Use semicolons for list separators

In the few places where F# needs an explicit separator character, such as lists and records,
the semicolon is used. Commas are never used. (Like a broken record, I will remind you that
commas are for tuples).
let list1 = [1,2,3] // wrong! This is a ONE-element list containing
// a three-element tuple
let list1 = [1;2;3] // correct
type Customer = {Name:string, Address: string} // wrong
type Customer = {Name:string; Address: string} // correct

Don't use ! for not or != for not-equal

Troubleshooting F#

The exclamation point symbol is not the "NOT" operator. It is the deferencing operator for
mutable references. If you use it by mistake, you will get the following error:
let y = true
let z = !y
// => error FS0001: This expression was expected to have
// type 'a ref but here has type bool

The correct construction is to use the "not" keyword. Think SQL or VB syntax rather than C
syntax.
let y = true
let z = not y //correct

And for "not equal", use "<>", again like SQL or VB.
let z = 1 <> 2 //correct

Don't use = for assignment

If you are using mutable values, the assignment operation is written " <- ". If you use the
equals symbol you might not even get an error, just an unexpected result.
let mutable x = 1
x = x + 1 // returns false. x is not equal to x+1
x <- x + 1 // assigns x+1 to x

Watch out for hidden tab characters

The indenting rules are very straightforward, and it is easy to get the hang of them. But you
are not allowed to use tabs, only spaces.
let add x y =
{tab}x + y
// => error FS1161: TABs are not allowed in F# code

Be sure to set your editor to convert tabs to spaces. And watch out if you are pasting code in
from elsewhere. If you do run into persistent problems with a bit of code, try removing the
whitespace and re-adding it.

Don't mistake simple values for function values

Troubleshooting F#

If you are trying to create a function pointer or delegate, watch out that you don't accidentally
create a simple value that has already been evaluated.
If you want a parameterless function that you can reuse, you will need to explicitly pass a
unit parameter, or define it as a lambda.
let reader = new System.IO.StringReader("hello")
let nextLineFn = reader.ReadLine() //wrong
let nextLineFn() = reader.ReadLine() //correct
let nextLineFn = fun() -> reader.ReadLine() //correct
let r = new System.Random()
let randomFn = r.Next() //wrong
let randomFn() = r.Next() //correct
let randomFn = fun () -> r.Next() //correct

See the series "thinking functionally" for more discussion of parameterless functions.

Tips for troubleshooting "not enough information" errors

The F# compiler is currently a one-pass left-to-right compiler, and so type information later in
the program is unavailable to the compiler if it hasn't been parsed yet.
A number of errors can be caused by this, such as "FS0072: Lookup on object of
indeterminate type" and "FS0041: A unique overload for could not be determined". The
suggested fixes for each of these specific cases are described below, but there are some
general principles that can help if the compiler is complaining about missing types or not
enough information. These guidelines are:
Define things before they are used (this includes making sure the files are compiled in
the right order)
Put the things that have "known types" earlier than things that have "unknown types". In
particular, you might be able reorder pipes and similar chained functions so that the
typed objects come first.
Annotate as needed. One common trick is to add annotations until everything works,
and then take them away one by one until you have the minimum needed.
Do try to avoid annotating if possible. Not only is it not aesthetically pleasing, but it makes
the code more brittle. It is a lot easier to change types if there are no explicit dependencies
on them.

F# compiler errors
31

Troubleshooting F#

A listing of common errors, ordered by error number

Here is a list of the major errors that seem to me worth documenting. I have not documented
any errors that are self explanatory, only those that seem obscure to beginners.
I will continue to add to the list in the future, and I welcome any suggestions for additions.
FS0001: The type 'X' does not match the type 'Y'
FS0003: This value is not a function and cannot be applied
FS0008: This runtime coercion or type test involves an indeterminate type
FS0010: Unexpected identifier in binding
FS0010: Incomplete structured construct
FS0013: The static coercion from type X to Y involves an indeterminate type
FS0020: This expression should have type 'unit'
FS0030: Value restriction
FS0035: This construct is deprecated
FS0039: The field, constructor or member X is not defined
FS0041: A unique overload for could not be determined
FS0049: Uppercase variable identifiers should not generally be used in patterns
FS0072: Lookup on object of indeterminate type
FS0588: Block following this 'let' is unfinished

FS0001: The type 'X' does not match the type

'Y'
This is probably the most common error you will run into. It can manifest itself in a wide
variety of contexts, so I have grouped the most common problems together with examples
and fixes. Do pay attention to the error message, as it is normally quite explicit about what
the problem is.

Troubleshooting F#

Error message

Possible causes

The type 'float' does not match the type 'int'

A. Can't mix floats and ints

The type 'int' does not support any operators named

'DivideByInt'

A. Can't mix floats and ints.

The type 'X' is not compatible with any of the types

B. Using the wrong

numeric type.

This type (function type) does not match the type (simple
type). Note: function types have a arrow in them, like 'a
-> 'b .

C. Passing too many

arguments to a function.

This expression was expected to have (function type) but

here has (simple type)

C. Passing too many

arguments to a function.

This expression was expected to have (N part function)

but here has (N-1 part function)

C. Passing too many

arguments to a function.

This expression was expected to have (simple type) but

here has (function type)

D. Passing too few

arguments to a function.

This expression was expected to have (type) but here

has (other type)

E. Straightforward type
mismatch.
F. Inconsistent returns in
branches or matches.
G. Watch out for type
inference effects buried in
a function.

Type mismatch. Expecting a (simple type) but given a

(tuple type). Note: tuple types have a star in them, like
'a * 'b .

H. Have you used a

comma instead of space or
semicolon?

Type mismatch. Expecting a (tuple type) but given a

(different tuple type).

I. Tuples must be the same

type to be compared.

This expression was expected to have type 'a ref but

here has type X

J. Don't use ! as the "not"

operator.

The type (type) does not match the type (other type)

K. Operator precedence
(especially functions and
pipes).

This expression was expected to have type (monadic

type) but here has type 'b * 'c

L. let! error in computation

expressions.

A. Can't mix ints and floats

Unlike C# and most imperative languages, ints and floats cannot be mixed in expressions.
You will get a type error if you attempt this:

Troubleshooting F#

1 + 2.0 //wrong
// => error FS0001: The type 'float' does not match the type 'int'

The fix is to cast the int into a float first:

float 1 + 2.0 //correct

This issue can also manifest itself in library functions and other places. For example, you
cannot do " average " on a list of ints.
[1..10] |> List.average // wrong
// => error FS0001: The type 'int' does not support any
// operators named 'DivideByInt'

You must cast each int to a float first, as shown below:

[1..10] |> List.map float |> List.average //correct
[1..10] |> List.averageBy float //correct (uses averageBy)

B. Using the wrong numeric type

You will get a "not compatible" error when a numeric cast failed.
printfn "hello %i" 1.0 // should be a int not a float
// error FS0001: The type 'float' is not compatible
// with any of the types byte,int16,int32...

One possible fix is to cast it if appropriate.

printfn "hello %i" (int 1.0)

C. Passing too many arguments to a function

let add x y = x + y
let result = add 1 2 3
// ==> error FS0001: The type ''a -> 'b' does not match the type 'int'

The clue is in the error.

The fix is to remove one of the arguments!

Troubleshooting F#

Similar errors are caused by passing too many arguments to printf .

printfn "hello" 42
// ==> error FS0001: This expression was expected to have type 'a -> 'b
// but here has type unit
printfn "hello %i" 42 43
// ==> Error FS0001: Type mismatch. Expecting a 'a -> 'b -> 'c
// but given a 'a -> unit
printfn "hello %i %i" 42 43 44
// ==> Error FS0001: Type mismatch. Expecting a 'a -> 'b -> 'c -> 'd
// but given a 'a -> 'b -> unit

D. Passing too few arguments to a function

If you do not pass enough arguments to a function, you will get a partial application. When
you later use it, you get an error because it is not a simple type.
let reader = new System.IO.StringReader("hello");
let line = reader.ReadLine //wrong but compiler doesn't complain
printfn "The line is %s" line //compiler error here!
// ==> error FS0001: This expression was expected to have type string
// but here has type unit -> string

This is particularly common for some .NET library functions that expect a unit parameter,
such as ReadLine above.
The fix is to pass the correct number of parameters. Check the type of the result value to
make sure that it is indeed a simple type. In the ReadLine case, the fix is to pass a ()
argument.
let line = reader.ReadLine() //correct
printfn "The line is %s" line //no compiler error

E. Straightforward type mismatch

The simplest case is that you have the wrong type, or you are using the wrong type in a print
format string.
printfn "hello %s" 1.0
// => error FS0001: This expression was expected to have type string
// but here has type float

Troubleshooting F#

F. Inconsistent return types in branches or matches

A common mistake is that if you have a branch or match expression, then every branch
MUST return the same type. If not, you will get a type error.
let f x =
if x > 1 then "hello"
else 42
// => error FS0001: This expression was expected to have type string
// but here has type int

let g x =
match x with
| 1 -> "hello"
| _ -> 42
// error FS0001: This expression was expected to have type
// string but here has type int

Obviously, the straightforward fix is to make each branch return the same type.
let f x =
if x > 1 then "hello"
else "42"
let g x =
match x with
| 1 -> "hello"
| _ -> "42"

Remember that if an "else" branch is missing, it is assumed to return unit, so the "true"
branch must also return unit.
let f x =
if x > 1 then "hello"
// error FS0001: This expression was expected to have type
// unit but here has type string

If both branches cannot return the same type, you may need to create a new union type that
can contain both types.
type StringOrInt = | S of string | I of int // new union type
let f x =
if x > 1 then S "hello"
else I 42

Troubleshooting F#

G. Watch out for type inference effects buried in a function

A function may cause an unexpected type inference that ripples around your code. For
example, in the following, the innocent print format string accidentally causes doSomething
to expect a string.
let doSomething x =
// do something
printfn "x is %s" x
// do something more
doSomething 1
// => error FS0001: This expression was expected to have type string
// but here has type int

The fix is to check the function signatures and drill down until you find the guilty party. Also,
use the most generic types possible, and avoid type annotations if possible.

H. Have you used a comma instead of space or semicolon?

If you are new to F#, you might accidentally use a comma instead of spaces to separate
function arguments:
// define a two parameter function
let add x y = x + 1
add(x,y) // FS0001: This expression was expected to have
// type int but here has type 'a * 'b

The fix is: don't use a comma!

add x y // OK

One area where commas are used is when calling .NET library functions. These all take
tuples as arguments, so the comma form is correct. In fact, these calls look just the same as
they would from C#:
// correct
System.String.Compare("a","b")
// incorrect
System.String.Compare "a" "b"

Troubleshooting F#

I. Tuples must be the same type to be compared or pattern

matched
Tuples with different types cannot be compared. Trying to compare a tuple of type int *
int , with a tuple of type int * string results in an error:

let t1 = (0, 1)
let t2 = (0, "hello")
t1 = t2
// => error FS0001: Type mismatch. Expecting a int * int
// but given a int * string
// The type 'int' does not match the type 'string'

And the length must be the same:

let t1 = (0, 1)
let t2 = (0, 1, "hello")
t1 = t2
// => error FS0001: Type mismatch. Expecting a int * int
// but given a int * int * string
// The tuples have differing lengths of 2 and 3

You can get the same issue when pattern matching tuples during binding:
let x,y = 1,2,3
// => error FS0001: Type mismatch. Expecting a 'a * 'b
// but given a 'a * 'b * 'c
// The tuples have differing lengths of 2 and 3
let f (x,y) = x + y
let z = (1,"hello")
let result = f z
// => error FS0001: Type mismatch. Expecting a int * int
// but given a int * string
// The type 'int' does not match the type 'string'

J. Don't use ! as the "not" operator

If you use ! as a "not" operator, you will get a type error mentioning the word "ref".
let y = true
let z = !y //wrong
// => error FS0001: This expression was expected to have
// type 'a ref but here has type bool

Troubleshooting F#

The fix is to use the "not" keyword instead.

let y = true
let z = not y //correct

K. Operator precedence (especially functions and pipes)

If you mix up operator precedence, you may get type errors. Generally, function application
is highest precedence compared to other operators, so you get an error in the case below:
String.length "hello" + "world"
// => error FS0001: The type 'string' does not match the type 'int'
// what is really happening
(String.length "hello") + "world"

The fix is to use parentheses.

String.length ("hello" + "world") // corrected

Conversely, the pipe operator is low precedence compared to other operators.

let result = 42 + [1..10] |> List.sum
// => => error FS0001: The type ''a list' does not match the type 'int'
// what is really happening
let result = (42 + [1..10]) |> List.sum

Again, the fix is to use parentheses.

let result = 42 + ([1..10] |> List.sum)

L. let! error in computation expressions (monads)

Here is a simple computation expression:

Troubleshooting F#

type Wrapper<'a> = Wrapped of 'a

type wrapBuilder() =
member this.Bind (wrapper:Wrapper<'a>) (func:'a->Wrapper<'b>) =
match wrapper with
| Wrapped(innerThing) -> func innerThing
member this.Return innerThing =
Wrapped(innerThing)
let wrap = new wrapBuilder()

However, if you try to use it, you get an error.

wrap {
let! x1 = Wrapped(1) // <== error here
let! y1 = Wrapped(2)
let z1 = x + y
return z
}
// error FS0001: This expression was expected to have type Wrapper<'a>
// but here has type 'b * 'c

The reason is that " Bind " expects a tuple (wrapper,func) , not two parameters. (Check the
signature for bind in the F# documentation).
The fix is to change the bind function to accept a tuple as its (single) parameter.
type wrapBuilder() =
member this.Bind (wrapper:Wrapper<'a>, func:'a->Wrapper<'b>) =
match wrapper with
| Wrapped(innerThing) -> func innerThing

FS0003: This value is not a function and

cannot be applied
This error typically occurs when passing too many arguments to a function.
let add1 x = x + 1
let x = add1 2 3
// ==> error FS0003: This value is not a function and cannot be applied

It can also occur when you do operator overloading, but the operators cannot be used as
prefix or infix.
40

Troubleshooting F#

let (!!) x y = x + y
(!!) 1 2 // ok
1 !! 2 // failed !! cannot be used as an infix operator
// error FS0003: This value is not a function and cannot be applied

FS0008: This runtime coercion or type test

involves an indeterminate type
You will often see this when attempting to use " :? " operator to match on a type.
let detectType v =
match v with
| :? int -> printfn "this is an int"
| _ -> printfn "something else"
// error FS0008: This runtime coercion or type test from type 'a to int
// involves an indeterminate type based on information prior to this program point.
// Runtime type tests are not allowed on some types. Further type annotations are need
ed.

The message tells you the problem: "runtime type tests are not allowed on some types".
The answer is to "box" the value which forces it into a reference type, and then you can type
check it:
let detectTypeBoxed v =
match box v with // used "box v"
| :? int -> printfn "this is an int"
| _ -> printfn "something else"
//test
detectTypeBoxed 1
detectTypeBoxed 3.14

FS0010: Unexpected identifier in binding

Typically caused by breaking the "offside" rule for aligning expressions in a block.
//3456789
let f =
let x=1 // offside line is at column 3
x+1 // oops! don't start at column 4
// error FS0010: Unexpected identifier in binding

Troubleshooting F#

The fix is to align the code correctly!

See also FS0588: Block following this 'let' is unfinished for another issue caused by
alignment.

FS0010: Incomplete structured construct

Often occurs if you are missing parentheses from a class constructor:
type Something() =
let field = ()
let x1 = new Something // Error FS0010
let x2 = new Something() // OK!

Can also occur if you forgot to put parentheses around an operator:

// define new operator
let (|+) a = -a
|+ 1 // error FS0010:
// Unexpected infix operator
(|+) 1 // with parentheses -- OK!

Can also occur if you are missing one side of an infix operator:
|| true // error FS0010: Unexpected symbol '||'
false || true // OK

Can also occur if you attempt to send a namespace definition to F# interactive. The
interactive console does not allow namespaces.
namespace Customer // FS0010: Incomplete structured construct
// declare a type
type Person= {First:string; Last:string}

FS0013: The static coercion from type X to Y

involves an indeterminate type

Troubleshooting F#

This is generally caused by implic

FS0020: This expression should have type

'unit'
This error is commonly found in two situations:
Expressions that are not the last expression in the block
Using wrong assignment operator

FS0020 with expressions that are not the last expression in

the block
Only the last expression in a block can return a value. All others must return unit. So this
typically occurs when you have a function in a place that is not the last function.
let something =
2+2 // => FS0020: This expression should have type 'unit'
"hello"

The easy fix is use ignore . But ask yourself why you are using a function and then
throwing away the answer ? it might be a bug.
let something =
2+2 |> ignore // ok
"hello"

This also occurs if you think you writing C# and you accidentally use semicolons to separate
expressions:
// wrong
let result = 2+2; "hello";
// fixed
let result = 2+2 |> ignore; "hello";

FS0020 with assignment

Another variant of this error occurs when assigning to a property.
This expression should have type 'unit', but has type 'Y'.

Troubleshooting F#

With this error, chances are you have confused the assignment operator " <- " for mutable
values, with the equality comparison operator " = ".
// '=' versus '<-'
let add() =
let mutable x = 1
x = x + 1 // warning FS0020
printfn "%d" x

The fix is to use the proper assignment operator.

// fixed
let add() =
let mutable x = 1
x <- x + 1
printfn "%d" x

FS0030: Value restriction

This is related to F#'s automatic generalization to generic types whenever possible.
For example, given :
let id x = x
let compose f g x = g (f x)
let opt = None

F#'s type inference will cleverly figure out the generic types.
val id : 'a -> 'a
val compose : ('a -> 'b) -> ('b -> 'c) -> 'a -> 'c
val opt : 'a option

However in some cases, the F# compiler feels that the code is ambiguous, and, even though
it looks like it is guessing the type correctly, it needs you to be more specific:
let idMap = List.map id // error FS0030
let blankConcat = String.concat "" // error FS0030

Almost always this will be caused by trying to define a partially applied function, and almost
always, the easiest fix is to explicitly add the missing parameter:

Troubleshooting F#

let idMap list = List.map id list // OK

let blankConcat list = String.concat "" list // OK

For more details see the MSDN article on "automatic generalization".

FS0035: This construct is deprecated

F# syntax has been cleaned up over the last few years, so if you are using examples from
an older F# book or webpage, you may run into this. See the MSDN documentation for the
correct syntax.
let x = 10
let rnd1 = System.Random x // Good
let rnd2 = new System.Random(x) // Good
let rnd3 = new System.Random x // error FS0035

FS0039: The field, constructor or member X is

not defined
This error is commonly found in four situations:
The obvious case where something really isn't defined! And make sure that you don't
have a typo or case mismatch either.
Interfaces
Recursion
Extension methods

FS0039 with interfaces

In F# all interfaces are "explicit" implementations rather than "implicit". (Read the C#
documentation on "explicit interface implementation" for an explanation of the difference).
The key point is that when a interface member is explicitly implemented, it cannot be
accessed through a normal class instance, but only through an instance of the interface, so
you have to cast to the interface type by using the :> operator.
Here's an example of a class that implements an interface:

Troubleshooting F#

type MyResource() =
interface System.IDisposable with
member this.Dispose() = printfn "disposed"

This doesn't work:

let x = new MyResource()
x.Dispose() // error FS0039: The field, constructor
// or member 'Dispose' is not defined

The fix is to cast the object to the interface, as below:

// fixed by casting to System.IDisposable
(x :> System.IDisposable).Dispose() // OK
let y = new MyResource() :> System.IDisposable
y.Dispose() // OK

FS0039 with recursion

Here's a standard Fibonacci implementation:
let fib i =
match i with
| 1 -> 1
| 2 -> 1
| n -> fib(n-1) + fib(n-2)

Unfortunately, this will not compile:

Error FS0039: The value or constructor 'fib' is not defined

The reason is that when the compiler sees 'fib' in the body, it doesn't know about the function
because it hasn't finished compiling it yet!
The fix is to use the " rec " keyword.
let rec fib i =
match i with
| 1 -> 1
| 2 -> 1
| n -> fib(n-1) + fib(n-2)

Troubleshooting F#

Note that this only applies to " let " functions. Member functions do not need this, because
the scope rules are slightly different.
type FibHelper() =
member this.fib i =
match i with
| 1 -> 1
| 2 -> 1
| n -> fib(n-1) + fib(n-2)

FS0039 with extension methods

If you have defined an extension method, you won't be able to use it unless the module is in
scope.
Here's a simple extension to demonstrate:
module IntExtensions =
type System.Int32 with
member this.IsEven = this % 2 = 0

If you try to use it the extension, you get the FS0039 error:
let i = 2
let result = i.IsEven
// FS0039: The field, constructor or
// member 'IsEven' is not defined

The fix is just to open the IntExtensions module.

open IntExtensions // bring module into scope
let i = 2
let result = i.IsEven // fixed!

FS0041: A unique overload for could not be

determined
This can be caused when calling a .NET library function that has multiple overloads:
let streamReader filename = new System.IO.StreamReader(filename) // FS0041

Troubleshooting F#

There a number of ways to fix this. One way is to use an explicit type annotation:
let streamReader filename = new System.IO.StreamReader(filename:string) // OK

You can sometimes use a named parameter to avoid the type annotation:
let streamReader filename = new System.IO.StreamReader(path=filename) // OK

Or you can try to create intermediate objects that help the type inference, again without
needing type annotations:
let streamReader filename =
let fileInfo = System.IO.FileInfo(filename)
new System.IO.StreamReader(fileInfo.FullName) // OK

FS0049: Uppercase variable identifiers should

not generally be used in patterns
When pattern matching, be aware of a subtle difference between the pure F# union types
which consist of a tag only, and a .NET Enum type.
Pure F# union type:
type ColorUnion = Red | Yellow
let redUnion = Red
match redUnion with
| Red -> printfn "red" // no problem
| _ -> printfn "something else"

But with .NET enums you must fully qualify them:

type ColorEnum = Green=0 | Blue=1 // enum
let blueEnum = ColorEnum.Blue
match blueEnum with
| Blue -> printfn "blue" // warning FS0049
| _ -> printfn "something else"

The fixed version:

Troubleshooting F#

match blueEnum with

| ColorEnum.Blue -> printfn "blue"
| _ -> printfn "something else"

FS0072: Lookup on object of indeterminate

type
This occurs when "dotting into" an object whose type is unknown.
Consider the following example:
let stringLength x = x.Length // Error FS0072

The compiler does not know what type "x" is, and therefore does not know if " Length " is a
valid method.
There a number of ways to fix this. The crudest way is to provide an explicit type annotation:
let stringLength (x:string) = x.Length // OK

In some cases though, judicious rearrangement of the code can help. For example, the
example below looks like it should work. It's obvious to a human that the List.map function
is being applied to a list of strings, so why does x.Length cause an error?
List.map (fun x -> x.Length) ["hello"; "world"] // Error FS0072

The reason is that the F# compiler is currently a one-pass compiler, and so type information
present later in the program cannot be used if it hasn't been parsed yet.
Yes, you can always explicitly annotate:
List.map (fun x:string -> x.Length) ["hello"; "world"] // OK

But another, more elegant way that will often fix the problem is to rearrange things so the
known types come first, and the compiler can digest them before it moves to the next clause.
["hello"; "world"] |> List.map (fun x -> x.Length) // OK

It's good practice to avoid explicit type annotations, so this approach is best, if it is feasible.

Troubleshooting F#

FS0588: Block following this 'let' is unfinished

Caused by outdenting an expression in a block, and thus breaking the "offside rule".
//3456789
let f =
let x=1 // offside line is at column 3
x+1 // offside! You are ahead of the ball!
// error FS0588: Block following this
// 'let' is unfinished

The fix is to align the code correctly.

See also FS0010: Unexpected identifier in binding for another issue caused by alignment.

Low-risk ways to use F# at work

So you're all excited about functional programming, and you've been learning F# in your
spare time, and you're annoying your co-workers by ranting about how great it is, and you're
itching to use it for serious stuff at work...
But then you hit a brick wall.
Your workplace has a "C# only" policy and won't let you use F#.
If you work in a typical enterprise environment, getting a new language approved will be a
long drawn out process, involving persuading your teammates, the QA guys, the ops guys,
your boss, your boss's boss, and the mysterious bloke down the hall who you've never
talked to. I would encourage you to start that process (a helpful link for your manager), but
still, you're impatient and thinking "what can I do now?"
On the other hand, perhaps you work in a flexible, easy going place, where you can do what
you like.
But you're conscientious, and don't want to be one of those people who re-write some
mission critical system in APL, and then vanish without trace, leaving your replacement
some mind-bendingly cryptic code to maintain. No, you want to make sure that you are not
doing anything that will affect your team's bus factor.
So in both these scenarios, you want to use F# at work, but you can't (or don't want to) use it
for core application code.
What can you do?
Well, don't worry! This series will suggest a number of ways you can get your hands dirty
with F# in a low-risk, incremental way, without affecting any mission critical code.
Twenty six low-risk ways to use F# at work. You can start right now -- no permission
needed.
Using F# for development and devops scripts. Twenty six low-risk ways to use F# at
work (part 2).
Using F# for testing. Twenty six low-risk ways to use F# at work (part 3).
Using F# for database related tasks. Twenty six low-risk ways to use F# at work (part 4).
Other interesting ways of using F# at work. Twenty six low-risk ways to use F# at work
(part 5).

Twenty six low-risk ways to use F# at work

So you're all excited about functional programming, and you've been learning F# in your
spare time, and you're annoying your co-workers by ranting about how great it is, and you're
itching to use it for serious stuff at work...
But then you hit a brick wall.
Your workplace has a "C# only" policy and won't let you use F#.
If you work in a typical enterprise environment, getting a new language approved will be a
long drawn out process, involving persuading your teammates, the QA guys, the ops guys,
your boss, your boss's boss, and the mysterious bloke down the hall who you've never
talked to. I would encourage you to start that process (a helpful link for your manager), but
still, you're impatient and thinking "what can I do now?"
On the other hand, perhaps you work in a flexible, easy going place, where you can do what
you like.
But you're conscientious, and don't want to be one of those people who re-write some
mission critical system in APL, and then vanish without trace, leaving your replacement
some mind-bendingly cryptic code to maintain. No, you want to make sure that you are not
doing anything that will affect your team's bus factor.
So in both these scenarios, you want to use F# at work, but you can't (or don't want to) use it
for core application code.
What can you do?
Well, don't worry! This series of articles will suggest a number of ways you can get your
hands dirty with F# in a low-risk, incremental way, without affecting any critical code.

Series contents
Here's a list of the twenty six ways so that you can go straight to any one that you find
particularly interesting.
Part 1 - Using F# to explore and develop interactively
1. Use F# to explore the .NET framework interactively
2. Use F# to test your own code interactively
3. Use F# to play with webservices interactively

Twenty six low-risk ways to use F# at work

4. Use F# to play with UI's interactively

Part 2 - Using F# for development and devops scripts
5. Use FAKE for build and CI scripts
6. An F# script to check that a website is responding
7. An F# script to convert an RSS feed into CSV
8. An F# script that uses WMI to check the stats of a process
9. Use F# for configuring and managing the cloud
Part 3 - Using F# for testing
10. Use F# to write unit tests with readable names
11. Use F# to run unit tests programmatically
12. Use F# to learn to write unit tests in other ways
13. Use FsCheck to write better unit tests
14. Use FsCheck to create random dummy data
15. Use F# to create mocks
16. Use F# to do automated browser testing
17. Use F# for Behaviour Driven Development
Part 4. Using F# for database related tasks
18. Use F# to replace LINQpad
19. Use F# to unit test stored procedures
20. Use FsCheck to generate random database records
21. Use F# to do simple ETL
22. Use F# to generate SQL Agent scripts
Part 5: Other interesting ways of using F#
23. Use F# for parsing
24. Use F# for diagramming and visualization
25. Use F# for accessing web-based data stores
26. Use F# for data science and machine learning
(BONUS) 27: Balance the generation schedule for the UK power station fleet

Getting started
If you're using Visual Studio, you've already got F# installed, so you're ready to go! No need
to ask anyone's permission.
If you're on a Mac or Linux, you will have to a bit of work, alas (instructions for Mac and
Linux).

Twenty six low-risk ways to use F# at work

There are two ways to use F# interactively: (1) typing in the F# interactive window directly, or
(2) creating a F# script file (.FSX) and then evaluating code snippets.
To use the F# interactive window in Visual Studio:
1. Show the window with Menu > View > Other Windows > F# Interactive
2. Type an expression, and use double semicolon ( ;; ) to tell the interpreter you're
finished.
For example:
let x = 1
let y = 2
x + y;;

Personally, I prefer to create a script file ( File > New > File then pick "F# script") and type
code there, because you get auto-complete and intellisense.

To run a bit of code, just highlight and right click, or simply do Alt+Enter .

Twenty six low-risk ways to use F# at work

Working with external libraries and NuGet

Most of the code samples reference external libraries which are expected to be under the
script directory.
You could download or compile these DLLs explicitly, but I think using NuGet from the
command line is simpler.
1. First, you need to install Chocolately (from chocolatey.org)
2. Next install the NuGet command line using cinst nuget.commandline
3. Finally, go to your script directory, and install the NuGet package from the command
line.
For example, nuget install FSharp.Data -o Packages -ExcludeVersion
As you see, I prefer to exclude versions from Nuget packages when using them from
scripts so that I can update later without breaking existing code.

Part 1: Using F# to explore and develop

interactively
The first area where F# is valuable is as a tool to interactively explore .NET libraries.
Before, in order to do this, you might have created unit tests and then stepped through them
with a debugger to understand what is happening. But with F#, you don't need to do that,
you can run the code directly.
Let's look at some examples.

Twenty six low-risk ways to use F# at work

1. Use F# to explore the .NET framework

interactively
The code for this section is available on github.
When I'm coding, I often have little questions about how the .NET library works.
For example, here are some questions that I have had recently that I answered by using F#
interactively:
Have I got a custom DateTime format string correct?
How does XML serialization handle local DateTimes vs. UTC DateTimes?
Is GetEnvironmentVariable case-sensitive?
All these questions can be found in the MSDN documentation, of course, but can also
answered in seconds by running some simple F# snippets, shown below.

Have I got a custom DateTime format string correct?

I want to use 24 hour clock in a custom format. I know that it's "h", but is it upper or
lowercase "h"?
open System
DateTime.Now.ToString("yyyy-MM-dd hh:mm") // "2014-04-18 01:08"
DateTime.Now.ToString("yyyy-MM-dd HH:mm") // "2014-04-18 13:09"

How does XML serialization handle local DateTimes vs.

UTC DateTimes?
How exactly, does XML serialization work with dates? Let's find out!

Twenty six low-risk ways to use F# at work

// TIP: sets the current directory to be same as the script directory

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
open System
[<CLIMutable>]
type DateSerTest = {Local:DateTime;Utc:DateTime}
let ser = new System.Xml.Serialization.XmlSerializer(typeof<DateSerTest>)
let testSerialization (dt:DateSerTest) =
let filename = "serialization.xml"
use fs = new IO.FileStream(filename , IO.FileMode.Create)
ser.Serialize(fs, o=dt)
fs.Close()
IO.File.ReadAllText(filename) |> printfn "%s"
let d = {
Local = DateTime.SpecifyKind(new DateTime(2014,7,4), DateTimeKind.Local)
Utc = DateTime.SpecifyKind(new DateTime(2014,7,4), DateTimeKind.Utc)
}
testSerialization d

The output is:

So I can see it uses "Z" for UTC times.

Is GetEnvironmentVariable case-sensitive?
This can be answered with a simple snippet:
Environment.GetEnvironmentVariable "ProgramFiles" =
Environment.GetEnvironmentVariable "PROGRAMFILES"
// answer => true

The answer is therefore "not case-sensitive".

2. Use F# to test your own code interactively

Twenty six low-risk ways to use F# at work

The code for this section is available on github.

You are not restricted to playing with the .NET libraries, of course. Sometimes it can be quite
useful to test your own code.
To do this, just reference the DLL and then open the namespace as shown below.

// set the current directory to be same as the script directory

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// pass in the relative path to the DLL
#r @"bin\debug\myapp.dll"
// open the namespace
open MyApp
// do something
MyApp.DoSomething()

WARNING: in older versions of F#, opening a reference to your DLL will lock it so that you
can't compile it! In which case, before recompiling, be sure to reset the interactive session to
release the lock. In newer versions of F#, the DLL is shadow-copied, and there is no lock.

3. Use F# to play with webservices interactively

The code for this section is available on github.
If you want to play with the WebAPI and Owin libraries, you don't need to create an
executable -- you can do it through script alone!
There is a little bit of setup involved, as you will need a number of library DLLs to make this
work.
So, assuming you have got the NuGet command line set up (see above), go to your script
directory, and install the self hosting libraries via nuget install
Microsoft.AspNet.WebApi.OwinSelfHost -o Packages -ExcludeVersion

Once these libraries are in place, you can use the code below as a skeleton for a simple
WebAPI app.
// sets the current directory to be same as the script directory
System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// assumes nuget install Microsoft.AspNet.WebApi.OwinSelfHost has been run
// so that assemblies are available under the current directory

Twenty six low-risk ways to use F# at work

#r @"Packages\Owin\lib\net40\Owin.dll"
#r @"Packages\Microsoft.Owin\lib\net40\Microsoft.Owin.dll"
#r @"Packages\Microsoft.Owin.Host.HttpListener\lib\net40\Microsoft.Owin.Host.HttpListe
ner.dll"
#r @"Packages\Microsoft.Owin.Hosting\lib\net40\Microsoft.Owin.Hosting.dll"
#r @"Packages\Microsoft.AspNet.WebApi.Owin\lib\net45\System.Web.Http.Owin.dll"
#r @"Packages\Microsoft.AspNet.WebApi.Core\lib\net45\System.Web.Http.dll"
#r @"Packages\Microsoft.AspNet.WebApi.Client\lib\net45\System.Net.Http.Formatting.dll"
#r @"Packages\Newtonsoft.Json\lib\net40\Newtonsoft.Json.dll"
#r "System.Net.Http.dll"
open System
open Owin
open Microsoft.Owin
open System.Web.Http
open System.Web.Http.Dispatcher
open System.Net.Http.Formatting
module OwinSelfhostSample =
/// a record to return
[<CLIMutable>]
type Greeting = { Text : string }
/// A simple Controller
type GreetingController() =
inherit ApiController()
// GET api/greeting
member this.Get() =
{Text="Hello!"}
/// Another Controller that parses URIs
type ValuesController() =
inherit ApiController()
// GET api/values
member this.Get() =
["value1";"value2"]
// GET api/values/5
member this.Get id =
sprintf "id is %i" id
// POST api/values
member this.Post ([<FromBody>]value:string) =
()
// PUT api/values/5
member this.Put(id:int, [<FromBody>]value:string) =
()
// DELETE api/values/5

Twenty six low-risk ways to use F# at work

member this.Delete(id:int) =
()
/// A helper class to store routes, etc.
type ApiRoute = { id : RouteParameter }
/// IMPORTANT: When running interactively, the controllers will not be found with
error:
/// "No type was found that matches the controller named 'XXX'."
/// The fix is to override the ControllerResolver to use the current assembly
type ControllerResolver() =
inherit DefaultHttpControllerTypeResolver()
override this.GetControllerTypes (assembliesResolver:IAssembliesResolver) =
let t = typeof<System.Web.Http.Controllers.IHttpController>
System.Reflection.Assembly.GetExecutingAssembly().GetTypes()
|> Array.filter t.IsAssignableFrom
:> Collections.Generic.ICollection<Type>
/// A class to manage the configuration
type MyHttpConfiguration() as this =
inherit HttpConfiguration()
let configureRoutes() =
this.Routes.MapHttpRoute(
name= "DefaultApi",
routeTemplate= "api/{controller}/{id}",
defaults= { id = RouteParameter.Optional }
) |> ignore
let configureJsonSerialization() =
let jsonSettings = this.Formatters.JsonFormatter.SerializerSettings
jsonSettings.Formatting <- Newtonsoft.Json.Formatting.Indented
jsonSettings.ContractResolver <-
Newtonsoft.Json.Serialization.CamelCasePropertyNamesContractResolver()
// Here is where the controllers are resolved
let configureServices() =
this.Services.Replace(
typeof<IHttpControllerTypeResolver>,
new ControllerResolver())
do configureRoutes()
do configureJsonSerialization()
do configureServices()
/// Create a startup class using the configuration
type Startup() =
// This code configures Web API. The Startup class is specified as a type
// parameter in the WebApp.Start method.
member this.Configuration (appBuilder:IAppBuilder) =
// Configure Web API for self-host.

Twenty six low-risk ways to use F# at work

let config = new MyHttpConfiguration()

appBuilder.UseWebApi(config) |> ignore

// Start OWIN host

do
// Create server
let baseAddress = "http://localhost:9000/"
use app = Microsoft.Owin.Hosting.WebApp.Start<OwinSelfhostSample.Startup>(url=base
Address)
// Create client and make some requests to the api
use client = new System.Net.Http.HttpClient()
let showResponse query =
let response = client.GetAsync(baseAddress + query).Result
Console.WriteLine(response)
Console.WriteLine(response.Content.ReadAsStringAsync().Result)
showResponse "api/greeting"
showResponse "api/values"
showResponse "api/values/42"
// for standalone scripts, pause so that you can test via your browser as well
Console.ReadLine() |> ignore

Here's the output:

Twenty six low-risk ways to use F# at work

StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamCont

ent, Headers:
{
Date: Fri, 18 Apr 2014 22:29:04 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 24
Content-Type: application/json; charset=utf-8
}
{
"text": "Hello!"
}
StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamCont
ent, Headers:
{
Date: Fri, 18 Apr 2014 22:29:04 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 29
Content-Type: application/json; charset=utf-8
}
[
"value1",
"value2"
]
StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.StreamCont
ent, Headers:
{
Date: Fri, 18 Apr 2014 22:29:04 GMT
Server: Microsoft-HTTPAPI/2.0
Content-Length: 10
Content-Type: application/json; charset=utf-8
}
"id is 42"

This example is just to demonstrate that you can use the OWIN and WebApi libraries "outof-the-box".
For a more F# friendly web framework, have a look at Suave or WebSharper. There is a lot
more webby stuff at fsharp.org.

4. Use F# to play with UI's interactively

The code for this section is available on github.
Another use for F# interactive is to play with UI's while they are running -- live!
Here's an example of developing a WinForms screen interactively.

Twenty six low-risk ways to use F# at work

open System.Windows.Forms
open System.Drawing
let form = new Form(Width= 400, Height = 300, Visible = true, Text = "Hello World")
form.TopMost <- true
form.Click.Add (fun _ ->
form.Text <- sprintf "form clicked at %i" DateTime.Now.Ticks)
form.Show()

Here's the window:

And here's the window after clicking, with the title bar changed:

Now let's add a FlowLayoutPanel and a button.

let panel = new FlowLayoutPanel()
form.Controls.Add(panel)
panel.Dock = DockStyle.Fill
panel.WrapContents <- false
let greenButton = new Button()
greenButton.Text <- "Make the background color green"
greenButton.Click.Add (fun _-> form.BackColor <- Color.LightGreen)
panel.Controls.Add(greenButton)

Here's the window now:

Twenty six low-risk ways to use F# at work

But the button is too small -- we need to set AutoSize to be true.

greenButton.AutoSize <- true

That's better!

Let's add a yellow button too:

let yellowButton = new Button()
yellowButton.Text <- "Make me yellow"
yellowButton.AutoSize <- true
yellowButton.Click.Add (fun _-> form.BackColor <- Color.Yellow)
panel.Controls.Add(yellowButton)

But the button is cut off, so let's change the flow direction:
panel.FlowDirection <- FlowDirection.TopDown

Twenty six low-risk ways to use F# at work

But now the yellow button is not the same width as the green button, which we can fix with
Dock :

yellowButton.Dock <- DockStyle.Fill

As you can see, it is really easy to play around with layouts interactively this way. Once
you're happy with the layout logic, you can convert the code back to C# for your real
application.
This example is WinForms specific. For other UI frameworks the logic would be different, of
course.

So that's the first four suggestions. We're not done yet! The next post will cover using F# for
development and devops scripts.

Using F# for development and devops scripts

Using F# for development and devops

scripts
This post is a continuation of the series on low-risk ways to use F# at work. I've been
suggesting a number of ways you can get your hands dirty with F# in a low-risk, incremental
way, without affecting any mission critical code.
In this one, we'll talk about using F# for builds and other development and devops scripts.
If you're new to F#, you might want to read the sections on getting started and working with
NuGet in the previous post.

Series contents
Here's a list of shortcuts to the twenty six ways:
Part 1 - Using F# to explore and develop interactively
1. Use F# to explore the .NET framework interactively
2. Use F# to test your own code interactively
3. Use F# to play with webservices interactively
4. Use F# to play with UI's interactively
Part 2 - Using F# for development and devops scripts
5. Use FAKE for build and CI scripts
6. An F# script to check that a website is responding
7. An F# script to convert an RSS feed into CSV
8. An F# script that uses WMI to check the stats of a process
9. Use F# for configuring and managing the cloud
Part 3 - Using F# for testing
10. Use F# to write unit tests with readable names
11. Use F# to run unit tests programmatically
12. Use F# to learn to write unit tests in other ways
13. Use FsCheck to write better unit tests
14. Use FsCheck to create random dummy data
15. Use F# to create mocks
16. Use F# to do automated browser testing
17. Use F# for Behaviour Driven Development

Using F# for development and devops scripts

Part 4. Using F# for database related tasks

18. Use F# to replace LINQpad
19. Use F# to unit test stored procedures
20. Use FsCheck to generate random database records
21. Use F# to do simple ETL
22. Use F# to generate SQL Agent scripts
Part 5: Other interesting ways of using F#
23. Use F# for parsing
24. Use F# for diagramming and visualization
25. Use F# for accessing web-based data stores
26. Use F# for data science and machine learning
(BONUS) 27: Balance the generation schedule for the UK power station fleet

Part 2: Using F# for development and devops

scripts
The next set of suggestions relates to using F# for the various scripts that revolve around
development activities: builds, continuous integration, deployment, etc.
For these kinds of small tasks, you need a good scripting language with a REPL. You could
use PowerShell, or ScriptCS, or even Python. But why not give F# a go?
F# feels lightweight like Python (few or no type declarations).
F# can access .NET libraries, both the core ones and those downloaded via NuGet.
F# has type providers (a big advantage over PowerShell and ScriptCS) that let you
easily access a wide range of data sources.
All this in a concise, type-safe manner, with intellisense too!
Using F# in this way will allow you and your fellow developers to use F# code to solve
practical problems. There shouldn't be any resistance from managers for this low-risk
approach -- in the worse case you can easily switch to using a different tool.
A hidden agenda, of course, is that once your fellow developers get a chance to play with
F#, they'll be hooked, and you'll be one step closer to using F# end to end!

What can you do with F# scripts?

In the next few sections we'll see three examples of F# scripts:

Using F# for development and devops scripts

An F# script to check that a website is responding

An F# script to convert an RSS feed into CSV
An F# script that uses WMI to check the stats of a process
But of course, you can integrate F# scripts with almost any .NET library. Here are other
suggestions for utilities that can be scripted:
Simple file copying, directory traversal, and archiving (e.g. of log files). If you're using
.NET 4.5, you can use the new System.IO.Compression.ZipArchive class to do zipping
and unzipping without needing a third party library.
Doing things with JSON, either with a known format (using the JSON Type Provider) or
unknown format (using the JSON parser).
Interacting with GitHub using Octokit.
Extracting data from, or manipulating data in, Excel. F# supports COM for doing Office
automation, or you can use one of the type providers or libraries.
Doing numerics with Math.NET.
Web crawling, link checking, and screenscraping. The built-in async workflows and
agents make this kind of "multithreaded" code very easy to write.
Scheduling things with Quartz.NET.
If these suggestions whet your interest, and you want to use more F#, then check out the F#
community projects page. It's a great source of useful libraries being written for F#, and most
of them will work well with F# scripting.

Debugging F# scripts
A great thing about using F# scripts is that you don't need to create a whole project, nor
launch Visual Studio.
But if you need to debug a script, and you're not in Visual Studio, what can you do? Here are
some tips:
First, you can just use tried and true printing to the console using printfn . I generally
wrap this in a simple log function so that I can turn logging on or off with a flag.
You can use the FsEye tool to inspect and watch variables in an interactive session.
Finally, you can still use the Visual Studio debugger. The trick is to attach the debugger
to the fsi.exe process, and then you can use Debugger.Break to halt at a certain point.

5. Use FAKE for build and CI scripts

The code for this section is available on github.

Using F# for development and devops scripts

Let's start with FAKE, which is a cross platform build automation tool written in F#,
analogous to Ruby's Rake.
FAKE has built-in support for git, NuGet, unit tests, Octopus Deploy, Xamarin and more, and
makes it easy to develop complex scripts with dependencies.
You can even use it with TFS to avoid using XAML.
One reason to use FAKE rather than something like Rake is that you can standardize on
.NET code throughout your tool chain. In theory, you could use NAnt instead, but in practice,
no thanks, because XML. PSake is also a possibility, but more complicated than FAKE, I
think.
You can also use FAKE to remove dependencies on a particular build server. For example,
rather than using TeamCity's integration to run tests and other tasks, you might consider
doing them in FAKE instead, which means you can run full builds without having TeamCity
installed.
Here's an example of a very simple FAKE script, taken from a more detailed example on the
FAKE site.
// Include Fake lib
// Assumes NuGet has been used to fetch the FAKE libraries
#r "packages/FAKE/tools/FakeLib.dll"
open Fake
// Properties
let buildDir = "./build/"
// Targets
Target "Clean" (fun _ ->
CleanDir buildDir
)
Target "Default" (fun _ ->
trace "Hello World from FAKE"
)
// Dependencies
"Clean"
==> "Default"
// start build
RunTargetOrDefault "Default"

The syntax takes a little getting used to, but that effort is well spent.
Some further reading on FAKE:

Using F# for development and devops scripts

Migrating to FAKE.
Hanselman on FAKE. Many of the comments are from people who are using FAKE
actively.
A NAnt user tries out FAKE.

6. An F# script to check that a website is

responding
The code for this section is available on github.
This script checks that a website is responding with a 200. This might be useful as the basis
for a post-deployment smoke test, for example.
// Requires FSharp.Data under script directory
// nuget install FSharp.Data -o Packages -ExcludeVersion
#r @"Packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data
let queryServer uri queryParams =
try
let response = Http.Request(uri, query=queryParams, silentHttpErrors = true)
Some response
with
| :? System.Net.WebException as ex -> None
let sendAlert uri message =
// send alert via email, say
printfn "Error for %s. Message=%O" uri message
let checkServer (uri,queryParams) =
match queryServer uri queryParams with
| Some response ->
printfn "Response for %s is %O" uri response.StatusCode
if (response.StatusCode <> 200) then
sendAlert uri response.StatusCode
| None ->
sendAlert uri "No response"
// test the sites
let google = "http://google.com", ["q","fsharp"]
let bad = "http://example.bad", []
[google;bad]
|> List.iter checkServer

The result is:

Using F# for development and devops scripts

Response for http://google.com is 200

Error for http://example.bad. Message=No response

Note that I'm using the Http utilities code in Fsharp.Data , which provides a nice wrapper
around HttpClient . More on HttpUtilities here.

7. An F# script to convert an RSS feed into CSV

The code for this section is available on github.
Here's a little script that uses the Xml type provider to parse an RSS feed (in this case, F#
questions on StackOverflow) and convert it to a CSV file for later analysis.
Note that the RSS parsing code is just one line of code! Most of the code is concerned with
writing the CSV. Yes, I could have used a CSV library (there are lots on NuGet) but I thought
I'd leave it as is to show you how simple it is.

Using F# for development and devops scripts

// sets the current directory to be same as the script directory

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// Requires FSharp.Data under script directory
// nuget install FSharp.Data -o Packages -ExcludeVersion
#r @"Packages\FSharp.Data\lib\net40\FSharp.Data.dll"
#r "System.Xml.Linq.dll"
open FSharp.Data
type Rss = XmlProvider<"http://stackoverflow.com/feeds/tag/f%23">
// prepare a string for writing to CSV
let prepareStr obj =
obj.ToString()
.Replace("\"","\"\"") // replace single with double quotes
|> sprintf "\"%s\"" // surround with quotes
// convert a list of strings to a CSV
let listToCsv list =
let combine s1 s2 = s1 + "," + s2
list
|> Seq.map prepareStr
|> Seq.reduce combine
// extract fields from Entry
let extractFields (entry:Rss.Entry) =
[entry.Title.Value;
entry.Author.Name;
entry.Published.ToShortDateString()]
// write the lines to a file
do
use writer = new System.IO.StreamWriter("fsharp-questions.csv")
let feed = Rss.GetSample()
feed.Entries
|> Seq.map (extractFields >> listToCsv)
|> Seq.iter writer.WriteLine
// writer will be closed automatically at the end of this scope

Note that the type provider generates intellisense (shown below) to show you the available
properties based on the actual contents of the feed. That's very cool.

Using F# for development and devops scripts

The result is something like this:

"Optimising F# answer for Euler #4","DropTheTable","18/04/2014"
"How to execute a function, that creates a lot of objects, in parallel?","Lawrence Woo
dman","01/04/2014"
"How to invoke a user defined function using R Type Provider","Dave","19/04/2014"
"Two types that use themselves","trn","19/04/2014"
"How does function [x] -> ... work","egerhard","19/04/2014"

For more on the XML type provider, see the FSharp.Data pages.

8. An F# script that uses WMI to check the

stats of a process
The code for this section is available on github.
If you use Windows, being able to access WMI is very useful. Luckily there is an F# type
provider for WMI that makes using it easy.
In this example, we'll get the system time and also check some stats for a process. This
could be useful during and after a load test, for example.

Using F# for development and devops scripts

// sets the current directory to be same as the script directory

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// Requires FSharp.Management under script directory
// nuget install FSharp.Management -o Packages -ExcludeVersion
#r @"System.Management.dll"
#r @"Packages\FSharp.Management\lib\net40\FSharp.Management.dll"
#r @"Packages\FSharp.Management\lib\net40\FSharp.Management.WMI.dll"
open FSharp.Management
// get data for the local machine
type Local = WmiProvider<"localhost">
let data = Local.GetDataContext()
// get the time and timezone on the machine
let time = data.Win32_UTCTime |> Seq.head
let tz = data.Win32_TimeZone |> Seq.head
printfn "Time=%O-%O-%O %O:%O:%O" time.Year time.Month time.Day time.Hour time.Minute t
ime.Second
printfn "Timezone=%O" tz.StandardName
// find the "explorer" process
let explorerProc =
data.Win32_PerfFormattedData_PerfProc_Process
|> Seq.find (fun proc -> proc.Name.Contains("explorer") )
// get stats about it
printfn "ElapsedTime=%O" explorerProc.ElapsedTime
printfn "ThreadCount=%O" explorerProc.ThreadCount
printfn "HandleCount=%O" explorerProc.HandleCount
printfn "WorkingSetPeak=%O" explorerProc.WorkingSetPeak
printfn "PageFileBytesPeak=%O" explorerProc.PageFileBytesPeak

The output is something like this:

Time=2014-4-20 14:2:35
Timezone=GMT Standard Time
ElapsedTime=2761906
ThreadCount=67
HandleCount=3700
WorkingSetPeak=168607744
PageFileBytesPeak=312565760

Again, using a type provider means that you get intellisense (shown below). Very useful for
the hundreds of WMI options.

Using F# for development and devops scripts

9. Use F# for configuring and managing the

cloud
One area which deserves special mention is using F# for configuring and managing cloud
services. The cloud page at fsharp.org has many helpful links.
For simple scripting, Fog is a nice wrapper for Azure.
So for example, to upload a blob, the code is as simple as this:
UploadBlob "testcontainer" "testblob" "This is a test" |> ignore

or to add and receive messages:

AddMessage "testqueue" "This is a test message" |> ignore
let result = GetMessages "testqueue" 20 5
for m in result do
DeleteMessage "testqueue" m

What's especially nice about using F# for this is that you can do it in micro scripts -- you
don't need any heavy tooling.

Using F# for development and devops scripts

Summary
I hope you found these suggestions useful. Let me know in the comments if you apply them
in practice.
Next up: using F# for testing.

Using F# for testing

This post is a continuation of the previous series on low-risk and incremental ways to use F#
at work -- how can you get your hands dirty with F# in a low-risk, incremental way, without
affecting any mission critical code?
In this one, we'll talk about using F# for testing.

Series contents
Before moving on to the content of the post, here's the full list of the twenty six ways:
Part 1 - Using F# to explore and develop interactively
1. Use F# to explore the .NET framework interactively
2. Use F# to test your own code interactively
3. Use F# to play with webservices interactively
4. Use F# to play with UI's interactively
Part 2 - Using F# for development and devops scripts
5. Use FAKE for build and CI scripts
6. An F# script to check that a website is responding
7. An F# script to convert an RSS feed into CSV
8. An F# script that uses WMI to check the stats of a process
9. Use F# for configuring and managing the cloud
Part 3 - Using F# for testing
10. Use F# to write unit tests with readable names
11. Use F# to run unit tests programmatically
12. Use F# to learn to write unit tests in other ways
13. Use FsCheck to write better unit tests
14. Use FsCheck to create random dummy data
15. Use F# to create mocks
16. Use F# to do automated browser testing
17. Use F# for Behaviour Driven Development
Part 4. Using F# for database related tasks

Using F# for testing

18. Use F# to replace LINQpad

19. Use F# to unit test stored procedures
20. Use FsCheck to generate random database records
21. Use F# to do simple ETL
22. Use F# to generate SQL Agent scripts
Part 5: Other interesting ways of using F#
23. Use F# for parsing
24. Use F# for diagramming and visualization
25. Use F# for accessing web-based data stores
26. Use F# for data science and machine learning
(BONUS) 27: Balance the generation schedule for the UK power station fleet

Part 3 - Using F# for testing

If you want to start writing useful code in F# without touching core code, writing tests is a
great way to start.
Not only does F# have a more compact syntax, it also has many nice features, such as the
"double backtick" syntax, that make test names much more readable.
As with all of the suggestions in this series, I think this is a low risk option. Test methods tend
to be short, so almost anyone will be able to read them without having to understand F#
deeply. In the worst-case, you can easily port them back to C#.

10. Use F# to write unit tests with readable

names
The code for this section is available on github.
Just like C#, F# can be used to write standard unit tests using the standard frameworks like
NUnit, MsUnit, xUnit, etc.
Here's an example of a test class written for use with NUnit.

Using F# for testing

[<TestFixture>]
type TestClass() =
[<Test>]
member this.When2IsAddedTo2Expect4() =
Assert.AreEqual(4, 2+2)

As you can see, there's a class with the TestFixture attribute, and a public void method
with the Test attribute. All very standard.
But there are some nice extras you get when you use F# rather than C#. First you can use
the double backtick syntax to create more readable names, and second, you can use let
bound functions in modules rather than classes, which simplifies the code.
[<Test>]
let ``When 2 is added to 2 expect 4``() =
Assert.AreEqual(4, 2+2)

The double backtick syntax makes the test results much easier to read. Here is the output of
the test with a standard class name:
TestClass.When2IsAddedTo2Expect4
Result: Success

vs. the output using the more friendly name:

MyUnitTests.When 2 is added to 2 expect 4
Result: Success

So if you want to write test names that are accessible to non-programmers, give F# a go!

11. Use F# to run unit tests programmatically

Often, you might want to run the unit tests programmatically. This can be for various
reasons, such as using custom filters, or doing custom logging, or not wanting to install
NUnit on test machines.
One simple way to do this is to use the Fuchu library which lets you organize tests directly,
especially parameterized tests, without any complex test attributes.
Here's an example:

Using F# for testing

let add1 x = x + 1
// a simple test using any assertion framework:
// Fuchu's own, Nunit, FsUnit, etc
let `Àssert that add1 is x+1`` x _notUsed =
NUnit.Framework.Assert.AreEqual(x+1, add1 x)
// a single test case with one value
let simpleTest =
testCase "Test with 42" <|
`Àssert that add1 is x+1`` 42
// a parameterized test case with one param
let parameterizedTest i =
testCase (sprintf "Test with %i" i) <|
`Àssert that add1 is x+1`` i

You can run these tests directly in F# interactive using code like this: run simpleTest .
You can also combine these tests into one or more lists, or hierarchical lists of lists:
// create a hierarchy of tests
// mark it as the start point with the "Tests" attribute
[<Fuchu.Tests>]
let tests =
testList "Test group A" [
simpleTest
testList "Parameterized 1..10" ([1..10] |> List.map parameterizedTest)
testList "Parameterized 11..20" ([11..20] |> List.map parameterizedTest)
]

The code above is available on github.

Finally, with Fuchu, the test assembly becomes its own test runner. Just make the assembly
a console app instead of a library and add this code to the program.fs file:
[<EntryPoint>]
let main args =
let exitCode = defaultMainThisAssembly args
Console.WriteLine("Press any key")
Console.ReadLine() |> ignore
// return the exit code
exitCode

Using F# for testing

Using the NUnit test runner

If you do need to use an existing test runner (such as the NUnit one), then it's very simple to
put together a simple script to do this.
I've made a little example, below, using the Nunit.Runners package.
All right, this might not be the most exciting use of F#, but it does show off F#'s "object
expression" syntax to create the NUnit.Core.EventListener interface, so I thought I'd leave it
in as a demo.
// sets the current directory to be same as the script directory
System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// Requires Nunit.Runners under script directory
// nuget install NUnit.Runners -o Packages -ExcludeVersion
#r @"Packages\NUnit.Runners\tools\lib\nunit.core.dll"
#r @"Packages\NUnit.Runners\tools\lib\nunit.core.interfaces.dll"
open System
open NUnit.Core
module Setup =
open System.Reflection
open NUnit.Core
open System.Diagnostics.Tracing
let configureTestRunner path (runner:TestRunner) =
let package = TestPackage("MyPackage")
package.Assemblies.Add(path) |> ignore
runner.Load(package) |> ignore
let createListener logger =
let replaceNewline (s:string) =
s.Replace(Environment.NewLine, "")
// This is an example of F#'s "object expression" syntax.
// You don't need to create a class to implement an interface
{new NUnit.Core.EventListener
with
member this.RunStarted(name:string, testCount:int) =
logger "Run started "
member this.RunFinished(result:TestResult ) =
logger ""
logger "-------------------------------"
result.ResultState
|> sprintf "Overall result: %O"

Using F# for testing

|> logger
member this.RunFinished(ex:Exception) =
ex.StackTrace
|> replaceNewline
|> sprintf "Exception occurred: %s"
|> logger
member this.SuiteFinished(result:TestResult) = ()
member this.SuiteStarted(testName:TestName) = ()
member this.TestFinished(result:TestResult)=
result.ResultState
|> sprintf "Result: %O"
|> logger
member this.TestOutput(testOutput:TestOutput) =
testOutput.Text
|> replaceNewline
|> logger
member this.TestStarted(testName:TestName) =
logger ""
testName.FullName
|> replaceNewline
|> logger
member this.UnhandledException(ex:Exception) =
ex.StackTrace
|> replaceNewline
|> sprintf "Unhandled exception occurred: %s"
|> logger
}

// run all the tests in the DLL

do
let dllPath = @".\bin\MyUnitTests.dll"
CoreExtensions.Host.InitializeService();
use runner = new NUnit.Core.SimpleTestRunner()
Setup.configureTestRunner dllPath runner
let logger = printfn "%s"
let listener = Setup.createListener logger
let result = runner.Run(listener, TestFilter.Empty, true, LoggingThreshold.All)
// if running from the command line, wait for user input
Console.ReadLine() |> ignore
// if running from the interactive session, reset session before recompiling MyUni
tTests.dll

Using F# for testing

The code above is available on github.

12. Use F# to learn to write unit tests in other

ways
The unit test code above is familiar to all of us, but there are other ways to write tests.
Learning to code in different styles is a great way to add some new techniques to your
repertoire and expand your thinking in general, so let's have a quick look at some of them.
First up is FsUnit, which replaces Assert with a more fluent and idiomatic approach
(natural language and piping).
Here's a snippet:
open NUnit.Framework
open FsUnit
let inline add x y = x + y
[<Test>]
let ``When 2 is added to 2 expect 4``() =
add 2 2 |> should equal 4
[<Test>]
let ``When 2.0 is added to 2.0 expect 4.01``() =
add 2.0 2.0 |> should (equalWithin 0.1) 4.01
[<Test>]
let ``When ToLower(), expect lowercase letters``() =
"FSHARP".ToLower() |> should startWith "fs"

The above code is available on github.

A very different approach is used by Unquote. The Unquote approach is to wrap any F#
expression in F# quotations and then evaluate it. If a test expression throws an exception,
the test will fail and print not just the exception, but each step up to the point of the
exception. This information could potentially give you much more insight in why the assert
fails.
Here's a very simple example:
[<Test>]
let ``When 2 is added to 2 expect 4``() =
test <@ 2 + 2 = 4 @>

Using F# for testing

There are also a number of shortcut operators such as =? and >? that allow you to write
your tests even more simply -- no asserts anywhere!
[<Test>]
let ``2 + 2 is 4``() =
let result = 2 + 2
result =? 4
[<Test>]
let ``2 + 2 is bigger than 5``() =
let result = 2 + 2
result >? 5

The above code is available on github.

13. Use FsCheck to write better unit tests

The code for this section is available on github.
Let's say that we have written a function that converts numbers to Roman numerals, and we
want to create some test cases for it.
We might start writing tests like this:
[<Test>]
let ``Test that 497 is CDXCVII``() =
arabicToRoman 497 |> should equal "CDXCVII"

But the problem with this approach is that it only tests a very specific example. There might
be some edge cases that we haven't thought of.
A much better approach is to find something that must be true for all cases. Then we can
create a test that checks that this something (a "property") is true for all cases, or at least a
large random subset.
For example, in the Roman numeral example, we can say that one property is "all Roman
numerals have at most one 'V' character" or "all Roman numerals have at most three 'X'
characters". We can then construct tests that check this property is indeed true.
This is where FsCheck can help. FsCheck is a framework designed for exactly this kind of
property-based testing. It's written in F# but it works equally well for testing C# code.
So, let's see how we'd use FsCheck for our Roman numerals.
First, we define some properties that we expect to hold for all Roman numerals.

Using F# for testing

let maxRepetitionProperty ch count (input:string) =

let find = String.replicate (count+1) ch
input.Contains find |> not
// a property that holds for all roman numerals
let ``has max rep of one V`` roman =
maxRepetitionProperty "V" 1 roman
// a property that holds for all roman numerals
let ``has max rep of three Xs`` roman =
maxRepetitionProperty "X" 3 roman

With this in place we create tests that:

1. Create a property checker function suitable for passing to FsCheck.
2. Use the Check.Quick function to generate hundreds of random test cases and send
them into that property checker.
[<Test>]
let ``Test that roman numerals have no more than one V``() =
let property num =
// convert the number to roman and check the property
arabicToRoman num |> ``has max rep of one V``
Check.QuickThrowOnFailure (testWithRange property)
[<Test>]
let ``Test that roman numerals have no more than three Xs``() =
let property num =
// convert the number to roman and check the property
arabicToRoman num |> ``has max rep of three Xs``
Check.QuickThrowOnFailure (testWithRange property)

Here are the results of the test. You can see that 100 random numbers have been tested,
not just one.
Test that roman numerals have no more than one V
Ok, passed 100 tests.
Test that roman numerals have no more than three Xs
Ok, passed 100 tests.

If we changed the test to be "Test that roman numerals have no more than TWO Xs", then
the test result is false, and looks like this:

Using F# for testing

Falsifiable, after 33 tests

In other words, after generating 33 different inputs, FsCheck has correctly found a number
(30) that does not meet the required property. Very nice!

Using FsCheck in practice

Not all situations have properties that can be tested this way, but you might find that it is
more common than you think.
For example, property based testing is especially useful for "algorithmic" code. Here a few
examples:
If you reverse a list and then reverse it again, you get the original list.
If you factorize an integer and then multiply the factors, you get the original number.
But even in Boring Line-Of-Business Applications?, you may find that property based testing
has a place. For example, here are some things that can be expressed as properties:
Roundtripping. For example, if you save a record to a database and then reload it, the
record's fields should be unchanged. Similarly, if you serialize and then deserialize
something, you should get the original thing back.
Invariants. If you add products to a sales order, the sum of the individual lines should
be the same as the order total. Or, the sum of word counts for each page should be the
sum of the word count for the entire book. More generally, if you calculate things via two
different paths, you should get the same answer (monoid homomorphisms!)
Rounding. If you add ingredients to a recipe, the sum of the ingredient percentages
(with 2 place precision) should always be exactly 100%. Similar rules are needed for
most partitioning logic, such as shares, tax calculations, etc. (e.g. the "share pie"
example in the DDD book).
Making sure you get the rounding right in situations like this is where FsCheck shines.
See this SO question for other ideas.
FsCheck is also very useful for doing refactoring, because once you trust that the tests are
extremely thorough, you can confidently work on tweaks and optimization.
Some more links for FsCheck:
I have written an introduction to property-based testing and a follow up on choosing
properties for property-based testing.
FsCheck documentation.

Using F# for testing

An article on using FsCheck in practice.

My post on the Roman Numerals kata that mentions FsCheck.
For more on property-based testing in general, look for articles and videos about
QuickCheck.
Intro to QuickCheck by John Hughes (PDF)
Fascinating talk on using QuickCheck to find bugs in Riak (another version) (videos)

14. Use FsCheck to create random dummy data

The code for this section is available on github.
In addition to doing testing, FsCheck can be used to create random dummy data.
For example, below is the complete code for generating random customers.
When you combine this with the SQL Type Provider (discussed later) or CSV writer, you can
easily generate thousands of rows of random customers in a database or CSV file. Or you
can use it with the JSON type provider to call a web service for testing validation logic, or
load testing.
(Dont worry about not understanding the code -- this sample is just to show you how easy it
is!)
// domain objects
type EmailAddress = EmailAddress of string
type PhoneNumber = PhoneNumber of string
type Customer = {
name: string
email: EmailAddress
phone: PhoneNumber
birthdate: DateTime
}
// a list of names to sample
let possibleNames = [
"Georgianne Stephan"
"Sharolyn Galban"
"Beatriz Applewhite"
"Merissa Cornwall"
"Kenneth Abdulla"
"Zora Feliz"
"Janeen Strunk"
"Oren Curlee"
]
// generate a random name by picking from the list at random

Using F# for testing

let generateName() =
FsCheck.Gen.elements possibleNames
// generate a random EmailAddress by combining random users and domains
let generateEmail() =
let userGen = FsCheck.Gen.elements ["a"; "b"; "c"; "d"; "e"; "f"]
let domainGen = FsCheck.Gen.elements ["gmail.com"; "example.com"; "outlook.com"]
let makeEmail u d = sprintf "%s@%s" u d |> EmailAddress
FsCheck.Gen.map2 makeEmail userGen domainGen
// generate a random PhoneNumber
let generatePhone() =
let areaGen = FsCheck.Gen.choose(100,999)
let n1Gen = FsCheck.Gen.choose(1,999)
let n2Gen = FsCheck.Gen.choose(1,9999)
let makeNumber area n1 n2 = sprintf "(%03i)%03i-%04i" area n1 n2 |> PhoneNumber
FsCheck.Gen.map3 makeNumber areaGen n1Gen n2Gen
// generate a random birthdate
let generateDate() =
let minDate = DateTime(1920,1,1).ToOADate() |> int
let maxDate = DateTime(2014,1,1).ToOADate() |> int
let oaDateGen = FsCheck.Gen.choose(minDate,maxDate)
let makeDate oaDate = float oaDate |> DateTime.FromOADate
FsCheck.Gen.map makeDate oaDateGen
// a function to create a customer
let createCustomer name email phone birthdate =
{name=name; email=email; phone=phone; birthdate=birthdate}
// use applicatives to create a customer generator
let generateCustomer =
createCustomer
<!> generateName()
<*> generateEmail()
<*> generatePhone()
<*> generateDate()
[<Test>]
let printRandomCustomers() =
let size = 0
let count = 10
let data = FsCheck.Gen.sample size count generateCustomer
// print it
data |> List.iter (printfn "%A")

And here is a sampling of the results:

Using F# for testing

{name = "Georgianne Stephan";

email = EmailAddress "d@outlook.com";
phone = PhoneNumber "(420)330-2080";
birthdate = 11/02/1976 00:00:00;}
{name = "Sharolyn Galban";
email = EmailAddress "e@outlook.com";
phone = PhoneNumber "(579)781-9435";
birthdate = 01/04/2011 00:00:00;}
{name = "Janeen Strunk";
email = EmailAddress "b@gmail.com";
phone = PhoneNumber "(265)405-6619";
birthdate = 21/07/1955 00:00:00;}

15. Use F# to create mocks

If you're using F# to write test cases for code written in C#, you may want to create mocks
and stubs for interfaces.
In C# you might use Moq or NSubstitute. In F# you can use object expressions to create
interfaces directly, or the Foq library.
Both are easy to do, and in a way that is similar to Moq.
Here's some Moq code in C#:
// Moq Method
var mock = new Mock<IFoo>();
mock.Setup(foo => foo.DoSomething("ping")).Returns(true);
var instance = mock.Object;
// Moq Matching Arguments:
mock.Setup(foo => foo.DoSomething(It.IsAny<string>())).Returns(true);
// Moq Property
mock.Setup(foo => foo.Name ).Returns("bar");

And here's the equivalent Foq code in F#:

Using F# for testing

// Foq Method
let mock =
Mock<IFoo>()
.Setup(fun foo -> <@ foo.DoSomething("ping") @>).Returns(true)
.Create()
// Foq Matching Arguments
mock.Setup(fun foo -> <@ foo.DoSomething(any()) @>).Returns(true)
// Foq Property
mock.Setup(fun foo -> <@ foo.Name @>).Returns("bar")

For more on mocking in F#, see:

F# as a Unit Testing Language
Mocking with Foq
Testing and mocking your C# code with F#
And you need to mock external services such as SMTP over the wire, there is an interesting
tool called mountebank, which is easy to interact with in F#.

16. Use F# to do automated browser testing

In addition to unit tests, you should be doing some kind of automated web testing, driving the
browser with Selenium or WatiN.
But what language should you write the automation in? Ruby? Python? C#? I think you know
the answer!
To make your life even easier, try using Canopy, a web testing framework built on top of
Selenium and written in F#. Their site claims "Quick to learn. Even if you've never done UI
Automation, and don't know F#.", and I'm inclined to believe them.
Below is a snippet taken from the Canopy site. As you can see, the code is simple and easy
to understand.
Also, FAKE integrates with Canopy, so you can run automated browser tests as part of a CI
build.

Using F# for testing

//start an instance of the firefox browser

start firefox
//this is how you define a test
"taking canopy for a spin" &&& fun _ ->
//go to url
url "http://lefthandedgoat.github.io/canopy/testpages/"
//assert that the element with an id of 'welcome' has
//the text 'Welcome'
"#welcome" == "Welcome"
//assert that the element with an id of 'firstName' has the value 'John'
"#firstName" == "John"
//change the value of element with
//an id of 'firstName' to 'Something Else'
"#firstName" << "Something Else"
//verify another element's value, click a button,
//verify the element is updated
"#button_clicked" == "button not clicked"
click "#button"
"#button_clicked" == "button clicked"
//run all tests
run()

17. Use F# for Behaviour Driven Development

The code for this section is available on github.
If you're not familiar with Behaviour Driven Development (BDD), the idea is that you express
requirements in a way that is both human-readable and executable.
The standard format (Gherkin) for writing these tests uses the Given/When/Then syntax -here's an example:
Feature: Refunded or replaced items should be returned to stock
Scenario 1: Refunded items should be returned to stock
Given a customer buys a black jumper
And I have 3 black jumpers left in stock
When they return the jumper for a refund
Then I should have 4 black jumpers in stock

If you are using BDD already with .NET, you're probably using SpecFlow or similar.

Using F# for testing

You should consider using TickSpec instead because, as with all things F#, the syntax is
much more lightweight.
For example, here's the full implementation of the scenario above.
type StockItem = { Count : int }
let mutable stockItem = { Count = 0 }
let [<Given>] `à customer buys a black jumper`` () =
()
let [<Given>] `Ì have (.*) black jumpers left in stock`` (n:int) =
stockItem <- { stockItem with Count = n }
let [<When>] ``they return the jumper for a refund`` () =
stockItem <- { stockItem with Count = stockItem.Count + 1 }
let [<Then>] `Ì should have (.*) black jumpers in stock`` (n:int) =
let passed = (stockItem.Count = n)
Assert.True(passed)

The C# equivalent has a lot more clutter, and the lack of double backtick syntax really hurts:
[Given(@"a customer buys a black jumper")]
public void GivenACustomerBuysABlackJumper()
{
// code
}
[Given(@"I have (.*) black jumpers left in stock")]
public void GivenIHaveNBlackJumpersLeftInStock(int n)
{
// code
}

Examples taken from the TickSpec site.

Summary of testing in F#
You can of course combine all the test techniques we've seen so far (as this slide deck
demonstrates):
Unit tests (FsUnit, Unquote) and property-based tests (FsCheck).
Automated acceptance tests (or at least a smoke test) written in BDD (TickSpec) driven
by browser automation (Canopy).

Using F# for testing

Both types of tests run on every build (with FAKE).

There's a lot of advice on test automation out there, and you'll find that it is easy to port
concepts from other languages to these F# tools. Have fun!

Using F# for database related tasks

This post is a continuation of the previous series on low-risk and incremental ways to use F#
at work.
In this one, we'll see how F# can be unexpectedly helpful when it comes to database related
tasks.

Using F# for database related tasks

18. Use F# to replace LINQpad

Part 4. Using F# for database related tasks

This next group of suggestions is all about working with databases, and MS SQL Server in
particular.
Relational databases are a critical part of most applications, but most teams do not
approach the management of these in the same way as with other development tasks.
For example, how many teams do you know that unit test their stored procedures?
Or their ETL jobs?
Or generate T-SQL admin scripts and other boilerplate using a non-SQL scripting language
that's stored in source control?
Here's where F# can shine over other scripting languages, and even over T-SQL itself.
The database type providers in F# give you the power to create simple, short scripts for
testing and admin, with the bonus that...
The scripts are type-checked and will fail at compile time if the database schema
changes, which means that...
The whole process works really well with builds and continuous integration processes,
which in turn means that...
You have really high confidence in your database related code!
We'll look at a few examples to demonstrate what I'm talking about:
Unit testing stored procedures
Using FsCheck to generate random records

Using F# for database related tasks

Doing simple ETL with F#

Generating SQL Agent scripts

Getting set up
The code for this section is available on github. In there, there are some SQL scripts to
create the sample database, tables and stored procs that I'll use in these examples.
To run the examples, then, you'll need SQL Express or SQL Server running locally or
somewhere accessible, with the relevant setup scripts having been run.

Which type provider?

There are a number of SQL Type Providers for F# -- see the fsharp.org Data Access page.
For these examples, I'm going to use the SqlDataConnection type provider, which is part of
the FSharp.Data.TypeProviders DLL. It uses SqlMetal behind the scenes and so only works
with SQL Server databases.
The SQLProvider project is another good choice -- it supports MySql, SQLite and other nonMicrosoft databases.

18. Use F# to replace LINQPad

The code for this section is available on github.
LINQPad is a great tool for doing queries against databases, and is also a general
scratchpad for C#/VB/F# code.
You can use F# interactive to do many of the same things -- you get queries,
autocompletion, etc., just like LINQPad.
For example, here's one that counts customers with a certain email domain.

Using F# for database related tasks

[<Literal>]
let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated
Security=True;"
type Sql = SqlDataConnection<connectionString>
let db = Sql.GetDataContext()
// find the number of customers with a gmail domain
query {
for c in db.Customer do
where (c.Email.EndsWith("gmail.com"))
select c
count
}

If you want to see what SQL code is generated, you can turn logging on, of course:
// optional, turn logging on
db.DataContext.Log <- Console.Out

The logged output for this query is:

SELECT COUNT(*) AS [value]
FROM [dbo].[Customer] AS [t0]
WHERE [t0].[Email] LIKE @p0
-- @p0: Input VarChar (Size = 8000; Prec = 0; Scale = 0) [%gmail.com]

You can also do more complicated things, such as using subqueries. Here's an example
from MSDN:
Note that, as befitting a functional approach, queries are nice and composable.
// Find students who have signed up at least one course.
query {
for student in db.Student do
where (query { for courseSelection in db.CourseSelection do
exists (courseSelection.StudentID = student.StudentID) })
select student
}

And if the SQL engine doesn't support certain functions such as regexes, and assuming the
size of the data is not too large, you can just stream the data out and do the processing in
F#.

Using F# for database related tasks

// find the most popular domain for people born in each decade
let getDomain email =
Regex.Match(email,".*@(.*)").Groups.[1].Value
let getDecade (birthdate:Nullable<DateTime>) =
if birthdate.HasValue then
birthdate.Value.Year / 10 * 10 |> Some
else
None
let topDomain list =
list
|> Seq.distinct
|> Seq.head
|> snd
db.Customer
|> Seq.map (fun c -> getDecade c.Birthdate, getDomain c.Email)
|> Seq.groupBy fst
|> Seq.sortBy fst
|> Seq.map (fun (decade, group) -> (decade,topDomain group))
|> Seq.iter (printfn "%A")

As you can see from the code above, the nice thing about doing the processing in F# is that
you can define helper functions separately and connect them together easily.

19. Use F# to unit test stored procedures

The code for this section is available on github.
Now let's look at how we can use the type provider to make creating unit tests for stored
procs really easy.
First, I create a helper module (which I'll call DbLib ) to set up the connection and to provide
shared utility functions such as resetDatabase , which will be called before each test.

Using F# for database related tasks

module DbLib
[<Literal>]
let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp;Integrated
Security=True;"
type Sql = SqlDataConnection<connectionString>
let removeExistingData (db:DbContext) =
let truncateTable name =
sprintf "TRUNCATE TABLE %s" name
|> db.DataContext.ExecuteCommand
|> ignore
["Customer"; "CustomerImport"]
|> List.iter truncateTable
let insertReferenceData (db:DbContext) =
[ "US","United States";
"GB","United Kingdom" ]
|> List.iter (fun (code,name) ->
let c = new Sql.ServiceTypes.Country()
c.IsoCode <- code; c.CountryName <- name
db.Country.InsertOnSubmit c
)
db.DataContext.SubmitChanges()
// removes all data and restores db to known starting point
let resetDatabase() =
use db = Sql.GetDataContext()
removeExistingData db
insertReferenceData db

Now I can write a unit test, using NUnit say, just like any other unit test.
Assume that we have Customer table, and a sproc called up_Customer_Upsert that either
inserts a new customer or updates an existing one, depending on whether the passed in
customer id is null or not.
Here's what a test looks like:

Using F# for database related tasks

[<Test>]
let ``When upsert customer called with null id, expect customer created with new id``(
) =
DbLib.resetDatabase()
use db = DbLib.Sql.GetDataContext()
// create customer
let newId = db.Up_Customer_Upsert(Nullable(),"Alice","x@example.com",Nullable())
// check new id
Assert.Greater(newId,0)
// check one customer exists
let customerCount = db.Customer |> Seq.length
Assert.AreEqual(1,customerCount)

Note that, because the setup is expensive, I do multiple asserts in the test. This could be
refactored if you find this too ugly!
Here's one that tests that updates work:
[<Test>]
let ``When upsert customer called with existing id, expect customer updated``() =
DbLib.resetDatabase()
use db = DbLib.Sql.GetDataContext()
// create customer
let custId = db.Up_Customer_Upsert(Nullable(),"Alice","x@example.com",Nullable())
// update customer
let newId = db.Up_Customer_Upsert(Nullable custId,"Bob","y@example.com",Nullable()
)
// check id hasnt changed
Assert.AreEqual(custId,newId)
// check still only one customer
let customerCount = db.Customer |> Seq.length
Assert.AreEqual(1,customerCount)
// check customer columns are updated
let customer = db.Customer |> Seq.head
Assert.AreEqual("Bob",customer.Name)

And one more, that checks for exceptions:

100

Using F# for database related tasks

[<Test>]
let ``When upsert customer called with blank name, expect validation error``() =
DbLib.resetDatabase()
use db = DbLib.Sql.GetDataContext()
try
// try to create customer will a blank name
db.Up_Customer_Upsert(Nullable(),"","x@example.com",Nullable()) |> ignore
Assert.Fail("expecting a SqlException")
with
| :? System.Data.SqlClient.SqlException as ex ->
Assert.That(ex.Message,Is.StringContaining("@Name"))
Assert.That(ex.Message,Is.StringContaining("blank"))

As you can see, the whole process is very straightforward.

These tests can be compiled and run as part of the continuous integration scripts. And what
is great is that, if the database schema gets out of sync with the code, then the tests will fail
to even compile!

20. Use FsCheck to generate random database

records
The code for this section is available on github.
As I showed in an earlier example, you can use FsCheck to generate random data. In this
case we'll use it to generate random records in the database.
Let's say we have a CustomerImport table, defined as below. (We'll use this table in the next
section on ETL)
CREATE TABLE dbo.CustomerImport (
CustomerId int NOT NULL IDENTITY(1,1)
,FirstName varchar(50) NOT NULL
,LastName varchar(50) NOT NULL
,EmailAddress varchar(50) NOT NULL
,Age int NULL
CONSTRAINT PK_CustomerImport PRIMARY KEY CLUSTERED (CustomerId)
)

Using the same code as before, we can then generate random instances of
CustomerImport .

101

Using F# for database related tasks

[<Literal>]
let connectionString = "Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated
Security=True;"
type Sql = SqlDataConnection<connectionString>
// a list of names to sample
let possibleFirstNames =
["Merissa";"Kenneth";"Zora";"Oren"]
let possibleLastNames =
["Applewhite";"Feliz";"Abdulla";"Strunk"]
// generate a random name by picking from the list at random
let generateFirstName() =
FsCheck.Gen.elements possibleFirstNames
let generateLastName() =
FsCheck.Gen.elements possibleLastNames
// generate a random email address by combining random users and domains
let generateEmail() =
let userGen = FsCheck.Gen.elements ["a"; "b"; "c"; "d"; "e"; "f"]
let domainGen = FsCheck.Gen.elements ["gmail.com"; "example.com"; "outlook.com"]
let makeEmail u d = sprintf "%s@%s" u d
FsCheck.Gen.map2 makeEmail userGen domainGen

So far so good.
Now we get to the age column, which is nullable. This means we can't generate random
int s, but instead we have to generate random Nullable<int> s. This is where type

checking is really useful -- the compiler has forced us to take that into account. So to make
sure we cover all the bases, we'll generate a null value one time out of twenty.
// Generate a random nullable age.
// Note that because age is nullable,
// the compiler forces us to take that into account
let generateAge() =
let nonNullAgeGenerator =
FsCheck.Gen.choose(1,99)
|> FsCheck.Gen.map (fun age -> Nullable age)
let nullAgeGenerator =
FsCheck.Gen.constant (Nullable())
// 19 out of 20 times choose a non null age
FsCheck.Gen.frequency [
(19,nonNullAgeGenerator)
(1,nullAgeGenerator)
]

102

Using F# for database related tasks

Putting it altogether...
// a function to create a customer
let createCustomerImport first last email age =
let c = new Sql.ServiceTypes.CustomerImport()
c.FirstName <- first
c.LastName <- last
c.EmailAddress <- email
c.Age <- age
c //return new record
// use applicatives to create a customer generator
let generateCustomerImport =
createCustomerImport
<!> generateFirstName()
<*> generateLastName()
<*> generateEmail()
<*> generateAge()

Once we have a random generator, we can fetch as many records as we like, and insert
them using the type provider.
In the code below, we'll generate 10,000 records, hitting the database in batches of 1,000
records.
let insertAll() =
use db = Sql.GetDataContext()
// optional, turn logging on or off
// db.DataContext.Log <- Console.Out
// db.DataContext.Log <- null
let insertOne counter customer =
db.CustomerImport.InsertOnSubmit customer
// do in batches of 1000
if counter % 1000 = 0 then
db.DataContext.SubmitChanges()
// generate the records
let count = 10000
let generator = FsCheck.Gen.sample 0 count generateCustomerImport
// insert the records
generator |> List.iteri insertOne
db.DataContext.SubmitChanges() // commit any remaining

Finally, let's do it and time it.

103

Using F# for database related tasks

#time
insertAll()
#time

It's not as fast as using BCP, but it is plenty adequate for testing. For example, it only takes a
few seconds to create the 10,000 records above.
I want to stress that this is a single standalone script, not a heavy binary, so it is really easy
to tweak and run on demand.
And of course you get all the goodness of a scripted approach, such as being able to store it
in source control, track changes, etc.

21. Use F# to do simple ETL

The code for this section is available on github.
Say that you need to transfer data from one table to another, but it is not a totally
straightforward copy, as you need to do some mapping and transformation.
This is a classic ETL (Extract/Transform/Load) situation, and most people will reach for
SSIS.
But for some situations, such as one off imports, and where the volumes are not large, you
could use F# instead. Let's have a look.
Say that we are importing data into a master table that looks like this:
CREATE TABLE dbo.Customer (
CustomerId int NOT NULL IDENTITY(1,1)
,Name varchar(50) NOT NULL
,Email varchar(50) NOT NULL
,Birthdate datetime NULL
)

But the system we're importing from has a different format, like this:
CREATE TABLE dbo.CustomerImport (
CustomerId int NOT NULL IDENTITY(1,1)
,FirstName varchar(50) NOT NULL
,LastName varchar(50) NOT NULL
,EmailAddress varchar(50) NOT NULL
,Age int NULL
)

104

Using F# for database related tasks

As part of this import then, we're going to have to:

Concatenate the FirstName and LastName columns into one Name column
Map the EmailAddress column to the Email column
Calculate a Birthdate given an Age
I'm going to skip the CustomerId for now -- hopefully we aren't using IDENTITY
columns in practice.
The first step is to define a function that maps source records to target records. In this case,
we'll call it makeTargetCustomer .
Here's some code for this:
[<Literal>]
let sourceConnectionString =
"Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
[<Literal>]
let targetConnectionString =
"Data Source=localhost; Initial Catalog=SqlInFsharp; Integrated Security=True;"
type SourceSql = SqlDataConnection<sourceConnectionString>
type TargetSql = SqlDataConnection<targetConnectionString>
let makeName first last =
sprintf "%s %s" first last
let makeBirthdate (age:Nullable<int>) =
if age.HasValue then
Nullable (DateTime.Today.AddYears(-age.Value))
else
Nullable()
let makeTargetCustomer (sourceCustomer:SourceSql.ServiceTypes.CustomerImport) =
let targetCustomer = new TargetSql.ServiceTypes.Customer()
targetCustomer.Name <- makeName sourceCustomer.FirstName sourceCustomer.LastName
targetCustomer.Email <- sourceCustomer.EmailAddress
targetCustomer.Birthdate <- makeBirthdate sourceCustomer.Age
targetCustomer // return it

With this transform in place, the rest of the code is easy, we just just read from the source
and write to the target.

105

Using F# for database related tasks

let transferAll() =
use sourceDb = SourceSql.GetDataContext()
use targetDb = TargetSql.GetDataContext()
let insertOne counter customer =
targetDb.Customer.InsertOnSubmit customer
// do in batches of 1000
if counter % 1000 = 0 then
targetDb.DataContext.SubmitChanges()
printfn "...%i records transferred" counter
// get the sequence of source records
sourceDb.CustomerImport
// transform to a target record
|> Seq.map makeTargetCustomer
// and insert
|> Seq.iteri insertOne
targetDb.DataContext.SubmitChanges() // commit any remaining
printfn "Done"

Because these are sequence operations, only one record at a time is in memory (excepting
the LINQ submit buffer), so even large data sets can be processed.
To see it in use, first insert a number of records using the dummy data script just discussed,
and then run the transfer as follows:
#time
transferAll()
#time

Again, it only takes a few seconds to transfer 10,000 records.

And again, this is a single standalone script -- it's a very lightweight way to create simple
ETL jobs.

22. Use F# to generate SQL Agent scripts

For the last database related suggestion, let me suggest the idea of generating SQL Agent
scripts from code.
In any decent sized shop you may have hundreds or thousands of SQL Agent jobs. In my
opinion, these should all be stored as script files, and loaded into the database when
provisioning/building the system.

106

Using F# for database related tasks

Alas, there are often subtle differences between dev, test and production environments:
connection strings, authorization, alerts, log configuration, etc.
That naturally leads to the problem of trying to keep three different copies of a script around,
which in turn makes you think: why not have one script and parameterize it for the
environment?
But now you are dealing with lots of ugly SQL code! The scripts that create SQL agent jobs
are typically hundreds of lines long and were not really designed to be maintained by hand.
F# to the rescue!
In F#, it's really easy to create some simple record types that store all the data you need to
generate and configure a job.
For example, in the script below:
I created a union type called Step that could store a Package , Executable ,
Powershell and so on.

Each of these step types in turn have their own specific properties, so that a Package
has a name and variables, and so on.
A JobInfo consists of a name plus a list of Step s.
An agent script is generated from a JobInfo plus a set of global properties associated
with an environment, such as the databases, shared folder locations, etc.
let thisDir = __SOURCE_DIRECTORY__
System.IO.Directory.SetCurrentDirectory (thisDir)
#load @"..\..\SqlAgentLibrary.Lib.fsx"
module MySqlAgentJob =
open SqlAgentLibrary.Lib.SqlAgentLibrary
let PackageFolder = @"\shared\etl\MyJob"
let step1 = Package {
Name = "An SSIS package"
Package = "AnSsisPackage.dtsx"
Variables =
[
"EtlServer", "EtlServer"
"EtlDatabase", "EtlDatabase"
"SsisLogServer", "SsisLogServer"
"SsisLogDatabase", "SsisLogDatabase"
]
}
let step2 = Package {

107

Using F# for database related tasks

Name = "Another SSIS package"

Package = "AnotherSsisPackage.dtsx"
Variables =
[
"EtlServer", "EtlServer2"
"EtlDatabase", "EtlDatabase2"
"SsisLogServer", "SsisLogServer2"
"SsisLogDatabase", "SsisLogDatabase2"
]
}
let jobInfo = {
JobName = "My SqlAgent Job"
JobDescription = "Copy data from one place to another"
JobCategory = "ETL"
Steps =
[
step1
step2
]
StepsThatContinueOnFailure = []
JobSchedule = None
JobAlert = None
JobNotification = None
}
let generate globals =
writeAgentScript globals jobInfo
module DevEnvironment =
let globals =
[
// global
"Environment", "DEV"
"PackageFolder", @"\shared\etl\MyJob"
"JobServer", "(local)"
// General variables
"JobName", "Some packages"
"SetStartFlag", "2"
"SetEndFlag", "0"
// databases
"Database", "mydatabase"
"Server", "localhost"
"EtlServer", "localhost"
"EtlDatabase", "etl_config"
"SsisLogServer", "localhost"
"SsisLogDatabase", "etl_config"
] |> Map.ofList

108

Using F# for database related tasks

let generateJob() =
MySqlAgentJob.generate globals
DevEnvironment.generateJob()

I can't share the actual F# code, but I think you get the idea. It's quite simple to create.
Once we have these .FSX files, we can generate the real SQL Agent scripts en-masse and
then deploy them to the appropriate servers.
Below is an example of a SQL Agent script that might be generated automatically from the
.FSX file.
As you can see, it is a nicely laid out and formatted T-SQL script. The idea is that a DBA can
review it and be confident that no magic is happening, and thus be willing to accept it as
input.
On the other hand, it would be risky to maintain scripts like. Editing the SQL code directly
could be risky. Better to use type-checked (and more concise) F# code than untyped T-SQL!
USE [msdb]
GO
-- =====================================================
-- Script that deletes and recreates the SQL Agent job 'My SqlAgent Job'
--
-- The job steps are:
-- 1) An SSIS package
-- {Continue on error=false}
-- 2) Another SSIS package
-- {Continue on error=false}
-- =====================================================

-- =====================================================
-- Environment is DEV
--
-- The other global variables are:
-- Database = mydatabase
-- EtlDatabase = etl_config
-- EtlServer = localhost
-- JobName = My SqlAgent Job
-- JobServer = (local)
-- PackageFolder = \\shared\etl\MyJob\
-- Server = localhost
-- SetEndFlag = 0
-- SetStartFlag = 2
-- SsisLogDatabase = etl_config

109

Using F# for database related tasks

-- SsisLogServer = localhost
-- =====================================================

-- =====================================================
-- Create job
-- =====================================================
-- ---------------------------------------------- Delete Job if it exists
-- --------------------------------------------IF EXISTS (SELECT job_id FROM msdb.dbo.sysjobs_view WHERE name = 'My SqlAgent Job')
BEGIN
PRINT 'Deleting job "My SqlAgent Job"'
EXEC msdb.dbo.sp_delete_job @job_name='My SqlAgent Job', @delete_unused_schedule=0
END
-- ---------------------------------------------- Create Job
-- --------------------------------------------BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
-- ---------------------------------------------- Create Category if needed
-- --------------------------------------------IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name='ETL' AND category_c
lass=1)
BEGIN
PRINT 'Creating category "ETL"'
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name='
ETL'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
END
-- ---------------------------------------------- Create Job
-- --------------------------------------------DECLARE @jobId BINARY(16)
PRINT 'Creating job "My SqlAgent Job"'
EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name='My SqlAgent Job',
@enabled=1,
@category_name='ETL',
@owner_login_name=N'sa',
@description='Copy data from one place to another',
@job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

110

Using F# for database related tasks

PRINT '-- ---------------------------------------------'

PRINT 'Create step 1: "An SSIS package"'
PRINT '-- ---------------------------------------------'
DECLARE @Step1_Name nvarchar(50) = 'An SSIS package'
DECLARE @Step1_Package nvarchar(170) = 'AnSsisPackage.dtsx'
DECLARE @Step1_Command nvarchar(1700) =
'/FILE "\\shared\etl\MyJob\AnSsisPackage.dtsx"' +
' /CHECKPOINTING OFF' +
' /SET "\Package.Variables[User::SetFlag].Value";"2"' +
' /SET "\Package.Variables[User::JobName].Value";""' +
' /SET "\Package.Variables[User::SourceServer].Value";"localhost"' +
' /SET "\Package.Variables[User::SourceDatabaseName].Value";"etl_config"' +
' /REPORTING E'
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=@Step1_Name,
@step_id=1,
@on_success_action=3,
@on_fail_action=2,
@subsystem=N'SSIS',
@command=@Step1_Command
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

PRINT '-- ---------------------------------------------'

PRINT 'Create step 2: "Another SSIS Package"'
PRINT '-- ---------------------------------------------'
DECLARE @Step2_Name nvarchar(50) = 'Another SSIS Package'
DECLARE @Step2_Package nvarchar(170) = 'AnotherSsisPackage.dtsx'
DECLARE @Step2_Command nvarchar(1700) =
'/FILE "\\shared\etl\MyJob\AnotherSsisPackage.dtsx.dtsx"' +
' /CHECKPOINTING OFF' +
' /SET "\Package.Variables[User::EtlServer].Value";"localhost"' +
' /SET "\Package.Variables[User::EtlDatabase].Value";"etl_config"' +
' /SET "\Package.Variables[User::SsisLogServer].Value";"localhost"' +
' /SET "\Package.Variables[User::SsisLogDatabase].Value";"etl_config"' +
' /REPORTING E'
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=@Step2_Name,
@step_id=2,
@on_success_action=3,
@on_fail_action=2,
@subsystem=N'SSIS',
@command=@Step2_Command
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
-- ---------------------------------------------- Job Schedule
-- ---------------------------------------------

111

Using F# for database related tasks

-- ----------------------------------------------- Job Alert

-- ----------------------------------------------

-- ---------------------------------------------- Set start step

-- --------------------------------------------EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
-- ---------------------------------------------- Set server
-- ---------------------------------------------

EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = '(local)'

IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

PRINT 'Done!'
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:
GO

Summary
I hope that this set of suggestions has thrown a new light on what F# can be used for.
In my opinion, the combination of concise syntax, lightweight scripting (no binaries) and SQL
type providers makes F# incredibly useful for database related tasks.
Please leave a comment and let me know what you think.

112

Other interesting ways of using F# at work

This post is the conclusion of the series on low-risk and incremental ways to use F# at work.
To wrap up, we'll look at a few more ways in which F# can help you with various
development tasks around the edges, without impacting any core or mission critical code.

113

Other interesting ways of using F# at work

21. Use F# to do simple ETL

22. Use F# to generate SQL Agent scripts
Part 5: Other interesting ways of using F#
23. Use F# for parsing
24. Use F# for diagramming and visualization
25. Use F# for accessing web-based data stores
26. Use F# for data science and machine learning
(BONUS) 27: Balance the generation schedule for the UK power station fleet

Part 5: Other ways of using F# outside the core

This last group of suggestions is a bit of a mish-mash I'm afraid. These are things that didn't
fit into earlier posts, mostly concerning using F# for analysis and data processing.

23. Use F# for parsing

It is surprising how often you need to parse something in the course of routine development:
splitting strings at spaces, reading a CSV file, doing substitutions in a template, finding
HTML links for a web crawler, parsing a query string in a URI, and so on.
F#, being an ML-derived language, is ideal for parsing tasks of all kinds, from simple
regexes to full fledged parsers.
Of course, there are many off-the-shelf libraries for common tasks, but sometimes you need
to write your own. A good example of this is TickSpec, the BDD framework that we saw
earlier.
TickSpec needs to parse the so-called "Gherkin" format of Given/When/Then. Rather than
create a dependency on another library, I imagine that it was easier (and more fun) for Phil
to write his own parser in a few hundred lines. You can see part of the source code here.
Another situation where it might be worth writing your own parser is when you have some
complex system, such as a rules engine, which has a horrible XML configuration format.
Rather than manually editing the configuration, you could create a very simple domain
specific language (DSL) that is parsed and then converted to the complex XML.
In his book on DSLs, Martin Fowler gives an example of this, a DSL that is parsed to create
a state machine. And here is an F# implementation of that DSL.

114

Other interesting ways of using F# at work

For more complicating parsing tasks, I highly recommend using FParsec, which is perfectly
suited for this kind of thing. For example, it has been used for parsing search queries for
FogCreek, CSV files, chess notation, and a custom DSL for load testing scenarios.

24. Use F# for diagramming and visualization

Once you have parsed or analyzed something, it is always nice if you can display the results
visually, rather than as tables full of data.
For example, in a previous post I used F# in conjunction with GraphViz to create diagrams of
dependency relationships. You can see a sample below:

The code to generate the diagram itself was short, only about 60 lines, which you can see
here.
As an alternative to GraphViz, you could also consider using FSGraph.
For more mathematical or data-centric visualizations, there are a number of good libraries:
FSharp.Charting for desktop visualizations that is well integrated with F# scripting.
FsPlot for interactive visualizations in HTML.
VegaHub, an F# library for working with Vega
F# for Visualization
And finally, there's the 800 lb gorilla -- Excel.

115

Other interesting ways of using F# at work

Using the built-in capabilities of Excel is great, if it is available. And F# scripting plays well
with Excel.
You can chart in Excel, plot functions in Excel, and for even more power and integration, you
have the FCell and Excel-DNA projects.

25. Use F# for accessing web-based data

stores
There is a lot of public data out on the web, just waiting to pulled down and loved. With the
magic of type providers, F# is a good choice for direct integrating these web-scale data
stores into your workflow.
Right now, we'll look at two data stores: Freebase and World Bank. More will be available
soon -- see the fsharp.org Data Access page for the latest information.

Freebase
The code for this section is available on github.
Freebase is a large collaborative knowledge base and online collection of structured data
harvested from many sources.
To get started, just link in the type provider DLL as we have seen before.
The site is throttled, so you'll probably need an API key if you're using it a lot (api details
here)

116

Other interesting ways of using F# at work

// sets the current directory to be same as the script directory

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// Requires FSharp.Data under script directory
// nuget install FSharp.Data -o Packages -ExcludeVersion
#r @"Packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data
// without a key
let data = FreebaseData.GetDataContext()
// with a key
(*
[<Literal>]
let FreebaseApiKey = "<enter your freebase-enabled google API key here>"
type FreebaseDataWithKey = FreebaseDataProvider<Key=FreebaseApiKey>
let data = FreebaseDataWithKey.GetDataContext()
*)

Once the type provider is loaded, you can start asking questions, such as...
"Who are the US presidents?"
data.Society.Government.``US Presidents``
|> Seq.map (fun p -> p.``President number`` |> Seq.head, p.Name)
|> Seq.sortBy fst
|> Seq.iter (fun (n,name) -> printfn "%s was number %i" name n )

Result:
George Washington was number 1
John Adams was number 2
Thomas Jefferson was number 3
James Madison was number 4
James Monroe was number 5
John Quincy Adams was number 6
...
Ronald Reagan was number 40
George H. W. Bush was number 41
Bill Clinton was number 42
George W. Bush was number 43
Barack Obama was number 44

Not bad for just four lines of code!

How about "what awards did Casablanca win?"

117

Other interesting ways of using F# at work

data.``Arts and Entertainment``.Film.Films.IndividualsAZ.C.Casablanca.``Awards Won``

|> Seq.map (fun award -> award.Year, award.``Award category``.Name)
|> Seq.sortBy fst
|> Seq.iter (fun (year,name) -> printfn "%s -- %s" year name)

The result is:

1943 -- Academy Award for Best Director
1943 -- Academy Award for Best Picture
1943 -- Academy Award for Best Screenplay

So that's Freebase. Lots of good information, both useful and frivolous.

More on how to use the Freebase type provider.

Using Freebase to generate realistic test data

We've seen how FsCheck can be used to generate test data. Well, you can also get the
same affect by getting data from Freebase, which makes the data much more realistic.
Kit Eason showed how to do this in a tweet, and here's an example based on his code:
let randomElement =
let random = new System.Random()
fun (arr:string array) -> arr.[random.Next(arr.Length)]
let surnames =
FreebaseData.GetDataContext().Society.People.``Family names``
|> Seq.truncate 1000
|> Seq.map (fun name -> name.Name)
|> Array.ofSeq
let firstnames =
FreebaseData.GetDataContext().Society.Celebrities.Celebrities
|> Seq.truncate 1000
|> Seq.map (fun celeb -> celeb.Name.Split([|' '|]).[0])
|> Array.ofSeq
// generate ten random people and print
type Person = {Forename:string; Surname:string}
Seq.init 10 ( fun _ ->
{Forename = (randomElement firstnames);
Surname = (randomElement surnames) }
)
|> Seq.iter (printfn "%A")

118

Other interesting ways of using F# at work

The results are:

{Forename = "Kelly"; Surname = "Deasy";}
{Forename = "Bam"; Surname = "Br?z?";}
{Forename = "Claire"; Surname = "Sludden";}
{Forename = "Kenneth"; Surname = "Kl?tz";}
{Forename = "?tienne"; Surname = "Defendi";}
{Forename = "Billy"; Surname = "Paleti";}
{Forename = "Alix"; Surname = "Nuin";}
{Forename = "Katherine"; Surname = "Desporte";}
{Forename = "Jasmine"; Surname = "Belousov";}
{Forename = "Josh"; Surname = "Kramarsic";}

World Bank
The code for this section is available on github.
On the other extreme from Freebase is the World Bank Open Data, which has lots of
detailed economic and social information from around the world.
The setup is identical to Freebase, but no API key is needed.
// sets the current directory to be same as the script directory
System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// Requires FSharp.Data under script directory
// nuget install FSharp.Data -o Packages -ExcludeVersion
#r @"Packages\FSharp.Data\lib\net40\FSharp.Data.dll"
open FSharp.Data
let data = WorldBankData.GetDataContext()

With the type provider set up, we can do a serious query, such as:
"How do malnutrition rates compare between low income and high income countries?"

119

Other interesting ways of using F# at work

// Create a list of countries to process

let groups =
[| data.Countries.``Low income``
data.Countries.``High income``
|]
// get data from an indicator for particular year
let getYearValue (year:int) (ind:Runtime.WorldBank.Indicator) =
ind.Name,year,ind.Item year
// get data
[ for c in groups ->
c.Name,
c.Indicators.``Malnutrition prevalence, weight for age (% of children under 5)`` |
> getYearValue 2010
]
// print the data
|> Seq.iter (
fun (group,(indName, indYear, indValue)) ->
printfn "%s -- %s %i %0.2f%% " group indName indYear indValue)

The result is:

Low income -- Malnutrition prevalence, weight for age (% of children under 5) 2010 23.
19%
High income -- Malnutrition prevalence, weight for age (% of children under 5) 2010 1.
36%

Similarly, here is the code to compare maternal mortality rates:

// Create a list of countries to process
let countries =
[| data.Countries.`Èuropean Union``
data.Countries.`Ùnited Kingdom``
data.Countries.`Ùnited States`` |]
/ get data
[ for c in countries ->
c.Name,
c.Indicators.``Maternal mortality ratio (modeled estimate, per 100,000 live births
)`` |> getYearValue 2010
]
// print the data
|> Seq.iter (
fun (group,(indName, indYear, indValue)) ->
printfn "%s -- %s %i %0.1f" group indName indYear indValue)

The result is:

120

Other interesting ways of using F# at work

European Union -- Maternal mortality ratio (modeled estimate, per 100,000 live births)
2010 9.0
United Kingdom -- Maternal mortality ratio (modeled estimate, per 100,000 live births)
2010 12.0
United States -- Maternal mortality ratio (modeled estimate, per 100,000 live births)
2010 21.0

26. Use F# for data science and machine

learning
So you're putting all these suggestions into practice. You're parsing your web logs with
FParsec, extracting stats from your internal databases with the SQL type provider, and
pulling down external data from web services. You've got all this data -- what can you do
with it?
Let's finish up by having a quick look at using F# for data science and machine learning.
As we have seen, F# is great for exploratory programming -- it has a REPL with intellisense.
But unlike Python and R, your code is type checked, so you know that your code is not going
to fail with an exception halfway through a two hour processing job!
If you are familiar with the Pandas library from Python or the ?tseries' package in R, then
you should take a serious look at Deedle, an easy-to-use, high quality package for data and
time series manipulation. Deedle is designed to work well for exploratory programming using
the REPL, but can be also used in efficient compiled .NET code.
And if you use R a lot, there's an R type provider(of course). This means you can use R
packages as if they were .NET libraries. How awesome is that!
There's lots of other F# friendly packages too. You can find out all about them at fsharp.org.
Data science
Math
Machine learning

Series summary
Phew! That was a long list of examples and a lot of code to look at. If you've made it to the
end, congratulations!

121

Other interesting ways of using F# at work

I hope that this has given you some new insights into the value of F#. It's not just a math-y or
financial language -- it's a practical one too. And it can help you with all sorts of things in
your development, testing, and data management workflows.
Finally, as I have stressed throughout this series, all these uses are safe, low risk and
incremental. What's the worst that can happen?
So go on, persuade your team mates and boss to give F# a try, and let me know how it
goes.

Postscript
After I posted this, Simon Cousins tweeted that I missed one -- I can't resist adding it.
@ScottWlaschin 27: balance the generation schedule for the uk power station fleet.
seriously, the alternative to #fsharp was way too risky
Simon Cousins (@simontcousins) April 25, 2014
You can read more about Simon's real-world of use of F# (for power generation) on his blog.
There are more testimonials to F# at fsharp.org.

122

The "Why use F#?" Series

This series of posts will give you a guided tour through the main features of F# and then
show you ways that F# can help you in your day-to-day development.
Introduction to the 'Why use F#' series. An overview of the benefits of F#.
F# syntax in 60 seconds. A very quick overview on how to read F# code.
Comparing F# with C#: A simple sum. In which we attempt to sum the squares from 1 to
N without using a loop.
Comparing F# with C#: Sorting. In which we see that F# is more declarative than C#,
and we are introduced to pattern matching..
Comparing F# with C#: Downloading a web page. In which we see that F# excels at
callbacks, and we are introduced to the 'use' keyword.
Four Key Concepts. The concepts that differentiate F# from a standard imperative
language.
Conciseness. Why is conciseness important?.
Type inference. How to avoid getting distracted by complex type syntax.
Low overhead type definitions. No penalty for making new types.
Using functions to extract boilerplate code. The functional approach to the DRY
principle.
Using functions as building blocks. Function composition and mini-languages make
code more readable.
Pattern matching for conciseness. Pattern matching can match and bind in a single
step.
Convenience. Features that reduce programming drudgery and boilerplate code.
Out-of-the-box behavior for types. Immutability and built-in equality with no coding.
Functions as interfaces. OO design patterns can be trivial when functions are used.
Partial Application. How to fix some of a function's parameters.
Active patterns. Dynamic patterns for powerful matching.
Correctness. How to write 'compile time unit tests'.
Immutability. Making your code predictable.
Exhaustive pattern matching. A powerful technique to ensure correctness.
Using the type system to ensure correct code. In F# the type system is your friend, not
your enemy.
Worked example: Designing for correctness. How to make illegal states
unrepresentable.
Concurrency. The next major revolution in how we write software?.
Asynchronous programming. Encapsulating a background task with the Async class.
Messages and Agents. Making it easier to think about concurrency.
Functional Reactive Programming. Turning events into streams.
Completeness. F# is part of the whole .NET ecosystem.
Seamless interoperation with .NET libraries. Some convenient features for working with
.NET libraries.
123

The "Why use F#?" Series

Anything C# can do.... A whirlwind tour of object-oriented code in F#.

Why use F#: Conclusion. .

124

Introduction to the 'Why use F#' series

This series of posts will give you a guided tour through the main features of F# and then
show you ways that F# can help you in your day-to-day development.

Key benefits of F# compared with C#

If you are already familiar with C# or Java, you might be wondering why it would be worth
learning yet another language. F# has some major benefits which I have grouped under the
following themes:
Conciseness. F# is not cluttered up with coding "noise" such as curly brackets,
semicolons and so on. You almost never have to specify the type of an object, thanks to
a powerful type inference system. And it generally takes less lines of code to solve the
same problem.
Convenience. Many common programming tasks are much simpler in F#. This includes
things like creating and using complex type definitions, doing list processing,
comparison and equality, state machines, and much more. And because functions are
first class objects, it is very easy to create powerful and reusable code by creating
functions that have other functions as parameters, or that combine existing functions to
create new functionality.
Correctness. F# has a very powerful type system which prevents many common errors
such as null reference exceptions. And in addition, you can often encode business logic
using the type system itself, so that it is actually impossible to write incorrect code,
because it is caught at compile time as a type error.
Concurrency. F# has a number of built-in tools and libraries to help with programming
systems when more than one thing at a time is happening. Asynchronous programming
is directly supported, as is parallelism. F# also has a message queuing system, and
excellent support for event handling and reactive programming. And because data
structures are immutable by default, sharing state and avoiding locks is much easier.
Completeness. Although F# is a functional language at heart, it does support other
styles which are not 100% pure, which makes it much easier to interact with the nonpure world of web sites, databases, other applications, and so on. In particular, F# is
designed as a hybrid functional/OO language, so it can do almost everything that C#
can do as well. Of course, F# integrates seamlessly with the .NET ecosystem, which
gives you access to all the third party .NET libraries and tools. Finally, it is part of Visual
Studio, which means you get a good editor with IntelliSense support, a debugger, and
many plug-ins for unit tests, source control, and other development tasks.

125

Introduction to the 'Why use F#' series

In the rest of this series of posts, I will try to demonstrate each of these F# benefits, using
standalone snippets of F# code (and often with C# code for comparison). I'll briefly cover all
the major features of F#, including pattern matching, function composition, and concurrent
programming. By the time you have finished this series, I hope that you will have been
impressed with the power and elegance of F#, and you will be encouraged to use it for your
next project!

How to read and use the example code

All the code snippets in these posts have been designed to be run interactively. I strongly
recommend that you evaluate the snippets as you read each post. The source for any large
code files will be linked to from the post.
This series is not a tutorial, so I will not go too much into why the code works. Don't worry if
you cannot understand some of the details; the goal of the series is just to introduce you to
F# and whet your appetitite for learning it more deeply.
If you have experience in languages such as C# and Java, you have probably found that you
can get a pretty good understanding of source code written in other similar languages, even
if you aren't familiar with the keywords or the libraries. You might ask "how do I assign a
variable?" or "how do I do a loop?", and with these answers be able to do some basic
programming quite quickly.
This approach will not work for F#, because in its pure form there are no variables, no loops,
and no objects. Don't be frustrated - it will eventually make sense! If you want to learn F# in
more depth, there are some helpful tips on the "learning F#" page.

126

Comparing F# with C#: A simple sum

To see what some real F# code looks like, let's start with a simple problem: "sum the
squares from 1 to N".
We'll compare an F# implementation with a C# implementation. First, the F# code:
// define the square function
let square x = x * x
// define the sumOfSquares function
let sumOfSquares n =
[1..n] |> List.map square |> List.sum
// try it
sumOfSquares 100

The mysterious looking |> is called the pipe operator. It just pipes the output of one
expression into the input of the next. So the code for sumOfSquares reads as:
1. Create a list of 1 to n (square brackets construct a list).
2. Pipe the list into the library function called List.map , transforming the input list into an
output list using the "square" function we just defined.
3. Pipe the resulting list of squares into the library function called List.sum . Can you
guess what it does?
4. There is no explicit "return" statement. The output of List.sum is the overall result of
the function.
Next, here's a C# implementation using the classic (non-functional) style of a C-based
language. (A more functional version using LINQ is discussed later.)

127

Comparing F# with C#: A simple sum

public static class SumOfSquaresHelper

{
public static int Square(int i)
{
return i * i;
}
public static int SumOfSquares(int n)
{
int sum = 0;
for (int i = 1; i <= n; i++)
{
sum += Square(i);
}
return sum;
}
}

What are the differences?

The F# code is more compact
The F# code didn't have any type declarations
F# can be developed interactively
Let's take each of these in turn.

Less code
The most obvious difference is that there is a lot more C# code. 13 C# lines compared with 3
F# lines (ignoring comments). The C# code has lots of "noise", things like curly braces,
semicolons, etc. And in C# the functions cannot stand alone, but need to be added to some
class ("SumOfSquaresHelper"). F# uses whitespace instead of parentheses, needs no line
terminator, and the functions can stand alone.
In F# it is common for entire functions to be written on one line, as the "square" function is.
The sumOfSquares function could also have been written on one line. In C# this is normally
frowned upon as bad practice.
When a function does have multiple lines, F# uses indentation to indicate a block of code,
which eliminates the need for braces. (If you have ever used Python, this is the same idea).
So the sumOfSquares function could also have been written this way:
let sumOfSquares n =
[1..n]
|> List.map square
|> List.sum

128

Comparing F# with C#: A simple sum

The only drawback is that you have to indent your code carefully. Personally, I think it is
worth the trade-off.

No type declarations
The next difference is that the C# code has to explicitly declare all the types used. For
example, the int i parameter and int SumOfSquares return type. Yes, C# does allow you
to use the "var" keyword in many places, but not for parameters and return types of
functions.
In the F# code we didn't declare any types at all. This is an important point: F# looks like an
untyped language, but it is actually just as type-safe as C#, in fact, even more so! F# uses a
technique called "type inference" to infer the types you are using from their context. It works
amazingly very well most of the time, and reduces the code complexity immensely.
In this case, the type inference algorithm notes that we started with a list of integers. That in
turn implies that the square function and the sum function must be taking ints as well, and
that the final value must be an int. You can see what the inferred types are by looking at the
result of the compilation in the interactive window. You'll see something like:
val square : int -> int

which means that the "square" function takes an int and returns an int.
If the original list had used floats instead, the type inference system would have deduced
that the square function used floats instead. Try it and see:
// define the square function
let squareF x = x * x
// define the sumOfSquares function
let sumOfSquaresF n =
[1.0 .. n] |> List.map squareF |> List.sum // "1.0" is a float
sumOfSquaresF 100.0

The type checking is very strict! If you try using a list of floats ( [1.0..n] ) in the original
sumOfSquares example, or a list of ints ( [1 ..n] ) in the sumOfSquaresF example, you will

get a type error from the compiler.

Interactive development

129

Comparing F# with C#: A simple sum

Finally, F# has an interactive window where you can test the code immediately and play
around with it. In C# there is no easy way to do this.
For example, I can write my square function and immediately test it:
// define the square function
let square x = x * x
// test
let s2 = square 2
let s3 = square 3
let s4 = square 4

When I am satisfied that it works, I can move on to the next bit of code.
This kind of interactivity encourages an incremental approach to coding that can become
addictive!
Furthermore, many people claim that designing code interactively enforces good design
practices such as decoupling and explicit dependencies, and therefore, code that is suitable
for interactive evaluation will also be code that is easy to test. Conversely, code that cannot
be tested interactively will probably be hard to test as well.

The C# code revisited

My original example was written using "old-style" C#. C# has incorporated a lot of functional
features, and it is possible to rewrite the example in a more compact way using the LINQ
extensions.
So here is another C# version -- a line-for-line translation of the F# code.
public static class FunctionalSumOfSquaresHelper
{
public static int SumOfSquares(int n)
{
return Enumerable.Range(1, n)
.Select(i => i * i)
.Sum();
}
}

However, in addition to the noise of the curly braces and periods and semicolons, the C#
version needs to declare the parameter and return types, unlike the F# version.

130

Comparing F# with C#: A simple sum

Many C# developers may find this a trivial example, but still resort back to loops when the
logic becomes more complicated. In F# though, you will almost never see explicit loops like
this. See for example, this post on eliminating boilerplate from more complicated loops.

131

Comparing F# with C#: Sorting

In this next example, we will implement a quicksort-like algorithm for sorting lists and
compare an F# implementation to a C# implementation.
Here is the logic for a simplified quicksort-like algorithm:
If the list is empty, there is nothing to do.
Otherwise:
1. Take the first element of the list
2. Find all elements in the rest of the list that
are less than the first element, and sort them.
3. Find all elements in the rest of the list that
are >= than the first element, and sort them
4. Combine the three parts together to get the final result:
(sorted smaller elements + firstElement +
sorted larger elements)

Note that this is a simplified algorithm and is not optimized (and it does not sort in place, like
a true quicksort); we want to focus on clarity for now.
Here is the code in F#:
let rec quicksort list =
match list with
| [] -> // If the list is empty
[] // return an empty list
| firstElem::otherElements -> // If the list is not empty
let smallerElements = // extract the smaller ones
otherElements
|> List.filter (fun e -> e < firstElem)
|> quicksort // and sort them
let largerElements = // extract the large ones
otherElements
|> List.filter (fun e -> e >= firstElem)
|> quicksort // and sort them
// Combine the 3 parts into a new list and return it
List.concat [smallerElements; [firstElem]; largerElements]
//test
printfn "%A" (quicksort [1;5;23;18;9;1;3])

132

Comparing F# with C#: Sorting

Again note that this is not an optimized implementation, but is designed to mirror the
algorithm closely.
Let's go through this code:
There are no type declarations anywhere. This function will work on any list that has
comparable items (which is almost all F# types, because they automatically have a
default comparison function).
The whole function is recursive -- this is signaled to the compiler using the rec
keyword in " let rec quicksort list = ".
The match..with is sort of like a switch/case statement. Each branch to test is signaled
with a vertical bar, like so:
match x with
| caseA -> something
| caseB -> somethingElse

The " match " with [] matches an empty list, and returns an empty list.
The " match " with firstElem::otherElements does two things.
First, it only matches a non-empty list.
Second, it creates two new values automatically. One for the first element called
" firstElem ", and one for the rest of the list, called " otherElements ". In C# terms,
this is like having a "switch" statement that not only branches, but does variable
declaration and assignment at the same time.
The -> is sort of like a lambda ( => ) in C#. The equivalent C# lambda would look
something like (firstElem, otherElements) => do something .
The " smallerElements " section takes the rest of the list, filters it against the first
element using an inline lambda expression with the " < " operator and then pipes the
result into the quicksort function recursively.
The " largerElements " line does the same thing, except using the " >= " operator
Finally the resulting list is constructed using the list concatenation function
" List.concat ". For this to work, the first element needs to be put into a list, which is
what the square brackets are for.
Again note there is no "return" keyword; the last value will be returned. In the " [] "
branch the return value is the empty list, and in the main branch, it is the newly
constructed list.
For comparison here is an old-style C# implementation (without using LINQ).

133

Comparing F# with C#: Sorting

public class QuickSortHelper

{
public static List<T> QuickSort<T>(List<T> values)
where T : IComparable
{
if (values.Count == 0)
{
return new List<T>();
}
//get the first element
T firstElement = values[0];
//get the smaller and larger elements
var smallerElements = new List<T>();
var largerElements = new List<T>();
for (int i = 1; i < values.Count; i++) // i starts at 1
{ // not 0!
var elem = values[i];
if (elem.CompareTo(firstElement) < 0)
{
smallerElements.Add(elem);
}
else
{
largerElements.Add(elem);
}
}
//return the result
var result = new List<T>();
result.AddRange(QuickSort(smallerElements.ToList()));
result.Add(firstElement);
result.AddRange(QuickSort(largerElements.ToList()));
return result;
}
}

Comparing the two sets of code, again we can see that the F# code is much more compact,
with less noise and no need for type declarations.
Furthermore, the F# code reads almost exactly like the actual algorithm, unlike the C# code.
This is another key advantage of F# -- the code is generally more declarative ("what to do")
and less imperative ("how to do it") than C#, and is therefore much more self-documenting.

A functional implementation in C#

134

Comparing F# with C#: Sorting

Here's a more modern "functional-style" implementation using LINQ and an extension

method:
public static class QuickSortExtension
{
/// <summary>
/// Implement as an extension method for IEnumerable
/// </summary>
public static IEnumerable<T> QuickSort<T>(
this IEnumerable<T> values) where T : IComparable
{
if (values == null || !values.Any())
{
return new List<T>();
}
//split the list into the first element and the rest
var firstElement = values.First();
var rest = values.Skip(1);
//get the smaller and larger elements
var smallerElements = rest
.Where(i => i.CompareTo(firstElement) < 0)
.QuickSort();
var largerElements = rest
.Where(i => i.CompareTo(firstElement) >= 0)
.QuickSort();
//return the result
return smallerElements
.Concat(new List<T>{firstElement})
.Concat(largerElements);
}
}

This is much cleaner, and reads almost the same as the F# version. But unfortunately there
is no way of avoiding the extra noise in the function signature.

Correctness
Finally, a beneficial side-effect of this compactness is that F# code often works the first time,
while the C# code may require more debugging.
Indeed, when coding these samples, the old-style C# code was incorrect initially, and
required some debugging to get it right. Particularly tricky areas were the for loop (starting
at 1 not zero) and the CompareTo comparison (which I got the wrong way round), and it

135

Comparing F# with C#: Sorting

would also be very easy to accidentally modify the inbound list. The functional style in the
second C# example is not only cleaner but was easier to code correctly.
But even the functional C# version has drawbacks compared to the F# version. For
example, because F# uses pattern matching, it is not possible to branch to the "non-empty
list" case with an empty list. On the other hand, in the C# code, if we forgot the test:
if (values == null || !values.Any()) ...

then the extraction of the first element:

var firstElement = values.First();

would fail with an exception. The compiler cannot enforce this for you. In your own code,
how often have you used FirstOrDefault rather than First because you are writing
"defensive" code. Here is an example of a code pattern that is very common in C# but is rare
in F#:
var item = values.FirstOrDefault(); // instead of .First()
if (item != null)
{
// do something if item is valid
}

The one-step "pattern match and branch" in F# allows you to avoid this in many cases.

Postscript
The example implementation in F# above is actually pretty verbose by F# standards!
For fun, here is what a more typically concise version would look like:
let rec quicksort2 = function
| [] -> []
| first::rest ->
let smaller,larger = List.partition ((>=) first) rest
List.concat [quicksort2 smaller; [first]; quicksort2 larger]
// test code
printfn "%A" (quicksort2 [1;5;23;18;9;1;3])

Not bad for 4 lines of code, and when you get used to the syntax, still quite readable.

136

Comparing F# with C#: Sorting

137

Comparing F# with C#: Downloading a web page

Comparing F# with C#: Downloading a

web page
In this example, we will compare the F# and C# code for downloading a web page, with a
callback to process the text stream.
We'll start with a straightforward F# implementation.
// "open" brings a .NET namespace into visibility
open System.Net
open System
open System.IO
// Fetch the contents of a web page
let fetchUrl callback url =
let req = WebRequest.Create(Uri(url))
use resp = req.GetResponse()
use stream = resp.GetResponseStream()
use reader = new IO.StreamReader(stream)
callback reader url

Let's go through this code:

The use of "open" at the top allows us to write "WebRequest" rather than
"System.Net.WebRequest". It is similar to a " using System.Net " header in C#.
Next, we define the fetchUrl function, which takes two arguments, a callback to
process the stream, and the url to fetch.
We next wrap the url string in a Uri. F# has strict type-checking, so if instead we had
written: let req = WebRequest.Create(url) the compiler would have complained that it
didn't know which version of WebRequest.Create to use.
When declaring the response , stream and reader values, the " use " keyword is
used instead of " let ". This can only be used in conjunction with classes that
implement IDisposable . It tells the compiler to automatically dispose of the resource
when it goes out of scope. This is equivalent to the C# " using " keyword.
The last line calls the callback function with the StreamReader and url as parameters.
Note that the type of the callback does not have to be specified anywhere.
Now here is the equivalent C# implementation.

138

Comparing F# with C#: Downloading a web page

class WebPageDownloader
{
public TResult FetchUrl<TResult>(
string url,
Func<string, StreamReader, TResult> callback)
{
var req = WebRequest.Create(url);
using (var resp = req.GetResponse())
{
using (var stream = resp.GetResponseStream())
{
using (var reader = new StreamReader(stream))
{
return callback(url, reader);
}
}
}
}
}

As usual, the C# version has more 'noise'.

There are ten lines just for curly braces, and there is the visual complexity of 5 levels of
nesting*
All the parameter types have to be explicitly declared, and the generic TResult type
has to be repeated three times.
* It's true that in this particular example, when all the using statements are adjacent, the
extra braces and indenting can be removed, but in the more general case they are needed.

Testing the code

Back in F# land, we can now test the code interactively:
let myCallback (reader:IO.StreamReader) url =
let html = reader.ReadToEnd()
let html1000 = html.Substring(0,1000)
printfn "Downloaded %s. First 1000 is %s" url html1000
html // return all the html
//test
let google = fetchUrl myCallback "http://google.com"

Finally, we have to resort to a type declaration for the reader parameter

( reader:IO.StreamReader ). This is required because the F# compiler cannot determine the
type of the "reader" parameter automatically.

139

Comparing F# with C#: Downloading a web page

A very useful feature of F# is that you can "bake in" parameters in a function so that they
don't have to be passed in every time. This is why the url parameter was placed last
rather than first, as in the C# version. The callback can be setup once, while the url varies
from call to call.
// build a function with the callback "baked in"
let fetchUrl2 = fetchUrl myCallback
// test
let google = fetchUrl2 "http://www.google.com"
let bbc = fetchUrl2 "http://news.bbc.co.uk"
// test with a list of sites
let sites = ["http://www.bing.com";
"http://www.google.com";
"http://www.yahoo.com"]
// process each site in the list
sites |> List.map fetchUrl2

The last line (using List.map ) shows how the new function can be easily used in
conjunction with list processing functions to download a whole list at once.
Here is the equivalent C# test code:

140

Comparing F# with C#: Downloading a web page

[Test]
public void TestFetchUrlWithCallback()
{
Func<string, StreamReader, string> myCallback = (url, reader) =>
{
var html = reader.ReadToEnd();
var html1000 = html.Substring(0, 1000);
Console.WriteLine(
"Downloaded {0}. First 1000 is {1}", url,
html1000);
return html;
};
var downloader = new WebPageDownloader();
var google = downloader.FetchUrl("http://www.google.com",
myCallback);
// test with a list of sites
var sites = new List<string> {
"http://www.bing.com",
"http://www.google.com",
"http://www.yahoo.com"};
// process each site in the list
sites.ForEach(site => downloader.FetchUrl(site, myCallback));
}

Again, the code is a bit noisier than the F# code, with many explicit type references. More
importantly, the C# code doesn't easily allow you to bake in some of the parameters in a
function, so the callback must be explicitly referenced every time.

141

Four Key Concepts

In the next few posts we'll move on to demonstrating the themes of this series: conciseness,
convenience, correctness, concurrency and completeness.
But before that, let's look at some of the key concepts in F# that we will meet over and over
again. F# is different in many ways from a standard imperative language like C#, but there
are a few major differences that are particularly important to understand:
Function-oriented rather than object-oriented
Expressions rather than statements
Algebraic types for creating domain models
Pattern matching for flow of control
In later posts, these will be dealt with in much greater depth -- this is just a taster to help you
understand the rest of this series.

Function-oriented rather than object-oriented

142

Four Key Concepts

As you might expect from the term "functional programming", functions are everywhere in
F#.
Of course, functions are first class entities, and can be passed around like any other value:
let square x = x * x
// functions as values
let squareclone = square
let result = [1..10] |> List.map squareclone
// functions taking other functions as parameters
let execFunction aFunc aParam = aFunc aParam
let result2 = execFunction square 12

But C# has first-class functions too, so what's so special about functional programming?
The short answer is that the function-oriented nature of F# infiltrates every part of the
language and type system in a way that it does not in C#, so that things that are awkward or
clumsy in C# are very elegant in F#.
It's hard to explain this in a few paragraphs, but here are some of the benefits that we will
see demonstrated over this series of posts:
Building with composition. Composition is the 'glue' that allows us build larger
systems from smaller ones. This is not an optional technique, but is at the very heart of
the functional style. Almost every line of code is a composable expression (see below).
Composition is used to build basic functions, and then functions that use those
functions, and so on. And the composition principle doesn't just apply to functions, but
also to types (the product and sum types discussed below).
Factoring and refactoring. The ability to factor a problem into parts depends how
easily the parts can be glued back together. Methods and classes that might seem to be
indivisible in an imperative language can often be broken down into surprisingly small
pieces in a functional design. These fine-grained components typically consist of (a) a
few very general functions that take other functions as parameters, and (b) other helper
functions that specialize the general case for a particular data structure or application.
Once factored out, the generalized functions allow many additional operations to be
programmed very easily without having to write new code. You can see a good example
of a general function like this (the fold function) in the post on extracting duplicate code
from loops.
Good design. Many of the principles of good design, such as "separation of concerns",
"single responsibility principle", "program to an interface, not an implementation", arise
naturally as a result of a functional approach. And functional code tends to be high level
and declarative in general.

143

Four Key Concepts

The following posts in this series will have examples of how functions can make code more
concise and convenient, and then for a deeper understanding, there is a whole series on
thinking functionally.

Expressions rather than statements

In functional languages, there are no statements, only expressions. That is, every chunk of
code always returns a value, and larger chunks are created by combining smaller chunks
using composition rather than a serialized list of statements.
If you have used LINQ or SQL you will already be familiar with expression-based languages.
For example, in pure SQL, you cannot have assignments. Instead, you must have
subqueries within larger queries.
SELECT EmployeeName
FROM Employees
WHERE EmployeeID IN
(SELECT DISTINCT ManagerID FROM Employees) -- subquery

F# works in the same way -- every function definition is a single expression, not a set of
statements.
And it might not be obvious, but code built from expressions is both safer and more compact
than using statements. To see this, let's compare some statement-based code in C# with the
equivalent expression-based code.
First, the statement-based code. Statements don't return values, so you have to use
temporary variables that are assigned to from within statement bodies.
// statement-based code in C#
int result;
if (aBool)
{
result = 42;
}
Console.WriteLine("result={0}", result);

Because the if-then block is a statement, the result variable must be defined outside
the statement but assigned to from inside the statement, which leads to issues such as:
What initial value should result be set to?
What if I forget to assign to the result variable?
What is the value of the result variable in the "else" case?
For comparison, here is the same code, rewritten in an expression-oriented style:
144

Four Key Concepts

// expression-based code in C#
int result = (aBool) ? 42 : 0;
Console.WriteLine("result={0}", result);

In the expression-oriented version, none of these issues apply:

The result variable is declared at the same time that it is assigned. No variables have
to be set up "outside" the expression and there is no worry about what initial value they
should be set to.
The "else" is explicitly handled. There is no chance of forgetting to do an assignment in
one of the branches.
It is not possible to forget to assign result , because then the variable would not even
exist!
Expression-oriented style is not a choice in F#, and it is one of the things that requires a
change of approach when coming from an imperative background.

Algebraic Types
The type system in F# is based on the concept of algebraic types. That is, new compound
types are built by combining existing types in two different ways:
First, a combination of values, each picked from a set of types. These are called
"product" types.
Of, alternately, as a disjoint union representing a choice between a set of types. These
are called "sum" types.
For example, given existing types int and bool , we can create a new product type that
must have one of each:
//declare it
type IntAndBool = {intPart: int; boolPart: bool}
//use it
let x = {intPart=1; boolPart=false}

Alternatively, we can create a new union/sum type that has a choice between each type:

145

Four Key Concepts

//declare it
type IntOrBool =
| IntChoice of int
| BoolChoice of bool
//use it
let y = IntChoice 42
let z = BoolChoice true

These "choice" types are not available in C#, but are incredibly useful for modeling many
real-world cases, such as states in a state machine (which is a surprisingly common theme
in many domains).
And by combining "product" and "sum" types in this way, it is easy to create a rich set of
types that accurately models any business domain. For examples of this in action, see the
posts on low overhead type definitions and using the type system to ensure correct code.

Pattern matching for flow of control

Most imperative languages offer a variety of control flow statements for branching and
looping:
if-then-else (and the ternary version bool ? if-true : if-false )
case or switch statements
for and foreach loops, with break and continue
while and until loops

and even the dreaded goto

F# does support some of these, but F# also supports the most general form of conditional
expression, which is pattern-matching.
A typical matching expression that replaces if-then-else looks like this:
match booleanExpression with
| true -> // true branch
| false -> // false branch

And the replacement of switch might look like this:

match aDigit with
| 1 -> // Case when digit=1
| 2 -> // Case when digit=2
| _ -> // Case otherwise

146

Four Key Concepts

Finally, loops are generally done using recursion, and typically look something like this:
match aList with
| [] ->
// Empty case
| first::rest ->
// Case with at least one element.
// Process first element, and then call
// recursively with the rest of the list

Although the match expression seems unnecessarily complicated at first, you'll see that in
practice it is both elegant and powerful.
For the benefits of pattern matching, see the post on exhaustive pattern matching, and for a
worked example that uses pattern matching heavily, see the roman numerals example.

Pattern matching with union types

We mentioned above that F# supports a "union" or "choice" type. This is used instead of
inheritance to work with different variants of an underlying type. Pattern matching works
seamlessly with these types to create a flow of control for each choice.
In the following example, we create a Shape type representing four different shapes and
then define a draw function with different behavior for each kind of shape. This is similar to
polymorphism in an object oriented language, but based on functions.

147

Four Key Concepts

type Shape = // define a "union" of alternative structures

| Circle of int
| Rectangle of int * int
| Polygon of (int * int) list
| Point of (int * int)
let draw shape = // define a function "draw" with a shape param
match shape with
| Circle radius ->
printfn "The circle has a radius of %d" radius
| Rectangle (height,width) ->
printfn "The rectangle is %d high by %d wide" height width
| Polygon points ->
printfn "The polygon is made of these points %A" points
| _ -> printfn "I don't recognize this shape"
let circle = Circle(10)
let rect = Rectangle(4,5)
let polygon = Polygon( [(1,1); (2,2); (3,3)])
let point = Point(2,3)
[circle; rect; polygon; point] |> List.iter draw

A few things to note:

As usual, we didn't have to specify any types. The compiler correctly determined that
the shape parameter for the "draw" function was of type Shape .
You can see that the match..with logic not only matches against the internal structure
of the shape, but also assigns values based on what is appropriate for the shape.
The underscore is similar to the "default" branch in a switch statement, except that in F#
it is required -- every possible case must always be handled. If you comment out the
line
| _ -> printfn "I don't recognize this shape"

see what happens when you compile!

These kinds of choice types can be simulated somewhat in C# by using subclasses or
interfaces, but there is no built in support in the C# type system for this kind of exhaustive
matching with error checking.

148

Conciseness

Conciseness
After having seen some simple code, we will now move on to demonstrating the major
themes (conciseness, convenience, correctness, concurrency and completeness), filtered
through the concepts of types, functions and pattern matching.
With the next few posts, we'll examine the features of F# that aid conciseness and
readability.
An important goal for most mainstream programming languages is a good balance of
readability and conciseness. Too much conciseness can result in hard-to-understand or
obfuscated code (APL anyone?), while too much verbosity can easily swamp the underlying
meaning. Ideally, we want a high signal-to-noise ratio, where every word and character in
the code contributes to the meaning of the code, and there is minimal boilerplate.
Why is conciseness important? Here are a few reasons:
A concise language tends to be more declarative, saying what the code should do
rather than how to do it. That is, declarative code is more focused on the high-level logic
rather than the nuts and bolts of the implementation.
It is easier to reason about correctness if there are fewer lines of code to reason
about!
And of course, you can see more code on a screen at a time. This might seem trivial,
but the more you can see, the more you can grasp as well.
As you have seen, compared with C#, F# is generally much more concise. This is due to
features such as:
Type inference and low overhead type definitions. One of the major reasons for F#'s
conciseness and readability is its type system. F# makes it very easy to create new
types as you need them. They don't cause visual clutter either in their definition or in
use, and the type inference system means that you can use them freely without getting
distracted by complex type syntax.
Using functions to extract boilerplate code. The DRY principle ("don't repeat
yourself") is a core principle of good design in functional languages as well as objectoriented languages. In F# it is extremely easy to extract repetitive code into common
utility functions, which allows you to focus on the important stuff.
Composing complex code from simple functions and creating mini-languages.
The functional approach makes it easy to create a set of basic operations and then
combine these building blocks in various ways to build up more complex behaviors. In
this way, even the most complex code is still very concise and readable.

149

Conciseness

Pattern matching. We've seen pattern matching as a glorified switch statement, but in
fact it is much more general, as it can compare expressions in a number of ways,
matching on values, conditions, and types, and then assign or extract values, all at the
same time.

150

Type inference

Type inference
As you have already seen, F# uses a technique called "type inference" to greatly reduce the
number of type annotations that need to be explicitly specified in normal code. And even
when types do need to be specified, the syntax is less longwinded compared to C#.
To see this, here are some C# methods that wrap two standard LINQ functions. The
implementations are trivial, but the method signatures are extremely complex:
public IEnumerable<TSource> Where<TSource>(
IEnumerable<TSource> source,
Func<TSource, bool> predicate
)
{
//use the standard LINQ implementation
return source.Where(predicate);
}
public IEnumerable<IGrouping<TKey, TSource>> GroupBy<TSource, TKey>(
IEnumerable<TSource> source,
Func<TSource, TKey> keySelector
)
{
//use the standard LINQ implementation
return source.GroupBy(keySelector);
}

And here are the exact F# equivalents, showing that no type annotations are needed at all!
let Where source predicate =
//use the standard F# implementation
Seq.filter predicate source
let GroupBy source keySelector =
//use the standard F# implementation
Seq.groupBy keySelector source

You might notice that the standard F# implementations for "filter" and "groupBy" have
the parameters in exactly the opposite order from the LINQ implementations used in C#.
The "source" parameter is placed last, rather than first. There is a reason for this, which
will be explained in the thinking functionally series.

151

Type inference

The type inference algorithm is excellent at gathering information from many sources to
determine the types. In the following example, it correctly deduces that the list value is a
list of strings.
let i = 1
let s = "hello"
let tuple = s,i // pack into tuple
let s2,i2 = tuple // unpack
let list = [s2] // type is string list

And in this example, it correctly deduces that the sumLengths function takes a list of strings
and returns an int.
let sumLengths strList =
strList |> List.map String.length |> List.sum
// function type is: string list -> int

152

Low overhead type definitions

In C#, there is a disincentive for creating new types ? the lack of type inference means you
need to explicitly specify types in most places, resulting in brittleness and more visual clutter.
As a result, there is always a temptation to create monolithic classes rather than
modularizing them.
In F# there is no penalty for making new types, so it is quite common to have hundreds if not
thousands of them. Every time you need to define a structure, you can create a special type,
rather than reusing (and overloading) existing types such as strings and lists.
This means that your programs will be more type-safe, more self documenting, and more
maintainable (because when the types change you will immediately get compile-time errors
rather than runtime errors).
Here are some examples of one-liner types in F#:
open System
// some "record" types
type Person = {FirstName:string; LastName:string; Dob:DateTime}
type Coord = {Lat:float; Long:float}
// some "union" (choice) types
type TimePeriod = Hour | Day | Week | Year
type Temperature = C of int | F of int
type Appointment = OneTime of DateTime
| Recurring of DateTime list

F# types and domain driven design

The conciseness of the type system in F# is particularly useful when doing domain driven
design (DDD). In DDD, for each real world entity and value object, you ideally want to have a
corresponding type. This can mean creating hundreds of "little" types, which can be tedious
in C#.
Furthermore, "value" objects in DDD should have structural equality, meaning that two
objects containing the same data should always be equal. In C# this can mean more tedium
in overriding IEquatable<T> , but in F#, you get this for free by default.
To show how easy it is to create DDD types in F#, here are some example types that might
be created for a simple "customer" domain.

153

Low overhead type definitions

type PersonalName = {FirstName:string; LastName:string}

// Addresses
type StreetAddress = {Line1:string; Line2:string; Line3:string }
type ZipCode = ZipCode of string
type StateAbbrev = StateAbbrev of string
type ZipAndState = {State:StateAbbrev; Zip:ZipCode }
type USAddress = {Street:StreetAddress; Region:ZipAndState}
type UKPostCode = PostCode of string
type UKAddress = {Street:StreetAddress; Region:UKPostCode}
type InternationalAddress = {
Street:StreetAddress; Region:string; CountryName:string}
// choice type -- must be one of these three specific types
type Address = USAddress | UKAddress | InternationalAddress
// Email
type Email = Email of string
// Phone
type CountryPrefix = Prefix of int
type Phone = {CountryPrefix:CountryPrefix; LocalNumber:string}
type Contact =
{
PersonalName: PersonalName;
// "option" means it might be missing
Address: Address option;
Email: Email option;
Phone: Phone option;
}
// Put it all together into a CustomerAccount type
type CustomerAccountId = AccountId of string
type CustomerType = Prospect | Active | Inactive
// override equality and deny comparison
[<CustomEquality; NoComparison>]
type CustomerAccount =
{
CustomerAccountId: CustomerAccountId;
CustomerType: CustomerType;
ContactInfo: Contact;
}
override this.Equals(other) =
match other with
| :? CustomerAccount as otherCust ->
(this.CustomerAccountId = otherCust.CustomerAccountId)

154

Low overhead type definitions

| _ -> false
override this.GetHashCode() = hash this.CustomerAccountId

This code fragment contains 17 type definitions in just a few lines, but with minimal
complexity. How many lines of C# code would you need to do the same thing?
Obviously, this is a simplified version with just the basic types ? in a real system, constraints
and other methods would be added. But note how easy it is to create lots of DDD value
objects, especially wrapper types for strings, such as " ZipCode " and " Email ". By using
these wrapper types, we can enforce certain constraints at creation time, and also ensure
that these types don't get confused with unconstrained strings in normal code. The only
"entity" type is the CustomerAccount , which is clearly indicated as having special treatment
for equality and comparison.
For a more in-depth discussion, see the series called "Domain driven design in F#".

155

Using functions to extract boilerplate code

In the very first example in this series, we saw a simple function that calculated the sum of
squares, implemented in both F# and C#. Now let's say we want some new functions which
are similar, such as:
Calculating the product of all the numbers up to N
Counting the sum of odd numbers up to N
The alternating sum of the numbers up to N
Obviously, all these requirements are similar, but how would you extract any common
functionality?
Let's start with some straightforward implementations in C# first:

156

Using functions to extract boilerplate code

public static int Product(int n)

{
int product = 1;
for (int i = 1; i <= n; i++)
{
product *= i;
}
return product;
}
public static int SumOfOdds(int n)
{
int sum = 0;
for (int i = 1; i <= n; i++)
{
if (i % 2 != 0) { sum += i; }
}
return sum;
}
public static int AlternatingSum(int n)
{
int sum = 0;
bool isNeg = true;
for (int i = 1; i <= n; i++)
{
if (isNeg)
{
sum -= i;
isNeg = false;
}
else
{
sum += i;
isNeg = true;
}
}
return sum;
}

What do all these implementations have in common? The looping logic! As programmers,
we are told to remember the DRY principle ("don't repeat yourself"), yet here we have
repeated almost exactly the same loop logic each time. Let's see if we can extract just the
differences between these three methods:

157

Using functions to extract boilerplate code

Function

Initial
value

Inner loop logic

Product

product=1

Multiply the i'th value with the running total

SumOfOdds

sum=0

Add the i'th value to the running total if not even

AlternatingSum

int sum =
0
bool
isNeg =
true

Use the isNeg flag to decide whether to add or

subtract, and flip the flag for the next pass.

Is there a way to strip the duplicate code and focus on the just the setup and inner loop
logic? Yes there is. Here are the same three functions in F#:
let product n =
let initialValue = 1
let action productSoFar x = productSoFar * x
[1..n] |> List.fold action initialValue
//test
product 10
let sumOfOdds n =
let initialValue = 0
let action sumSoFar x = if x%2=0 then sumSoFar else sumSoFar+x
[1..n] |> List.fold action initialValue
//test
sumOfOdds 10
let alternatingSum n =
let initialValue = (true,0)
let action (isNeg,sumSoFar) x = if isNeg then (false,sumSoFar-x)
else (true ,sumSoFar+x)
[1..n] |> List.fold action initialValue |> snd
//test
alternatingSum 100

All three of these functions have the same pattern:

1. Set up the initial value
2. Set up an action function that will be performed on each element inside the loop.
3. Call the library function List.fold . This is a powerful, general purpose function which
starts with the initial value and then runs the action function for each element in the list
in turn.

158

Using functions to extract boilerplate code

The action function always has two parameters: a running total (or state) and the list element
to act on (called "x" in the above examples).
In the last function, alternatingSum , you will notice that it used a tuple (pair of values) for
the initial value and the result of the action. This is because both the running total and the
isNeg flag must be passed to the next iteration of the loop -- there are no "global" values

that can be used. The final result of the fold is also a tuple so we have to use the "snd"
(second) function to extract the final total that we want.
By using List.fold and avoiding any loop logic at all, the F# code gains a number of
benefits:
The key program logic is emphasized and made explicit. The important differences
between the functions become very clear, while the commonalities are pushed to the
background.
The boilerplate loop code has been eliminated, and as a result the code is more
condensed than the C# version (4-5 lines of F# code vs. at least 9 lines of C# code)
There can never be a error in the loop logic (such as off-by-one) because that logic
is not exposed to us.
By the way, the sum of squares example could also be written using fold as well:
let sumOfSquaresWithFold n =
let initialValue = 0
let action sumSoFar x = sumSoFar + (x*x)
[1..n] |> List.fold action initialValue
//test
sumOfSquaresWithFold 100

"Fold" in C#
Can you use the "fold" approach in C#? Yes. LINQ does have an equivalent to fold , called
Aggregate . And here is the C# code rewritten to use it:

159

Using functions to extract boilerplate code

public static int ProductWithAggregate(int n)

{
var initialValue = 1;
Func<int, int, int> action = (productSoFar, x) =>
productSoFar * x;
return Enumerable.Range(1, n)
.Aggregate(initialValue, action);
}
public static int SumOfOddsWithAggregate(int n)
{
var initialValue = 0;
Func<int, int, int> action = (sumSoFar, x) =>
(x % 2 == 0) ? sumSoFar : sumSoFar + x;
return Enumerable.Range(1, n)
.Aggregate(initialValue, action);
}
public static int AlternatingSumsWithAggregate(int n)
{
var initialValue = Tuple.Create(true, 0);
Func<Tuple<bool, int>, int, Tuple<bool, int>> action =
(t, x) => t.Item1
? Tuple.Create(false, t.Item2 - x)
: Tuple.Create(true, t.Item2 + x);
return Enumerable.Range(1, n)
.Aggregate(initialValue, action)
.Item2;
}

Well, in some sense these implementations are simpler and safer than the original C#
versions, but all the extra noise from the generic types makes this approach much less
elegant than the equivalent code in F#. You can see why most C# programmers prefer to
stick with explicit loops.

A more relevant example

A slightly more relevant example that crops up frequently in the real world is how to get the
"maximum" element of a list when the elements are classes or structs. The LINQ method
'max' only returns the maximum value, not the whole element that contains the maximum
value.
Here's a solution using an explicit loop:

160

Using functions to extract boilerplate code

public class NameAndSize

{
public string Name;
public int Size;
}
public static NameAndSize MaxNameAndSize(IList<NameAndSize> list)
{
if (list.Count() == 0)
{
return default(NameAndSize);
}
var maxSoFar = list[0];
foreach (var item in list)
{
if (item.Size > maxSoFar.Size)
{
maxSoFar = item;
}
}
return maxSoFar;
}

Doing this in LINQ seems hard to do efficiently (that is, in one pass), and has come up as a
Stack Overflow question. Jon Skeet event wrote an article about it.
Again, fold to the rescue!
And here's the C# code using Aggregate :
public class NameAndSize
{
public string Name;
public int Size;
}
public static NameAndSize MaxNameAndSize(IList<NameAndSize> list)
{
if (!list.Any())
{
return default(NameAndSize);
}
var initialValue = list[0];
Func<NameAndSize, NameAndSize, NameAndSize> action =
(maxSoFar, x) => x.Size > maxSoFar.Size ? x : maxSoFar;
return list.Aggregate(initialValue, action);
}

161

Using functions to extract boilerplate code

Note that this C# version returns null for an empty list. That seems dangerous -- so what
should happen instead? Throwing an exception? That doesn't seem right either.
Here's the F# code using fold:
type NameAndSize= {Name:string;Size:int}
let maxNameAndSize list =
let innerMaxNameAndSize initialValue rest =
let action maxSoFar x = if maxSoFar.Size < x.Size then x else maxSoFar
rest |> List.fold action initialValue
// handle empty lists
match list with
| [] ->
None
| first::rest ->
let max = innerMaxNameAndSize first rest
Some max

The F# code has two parts:

the innerMaxNameAndSize function is similar to what we have seen before.
the second bit, match list with , branches on whether the list is empty or not. With an
empty list, it returns a None , and in the non-empty case, it returns a Some . Doing this
guarantees that the caller of the function has to handle both cases.
And a test:
//test
let list = [
{Name="Alice"; Size=10}
{Name="Bob"; Size=1}
{Name="Carol"; Size=12}
{Name="David"; Size=5}
]
maxNameAndSize list
maxNameAndSize []

Actually, I didn't need to write this at all, because F# already has a maxBy function!
// use the built in function
list |> List.maxBy (fun item -> item.Size)
[] |> List.maxBy (fun item -> item.Size)

162

Using functions to extract boilerplate code

But as you can see, it doesn't handle empty lists well. Here's a version that wraps the
maxBy safely.

let maxNameAndSize list =

match list with
| [] ->
None
| _ ->
let max = list |> List.maxBy (fun item -> item.Size)
Some max

163

Using functions as building blocks

A well-known principle of good design is to create a set of basic operations and then
combine these building blocks in various ways to build up more complex behaviors. In
object-oriented languages, this goal gives rise to a number of implementation approaches
such as "fluent interfaces", "strategy pattern", "decorator pattern", and so on. In F#, they are
all done the same way, via function composition.
Let's start with a simple example using integers. Say that we have created some basic
functions to do arithmetic:
// building blocks
let add2 x = x + 2
let mult3 x = x * 3
let square x = x * x
// test
[1..10] |> List.map add2 |> printfn "%A"
[1..10] |> List.map mult3 |> printfn "%A"
[1..10] |> List.map square |> printfn "%A"

Now we want to create new functions that build on these:

// new composed functions
let add2ThenMult3 = add2 >> mult3
let mult3ThenSquare = mult3 >> square

The " >> " operator is the composition operator. It means: do the first function, and then do
the second.
Note how concise this way of combining functions is. There are no parameters, types or
other irrelevant noise.
To be sure, the examples could also have been written less concisely and more explicitly as:
let add2ThenMult3 x = mult3 (add2 x)
let mult3ThenSquare x = square (mult3 x)

But this more explicit style is also a bit more cluttered:

In the explicit style, the x parameter and the parentheses must be added, even though
they don't add to the meaning of the code.

164

Using functions as building blocks

And in the explicit style, the functions are written back-to-front from the order they are
applied. In my example of add2ThenMult3 I want to add 2 first, and then multiply. The
add2 >> mult3 syntax makes this visually clearer than mult3(add2 x) .

Now let's test these compositions:

// test
add2ThenMult3 5
mult3ThenSquare 5
[1..10] |> List.map add2ThenMult3 |> printfn "%A"
[1..10] |> List.map mult3ThenSquare |> printfn "%A"

Extending existing functions

Now say that we want to decorate these existing functions with some logging behavior. We
can compose these as well, to make a new function with the logging built in.
// helper functions;
let logMsg msg x = printf "%s%i" msg x; x //without linefeed
let logMsgN msg x = printfn "%s%i" msg x; x //with linefeed
// new composed function with new improved logging!
let mult3ThenSquareLogged =
logMsg "before="
>> mult3
>> logMsg " after mult3="
>> square
>> logMsgN " result="
// test
mult3ThenSquareLogged 5
[1..10] |> List.map mult3ThenSquareLogged //apply to a whole list

Our new function, mult3ThenSquareLogged , has an ugly name, but it is easy to use and nicely
hides the complexity of the functions that went into it. You can see that if you define your
building block functions well, this composition of functions can be a powerful way to get new
functionality.
But wait, there's more! Functions are first class entities in F#, and can be acted on by any
other F# code. Here is an example of using the composition operator to collapse a list of
functions into a single operation.

165

Using functions as building blocks

let listOfFunctions = [
mult3;
square;
add2;
logMsgN "result=";
]
// compose all functions in the list into a single one
let allFunctions = List.reduce (>>) listOfFunctions
//test
allFunctions 5

Mini languages
Domain-specific languages (DSLs) are well recognized as a technique to create more
readable and concise code. The functional approach is very well suited for this.
If you need to, you can go the route of having a full "external" DSL with its own lexer, parser,
and so on, and there are various toolsets for F# that make this quite straightforward.
But in many cases, it is easier to stay within the syntax of F#, and just design a set of "verbs"
and "nouns" that encapsulate the behavior we want.
The ability to create new types concisely and then match against them makes it very easy to
set up fluent interfaces quickly. For example, here is a little function that calculates dates
using a simple vocabulary. Note that two new enum-style types are defined just for this one
function.

166

Using functions as building blocks

// set up the vocabulary

type DateScale = Hour | Hours | Day | Days | Week | Weeks
type DateDirection = Ago | Hence
// define a function that matches on the vocabulary
let getDate interval scale direction =
let absHours = match scale with
| Hour | Hours -> 1 * interval
| Day | Days -> 24 * interval
| Week | Weeks -> 24 * 7 * interval
let signedHours = match direction with
| Ago -> -1 * absHours
| Hence -> absHours
System.DateTime.Now.AddHours(float signedHours)
// test some examples
let example1 = getDate 5 Days Ago
let example2 = getDate 1 Hour Hence
// the C# equivalent would probably be more like this:
// getDate().Interval(5).Days().Ago()
// getDate().Interval(1).Hour().Hence()

The example above only has one "verb", using lots of types for the "nouns".
The following example demonstrates how you might build the functional equivalent of a
fluent interface with many "verbs".
Say that we are creating a drawing program with various shapes. Each shape has a color,
size, label and action to be performed when clicked, and we want a fluent interface to
configure each shape.
Here is an example of what a simple method chain for a fluent interface in C# might look
like:
FluentShape.Default
.SetColor("red")
.SetLabel("box")
.OnClick( s => Console.Write("clicked") );

Now the concept of "fluent interfaces" and "method chaining" is really only relevant for
object-oriented design. In a functional language like F#, the nearest equivalent would be the
use of the pipeline operator to chain a set of functions together.
Let's start with the underlying Shape type:

167

Using functions as building blocks

// create an underlying type

type FluentShape = {
label : string;
color : string;
onClick : FluentShape->FluentShape // a function type
}

We'll add some basic functions:

let defaultShape =
{label=""; color=""; onClick=fun shape->shape}
let click shape =
shape.onClick shape
let display shape =
printfn "My label=%s and my color=%s" shape.label shape.color
shape //return same shape

For "method chaining" to work, every function should return an object that can be used next
in the chain. So you will see that the " display " function returns the shape, rather than
nothing.
Next we create some helper functions which we expose as the "mini-language", and will be
used as building blocks by the users of the language.
let setLabel label shape =
{shape with FluentShape.label = label}
let setColor color shape =
{shape with FluentShape.color = color}
//add a click action to what is already there
let appendClickAction action shape =
{shape with FluentShape.onClick = shape.onClick >> action}

Notice that appendClickAction takes a function as a parameter and composes it with the
existing click action. As you start getting deeper into the functional approach to reuse, you
start seeing many more "higher order functions" like this, that is, functions that act on other
functions. Combining functions like this is one of the keys to understanding the functional
way of programming.
Now as a user of this "mini-language", I can compose the base helper functions into more
complex functions of my own, creating my own function library. (In C# this kind of thing might
be done using extension methods.)

168

Using functions as building blocks

// Compose two "base" functions to make a compound function.

let setRedBox = setColor "red" >> setLabel "box"
// Create another function by composing with previous function.
// It overrides the color value but leaves the label alone.
let setBlueBox = setRedBox >> setColor "blue"
// Make a special case of appendClickAction
let changeColorOnClick color = appendClickAction (setColor color)

I can then combine these functions together to create objects with the desired behavior.
//setup some test values
let redBox = defaultShape |> setRedBox
let blueBox = defaultShape |> setBlueBox
// create a shape that changes color when clicked
redBox
|> display
|> changeColorOnClick "green"
|> click
|> display // new version after the click
// create a shape that changes label and color when clicked
blueBox
|> display
|> appendClickAction (setLabel "box2" >> setColor "green")
|> click
|> display // new version after the click

In the second case, I actually pass two functions to appendClickAction , but I compose them
into one first. This kind of thing is trivial to do with a well structured functional library, but it is
quite hard to do in C# without having lambdas within lambdas.
Here is a more complex example. We will create a function " showRainbow " that, for each
color in the rainbow, sets the color and displays the shape.
let rainbow =
["red";"orange";"yellow";"green";"blue";"indigo";"violet"]
let showRainbow =
let setColorAndDisplay color = setColor color >> display
rainbow
|> List.map setColorAndDisplay
|> List.reduce (>>)
// test the showRainbow function
defaultShape |> showRainbow

169

Using functions as building blocks

Notice that the functions are getting more complex, but the amount of code is still quite
small. One reason for this is that the function parameters can often be ignored when doing
function composition, which reduces visual clutter. For example, the " showRainbow " function
does take a shape as a parameter, but it is not explicitly shown! This elision of parameters is
called "point-free" style and will be discussed further in the "thinking functionally" series

170

Pattern matching for conciseness

So far we have seen the pattern matching logic in the match..with expression, where it
seems to be just a switch/case statement. But in fact pattern matching is much more general
? it can compare expressions in a number of ways, matching on values, conditions, and
types, and then assign or extract values, all at the same time.
Pattern matching will be discussed in depth in later posts, but to start with, here is a little
taster of one way that it aids conciseness. We'll look at the way pattern matching is used for
binding values to expressions (the functional equivalent of assigning to variables).
In the following examples, we are binding to the internal members of tuples and lists directly:
//matching tuples directly
let firstPart, secondPart, _ = (1,2,3) // underscore means ignore
//matching lists directly
let elem1::elem2::rest = [1..10] // ignore the warning for now
//matching lists inside a match..with
let listMatcher aList =
match aList with
| [] -> printfn "the list is empty"
| [firstElement] -> printfn "the list has one element %A " firstElement
| [first; second] -> printfn "list is %A and %A" first second
| _ -> printfn "the list has more than two elements"
listMatcher [1;2;3;4]
listMatcher [1;2]
listMatcher [1]
listMatcher []

You can also bind values to the inside of complex structures such as records. In the
following example, we will create an " Address " type, and then a " Customer " type which
contains an address. Next, we will create a customer value, and then match various
properties against it.

171

Pattern matching for conciseness

// create some types

type Address = { Street: string; City: string; }
type Customer = { ID: int; Name: string; Address: Address}
// create a customer
let customer1 = { ID = 1; Name = "Bob";
Address = {Street="123 Main"; City="NY" } }
// extract name only
let { Name=name1 } = customer1
printfn "The customer is called %s" name1
// extract name and id
let { ID=id2; Name=name2; } = customer1
printfn "The customer called %s has id %i" name2 id2
// extract name and address
let { Name=name3; Address={Street=street3} } = customer1
printfn "The customer is called %s and lives on %s" name3 street3

In the last example, note how we could reach right into the Address substructure and pull
out the street as well as the customer name.
This ability to process a nested structure, extract only the fields you want, and assign them
to values, all in a single step, is very useful. It removes quite a bit of coding drudgery, and is
another factor in the conciseness of typical F# code.

172

Convenience

Convenience
In the next set of posts, we will explore a few more features of F# that I have grouped under
the theme of "convenience". These features do not necessarily result in more concise code,
but they do remove much of the drudgery and boilerplate code that would be needed in C#.
Useful "out-of-the-box" behavior for types. Most types that you create will
immediately have some useful behavior, such as immutability and built-in equality ?
functionality that has to be explicitly coded for in C#.
All functions are "interfaces", meaning that many of the roles that interfaces play in
object-oriented design are implicit in the way that functions work. And similarly, many
object-oriented design patterns are unnecessary or trivial within a functional paradigm.
Partial application. Complicated functions with many parameters can have some of the
parameters fixed or "baked in" and yet leave other parameters open.
Active patterns. Active patterns are a special kind of pattern where the pattern can be
matched or detected dynamically, rather than statically. They are great for simplifying
frequently used parsing and grouping behaviors.

173

Out-of-the-box behavior for types

One nice thing about F# is that most types immediately have some useful "out-of-the-box"
behavior such as immutability and built-in equality, functionality that often has to be explicitly
coded for in C#.
By "most" F# types, I mean the core "structural" types such as tuples, records, unions,
options, lists, etc. Classes and some other types have been added to help with .NET
integration, but lose some of the power of the structural types.
This built-in functionality for these core types includes:
Immutability
Pretty printing when debugging
Equality
Comparisons
Each of these is addressed below.

F# types have built-in immutability

In C# and Java, it is has become good practice to create immutable classes whenever
possible. In F#, you get this for free.
Here is an immutable type in F#:
type PersonalName = {FirstName:string; LastName:string}

And here is how the same type is typically coded in C#:

class ImmutablePersonalName
{
public ImmutablePersonalName(string firstName, string lastName)
{
this.FirstName = firstName;
this.LastName = lastName;
}
public string FirstName { get; private set; }
public string LastName { get; private set; }
}

174

Out-of-the-box behavior for types

That's 10 lines to do the same thing as 1 line of F#.

Most F# types have built-in pretty printing

In F#, you don't have to override ToString() for most types -- you get pretty printing for
free!
You have probably already seen this when running the earlier examples. Here is another
simple example:
type USAddress =
{Street:string; City:string; State:string; Zip:string}
type UKAddress =
{Street:string; Town:string; PostCode:string}
type Address = US of USAddress | UK of UKAddress
type Person =
{Name:string; Address:Address}
let alice = {
Name="Alice";
Address=US {Street="123 Main";City="LA";State="CA";Zip="91201"}}
let bob = {
Name="Bob";
Address=UK {Street="221b Baker St";Town="London";PostCode="NW1 6XE"}}
printfn "Alice is %A" alice
printfn "Bob is %A" bob

The output is:

Alice is {Name = "Alice";
Address = US {Street = "123 Main";
City = "LA";
State = "CA";
Zip = "91201";};}

Most F# types have built-in structural equality

In C#, you often have to implement the IEquatable interface so that you can test for
equality between objects. This is needed when using objects for Dictionary keys, for
example.
In F#, you get this for free with most F# types. For example, using the PersonalName type
from above, we can compare two names straight away.

175

Out-of-the-box behavior for types

type PersonalName = {FirstName:string; LastName:string}

Most F# types are automatically comparable

In C#, you often have to implement the IComparable interface so that you can sort objects.
Again, in F#, you get this for free with most F# types. For example, here is a simple
definition of a deck of cards.

type Suit = Club | Diamond | Spade | Heart

We can write a function to test the comparison logic:

let compareCard card1 card2 =
if card1 < card2
then printfn "%A is greater than %A" card2 card1
else printfn "%A is greater than %A" card1 card2

And let's see how it works:

let aceHearts = Heart, Ace
let twoHearts = Heart, Two
let aceSpades = Spade, Ace
compareCard aceHearts twoHearts
compareCard twoHearts aceSpades

Note that the Ace of Hearts is automatically greater than the Two of Hearts, because the
"Ace" rank value comes after the "Two" rank value.
But also note that the Two of Hearts is automatically greater than the Ace of Spades,
because the Suit part is compared first, and the "Heart" suit value comes after the "Spade"
value.

176

Out-of-the-box behavior for types

Here's an example of a hand of cards:

let hand = [ Club,Ace; Heart,Three; Heart,Ace;
Spade,Jack; Diamond,Two; Diamond,Ace ]
//instant sorting!
List.sort hand |> printfn "sorted hand is (low to high) %A"

And as a side benefit, you get min and max for free too!
List.max hand |> printfn "high card is %A"
List.min hand |> printfn "low card is %A"

177

Functions as interfaces

Functions as interfaces
An important aspect of functional programming is that, in a sense, all functions are
"interfaces", meaning that many of the roles that interfaces play in object-oriented design are
implicit in the way that functions work.
In fact, one of the critical design maxims, "program to an interface, not an implementation",
is something you get for free in F#.
To see how this works, let's compare the same design pattern in C# and F#. For example, in
C# we might want to use the "decorator pattern" to enhance some core code.
Let's say that we have a calculator interface:
interface ICalculator
{
int Calculate(int input);
}

And then a specific implementation:

class AddingCalculator: ICalculator
{
public int Calculate(int input) { return input + 1; }
}

And then if we want to add logging, we can wrap the core calculator implementation inside a
logging wrapper.

178

Functions as interfaces

class LoggingCalculator: ICalculator

{
ICalculator _innerCalculator;
LoggingCalculator(ICalculator innerCalculator)
{
_innerCalculator = innerCalculator;
}
public int Calculate(int input)
{
Console.WriteLine("input is {0}", input);
var result = _innerCalculator.Calculate(input);
Console.WriteLine("result is {0}", result);
return result;
}
}

So far, so straightforward. But note that, for this to work, we must have defined an interface
for the classes. If there had been no ICalculator interface, it would be necessary to retrofit
the existing code.
And here is where F# shines. In F#, you can do the same thing without having to define the
interface first. Any function can be transparently swapped for any other function as long as
the signatures are the same.
Here is the equivalent F# code.
let addingCalculator input = input + 1
let loggingCalculator innerCalculator input =
printfn "input is %A" input
let result = innerCalculator input
printfn "result is %A" result
result

In other words, the signature of the function is the interface.

Generic wrappers
Even nicer is that by default, the F# logging code can be made completely generic so that it
will work for any function at all. Here are some examples:

179

Functions as interfaces

let add1 input = input + 1

let times2 input = input * 2
let genericLogger anyFunc input =
printfn "input is %A" input //log the input
let result = anyFunc input //evaluate the function
printfn "result is %A" result //log the result
result //return the result
let add1WithLogging = genericLogger add1
let times2WithLogging = genericLogger times2

The new "wrapped" functions can be used anywhere the original functions could be used ?
no one can tell the difference!
// test
add1WithLogging 3
times2WithLogging 3
[1..5] |> List.map add1WithLogging

Exactly the same generic wrapper approach can be used for other things. For example, here
is a generic wrapper for timing a function.
let genericTimer anyFunc input =
let stopwatch = System.Diagnostics.Stopwatch()
stopwatch.Start()
let result = anyFunc input //evaluate the function
printfn "elapsed ms is %A" stopwatch.ElapsedMilliseconds
result
let add1WithTimer = genericTimer add1WithLogging
// test
add1WithTimer 3

The ability to do this kind of generic wrapping is one of the great conveniences of the
function-oriented approach. You can take any function and create a similar function based
on it. As long as the new function has exactly the same inputs and outputs as the original
function, the new can be substituted for the original anywhere. Some more examples:
It is easy to write a generic caching wrapper for a slow function, so that the value is only
calculated once.
It is also easy to write a generic "lazy" wrapper for a function, so that the inner function
is only called when a result is needed

180

Functions as interfaces

The strategy pattern

We can apply this same approach to another common design pattern, the "strategy pattern."
Let's use the familiar example of inheritance: an Animal superclass with Cat and Dog
subclasses, each of which overrides a MakeNoise() method to make different noises.
In a true functional design, there are no subclasses, but instead the Animal class would
have a NoiseMaking function that would be passed in with the constructor. This approach is
exactly the same as the "strategy" pattern in OO design.
type Animal(noiseMakingStrategy) =
member this.MakeNoise =
noiseMakingStrategy() |> printfn "Making noise %s"
// now create a cat
let meowing() = "Meow"
let cat = Animal(meowing)
cat.MakeNoise
// .. and a dog
let woofOrBark() = if (System.DateTime.Now.Second % 2 = 0)
then "Woof" else "Bark"
let dog = Animal(woofOrBark)
dog.MakeNoise
dog.MakeNoise //try again a second later

Note that again, we do not have to define any kind of INoiseMakingStrategy interface first.
Any function with the right signature will work. As a consequence, in the functional model,
the standard .NET "strategy" interfaces such as IComparer , IFormatProvider , and
IServiceProvider become irrelevant.

Many other design patterns can be simplified in the same way.

181

Partial Application

Partial Application
A particularly convenient feature of F# is that complicated functions with many parameters
can have some of the parameters fixed or "baked in" and yet leave other parameters open.
In this post, we'll take a quick look at how this might be used in practice.
Let's start with a very simple example of how this works. We'll start with a trivial function:
// define a adding function
let add x y = x + y
// normal use
let z = add 1 2

But we can do something strange as well ? we can call the function with only one parameter!
let add42 = add 42

The result is a new function that has the "42" baked in, and now takes only one parameter
instead of two! This technique is called "partial application", and it means that, for any
function, you can "fix" some of the parameters and leave other ones open to be filled in later.
// use the new function
add42 2
add42 3

With that under our belt, let's revisit the generic logger that we saw earlier:
let genericLogger anyFunc input =
printfn "input is %A" input //log the input
let result = anyFunc input //evaluate the function
printfn "result is %A" result //log the result
result //return the result

Unfortunately, I have hard-coded the logging operations. Ideally, I'd like to make this more
generic so that I can choose how logging is done.
Of course, F# being a functional programming language, we will do this by passing functions
around.

182

Partial Application

In this case we would pass "before" and "after" callback functions to the library function, like
this:
let genericLogger before after anyFunc input =
before input //callback for custom behavior
let result = anyFunc input //evaluate the function
after result //callback for custom behavior
result //return the result

You can see that the logging function now has four parameters. The "before" and "after"
actions are passed in as explicit parameters as well as the function and its input. To use this
in practice, we just define the functions and pass them in to the library function along with
the final int parameter:
let add1 input = input + 1
// reuse case 1
genericLogger
(fun x -> printf "before=%i. " x) // function to call before
(fun x -> printfn " after=%i." x) // function to call after
add1 // main function
2 // parameter
// reuse case 2
genericLogger
(fun x -> printf "started with=%i " x) // different callback
(fun x -> printfn " ended with=%i" x)
add1 // main function
2 // parameter

This is a lot more flexible. I don't have to create a new function every time I want to change
the behavior -- I can define the behavior on the fly.
But you might be thinking that this is a bit ugly. A library function might expose a number of
callback functions and it would be inconvenient to have to pass the same functions in over
and over.
Luckily, we know the solution for this. We can use partial application to fix some of the
parameters. So in this case, let's define a new function which fixes the before and after
functions, as well as the add1 function, but leaves the final parameter open.

183

Partial Application

// define a reusable function with the "callback" functions fixed

let add1WithConsoleLogging =
genericLogger
(fun x -> printf "input=%i. " x)
(fun x -> printfn " result=%i" x)
add1
// last parameter NOT defined here yet!

The new "wrapper" function is called with just an int now, so the code is much cleaner. As in
the earlier example, it can be used anywhere the original add1 function could be used
without any changes.
add1WithConsoleLogging 2
add1WithConsoleLogging 3
add1WithConsoleLogging 4
[1..5] |> List.map add1WithConsoleLogging

The functional approach in C#

In a classical object-oriented approach, we would probably have used inheritance to do this
kind of thing. For instance, we might have had an abstract LoggerBase class, with virtual
methods for " before " and " after " and the function to execute. And then to implement a
particular kind of behavior, we would have created a new subclass and overridden the virtual
methods as needed.
But classical style inheritance is now becoming frowned upon in object-oriented design, and
composition of objects is much preferred. And indeed, in "modern" C#, we would probably
write the code in the same way as F#, either by using events or by passing functions in.
Here's the F# code translated into C# (note that I had to specify the types for each Action)

184

Partial Application

public class GenericLoggerHelper<TInput, TResult>

{
public TResult GenericLogger(
Action<TInput> before,
Action<TResult> after,
Func<TInput, TResult> aFunc,
TInput input)
{
before(input); //callback for custom behavior
var result = aFunc(input); //do the function
after(result); //callback for custom behavior
return result;
}
}

And here it is in use:

[NUnit.Framework.Test]
public void TestGenericLogger()
{
var sut = new GenericLoggerHelper<int, int>();
sut.GenericLogger(
x => Console.Write("input={0}. ", x),
x => Console.WriteLine(" result={0}", x),
x => x + 1,
3);
}

In C#, this style of programming is required when using the LINQ libraries, but many
developers have not embraced it fully to make their own code more generic and adaptable.
And it's not helped by the ugly Action<> and Func<> type declarations that are required.
But it can certainly make the code much more reusable.

185

Active patterns

Active patterns
F# has a special type of pattern matching called "active patterns" where the pattern can be
parsed or detected dynamically. As with normal patterns, the matching and output are
combined into a single step from the caller's point of view.
Here is an example of using active patterns to parse a string into an int or bool.
// create an active pattern
let (|Int|_|) str =
match System.Int32.TryParse(str) with
| (true,int) -> Some(int)
| _ -> None
// create an active pattern
let (|Bool|_|) str =
match System.Boolean.TryParse(str) with
| (true,bool) -> Some(bool)
| _ -> None

You don't need to worry about the complex syntax used to define the active pattern right
now ? this is just an example so that you can see how they are used.

Once these patterns have been set up, they can be used as part of a normal " match..with "
expression.
// create a function to call the patterns
let testParse str =
match str with
| Int i -> printfn "The value is an int '%i'" i
| Bool b -> printfn "The value is a bool '%b'" b
| _ -> printfn "The value '%s' is something else" str
// test
testParse "12"
testParse "true"
testParse "abc"

You can see that from the caller's point of view, the matching with an Int or Bool is
transparent, even though there is parsing going on behind the scenes.

186

Active patterns

A similar example is to use active patterns with regular expressions in order to both match
on a regex pattern and return the matched value in a single step.
// create an active pattern
open System.Text.RegularExpressions
let (|FirstRegexGroup|_|) pattern input =
let m = Regex.Match(input,pattern)
if (m.Success) then Some m.Groups.[1].Value else None

Again, once this pattern has been set up, it can be used transparently as part of a normal
match expression.
// create a function to call the pattern
let testRegex str =
match str with
| FirstRegexGroup "http://(.*?)/(.*)" host ->
printfn "The value is a url and the host is %s" host
| FirstRegexGroup ".*?@(.*)" host ->
printfn "The value is an email and the host is %s" host
| _ -> printfn "The value '%s' is something else" str
// test
testRegex "http://google.com/test"
testRegex "alice@hotmail.com"

187

Correctness

Correctness
As a programmer, you are constantly judging the code that you and others write. In an ideal
world, you should be able to look at a piece of code and easily understand exactly what it
does; and of course, being concise, clear and readable is a major factor in this.
But more importantly, you have to be able to convince yourself that the code does what it is
supposed to do. As you program, you are constantly reasoning about code correctness, and
the little compiler in your brain is checking the code for errors and possible mistakes.
So how can a programming language help you with this?
A modern imperative language like C# provides many ways that you are already familiar
with: type checking, scoping and naming rules, access modifiers and so on. And, in recent
versions, static code analysis and code contracts.
All these techniques mean that the compiler can take on a lot of the burden of checking for
correctness. If you make a mistake, the compiler will warn you.
But F# has some additional features that can have a huge impact on ensuring correctness.
The next few posts will be devoted to four of them:
Immutability, which enables code to behave much more predictably.
Exhaustive pattern matching, which traps many common errors at compile time.
A strict type system, which is your friend, not your enemy. You can use the static type
checking almost as an instant "compile time unit test".
An expressive type system that can help you "make illegal states unrepresentable"* .
We'll see how to design a real-world example that demonstrates this.
* Thanks to Yaron Minsky at Jane Street for this phrase.

188

Immutability

Immutability
To see why immutability is important, let's start with a small example.
Here's some simple C# code that processes a list of numbers.
public List<int> MakeList()
{
return new List<int> {1,2,3,4,5,6,7,8,9,10};
}
public List<int> OddNumbers(List<int> list)
{
// some code
}
public List<int> EvenNumbers(List<int> list)
{
// some code
}

Now let me test it:

public void Test()
{
var odds = OddNumbers(MakeList());
var evens = EvenNumbers(MakeList());
// assert odds = 1,3,5,7,9 -- OK!
// assert evens = 2,4,6,8,10 -- OK!
}

Everything works great, and the test passes, but I notice that I am creating the list twice ?
surely I should refactor this out? So I do the refactoring, and here's the new improved
version:
public void RefactoredTest()
{
var list = MakeList();
var odds = OddNumbers(list);
var evens = EvenNumbers(list);
// assert odds = 1,3,5,7,9 -- OK!
// assert evens = 2,4,6,8,10 -- FAIL!
}

189

Immutability

But now the test suddenly fails! Why would a refactoring break the test? Can you tell just by
looking at the code?
The answer is, of course, that the list is mutable, and it is probable that the OddNumbers
function is making destructive changes to the list as part of its filtering logic. Of course, in
order to be sure, we would have to examine the code inside the OddNumbers function.
In other words, when I call the OddNumbers function, I am unintentionally creating
undesirable side effects.
Is there a way to ensure that this cannot happen? Yes -- if the functions had used
IEnumerable instead:

public IEnumerable<int> MakeList() {}

public List<int> OddNumbers(IEnumerable<int> list) {}
public List<int> EvenNumbers(IEnumerable <int> list) {}

In this case we can be confident that calling the OddNumbers function could not possibly
have any effect on the list, and EvenNumbers would work correctly. What's more, we can
know this just by looking at the signatures, without having to examine the internals of the
functions. And if you try to make one of the functions misbehave by assigning to the list then
you will get an error straight away, at compile time.
So IEnumerable can help in this case, but what if I had used a type such as
IEnumerable<Person> instead of IEnumerable<int> ? Could I still be as confident that the

functions wouldn't have unintentional side effects?

Reasons why immutability is important

The example above shows why immutability is helpful. In fact, this is just the tip of the
iceberg. There are a number of reasons why immutability is important:
Immutable data makes the code predictable
Immutable data is easier to work with
Immutable data forces you to use a "transformational" approach
First, immutability makes the code predictable. If data is immutable, there can be no sideeffects. If there are no side-effects, it is much, much, easier to reason about the correctness
of the code.
And when you have two functions that work on immutable data, you don't have to worry
about which order to call them in, or whether one function will mess with the input of the
other function. And you have peace of mind when passing data around (for example, you

190

Immutability

don't have to worry about using an object as a key in a hashtable and having its hash code
change).
In fact, immutability is a good idea for the same reasons that global variables are a bad idea:
data should be kept as local as possible and side-effects should be avoided.
Second, immutability is easier to work with. If data is immutable, many common tasks
become much easier. Code is easier to write and easier to maintain. Fewer unit tests are
needed (you only have to check that a function works in isolation), and mocking is much
easier. Concurrency is much simpler, as you don't have to worry about using locks to avoid
update conflicts (because there are no updates).
Finally, using immutability by default means that you start thinking differently about
programming. You tend to think about transforming the data rather than mutating it in
place.
SQL queries and LINQ queries are good examples of this "transformational" approach. In
both cases, you always transform the original data through various functions (selects, filters,
sorts) rather than modifying the original data.
When a program is designed using a transformation approach, the result tends to be more
elegant, more modular, and more scalable. And as it happens, the transformation approach
is also a perfect fit with a function-oriented paradigm.

How F# does immutability

We saw earlier that immutable values and types are the default in F#:
// immutable list
let list = [1;2;3;4]
type PersonalName = {FirstName:string; LastName:string}
// immutable person
let john = {FirstName="John"; LastName="Doe"}

Because of this, F# has a number of tricks to make life easier and to optimize the underlying
code.
First, since you can't modify a data structure, you must copy it when you want to change it.
F# makes it easy to copy another data structure with only the changes you want:
let alice = {john with FirstName="Alice"}

191

Immutability

And complex data structures are implemented as linked lists or similar, so that common
parts of the structure are shared.
// create an immutable list
let list1 = [1;2;3;4]
// prepend to make a new list
let list2 = 0::list1
// get the last 4 of the second list
let list3 = list2.Tail
// the two lists are the identical object in memory!
System.Object.ReferenceEquals(list1,list3)

This technique ensures that, while you might appear to have hundreds of copies of a list in
your code, they are all sharing the same memory behind the scenes.

Mutable data
F# is not dogmatic about immutability; it does support mutable data with the mutable
keyword. But turning on mutability is an explicit decision, a deviation from the default, and it
is generally only needed for special cases such as optimization, caching, etc, or when
dealing with the .NET libraries.
In practice, a serious application is bound to have some mutable state if it deals with messy
world of user interfaces, databases, networks and so on. But F# encourages the
minimization of such mutable state. You can generally still design your core business logic to
use immutable data, with all the corresponding benefits.

192

Exhaustive pattern matching

We briefly noted earlier that when pattern matching there is a requirement to match all
possible cases. This turns out be a very powerful technique to ensure correctness.
Let's compare some C# to F# again. Here's some C# code that uses a switch statement to
handle different types of state.
enum State { New, Draft, Published, Inactive, Discontinued }
void HandleState(State state)
{
switch (state)
{
case State.Inactive: // code for Inactive
break;
case State.Draft: // code for Draft
break;
case State.New: // code for New
break;
case State.Discontinued: // code for Discontinued
break;
}
}

This code will compile, but there is an obvious bug! The compiler couldn't see it ? can you?
If you can, and you fixed it, would it stay fixed if I added another State to the list?
Here's the F# equivalent:
type State = New | Draft | Published | Inactive | Discontinued
let handleState state =
match state with
| Inactive -> () // code for Inactive
| Draft -> () // code for Draft
| New -> () // code for New
| Discontinued -> () // code for Discontinued

Now try running this code. What does the compiler tell you?
The fact that exhaustive matching is always done means that certain common errors will be
detected by the compiler immediately:
A missing case (often caused when a new choice has been added due to changed
requirements or refactoring).
An impossible case (when an existing choice has been removed).
193

Exhaustive pattern matching

A redundant case that could never be reached (the case has been subsumed in a
previous case -- this can sometimes be non-obvious).
Now let's look at some real examples of how exhaustive matching can help you write correct
code.

Avoiding nulls with the Option type

We'll start with an extremely common scenario where the caller should always check for an
invalid case, namely testing for nulls. A typical C# program is littered with code like this:
if (myObject != null)
{
// do something
}

Unfortunately, this test is not required by the compiler. All it takes is for one piece of code to
forget to do this, and the program can crash. Over the years, a huge amount of
programming effort has been devoted to handling nulls ? the invention of nulls has even
been called a billion dollar mistake!
In pure F#, nulls cannot exist accidentally. A string or object must always be assigned to
something at creation, and is immutable thereafter.
However, there are many situations where the design intent is to distinguish between valid
and invalid values, and you require the caller to handle both cases.
In C#, this can be managed in certain situations by using nullable value types (such as
Nullable<int> ) to make the design decision clear. When a nullable is encountered the

compiler will force you to be aware of it. You can then test the validity of the value before
using it. But nullables do not work for standard classes (i.e. reference types), and it is easy
to accidentally bypass the tests too and just call Value directly.
In F# there is a similar but more powerful concept to convey the design intent: the generic
wrapper type called Option , with two choices: Some or None . The Some choice wraps a
valid value, and None represents a missing value.
Here's an example where Some is returned if a file exists, but a missing file returns None .

194

Exhaustive pattern matching

let getFileInfo filePath =

let fi = new System.IO.FileInfo(filePath)
if fi.Exists then Some(fi) else None
let goodFileName = "good.txt"
let badFileName = "bad.txt"
let goodFileInfo = getFileInfo goodFileName // Some(fileinfo)
let badFileInfo = getFileInfo badFileName // None

If we want to do anything with these values, we must always handle both possible cases.
match goodFileInfo with
| Some fileInfo ->
printfn "the file %s exists" fileInfo.FullName
| None ->
printfn "the file doesn't exist"
match badFileInfo with
| Some fileInfo ->
printfn "the file %s exists" fileInfo.FullName
| None ->
printfn "the file doesn't exist"

We have no choice about this. Not handling a case is a compile-time error, not a run-time
error. By avoiding nulls and by using Option types in this way, F# completely eliminates a
large class of null reference exceptions.
Caveat: F# does allow you to access the value without testing, just like C#, but that is
considered extremely bad practice.

Exhaustive pattern matching for edge cases

Here's some C# code that creates a list by averaging pairs of numbers from an input list:
public IList<float> MovingAverages(IList<int> list)
{
var averages = new List<float>();
for (int i = 0; i < list.Count; i++)
{
var avg = (list[i] + list[i+1]) / 2;
averages.Add(avg);
}
return averages;
}

195

Exhaustive pattern matching

It compiles correctly, but it actually has a couple of issues. Can you find them quickly? If
you're lucky, your unit tests will find them for you, assuming you have thought of all the edge
cases.
Now let's try the same thing in F#:
let rec movingAverages list =
match list with
// if input is empty, return an empty list
| [] -> []
// otherwise process pairs of items from the input
| x::y::rest ->
let avg = (x+y)/2.0
//build the result by recursing the rest of the list
avg :: movingAverages (y::rest)

This code also has a bug. But unlike C#, this code will not even compile until I fix it. The
compiler will tell me that I haven't handled the case when I have a single item in my list. Not
only has it found a bug, it has revealed a gap in the requirements: what should happen when
there is only one item?
Here's the fixed up version:
let rec movingAverages list =
match list with
// if input is empty, return an empty list
| [] -> []
// otherwise process pairs of items from the input
| x::y::rest ->
let avg = (x+y)/2.0
//build the result by recursing the rest of the list
avg :: movingAverages (y::rest)
// for one item, return an empty list
| [_] -> []
// test
movingAverages [1.0]
movingAverages [1.0; 2.0]
movingAverages [1.0; 2.0; 3.0]

As an additional benefit, the F# code is also much more self-documenting. It explicitly

describes the consequences of each case. In the C# code, it is not at all obvious what
happens if a list is empty or only has one item. You would have to read the code carefully to
find out.

196

Exhaustive pattern matching

Exhaustive pattern matching as an error

handling technique
The fact that all choices must be matched can also be used as a useful alternative to
throwing exceptions. For example consider the following common scenario:
There is a utility function in the lowest tier of your app that opens a file and performs an
arbitrary operation on it (that you pass in as a callback function)
The result is then passed back up through to tiers to the top level.
A client calls the top level code, and the result is processed and any error handling
done.
In a procedural or OO language, propagating and handling exceptions across layers of code
is a common problem. Top level functions are not easily able to tell the difference between
an exception that they should recover from ( FileNotFound say) vs. an exception that they
needn't handle ( OutOfMemory say). In Java, there has been an attempt to do this with
checked exceptions, but with mixed results.
In the functional world, a common technique is to create a new structure to hold both the
good and bad possibilities, rather than throwing an exception if the file is missing.
// define a "union" of two different alternatives
type Result<'a, 'b> =
| Success of 'a // 'a means generic type. The actual type
// will be determined when it is used.
| Failure of 'b // generic failure type as well
// define all possible errors
type FileErrorReason =
| FileNotFound of string
| UnauthorizedAccess of string * System.Exception
// define a low level function in the bottom layer
let performActionOnFile action filePath =
try
//open file, do the action and return the result
use sr = new System.IO.StreamReader(filePath:string)
let result = action sr //do the action to the reader
sr.Close()
Success (result) // return a Success
with // catch some exceptions and convert them to errors
| :? System.IO.FileNotFoundException as ex
-> Failure (FileNotFound filePath)
| :? System.Security.SecurityException as ex
-> Failure (UnauthorizedAccess (filePath,ex))
// other exceptions are unhandled

197

Exhaustive pattern matching

The code demonstrates how performActionOnFile returns a Result object which has two
alternatives: Success and Failure . The Failure alternative in turn has two alternatives
as well: FileNotFound and UnauthorizedAccess .
Now the intermediate layers can call each other, passing around the result type without
worrying what its structure is, as long as they don't access it:
// a function in the middle layer
let middleLayerDo action filePath =
let fileResult = performActionOnFile action filePath
// do some stuff
fileResult //return
// a function in the top layer
let topLayerDo action filePath =
let fileResult = middleLayerDo action filePath
// do some stuff
fileResult //return

Because of type inference, the middle and top layers do not need to specify the exact types
returned. If the lower layer changes the type definition at all, the intermediate layers will not
be affected.
Obviously at some point, a client of the top layer does want to access the result. And here is
where the requirement to match all patterns is enforced. The client must handle the case
with a Failure or else the compiler will complain. And furthermore, when handling the
Failure branch, it must handle the possible reasons as well. In other words, special case

handling of this sort can be enforced at compile time, not at runtime! And in addition the
possible reasons are explicitly documented by examining the reason type.
Here is an example of a client function that accesses the top layer:
/// get the first line of the file
let printFirstLineOfFile filePath =
let fileResult = topLayerDo (fun fs->fs.ReadLine()) filePath
match fileResult with
| Success result ->
// note type-safe string printing with %s
printfn "first line is: '%s'" result
| Failure reason ->
match reason with // must match EVERY reason
| FileNotFound file ->
printfn "File not found: %s" file
| UnauthorizedAccess (file,_) ->
printfn "You do not have access to the file: %s" file

198

Exhaustive pattern matching

You can see that this code must explicitly handle the Success and Failure cases, and
then for the failure case, it explicitly handles the different reasons. If you want to see what
happens if it does not handle one of the cases, try commenting out the line that handles
UnauthorizedAccess and see what the compiler says.

Now it is not required that you always handle all possible cases explicitly. In the example
below, the function uses the underscore wildcard to treat all the failure reasons as one. This
can considered bad practice if we want to get the benefits of the strictness, but at least it is
clearly done.
/// get the length of the text in the file
let printLengthOfFile filePath =
let fileResult =
topLayerDo (fun fs->fs.ReadToEnd().Length) filePath
match fileResult with
| Success result ->
// note type-safe int printing with %i
printfn "length is: %i" result
| Failure _ ->
printfn "An error happened but I don't want to be specific"

Now let's see all this code work in practice with some interactive tests.
First set up a good file and a bad file.
/// write some text to a file
let writeSomeText filePath someText =
use writer = new System.IO.StreamWriter(filePath:string)
writer.WriteLine(someText:string)
writer.Close()
let goodFileName = "good.txt"
let badFileName = "bad.txt"
writeSomeText goodFileName "hello"

And now test interactively:

printFirstLineOfFile goodFileName
printLengthOfFile goodFileName
printFirstLineOfFile badFileName
printLengthOfFile badFileName

I think you can see that this approach is very attractive:

199

Exhaustive pattern matching

Functions return error types for each expected case (such as FileNotFound ), but the
handling of these types does not need to make the calling code ugly.
Functions continue to throw exceptions for unexpected cases (such as OutOfMemory ),
which will generally be caught and logged at the top level of the program.
This technique is simple and convenient. Similar (and more generic) approaches are
standard in functional programming.
It is feasible to use this approach in C# too, but it is normally impractical, due to the lack of
union types and the lack of type inference (we would have to specify generic types
everywhere).

Exhaustive pattern matching as a change

management tool
Finally, exhaustive pattern matching is a valuable tool for ensuring that code stays correct as
requirements change, or during refactoring.
Let's say that the requirements change and we need to handle a third type of error:
"Indeterminate". To implement this new requirement, change the first Result type as
follows, and re-evaluate all the code. What happens?
type Result<'a, 'b> =
| Success of 'a
| Failure of 'b
| Indeterminate

Or sometimes a requirements change will remove a possible choice. To emulate this,

change the first Result type to eliminate all but one of the choices.
type Result<'a> =
| Success of 'a

Now re-evaluate the rest of the code. What happens now?

This is very powerful! When we adjust the choices, we immediately know all the places
which need to be fixed to handle the change. This is another example of the power of
statically checked type errors. It is often said about functional languages like F# that "if it
compiles, it must be correct".

200

Exhaustive pattern matching

201

Using the type system to ensure correct code

Using the type system to ensure correct

code
You are familiar with static type checking through languages such as C# and Java. In these
languages, the type checking is straightforward but rather crude, and can be seen as an
annoyance compared with the freedom of dynamic languages such as Python and Ruby.
But in F# the type system is your friend, not your enemy. You can use static type checking
almost as an instant unit test ? making sure that your code is correct at compile time.
In the earlier posts we have already seen some of the things that you can do with the type
system in F#:
The types and their associated functions provide an abstraction to model the problem
domain. Because creating types is so easy, there is rarely an excuse to avoid designing
them as needed for a given problem, and unlike C# classes it is hard to create "kitchensink" types that do everything.
Well defined types aid in maintenance. Since F# uses type inference, you can normally
rename or restructure types easily without using a refactoring tool. And if the type is
changed in an incompatible way, this will almost certainly create compile-time errors
that aid in tracking down any problems.
Well named types provide instant documentation about their roles in the program (and
this documentation can never be out of date).
In this post and the next we will focus on using the type system as an aid to writing correct
code. I will demonstrate that you can create designs such that, if your code actually
compiles, it will almost certainly work as designed.

Using standard type checking

In C#, you use the compile-time checks to validate your code without even thinking about it.
For example, would you give up List<string> for a plain List ? Or give up Nullable<int>
and be forced to used object with casting? Probably not.
But what if you could have even more fine-grained types? You could have even better
compile-time checks. And this is exactly what F# offers.
The F# type checker is not that much stricter than the C# type checker. But because it is so
easy to create new types without clutter, you can represent the domain better, and, as a
useful side-effect, avoid many common errors.

202

Using the type system to ensure correct code

Here is a simple example:

//define a "safe" email address type
type EmailAddress = EmailAddress of string
//define a function that uses it
let sendEmail (EmailAddress email) =
printfn "sent an email to %s" email
//try to send one
let aliceEmail = EmailAddress "alice@example.com"
sendEmail aliceEmail
//try to send a plain string
sendEmail "bob@example.com" //error

By wrapping the email address in a special type, we ensure that normal strings cannot be
used as arguments to email specific functions. (In practice, we would also hide the
constructor of the EmailAddress type as well, to ensure that only valid values could be
created in the first place.)
There is nothing here that couldn't be done in C#, but it would be quite a lot of work to create
a new value type just for this one purpose, so in C#, it is easy to be lazy and just pass
strings around.

Additional type safety features in F#

Before moving on to the major topic of "designing for correctness", let's see a few of the
other minor, but cool, ways that F# is type-safe.

Type-safe formatting with printf

Here is a minor feature that demonstrates one of the ways that F# is more type-safe than
C#, and how the F# compiler can catch errors that would only be detected at runtime in C#.
Try evaluating the following and look at the errors generated:

203

Using the type system to ensure correct code

let printingExample =
printf "an int %i" 2 // ok
printf "an int %i" 2.0 // wrong type
printf "an int %i" "hello" // wrong type
printf "an int %i" // missing param
printf "a string %s" "hello" // ok
printf "a string %s" 2 // wrong type
printf "a string %s" // missing param
printf "a string %s" "he" "lo" // too many params
printf "an int %i and string %s" 2 "hello" // ok
printf "an int %i and string %s" "hello" 2 // wrong type
printf "an int %i and string %s" 2 // missing param

Unlike C#, the compiler analyses the format string and determines what the number and
types of the arguments are supposed to be.
This can be used to constrain the types of parameters without explicitly having to specify
them. So for example, in the code below, the compiler can deduce the types of the
arguments automatically.
let printAString x = printf "%s" x
let printAnInt x = printf "%i" x
// the result is:
// val printAString : string -> unit //takes a string parameter
// val printAnInt : int -> unit //takes an int parameter

Units of measure
F# has the ability to define units of measure and associate them with floats. The unit of
measure is then "attached" to the float as a type and prevents mixing different types. This is
another feature that can be very handy if you need it.

204

Using the type system to ensure correct code

// define some measures

[<Measure>]
type cm
[<Measure>]
type inches
[<Measure>]
type feet =
// add a conversion function
static member toInches(feet : float<feet>) : float<inches> =
feet * 12.0<inches/feet>
// define some values
let meter = 100.0<cm>
let yard = 3.0<feet>
//convert to different measure
let yardInInches = feet.toInches(yard)
// can't mix and match!
yard + meter
// now define some currencies
[<Measure>]
type GBP
[<Measure>]
type USD
let gbp10 = 10.0<GBP>
let usd10 = 10.0<USD>
gbp10 + gbp10 // allowed: same currency
gbp10 + usd10 // not allowed: different currency
gbp10 + 1.0 // not allowed: didn't specify a currency
gbp10 + 1.0<_> // allowed using wildcard

Type-safe equality
One final example. In C# any class can be equated with any other class (using reference
equality by default). In general, this is a bad idea! For example, you shouldn't really be able
to compare a string with a person at all.
Here is some C# code which is perfectly valid and compiles fine:
using System;
var obj = new Object();
var ex = new Exception();
var b = (obj == ex);

205

Using the type system to ensure correct code

If we write the identical code in F#, we get a compile-time error:

open System
let obj = new Object()
let ex = new Exception()
let b = (obj = ex)

Chances are, if you are testing equality between two different types, you are doing
something wrong.
In F#, you can even stop a type being compared at all! This is not as silly as it seems. For
some types, there may not be a useful default, or you may want to force equality to be based
on a specific field rather than the object as whole.
Here is an example of this:
// deny comparison
[<NoEquality; NoComparison>]
type CustomerAccount = {CustomerAccountId: int}
let x = {CustomerAccountId = 1}
x = x // error!
x.CustomerAccountId = x.CustomerAccountId // no error

206

Worked example: Designing for correctness

Worked example: Designing for

correctness
In this post, we'll see how you can design for correctness (or at least, for the requirements
as you currently understand them), by which I mean that a client of a well designed model
will not be able to put the system into an illegal state ? a state that doesn't meet the
requirements. You literally cannot create incorrect code because the compiler will not let
you.
For this to work, we do have to spend some time up front thinking about design and making
an effort to encode the requirements into the types that you use. If you just use strings or
lists for all your data structures, you will not get any benefit from the type checking.
We'll use a simple example. Let's say that you are designing an e-commerce site which has
a shopping cart and you are given the following requirements.
You can only pay for a cart once.
Once a cart is paid for, you cannot change the items in it.
Empty carts cannot be paid for.

A bad design in C#
In C#, we might think that this is simple enough and dive straight into coding. Here is a
straightforward implementation in C# that seems OK at first glance.

207

Worked example: Designing for correctness

public class NaiveShoppingCart<TItem>

{
private List<TItem> items;
private decimal paidAmount;
public NaiveShoppingCart()
{
this.items = new List<TItem>();
this.paidAmount = 0;
}
/// Is cart paid for?
public bool IsPaidFor { get { return this.paidAmount > 0; } }
/// Readonly list of items
public IEnumerable<TItem> Items { get {return this.items; } }
/// add item only if not paid for
public void AddItem(TItem item)
{
if (!this.IsPaidFor)
{
this.items.Add(item);
}
}
/// remove item only if not paid for
public void RemoveItem(TItem item)
{
if (!this.IsPaidFor)
{
this.items.Remove(item);
}
}
/// pay for the cart
public void Pay(decimal amount)
{
if (!this.IsPaidFor)
{
this.paidAmount = amount;
}
}
}

Unfortunately, it's actually a pretty bad design:

One of the requirements is not even met. Can you see which one?
It has a major design flaw, and a number of minor ones. Can you see what they are?
So many problems in such a short piece of code!

208

Worked example: Designing for correctness

What would happen if we had even more complicated requirements and the code was
thousands of lines long? For example, the fragment that is repeated everywhere:
if (!this.IsPaidFor) { do something }

looks like it will be quite brittle if requirements change in some methods but not others.
Before you read the next section, think for a minute how you might better implement the
requirements above in C#, with these additional requirements:
If you try to do something that is not allowed in the requirements, you will get a compile
time error, not a run time error. For example, you must create a design such that you
cannot even call the RemoveItem method from an empty cart.
The contents of the cart in any state should be immutable. The benefit of this is that if I
am in the middle of paying for a cart, the cart contents can't change even if some other
process is adding or removing items at the same time.

A correct design in F#
Let's step back and see if we can come up with a better design. Looking at these
requirements, it's obvious that we have a simple state machine with three states and some
state transitions:
A Shopping Cart can be Empty, Active or PaidFor
When you add an item to an Empty cart, it becomes Active
When you remove the last item from an Active cart, it becomes Empty
When you pay for an Active cart, it becomes PaidFor
And now we can add the business rules to this model:
You can add an item only to carts that are Empty or Active
You can remove an item only from carts that are Active
You can only pay for carts that are Active
Here is the state diagram:

209

Worked example: Designing for correctness

It's worth noting that these kinds of state-oriented models are very common in business
systems. Product development, customer relationship management, order processing, and
other workflows can often be modeled this way.
Now we have the design, we can reproduce it in F#:
type CartItem = string // placeholder for a more complicated type
type EmptyState = NoItems // don't use empty list! We want to
// force clients to handle this as a
// separate case. E.g. "you have no
// items in your cart"
type ActiveState = { UnpaidItems : CartItem list; }
type PaidForState = { PaidItems : CartItem list;
Payment : decimal}
type Cart =
| Empty of EmptyState
| Active of ActiveState
| PaidFor of PaidForState

We create a type for each state, and Cart type that is a choice of any one of the states. I
have given everything a distinct name (e.g. PaidItems and UnpaidItems rather than just
Items ) because this helps the inference engine and makes the code more self

documenting.
This is a much longer example than the earlier ones! Don't worry too much about the F#
syntax right now, but I hope that you can get the gist of the code, and see how it fits into
the overall design.
Also, do paste the snippets into a script file and evaluate them for yourself as they come
up.

Next we can create the operations for each state. The main thing to note is each operation
will always take one of the States as input and return a new Cart. That is, you start off with a
particular known state, but you return a Cart which is a wrapper for a choice of three
possible states.

210

Worked example: Designing for correctness

// =============================
// operations on empty state
// =============================
let addToEmptyState item =
// returns a new Active Cart
Cart.Active {UnpaidItems=[item]}
// =============================
// operations on active state
// =============================
let addToActiveState state itemToAdd =
let newList = itemToAdd :: state.UnpaidItems
Cart.Active {state with UnpaidItems=newList }
let removeFromActiveState state itemToRemove =
let newList = state.UnpaidItems
|> List.filter (fun i -> i<>itemToRemove)
match newList with
| [] -> Cart.Empty NoItems
| _ -> Cart.Active {state with UnpaidItems=newList}
let payForActiveState state amount =
// returns a new PaidFor Cart
Cart.PaidFor {PaidItems=state.UnpaidItems; Payment=amount}

Next, we attach the operations to the states as methods

type EmptyState with
member this.Add = addToEmptyState
type ActiveState with
member this.Add = addToActiveState this
member this.Remove = removeFromActiveState this
member this.Pay = payForActiveState this

And we can create some cart level helper methods as well. At the cart level, we have to
explicitly handle each possibility for the internal state with a match..with expression.

211

Worked example: Designing for correctness

let addItemToCart cart item =

match cart with
| Empty state -> state.Add item
| Active state -> state.Add item
| PaidFor state ->
printfn "ERROR: The cart is paid for"
cart
let removeItemFromCart cart item =
match cart with
| Empty state ->
printfn "ERROR: The cart is empty"
cart // return the cart
| Active state ->
state.Remove item
| PaidFor state ->
printfn "ERROR: The cart is paid for"
cart // return the cart
let displayCart cart =
match cart with
| Empty state ->
printfn "The cart is empty" // can't do state.Items
| Active state ->
printfn "The cart contains %A unpaid items"
state.UnpaidItems
| PaidFor state ->
printfn "The cart contains %A paid items. Amount paid: %f"
state.PaidItems state.Payment
type Cart with
static member NewCart = Cart.Empty NoItems
member this.Add = addItemToCart this
member this.Remove = removeItemFromCart this
member this.Display = displayCart this

Testing the design

Let's exercise this code now:
let emptyCart = Cart.NewCart
printf "emptyCart="; emptyCart.Display
let cartA = emptyCart.Add "A"
printf "cartA="; cartA.Display

We now have an active cart with one item in it. Note that " cartA " is a completely different
object from " emptyCart " and is in a different state.
212

Worked example: Designing for correctness

Let's keep going:

let cartAB = cartA.Add "B"
printf "cartAB="; cartAB.Display
let cartB = cartAB.Remove "A"
printf "cartB="; cartB.Display
let emptyCart2 = cartB.Remove "B"
printf "emptyCart2="; emptyCart2.Display

So far, so good. Again, all these are distinct objects in different states,
Let's test the requirement that you cannot remove items from an empty cart:
let emptyCart3 = emptyCart2.Remove "B" //error
printf "emptyCart3="; emptyCart3.Display

An error ? just what we want!

Now say that we want to pay for a cart. We didn't create this method at the Cart level,
because we didn't want to tell the client how to handle all the cases. This method only exists
for the Active state, so the client will have to explicitly handle each case and only call the
Pay method when an Active state is matched.

First we'll try to pay for cartA.

// try to pay for cartA
let cartAPaid =
match cartA with
| Empty _ | PaidFor _ -> cartA
| Active state -> state.Pay 100m
printf "cartAPaid="; cartAPaid.Display

The result was a paid cart.

Now we'll try to pay for the emptyCart.
// try to pay for emptyCart
let emptyCartPaid =
match emptyCart with
| Empty _ | PaidFor _ -> emptyCart
| Active state -> state.Pay 100m
printf "emptyCartPaid="; emptyCartPaid.Display

213

Worked example: Designing for correctness

Nothing happens. The cart is empty, so the Active branch is not called. We might want to
raise an error or log a message in the other branches, but no matter what we do we cannot
accidentally call the Pay method on an empty cart, because that state does not have a
method to call!
The same thing happens if we accidentally try to pay for a cart that is already paid.
// try to pay for cartAB
let cartABPaid =
match cartAB with
| Empty _ | PaidFor _ -> cartAB // return the same cart
| Active state -> state.Pay 100m
// try to pay for cartAB again
let cartABPaidAgain =
match cartABPaid with
| Empty _ | PaidFor _ -> cartABPaid // return the same cart
| Active state -> state.Pay 100m

You might argue that the client code above might not be representative of code in the real
world ? it is well-behaved and already dealing with the requirements.
So what happens if we have badly written or malicious client code that tries to force
payment:
match cartABPaid with
| Empty state -> state.Pay 100m
| PaidFor state -> state.Pay 100m
| Active state -> state.Pay 100m

If we try to force it like this, we will get compile errors. There is no way the client can create
code that does not meet the requirements.

Summary
We have designed a simple shopping cart model which has many benefits over the C#
design.
It maps to the requirements quite clearly. It is impossible for a client of this API to call
code that doesn't meet the requirements.
Using states means that the number of possible code paths is much smaller than the C#
version, so there will be many fewer unit tests to write.
Each function is simple enough to probably work the first time, as, unlike the C# version,
there are no conditionals anywhere.

214

Worked example: Designing for correctness

Analysis of the original C# code

Now that you have seen the F# code, we can revisit the original C# code with fresh eyes. In
case you were wondering, here are my thoughts as to what is wrong with the C# shopping
cart example as designed.
Requirement not met: An empty cart can still be paid for.
Major design flaw: Overloading the payment amount to be a signal for IsPaidFor means that
a zero paid amount can never lock down the cart. Are you sure it would never be possible to
have a cart which is paid for but free of charge? The requirements are not clear, but what if
this did become a requirement later? How much code would have to be changed?
Minor design flaws: What should happen when trying to remove an item from an empty cart?
And what should happen when attempting to pay for a cart that is already paid for? Should
we throw exceptions in these cases, or just silently ignore them? And does it make sense
that a client should be able to enumerate the items in an empty cart? And this is not thread
safe as designed; so what happens if a secondary thread adds an item to the cart while a
payment is being made on the main thread?
That's quite a lot of things to worry about.
The nice thing about the F# design is none of these problems can even exist. So designing
this way not only ensures correct code, but it also really reduces the cognitive effort to
ensure that the design is bullet proof in the first place.
Compile time checking: The original C# design mixes up all the states and transitions in a
single class, which makes it very error prone. A better approach would be to create separate
state classes (with a common base class say) which reduces complexity, but still, the lack of
a built in "union" type means that you cannot statically verify that the code is correct. There
are ways of doing "union" types in C#, but it is not idiomatic at all, while in F# it is
commonplace.

Appendix: C# code for a correct solution

When faced with these requirements in C#, you might immediately think -- just create an
interface!
But it is not as easy as you might think. I have written a follow up post on this to explain why:
The shopping cart example in C#.

215

Worked example: Designing for correctness

If you are interested to see what the C# code for a solution looks like, here it is below. This
code meets the requirements above and guarantees correctness at compile time, as
desired.
The key thing to note is that, because C# doesn't have union types, the implementation uses
a "fold" function, a function that has three function parameters, one for each state. To use
the cart, the caller passes a set of three lambdas in, and the (hidden) state determines what
happens.
var paidCart = cartA.Do(
// lambda for Empty state
state => cartA,
// lambda for Active state
state => state.Pay(100),
// lambda for Paid state
state => cartA);

This approach means that the caller can never call the "wrong" function, such as "Pay" for
the Empty state, because the parameter to the lambda will not support it. Try it and see!
using System;
using System.Collections.Generic;
using System.Linq;
namespace WhyUseFsharp
{
public class ShoppingCart<TItem>
{
#region ShoppingCart State classes
/// <summary>
/// Represents the Empty state
/// </summary>
public class EmptyState
{
public ShoppingCart<TItem> Add(TItem item)
{
var newItems = new[] { item };
var newState = new ActiveState(newItems);
return FromState(newState);
}
}
/// <summary>
/// Represents the Active state
/// </summary>
public class ActiveState

216

Worked example: Designing for correctness

{
public ActiveState(IEnumerable<TItem> items)
{
Items = items;
}
public IEnumerable<TItem> Items { get; private set; }
public ShoppingCart<TItem> Add(TItem item)
{
var newItems = new List<TItem>(Items) {item};
var newState = new ActiveState(newItems);
return FromState(newState);
}
public ShoppingCart<TItem> Remove(TItem item)
{
var newItems = new List<TItem>(Items);
newItems.Remove(item);
if (newItems.Count > 0)
{
var newState = new ActiveState(newItems);
return FromState(newState);
}
else
{
var newState = new EmptyState();
return FromState(newState);
}
}
public ShoppingCart<TItem> Pay(decimal amount)
{
var newState = new PaidForState(Items, amount);
return FromState(newState);
}

}
/// <summary>
/// Represents the Paid state
/// </summary>
public class PaidForState
{
public PaidForState(IEnumerable<TItem> items, decimal amount)
{
Items = items.ToList();
Amount = amount;
}
public IEnumerable<TItem> Items { get; private set; }
public decimal Amount { get; private set; }

217

Worked example: Designing for correctness

}
#endregion ShoppingCart State classes
//====================================
// Execute of shopping cart proper
//====================================
private enum Tag { Empty, Active, PaidFor }
private readonly Tag _tag = Tag.Empty;
private readonly object _state; //has to be a generic object
/// <summary>
/// Private ctor. Use FromState instead
/// </summary>
private ShoppingCart(Tag tagValue, object state)
{
_state = state;
_tag = tagValue;
}
public static ShoppingCart<TItem> FromState(EmptyState state)
{
return new ShoppingCart<TItem>(Tag.Empty, state);
}
public static ShoppingCart<TItem> FromState(ActiveState state)
{
return new ShoppingCart<TItem>(Tag.Active, state);
}
public static ShoppingCart<TItem> FromState(PaidForState state)
{
return new ShoppingCart<TItem>(Tag.PaidFor, state);
}
/// <summary>
/// Create a new empty cart
/// </summary>
public static ShoppingCart<TItem> NewCart()
{
var newState = new EmptyState();
return FromState(newState);
}
/// <summary>
/// Call a function for each case of the state
/// </summary>
/// <remarks>
/// Forcing the caller to pass a function for each possible case means that al
l cases are handled at all times.
/// </remarks>
public TResult Do<TResult>(

218

Worked example: Designing for correctness

Func<EmptyState, TResult> emptyFn,

Func<ActiveState, TResult> activeFn,
Func<PaidForState, TResult> paidForyFn
)
{
switch (_tag)
{
case Tag.Empty:
return emptyFn(_state as EmptyState);
case Tag.Active:
return activeFn(_state as ActiveState);
case Tag.PaidFor:
return paidForyFn(_state as PaidForState);
default:
throw new InvalidOperationException(string.Format("Tag {0} not rec
ognized", _tag));
}
}
/// <summary>
/// Do an action without a return value
/// </summary>
public void Do(
Action<EmptyState> emptyFn,
Action<ActiveState> activeFn,
Action<PaidForState> paidForyFn
)
{
//convert the Actions into Funcs by returning a dummy value
Do(
state => { emptyFn(state); return 0; },
state => { activeFn(state); return 0; },
state => { paidForyFn(state); return 0; }
);
}

}
/// <summary>
/// Extension methods for my own personal library
/// </summary>
public static class ShoppingCartExtension
{
/// <summary>
/// Helper method to Add
/// </summary>
public static ShoppingCart<TItem> Add<TItem>(this ShoppingCart<TItem> cart, TI
tem item)
{
return cart.Do(
state => state.Add(item), //empty case

219

Worked example: Designing for correctness

state => state.Add(item), //active case

state => { Console.WriteLine("ERROR: The cart is paid for and items ca
nnot be added"); return cart; } //paid for case
);
}
/// <summary>
/// Helper method to Remove
/// </summary>
public static ShoppingCart<TItem> Remove<TItem>(this ShoppingCart<TItem> cart,
TItem item)
{
return cart.Do(
state => { Console.WriteLine("ERROR: The cart is empty and items canno
t be removed"); return cart; }, //empty case
state => state.Remove(item), //active case
state => { Console.WriteLine("ERROR: The cart is paid for and items ca
nnot be removed"); return cart; } //paid for case
);
}
/// <summary>
/// Helper method to Display
/// </summary>
public static void Display<TItem>(this ShoppingCart<TItem> cart)
{
cart.Do(
state => Console.WriteLine("The cart is empty"),
state => Console.WriteLine("The active cart contains {0} items", state
.Items.Count()),
state => Console.WriteLine("The paid cart contains {0} items. Amount p
aid {1}", state.Items.Count(), state.Amount)
);
}
}
[NUnit.Framework.TestFixture]
public class CorrectShoppingCartTest
{
[NUnit.Framework.Test]
public void TestCart()
{
var emptyCart = ShoppingCart<string>.NewCart();
emptyCart.Display();
var cartA = emptyCart.Add("A"); //one item
cartA.Display();
var cartAb = cartA.Add("B"); //two items
cartAb.Display();
var cartB = cartAb.Remove("A"); //one item
cartB.Display();

220

Worked example: Designing for correctness

var emptyCart2 = cartB.Remove("B"); //empty

emptyCart2.Display();
Console.WriteLine("Removing from emptyCart");
emptyCart.Remove("B"); //error

// try to pay for cartA

Console.WriteLine("paying for cartA");
var paidCart = cartA.Do(
state => cartA,
state => state.Pay(100),
state => cartA);
paidCart.Display();
Console.WriteLine("Adding to paidCart");
paidCart.Add("C");
// try to pay for emptyCart
Console.WriteLine("paying for emptyCart");
var emptyCartPaid = emptyCart.Do(
state => emptyCart,
state => state.Pay(100),
state => emptyCart);
emptyCartPaid.Display();
}
}
}

221

Concurrency

Concurrency
We hear a lot about concurrency nowadays, how important it is, and how it is "the next major
revolution in how we write software".
So what do we actually mean by "concurrency" and how can F# help?
The simplest definition of concurrency is just "several things happening at once, and maybe
interacting with each other". It seems a trivial definition, but the key point is that most
computer programs (and languages) are designed to work serially, on one thing at a time,
and are not well-equipped to handle concurrency.
And even if computer programs are written to handle concurrency, there is an even more
serious problem: our brains do not do well when thinking about concurrency. It is commonly
acknowledged that writing code that handles concurrency is extremely hard. Or I should say,
writing concurrent code that is correct is extremely hard! It's very easy to write concurrent
code that is buggy; there might be race conditions, or operations might not be atomic, or
tasks might be starved or blocked unnecessarily, and these issues are hard to find by
looking at the code or using a debugger.
Before talking about the specifics of F#, let's try to classify some of the common types of
concurrency scenarios that we have to deal with as developers:
"Concurrent Multitasking". This is when we have a number of concurrent tasks (e.g.
processes or threads) within our direct control, and we want them to communicate with
each other and share data safely.
"Asynchronous" programming. This is when we initiate a conversation with a
separate system outside our direct control, and then wait for it to get back to us.
Common cases of this are when talking to the filesystem, a database, or the network.
These situations are typically I/O bound, so you want to do something else useful while
you are waiting. These types of tasks are often non-deterministic as well, meaning that
running the same program twice might give a different result.
"Parallel" programming. This is when we have a single task that we want to split into
independant subtasks, and then run the subtasks in parallel, ideally using all the cores
or CPUs that are available. These situations are typically CPU bound. Unlike the async
tasks, parallelism is typically deterministic, so running the same program twice will give
the same result.
"Reactive" programming. This is when we do not initiate tasks ourselves, but are
focused on listening for events which we then process as fast as possible. This situation
occurs when designing servers, and when working with a user interface.

222

Concurrency

Of course, these are vague definitions and overlap in practice. In general, though, for all
these cases, the actual implementations that address these scenarios tend to use two
distinct approaches:
If there are lots of different tasks that need to share state or resources without waiting,
then use a "buffered asynchronous" design.
If there are lots of identical tasks that do not need to share state, then use parallel tasks
using "fork/join" or "divide and conquer" approaches.

F# tools for concurrent programming

F# offers a number of different approaches to writing concurrent code:
For multitasking and asynchronous problems, F# can directly use all the usual .NET
suspects, such as Thread AutoResetEvent , BackgroundWorker and IAsyncResult . But
it also offers a much simpler model for all types of async IO and background task
management, called "asynchronous workflows". We will look at these in the next post.
An alternative approach for asynchronous problems is to use message queues and the
"actor model" (this is the "buffered asynchronous" design mentioned above). F# has a
built in implementation of the actor model called MailboxProcessor . I am a big
proponent of designing with actors and message queues, as it decouples the various
components and allows you to think serially about each one.
For true CPU parallelism, F# has convenient library code that builds on the
asynchronous workflows mentioned above, and it can also use the .NET Task Parallel
Library.
Finally, the functional approach to event handling and reactive programming is quite
different from the traditional approach. The functional approach treats events as
"streams" which can be filtered, split, and combined in much the the same way that
LINQ handles collections. F# has built in support for this model, as well as for the
standard event-driven model.

223

Asynchronous programming

Asynchronous programming
In this post we'll have a look at a few ways to write asynchronous code in F#, and a very
brief example of parallelism as well.

Traditional asynchronous programming

As noted in the previous post, F# can directly use all the usual .NET suspects, such as
Thread AutoResetEvent , BackgroundWorker and IAsyncResult .

Let's see a simple example where we wait for a timer event to go off:
open System
let userTimerWithCallback =
// create an event to wait on
let event = new System.Threading.AutoResetEvent(false)
// create a timer and add an event handler that will signal the event
let timer = new System.Timers.Timer(2000.0)
timer.Elapsed.Add (fun _ -> event.Set() |> ignore )
//start
printfn "Waiting for timer at %O" DateTime.Now.TimeOfDay
timer.Start()
// keep working
printfn "Doing something useful while waiting for event"
// block on the timer via the AutoResetEvent
event.WaitOne() |> ignore
//done
printfn "Timer ticked at %O" DateTime.Now.TimeOfDay

This shows the use of AutoResetEvent as a synchronization mechanism.

A lambda is registered with the Timer.Elapsed event, and when the event is triggered,
the AutoResetEvent is signalled.
The main thread starts the timer, does something else while waiting, and then blocks
until the event is triggered.
Finally, the main thread continues, about 2 seconds later.

224

Asynchronous programming

The code above is reasonably straightforward, but does require you to instantiate an
AutoResetEvent, and could be buggy if the lambda is defined incorrectly.

Introducing asynchronous workflows

F# has a built-in construct called "asynchronous workflows" which makes async code much
easier to write. These workflows are objects that encapsulate a background task, and
provide a number of useful operations to manage them.
Here's the previous example rewritten to use one:
open System
//open Microsoft.FSharp.Control // Async.* is in this module.
let userTimerWithAsync =
// create a timer and associated async event
let timer = new System.Timers.Timer(2000.0)
let timerEvent = Async.AwaitEvent (timer.Elapsed) |> Async.Ignore
// start
printfn "Waiting for timer at %O" DateTime.Now.TimeOfDay
timer.Start()
// keep working
printfn "Doing something useful while waiting for event"
// block on the timer event now by waiting for the async to complete
Async.RunSynchronously timerEvent
// done
printfn "Timer ticked at %O" DateTime.Now.TimeOfDay

Here are the changes:

the AutoResetEvent and lambda have disappeared, and are replaced by let
timerEvent = Control.Async.AwaitEvent (timer.Elapsed) , which creates an async object

directly from the event, without needing a lambda. The ignore is added to ignore the
result.
the event.WaitOne() has been replaced by Async.RunSynchronously timerEvent which
blocks on the async object until it has completed.
That's it. Both simpler and easier to understand.
The async workflows can also be used with IAsyncResult , begin/end pairs, and other
standard .NET methods.

225

Asynchronous programming

For example, here's how you might do an async file write by wrapping the IAsyncResult
generated from BeginWrite .
let fileWriteWithAsync =
// create a stream to write to
use stream = new System.IO.FileStream("test.txt",System.IO.FileMode.Create)
// start
printfn "Starting async write"
let asyncResult = stream.BeginWrite(Array.empty,0,0,null,null)
// create an async wrapper around an IAsyncResult
let async = Async.AwaitIAsyncResult(asyncResult) |> Async.Ignore
// keep working
printfn "Doing something useful while waiting for write to complete"
// block on the timer now by waiting for the async to complete
Async.RunSynchronously async
// done
printfn "Async write completed"

Creating and nesting asynchronous workflows

Asynchronous workflows can also be created manually. A new workflow is created using the
async keyword and curly braces. The braces contain a set of expressions to be executed in

the background.
This simple workflow just sleeps for 2 seconds.
let sleepWorkflow = async{
printfn "Starting sleep workflow at %O" DateTime.Now.TimeOfDay
do! Async.Sleep 2000
printfn "Finished sleep workflow at %O" DateTime.Now.TimeOfDay
}
Async.RunSynchronously sleepWorkflow

Note: the code do! Async.Sleep 2000 is similar to Thread.Sleep but designed to work with
asynchronous workflows.
Workflows can contain other async workflows nested inside them. Within the braces, the
nested workflows can be blocked on by using the let! syntax.

226

Asynchronous programming

let nestedWorkflow = async{

printfn "Starting parent"
let! childWorkflow = Async.StartChild sleepWorkflow
// give the child a chance and then keep working
do! Async.Sleep 100
printfn "Doing something useful while waiting "
// block on the child
let! result = childWorkflow
// done
printfn "Finished parent"
}
// run the whole workflow
Async.RunSynchronously nestedWorkflow

Cancelling workflows
One very convenient thing about async workflows is that they support a built-in cancellation
mechanism. No special code is needed.
Consider a simple task that prints numbers from 1 to 100:
let testLoop = async {
for i in [1..100] do
// do something
printf "%i before.." i
// sleep a bit
do! Async.Sleep 10
printfn "..after"
}

We can test it in the usual way:

Async.RunSynchronously testLoop

Now let's say we want to cancel this task half way through. What would be the best way of
doing it?
In C#, we would have to create flags to pass in and then check them frequently, but in F#
this technique is built in, using the CancellationToken class.

227

Asynchronous programming

Here an example of how we might cancel the task:

open System
open System.Threading
// create a cancellation source
let cancellationSource = new CancellationTokenSource()
// start the task, but this time pass in a cancellation token
Async.Start (testLoop,cancellationSource.Token)
// wait a bit
Thread.Sleep(200)
// cancel after 200ms
cancellationSource.Cancel()

In F#, any nested async call will check the cancellation token automatically!
In this case it was the line:
do! Async.Sleep(10)

As you can see from the output, this line is where the cancellation happened.

Composing workflows in series and parallel

Another useful thing about async workflows is that they can be easily combined in various
ways: both in series and in parallel.
Let's again create a simple workflow that just sleeps for a given time:
// create a workflow to sleep for a time
let sleepWorkflowMs ms = async {
printfn "%i ms workflow started" ms
do! Async.Sleep ms
printfn "%i ms workflow finished" ms
}

Here's a version that combines two of these in series:

228

Asynchronous programming

let workflowInSeries = async {

let! sleep1 = sleepWorkflowMs 1000
printfn "Finished one"
let! sleep2 = sleepWorkflowMs 2000
printfn "Finished two"
}
#time
Async.RunSynchronously workflowInSeries
#time

And here's a version that combines two of these in parallel:

// Create them
let sleep1 = sleepWorkflowMs 1000
let sleep2 = sleepWorkflowMs 2000
// run them in parallel
#time
[sleep1; sleep2]
|> Async.Parallel
|> Async.RunSynchronously
#time

Note: The #time command toggles the timer on and off. It only works in the interactive
window, so this example must be sent to the interactive window in order to work
corrrectly.

We're using the #time option to show the total elapsed time, which, because they run in
parallel, is 2 secs. If they ran in series instead, it would take 3 seconds.
Also you might see that the output is garbled sometimes because both tasks are writing to
the console at the same time!
This last sample is a classic example of a "fork/join" approach, where a number of a child
tasks are spawned and then the parent waits for them all to finish. As you can see, F#
makes this very easy!

Example: an async web downloader

In this more realistic example, we'll see how easy it is to convert some existing code from a
non-asynchronous style to an asynchronous style, and the corresponding performance
increase that can be achieved.

229

Asynchronous programming

So here is a simple URL downloader, very similar to the one we saw at the start of the
series:
open System.Net
open System
open System.IO
let fetchUrl url =
let req = WebRequest.Create(Uri(url))
use resp = req.GetResponse()
use stream = resp.GetResponseStream()
use reader = new IO.StreamReader(stream)
let html = reader.ReadToEnd()
printfn "finished downloading %s" url

And here is some code to time it:

// a list of sites to fetch
let sites = ["http://www.bing.com";
"http://www.google.com";
"http://www.microsoft.com";
"http://www.amazon.com";
"http://www.yahoo.com"]
#time // turn interactive timer on
sites // start with the list of sites
|> List.map fetchUrl // loop through each site and download
#time // turn timer off

Make a note of the time taken, and let's if we can improve on it!
Obviously the example above is inefficient -- only one web site at a time is visited. The
program would be faster if we could visit them all at the same time.
So how would we convert this to a concurrent algorithm? The logic would be something like:
Create a task for each web page we are downloading, and then for each task, the
download logic would be something like:
Start downloading a page from a website. While that is going on, pause and let
other tasks have a turn.
When the download is finished, wake up and continue on with the task
Finally, start all the tasks up and let them go at it!
Unfortunately, this is quite hard to do in a standard C-like language. In C# for example, you
have to create a callback for when an async task completes. Managing these callbacks is
painful and creates a lot of extra support code that gets in the way of understanding the

230

Asynchronous programming

logic. There are some elegant solutions to this, but in general, the signal to noise ratio for
concurrent programming in C# is very high*.
* As of the time of this writing. Future versions of C# will have the await keyword, which is
similar to what F# has now.
But as you can guess, F# makes this easy. Here is the concurrent F# version of the
downloader code:
open Microsoft.FSharp.Control.CommonExtensions
// adds AsyncGetResponse
// Fetch the contents of a web page asynchronously
let fetchUrlAsync url =
async {
let req = WebRequest.Create(Uri(url))
use! resp = req.AsyncGetResponse() // new keyword "use!"
use stream = resp.GetResponseStream()
use reader = new IO.StreamReader(stream)
let html = reader.ReadToEnd()
printfn "finished downloading %s" url
}

Note that the new code looks almost exactly the same as the original. There are only a few
minor changes.
The change from " use resp = " to " use! resp = " is exactly the change that we talked
about above -- while the async operation is going on, let other tasks have a turn.
We also used the extension method AsyncGetResponse defined in the CommonExtensions
namespace. This returns an async workflow that we can nest inside the main workflow.
In addition the whole set of steps is contained in the " async {...} " wrapper which turns
it into a block that can be run asynchronously.
And here is a timed download using the async version.
// a list of sites to fetch
let sites = ["http://www.bing.com";
"http://www.google.com";
"http://www.microsoft.com";
"http://www.amazon.com";
"http://www.yahoo.com"]
#time // turn interactive timer on
sites
|> List.map fetchUrlAsync // make a list of async tasks
|> Async.Parallel // set up the tasks to run in parallel
|> Async.RunSynchronously // start them off
#time // turn timer off

231

Asynchronous programming

The way this works is:

fetchUrlAsync is applied to each site. It does not immediately start the download, but

returns an async workflow for running later.

To set up all the tasks to run at the same time we use the Async.Parallel function
Finally we call Async.RunSynchronously to start all the tasks, and wait for them all to
stop.
If you try out this code yourself, you will see that the async version is much faster than the
sync version. Not bad for a few minor code changes! Most importantly, the underlying logic
is still very clear and is not cluttered up with noise.

Example: a parallel computation

To finish up, let's have another quick look at a parallel computation again.
Before we start, I should warn you that the example code below is just to demonstrate the
basic principles. Benchmarks from "toy" versions of parallelization like this are not
meaningful, because any kind of real concurrent code has so many dependencies.
And also be aware that parallelization is rarely the best way to speed up your code. Your
time is almost always better spent on improving your algorithms. I'll bet my serial version of
quicksort against your parallel version of bubblesort any day! (For more details on how to
improve performance, see the optimization series)
Anyway, with that caveat, let's create a little task that chews up some CPU. We'll test this
serially and in parallel.
let childTask() =
// chew up some CPU.
for i in [1..1000] do
for i in [1..1000] do
do "Hello".Contains("H") |> ignore
// we don't care about the answer!
// Test the child task on its own.
// Adjust the upper bounds as needed
// to make this run in about 0.2 sec
#time
childTask()
#time

Adjust the upper bounds of the loops as needed to make this run in about 0.2 seconds.

232

Asynchronous programming

Now let's combine a bunch of these into a single serial task (using composition), and test it
with the timer:
let parentTask =
childTask
|> List.replicate 20
|> List.reduce (>>)
//test
#time
parentTask()
#time

This should take about 4 seconds.

Now in order to make the childTask parallelizable, we have to wrap it inside an async :
let asyncChildTask = async { return childTask() }

And to combine a bunch of asyncs into a single parallel task, we use Async.Parallel .
Let's test this and compare the timings:
let asyncParentTask =
asyncChildTask
|> List.replicate 20
|> Async.Parallel
//test
#time
asyncParentTask
|> Async.RunSynchronously
#time

On a dual-core machine, the parallel version is about 50% faster. It will get faster in
proportion to the number of cores or CPUs, of course, but sublinearly. Four cores will be
faster than one core, but not four times faster.
On the other hand, as with the async web download example, a few minor code changes
can make a big difference, while still leaving the code easy to read and understand. So in
cases where parallelism will genuinely help, it is nice to know that it is easy to arrange.

233

Messages and Agents

In this post, we'll look at the message-based (or actor-based) approach to concurrency.
In this approach, when one task wants to communicate with another, it sends it a message,
rather than contacting it directly. The messages are put on a queue, and the receiving task
(known as an "actor" or "agent") pulls the messages off the queue one at a time to process
them.
This message-based approach has been applied to many situations, from low-level network
sockets (built on TCP/IP) to enterprise wide application integration systems (for example
MSMQ or IBM WebSphere MQ).
From a software design point of view, a message-based approach has a number of benefits:
You can manage shared data and resources without locks.
You can easily follow the "single responsibility principle", because each agent can be
designed to do only one thing.
It encourages a "pipeline" model of programming with "producers" sending messages to
decoupled "consumers", which has additional benefits:
The queue acts as a buffer, eliminating waiting on the client side.
It is straightforward to scale up one side or the other of the queue as needed in
order to maximize throughput.
Errors can be handled gracefully, because the decoupling means that agents can
be created and destroyed without affecting their clients.
From a practical developer's point of view, what I find most appealing about the messagebased approach is that when writing the code for any given actor, you don't have to hurt your
brain by thinking about concurrency. The message queue forces a "serialization" of
operations that otherwise might occur concurrently. And this in turn makes it much easier to
think about (and write code for) the logic for processing a message, because you can be
sure that your code will be isolated from other events that might interrupt your flow.
With these advantages, it is not surprising that when a team inside Ericsson wanted to
design a programming language for writing highly-concurrent telephony applications, they
created one with a message-based approach, namely Erlang. Erlang has now become the
poster child for the whole topic, and has created a lot of interest in implementing the same
approach in other languages.

234

Messages and Agents

How F# implements a message-based

approach
F# has a built-in agent class called MailboxProcessor . These agents are very lightweight
compared with threads - you can instantiate tens of thousands of them at the same time.
These are similar to the agents in Erlang, but unlike the Erlang ones, they do not work
across process boundaries, only in the same process. And unlike a heavyweight queueing
system such as MSMQ, the messages are not persistent. If your app crashes, the messages
are lost.
But these are minor issues, and can be worked around. In a future series, I will go into
alternative implementations of message queues. The fundamental approach is the same in
all cases.
Let's see a simple agent implementation in F#:

#nowarn "40"
let printerAgent = MailboxProcessor.Start(fun inbox->
// the message processing function
let rec messageLoop = async{
// read a message
let! msg = inbox.Receive()
// process a message
printfn "message is: %s" msg
// loop to top
return! messageLoop
}
// start the loop
messageLoop
)

The MailboxProcessor.Start function takes a simple function parameter. That function loops
forever, reading messages from the queue (or "inbox") and processing them.
Note: I have added the #nowarn "40" pragma to avoid the warning "FS0040", which can be
safely ignored in this case.
Here's the example in use:

235

Messages and Agents

// test it
printerAgent.Post "hello"
printerAgent.Post "hello again"
printerAgent.Post "hello a third time"

In the rest of this post we'll look at two slightly more useful examples:
Managing shared state without locks
Serialized and buffered access to shared IO
In both of these cases, a message based approach to concurrency is elegant, efficient, and
easy to program.

Managing shared state

Let's look at the shared state problem first.
A common scenario is that you have some state that needs to be accessed and changed by
multiple concurrent tasks or threads. We'll use a very simple case, and say that the
requirements are:
A shared "counter" and "sum" that can be incremented by multiple tasks concurrently.
Changes to the counter and sum must be atomic -- we must guarantee that they will
both be updated at the same time.

The locking approach to shared state

Using locks or mutexes is a common solution for these requirements, so let's write some
code using a lock, and see how it performs.
First let's write a static LockedCounter class that protects the state with locks.

236

Messages and Agents

open System
open System.Threading
open System.Diagnostics
// a utility function
type Utility() =
static let rand = new Random()
static member RandomSleep() =
let ms = rand.Next(1,10)
Thread.Sleep ms
// an implementation of a shared counter using locks
type LockedCounter () =
static let _lock = new Object()
static let mutable count = 0
static let mutable sum = 0
static let updateState i =
// increment the counters and...
sum <- sum + i
count <- count + 1
printfn "Count is: %i. Sum is: %i" count sum
// ...emulate a short delay
Utility.RandomSleep()

// public interface to hide the state

static member Add i =
// see how long a client has to wait
let stopwatch = new Stopwatch()
stopwatch.Start()
// start lock. Same as C# lock{...}
lock _lock (fun () ->
// see how long the wait was
stopwatch.Stop()
printfn "Client waited %i" stopwatch.ElapsedMilliseconds
// do the core logic
updateState i
)
// release lock

Some notes on this code:

This code is written using a very imperative approach, with mutable variables and locks

237

Messages and Agents

The public Add method has explicit Monitor.Enter and Monitor.Exit expressions to
get and release the lock. This is the same as the lock{...} statement in C#.
We've also added a stopwatch to measure how long a client has to wait to get the lock.
The core "business logic" is the updateState method, which not only updates the state,
but adds a small random wait as well to emulate the time taken to do the processing.
Let's test it in isolation:
// test in isolation
LockedCounter.Add 4
LockedCounter.Add 5

Next, we'll create a task that will try to access the counter:
let makeCountingTask addFunction taskId = async {
let name = sprintf "Task%i" taskId
for i in [1..3] do
addFunction i
}
// test in isolation
let task = makeCountingTask LockedCounter.Add 1
Async.RunSynchronously task

In this case, when there is no contention at all, the wait times are all 0.
But what happens when we create 10 child tasks that all try to access the counter at once:
let lockedExample5 =
[1..10]
|> List.map (fun i -> makeCountingTask LockedCounter.Add i)
|> Async.Parallel
|> Async.RunSynchronously
|> ignore

Oh dear! Most tasks are now waiting quite a while. If two tasks want to update the state at
the same time, one must wait for the other's work to complete before it can do its own work,
which affects performance.
And if we add more and more tasks, the contention will increase, and the tasks will spend
more and more time waiting rather than working.

The message-based approach to shared state

Let's see how a message queue might help us. Here's the message based version:

238

Messages and Agents

type MessageBasedCounter () =
static let updateState (count,sum) msg =
// increment the counters and...
let newSum = sum + msg
let newCount = count + 1
printfn "Count is: %i. Sum is: %i" newCount newSum
// ...emulate a short delay
Utility.RandomSleep()
// return the new state
(newCount,newSum)
// create the agent
static let agent = MailboxProcessor.Start(fun inbox ->
// the message processing function
let rec messageLoop oldState = async{
// read a message
let! msg = inbox.Receive()
// do the core logic
let newState = updateState oldState msg
// loop to top
return! messageLoop newState
}
// start the loop
messageLoop (0,0)
)
// public interface to hide the implementation
static member Add i = agent.Post i

Some notes on this code:

The core "business logic" is again in the updateState method, which has almost the
same implementation as the earlier example, except the state is immutable, so that a
new state is created and returned to the main loop.
The agent reads messages (simple ints in this case) and then calls updateState
method
The public method Add posts a message to the agent, rather than calling the
updateState method directly

This code is written in a more functional way; there are no mutable variables and no
locks anywhere. In fact, there is no code dealing with concurrency at all! The code only

239

Messages and Agents

has to focus on the business logic, and is consequently much easier to understand.
Let's test it in isolation:
// test in isolation
MessageBasedCounter.Add 4
MessageBasedCounter.Add 5

Next, we'll reuse a task we defined earlier, but calling MessageBasedCounter.Add instead:
let task = makeCountingTask MessageBasedCounter.Add 1
Async.RunSynchronously task

Finally let's create 5 child tasks that try to access the counter at once.
let messageExample5 =
[1..5]
|> List.map (fun i -> makeCountingTask MessageBasedCounter.Add i)
|> Async.Parallel
|> Async.RunSynchronously
|> ignore

We can't measure the waiting time for the clients, because there is none!

Shared IO
A similar concurrency problem occurs when accessing a shared IO resource such as a file:
If the IO is slow, the clients can spend a lot of time waiting, even without locks.
If multiple threads write to the resource at the same time, you can get corrupted data.
Both problems can be solved by using asynchronous calls combined with buffering -- exactly
what a message queue does.
In this next example, we'll consider the example of a logging service that many clients will
write to concurrently. (In this trivial case, we'll just write directly to the Console.)
We'll first look at an implementation without concurrency control, and then at an
implementation that uses message queues to serialize all requests.

IO without serialization

240

Messages and Agents

In order to make the corruption very obvious and repeatable, let's first create a "slow"
console that writes each individual character in the log message and pauses for a
millisecond between each character. During that millisecond, another thread could be writing
as well, causing an undesirable interleaving of messages.
let slowConsoleWrite msg =
msg |> String.iter (fun ch->
System.Threading.Thread.Sleep(1)
System.Console.Write ch
)
// test in isolation
slowConsoleWrite "abc"

Next, we will create a simple task that loops a few times, writing its name each time to the
logger:
let makeTask logger taskId = async {
let name = sprintf "Task%i" taskId
for i in [1..3] do
let msg = sprintf "-%s:Loop%i-" name i
logger msg
}
// test in isolation
let task = makeTask slowConsoleWrite 1
Async.RunSynchronously task

Next, we write a logging class that encapsulates access to the slow console. It has no
locking or serialization, and is basically not thread-safe:
type UnserializedLogger() =
// interface
member this.Log msg = slowConsoleWrite msg
// test in isolation
let unserializedLogger = UnserializedLogger()
unserializedLogger.Log "hello"

Now let's combine all these into a real example. We will create five child tasks and run them
in parallel, all trying to write to the slow console.

241

Messages and Agents

let unserializedExample =
let logger = new UnserializedLogger()
[1..5]
|> List.map (fun i -> makeTask logger.Log i)
|> Async.Parallel
|> Async.RunSynchronously
|> ignore

Ouch! The output is very garbled!

Serialized IO with messages

So what happens when we replace UnserializedLogger with a SerializedLogger class that
encapsulates a message queue.
The agent inside SerializedLogger simply reads a message from its input queue and writes
it to the slow console. Again there is no code dealing with concurrency and no locks are
used.
type SerializedLogger() =
// create the mailbox processor
let agent = MailboxProcessor.Start(fun inbox ->
// the message processing function
let rec messageLoop () = async{
// read a message
let! msg = inbox.Receive()
// write it to the log
slowConsoleWrite msg
// loop to top
return! messageLoop ()
}
// start the loop
messageLoop ()
)
// public interface
member this.Log msg = agent.Post msg
// test in isolation
let serializedLogger = SerializedLogger()
serializedLogger.Log "hello"

242

Messages and Agents

So now we can repeat the earlier unserialized example but using the SerializedLogger
instead. Again, we create five child tasks and run them in parallel:
let serializedExample =
let logger = new SerializedLogger()
[1..5]
|> List.map (fun i -> makeTask logger.Log i)
|> Async.Parallel
|> Async.RunSynchronously
|> ignore

What a difference! This time the output is perfect.

Summary
There is much more to say about this message based approach. In a future series, I hope to
go into much more detail, including discussion of topics such as:
alternative implementations of message queues with MSMQ and TPL Dataflow.
cancellation and out of band messages.
error handling and retries, and handling exceptions in general.
how to scale up and down by creating or removing child agents.
avoiding buffer overruns and detecting starvation or inactivity.

243

Functional Reactive Programming

Events are everywhere. Almost every program has to handle events, whether it be button
clicks in the user interface, listening to sockets in a server, or even a system shutdown
notification.
And events are the basis of one of the most common OO design patterns: the "Observer"
pattern.
But as we know, event handling, like concurrency in general, can be tricky to implement.
Simple event logic is straightforward, but what about logic like "do something if two events
happen in a row but do something different if only one event happens" or "do something if
two events happen at roughly the same time". And how easy is it to combine these
requirements in other, more complex ways?
Even you can successfully implement these requirements, the code tends to be spaghetti
like and hard to understand, even with the best intentions.
Is there a approach that can make event handling easier?
We saw in the previous post on message queues that one of the advantages of that
approach was that the requests were "serialized" making it conceptually easier to deal with.
There is a similar approach that can be used with events. The idea is to turn a series of
events into an "event stream". Event streams then become quite like IEnumerables, and so
the obvious next step is to treat them in much the the same way that LINQ handles
collections, so that they can be filtered, mapped, split and combined.
F# has built in support for this model, as well as for the more tradition approach.

A simple event stream

Let's start with a simple example to compare the two approaches. We'll implement the
classic event handler approach first.
First, we define a utility function that will:
create a timer
register a handler for the Elapsed event
run the timer for five seconds and then stop it
Here's the code:

244

Functional Reactive Programming

open System
open System.Threading
/// create a timer and register an event handler,
/// then run the timer for five seconds
let createTimer timerInterval eventHandler =
// setup a timer
let timer = new System.Timers.Timer(float timerInterval)
timer.AutoReset <- true
// add an event handler
timer.Elapsed.Add eventHandler
// return an async task
async {
// start timer...
timer.Start()
// ...run for five seconds...
do! Async.Sleep 5000
// ... and stop
timer.Stop()
}

Now test it interactively:

// create a handler. The event args are ignored
let basicHandler _ = printfn "tick %A" DateTime.Now
// register the handler
let basicTimer1 = createTimer 1000 basicHandler
// run the task now
Async.RunSynchronously basicTimer1

Now let's create a similar utility method to create a timer, but this time it will return an
"observable" as well, which is the stream of events.

245

Functional Reactive Programming

let createTimerAndObservable timerInterval =

// setup a timer
let timer = new System.Timers.Timer(float timerInterval)
timer.AutoReset <- true
// events are automatically IObservable
let observable = timer.Elapsed
// return an async task
let task = async {
timer.Start()
do! Async.Sleep 5000
timer.Stop()
}
// return a async task and the observable
(task,observable)

And again test it interactively:

// create the timer and the corresponding observable
let basicTimer2 , timerEventStream = createTimerAndObservable 1000
// register that everytime something happens on the
// event stream, print the time.
timerEventStream
|> Observable.subscribe (fun _ -> printfn "tick %A" DateTime.Now)
// run the task now
Async.RunSynchronously basicTimer2

The difference is that instead of registering a handler directly with an event, we are
"subscribing" to an event stream. Subtly different, and important.

Counting events
In this next example, we'll have a slightly more complex requirement:
Create a timer that ticks every 500ms.
At each tick, print the number of ticks so far and the current time.

To do this in a classic imperative way, we would probably create a class with a mutable
counter, as below:

246

Functional Reactive Programming

type ImperativeTimerCount() =
let mutable count = 0
// the event handler. The event args are ignored
member this.handleEvent _ =
count <- count + 1
printfn "timer ticked with count %i" count

We can reuse the utility functions we created earlier to test it:

// create a handler class
let handler = new ImperativeTimerCount()
// register the handler method
let timerCount1 = createTimer 500 handler.handleEvent
// run the task now
Async.RunSynchronously timerCount1

Let's see how we would do this same thing in a functional way:

// create the timer and the corresponding observable
let timerCount2, timerEventStream = createTimerAndObservable 500
// set up the transformations on the event stream
timerEventStream
|> Observable.scan (fun count _ -> count + 1) 0
|> Observable.subscribe (fun count -> printfn "timer ticked with count %i" count)
// run the task now
Async.RunSynchronously timerCount2

Here we see how you can build up layers of event transformations, just as you do with list
transformations in LINQ.
The first transformation is scan , which accumulates state for each event. It is roughly
equivalent to the List.fold function that we have seen used with lists. In this case, the
accumulated state is just a counter.
And then, for each event, the count is printed out.
Note that in this functional approach, we didn't have any mutable state, and we didn't need
to create any special classes.

247

Functional Reactive Programming

Merging multiple event streams

For a final example, we'll look at merging multiple event streams.
Let's make a requirement based on the well-known "FizzBuzz" problem:
Create two timers, called '3' and '5'. The '3' timer ticks every 300ms and the '5' tim
er ticks
every 500ms.
Handle the events as follows:
a) for all events, print the id of the time and the time
b) when a tick is simultaneous with a previous tick, print 'FizzBuzz'
otherwise:
c) when the '3' timer ticks on its own, print 'Fizz'
d) when the '5' timer ticks on its own, print 'Buzz'

First let's create some code that both implementations can use.
We'll want a generic event type that captures the timer id and the time of the tick.
type FizzBuzzEvent = {label:int; time: DateTime}

And then we need a utility function to see if two events are simultaneous. We'll be generous
and allow a time difference of up to 50ms.
let areSimultaneous (earlierEvent,laterEvent) =
let {label=_;time=t1} = earlierEvent
let {label=_;time=t2} = laterEvent
t2.Subtract(t1).Milliseconds < 50

In the imperative design, we'll need to keep track of the previous event, so we can compare
them. And we'll need special case code for the first time, when the previous event doesn't
exist

248

Functional Reactive Programming

type ImperativeFizzBuzzHandler() =
let mutable previousEvent: FizzBuzzEvent option = None
let printEvent thisEvent =
let {label=id; time=t} = thisEvent
printf "[%i] %i.%03i " id t.Second t.Millisecond
let simultaneous = previousEvent.IsSome && areSimultaneous (previousEvent.Value,
thisEvent)
if simultaneous then printfn "FizzBuzz"
elif id = 3 then printfn "Fizz"
elif id = 5 then printfn "Buzz"
member this.handleEvent3 eventArgs =
let event = {label=3; time=DateTime.Now}
printEvent event
previousEvent <- Some event
member this.handleEvent5 eventArgs =
let event = {label=5; time=DateTime.Now}
printEvent event
previousEvent <- Some event

Now the code is beginning to get ugly fast! Already we have mutable state, complex
conditional logic, and special cases, just for such a simple requirement.
Let's test it:
// create the class
let handler = new ImperativeFizzBuzzHandler()
// create the two timers and register the two handlers
let timer3 = createTimer 300 handler.handleEvent3
let timer5 = createTimer 500 handler.handleEvent5
// run the two timers at the same time
[timer3;timer5]
|> Async.Parallel
|> Async.RunSynchronously

It does work, but are you sure the code is not buggy? Are you likely to accidentally break
something if you change it?
The problem with this imperative code is that it has a lot of noise that obscures the the
requirements.
Can the functional version do better? Let's see!
First, we create two event streams, one for each timer:

249

Functional Reactive Programming

let timer3, timerEventStream3 = createTimerAndObservable 300

let timer5, timerEventStream5 = createTimerAndObservable 500

Next, we convert each event on the "raw" event streams into our FizzBuzz event type:
// convert the time events into FizzBuzz events with the appropriate id
let eventStream3 =
timerEventStream3
|> Observable.map (fun _ -> {label=3; time=DateTime.Now})
let eventStream5 =
timerEventStream5
|> Observable.map (fun _ -> {label=5; time=DateTime.Now})

Now, to see if two events are simultaneous, we need to compare them from the two different
streams somehow.
It's actually easier than it sounds, because we can:
combine the two streams into a single stream:
then create pairs of sequential events
then test the pairs to see if they are simultaneous
then split the input stream into two new output streams based on that test
Here's the actual code to do this:
// combine the two streams
let combinedStream =
Observable.merge eventStream3 eventStream5
// make pairs of events
let pairwiseStream =
combinedStream |> Observable.pairwise
// split the stream based on whether the pairs are simultaneous
let simultaneousStream, nonSimultaneousStream =
pairwiseStream |> Observable.partition areSimultaneous

Finally, we can split the nonSimultaneousStream again, based on the event id:

250

Functional Reactive Programming

// split the non-simultaneous stream based on the id

let fizzStream, buzzStream =
nonSimultaneousStream
// convert pair of events to the first event
|> Observable.map (fun (ev1,_) -> ev1)
// split on whether the event id is three
|> Observable.partition (fun {label=id} -> id=3)

Let's review so far. We have started with the two original event streams and from them
created four new ones:
combinedStream contains all the events
simultaneousStream contains only the simultaneous events
fizzStream contains only the non-simultaneous events with id=3
buzzStream contains only the non-simultaneous events with id=5

Now all we need to do is attach behavior to each stream:

//print events from the combinedStream
combinedStream
|> Observable.subscribe (fun {label=id;time=t} ->
printf "[%i] %i.%03i " id t.Second t.Millisecond)
//print events from the simultaneous stream
simultaneousStream
|> Observable.subscribe (fun _ -> printfn "FizzBuzz")
//print events from the nonSimultaneous streams
fizzStream
|> Observable.subscribe (fun _ -> printfn "Fizz")
buzzStream
|> Observable.subscribe (fun _ -> printfn "Buzz")

Let's test it:

// run the two timers at the same time
[timer3;timer5]
|> Async.Parallel
|> Async.RunSynchronously

Here's all the code in one complete set:

251

Functional Reactive Programming

// create the event streams and raw observables

let timer3, timerEventStream3 = createTimerAndObservable 300
let timer5, timerEventStream5 = createTimerAndObservable 500
// convert the time events into FizzBuzz events with the appropriate id
let eventStream3 = timerEventStream3
|> Observable.map (fun _ -> {label=3; time=DateTime.Now})
let eventStream5 = timerEventStream5
|> Observable.map (fun _ -> {label=5; time=DateTime.Now})
// combine the two streams
let combinedStream =
Observable.merge eventStream3 eventStream5
// make pairs of events
let pairwiseStream =
combinedStream |> Observable.pairwise
// split the stream based on whether the pairs are simultaneous
let simultaneousStream, nonSimultaneousStream =
pairwiseStream |> Observable.partition areSimultaneous
// split the non-simultaneous stream based on the id
let fizzStream, buzzStream =
nonSimultaneousStream
// convert pair of events to the first event
|> Observable.map (fun (ev1,_) -> ev1)
// split on whether the event id is three
|> Observable.partition (fun {label=id} -> id=3)
//print events from the combinedStream
combinedStream
|> Observable.subscribe (fun {label=id;time=t} ->
printf "[%i] %i.%03i " id t.Second t.Millisecond)
//print events from the simultaneous stream
simultaneousStream
|> Observable.subscribe (fun _ -> printfn "FizzBuzz")
//print events from the nonSimultaneous streams
fizzStream
|> Observable.subscribe (fun _ -> printfn "Fizz")
buzzStream
|> Observable.subscribe (fun _ -> printfn "Buzz")
// run the two timers at the same time
[timer3;timer5]
|> Async.Parallel
|> Async.RunSynchronously

252

Functional Reactive Programming

The code might seem a bit long winded, but this kind of incremental, step-wise approach is
very clear and self-documenting.
Some of the benefits of this style are:
I can see that it meets the requirements just by looking at it, without even running it. Not
so with the imperative version.
From a design point of view, each final "output" stream follows the single responsibility
principle -- it only does one thing -- so it is very easy to associate behavior with it.
This code has no conditionals, no mutable state, no edge cases. It would be easy to
maintain or change, I hope.
It is easy to debug. For example, I could easily "tap" the output of the
simultaneousStream to see if it contains what I think it contains:

// debugging code
//simultaneousStream |> Observable.subscribe (fun e -> printfn "sim %A" e)
//nonSimultaneousStream |> Observable.subscribe (fun e -> printfn "non-sim %A" e)

This would be much harder in the imperative version.

Summary
Functional Reactive Programming (known as FRP) is a big topic, and we've only just
touched on it here. I hope this introduction has given you a glimpse of the usefulness of this
way of doing things.
If you want to learn more, see the documentation for the F# Observable module, which has
the basic transformations used above. And there is also the Reactive Extensions (Rx) library
which shipped as part of .NET 4. That contains many other additional transformations.

253

Completeness

Completeness
In this final set of posts, we will look at some other aspects of F# under the theme of
"completeness".
Programming languages coming from the academic world tend to focus on elegance and
purity over real-world usefulness, while more mainstream business languages such as C#
and Java are valued precisely because they are pragmatic; they can work in a wide array of
situations and have extensive tools and libraries to meet almost every need. In other words,
to be useful in the enterprise, a language needs to be complete, not just well-designed.
F# is unusual in that it successfully bridges both worlds. Although all the examples so far
have focused on F# as an elegant functional language, it does support an object-oriented
paradigm as well, and can integrate easily with other .NET languages and tools. As a result,
F# is not a isolated island, but benefits from being part of the whole .NET ecosystem.
The other aspects that make F# "complete" are being an official .NET language (with all the
support and documentation that that entails) and being designed to work in Visual Studio
(which provides a nice editor with IntelliSense support, a debugger, and so on). These
benefits should be obvious and won't be discussed here.
So, in this last section, we'll focus on two particular areas:
Seamless interoperation with .NET libraries. Obviously, there can be a mismatch
between the functional approach of F# and the imperative approach that is designed
into the base libraries. We'll look at some of the features of F# that make this integration
easier.
Full support for classes and other C# style code. F# is designed as a hybrid
functional/OO language, so it can do almost everything that C# can do as well. We'll
have a quick tour of the syntax for these other features.

254

Seamless interoperation with .NET libraries

Seamless interoperation with .NET

libraries
We have already seen many examples of F#'s use with the .NET libraries, such as using
System.Net.WebRequest and System.Text.RegularExpressions . And the integration was

indeed seamless.
For more complex requirements, F# natively supports .NET classes, interfaces, and
structures, so the interop is still very straightforward. For example, you can write an
ISomething interface in C# and have the implementation be done in F#.

But not only can F# call into existing .NET code, it can also expose almost any .NET API
back to other languages. For example, you can write classes and methods in F# and expose
them to C#, VB or COM. You can even do the above example backwards -- define an
ISomething interface in F# and have the implementation be done in C#! The benefit of all

this is that you don't have to discard any of your existing code base; you can start using F#
for some things while retaining C# or VB for others, and pick the best tool for the job.
In addition to the tight integration though, there are a number of nice features in F# that often
make working with .NET libraries more convenient than C# in some ways. Here are some of
my favorites:
You can use TryParse and TryGetValue without passing an "out" parameter.
You can resolve method overloads by using argument names, which also helps with
type inference.
You can use "active patterns" to convert .NET APIs into more friendly code.
You can dynamically create objects from an interface such as IDisposable without
creating a concrete class.
You can mix and match "pure" F# objects with existing .NET APIs

TryParse and TryGetValue

The TryParse and TryGetValue functions for values and dictionaries are frequently used to
avoid extra exception handling. But the C# syntax is a bit clunky. Using them from F# is
more elegant because F# will automatically convert the function into a tuple where the first
element is the function return value and the second is the "out" parameter.

255

Seamless interoperation with .NET libraries

//using an Int32
let (i1success,i1) = System.Int32.TryParse("123");
if i1success then printfn "parsed as %i" i1 else printfn "parse failed"
let (i2success,i2) = System.Int32.TryParse("hello");
if i2success then printfn "parsed as %i" i2 else printfn "parse failed"
//using a DateTime
let (d1success,d1) = System.DateTime.TryParse("1/1/1980");
let (d2success,d2) = System.DateTime.TryParse("hello");
//using a dictionary
let dict = new System.Collections.Generic.Dictionary<string,string>();
dict.Add("a","hello")
let (e1success,e1) = dict.TryGetValue("a");
let (e2success,e2) = dict.TryGetValue("b");

Named arguments to help type inference

In C# (and .NET in general), you can have overloaded methods with many different
parameters. F# can have trouble with this. For example, here is an attempt to create a
StreamReader :

let createReader fileName = new System.IO.StreamReader(fileName)

// error FS0041: A unique overload for method 'StreamReader'
// could not be determined

The problem is that F# does not know if the argument is supposed to be a string or a stream.
You could explicitly specify the type of the argument, but that is not the F# way!
Instead, a nice workaround is enabled by the fact that in F#, when calling methods in .NET
libraries, you can specify named arguments.
let createReader2 fileName = new System.IO.StreamReader(path=fileName)

In many cases, such as the one above, just using the argument name is enough to resolve
the type issue. And using explicit argument names can often help to make the code more
legible anyway.

Active patterns for .NET functions

256

Seamless interoperation with .NET libraries

There are many situations where you want to use pattern matching against .NET types, but
the native libraries do not support this. Earlier, we briefly touched on the F# feature called
"active patterns" which allows you to dynamically create choices to match on. This can be
very for useful .NET integration.
A common case is that a .NET library class has a number of mutually exclusive
isSomething , isSomethingElse methods, which have to be tested with horrible looking

cascading if-else statements. Active patterns can hide all the ugly testing, letting the rest of
your code use a more natural approach.
For example, here's the code to test for various isXXX methods for System.Char .
let (|Digit|Letter|Whitespace|Other|) ch =
if System.Char.IsDigit(ch) then Digit
else if System.Char.IsLetter(ch) then Letter
else if System.Char.IsWhiteSpace(ch) then Whitespace
else Other

Once the choices are defined, the normal code can be straightforward:
let printChar ch =
match ch with
| Digit -> printfn "%c is a Digit" ch
| Letter -> printfn "%c is a Letter" ch
| Whitespace -> printfn "%c is a Whitespace" ch
| _ -> printfn "%c is something else" ch
// print a list
['a';'b';'1';' ';'-';'c'] |> List.iter printChar

Another common case is when you have to parse text or error codes to determine the type
of an exception or result. Here's an example that uses an active pattern to parse the error
number associated with SqlExceptions , making them more palatable.
First, set up the active pattern matching on the error number:
open System.Data.SqlClient
let (|ConstraintException|ForeignKeyException|Other|) (ex:SqlException) =
if ex.Number = 2601 then ConstraintException
else if ex.Number = 2627 then ConstraintException
else if ex.Number = 547 then ForeignKeyException
else Other

Now we can use these patterns when processing SQL commands:

257

Seamless interoperation with .NET libraries

let executeNonQuery (sqlCommmand:SqlCommand) =

try
let result = sqlCommmand.ExecuteNonQuery()
// handle success
with
| :?SqlException as sqlException -> // if a SqlException
match sqlException with // nice pattern matching
| ConstraintException -> // handle constraint error
| ForeignKeyException -> // handle FK error
| _ -> reraise() // don't handle any other cases
// all non SqlExceptions are thrown normally

Creating objects directly from an interface

F# has another useful feature called "object expressions". This is the ability to directly create
objects from an interface or abstract class without having to define a concrete class first.
In the example below, we create some objects that implement IDisposable using a
makeResource helper function.

// create a new object that implements IDisposable

let makeResource name =
{ new System.IDisposable
with member this.Dispose() = printfn "%s disposed" name }
let useAndDisposeResources =
use r1 = makeResource "first resource"
printfn "using first resource"
for i in [1..3] do
let resourceName = sprintf "\tinner resource %d" i
use temp = makeResource resourceName
printfn "\tdo something with %s" resourceName
use r2 = makeResource "second resource"
printfn "using second resource"
printfn "done."

The example also demonstrates how the " use " keyword automatically disposes a resource
when it goes out of scope. Here is the output:

258

Seamless interoperation with .NET libraries

using first resource

do something with inner resource 1
inner resource 1 disposed
do something with inner resource 2
inner resource 2 disposed
do something with inner resource 3
inner resource 3 disposed
using second resource
done.
second resource disposed
first resource disposed

Mixing .NET interfaces with pure F# types

The ability to create instances of an interface on the fly means that it is easy to mix and
match interfaces from existing APIs with pure F# types.
For example, say that you have a preexisting API which uses the IAnimal interface, as
shown below.
type IAnimal =
abstract member MakeNoise : unit -> string
let showTheNoiseAnAnimalMakes (animal:IAnimal) =
animal.MakeNoise() |> printfn "Making noise %s"

But we want to have all the benefits of pattern matching, etc, so we have created pure F#
types for cats and dogs instead of classes.
type Cat = Felix | Socks
type Dog = Butch | Lassie

But using this pure F# approach means that that we cannot pass the cats and dogs to the
showTheNoiseAnAnimalMakes function directly.

However, we don't have to create new sets of concrete classes just to implement IAnimal .
Instead, we can dynamically create the IAnimal interface by extending the pure F# types.

259

Seamless interoperation with .NET libraries

// now mixin the interface with the F# types

type Cat with
member this.AsAnimal =
{ new IAnimal
with member a.MakeNoise() = "Meow" }
type Dog with
member this.AsAnimal =
{ new IAnimal
with member a.MakeNoise() = "Woof" }

Here is some test code:

let dog = Lassie
showTheNoiseAnAnimalMakes (dog.AsAnimal)
let cat = Felix
showTheNoiseAnAnimalMakes (cat.AsAnimal)

This approach gives us the best of both worlds. Pure F# types internally, but the ability to
convert them into interfaces as needed to interface with libraries.

Using reflection to examine F# types

F# gets the benefit of the .NET reflection system, which means that you can do all sorts of
interesting things that are not directly available to you using the syntax of the language itself.
The Microsoft.FSharp.Reflection namespace has a number of functions that are designed
to help specifically with F# types.
For example, here is a way to print out the fields in a record type, and the choices in a union
type.

260

Seamless interoperation with .NET libraries

open System.Reflection
open Microsoft.FSharp.Reflection
// create a record type...
type Account = {Id: int; Name: string}
// ... and show the fields
let fields =
FSharpType.GetRecordFields(typeof<Account>)
|> Array.map (fun propInfo -> propInfo.Name, propInfo.PropertyType.Name)
// create a union type...
type Choices = | A of int | B of string
// ... and show the choices
let choices =
FSharpType.GetUnionCases(typeof<Choices>)
|> Array.map (fun choiceInfo -> choiceInfo.Name)

261

Anything C# can do...

As should be apparent, you should generally try to prefer functional-style code over objectoriented code in F#, but in some situations, you may need all the features of a fully fledged
OO language ? classes, inheritance, virtual methods, etc.
So just to conclude this section, here is a whirlwind tour of the F# versions of these features.
Some of these will be dealt with in much more depth in a later series on .NET integration.
But I won't cover some of the more obscure ones, as you can read about them in the MSDN
documentation if you ever need them.

Classes and interfaces

First, here are some examples of an interface, an abstract class, and a concrete class that
inherits from the abstract class.

262

Anything C# can do...

// interface
type IEnumerator<'a> =
abstract member Current : 'a
abstract MoveNext : unit -> bool
// abstract base class with virtual methods
[<AbstractClass>]
type Shape() =
//readonly properties
abstract member Width : int with get
abstract member Height : int with get
//non-virtual method
member this.BoundingArea = this.Height * this.Width
//virtual method with base implementation
abstract member Print : unit -> unit
default this.Print () = printfn "I'm a shape"
// concrete class that inherits from base class and overrides
type Rectangle(x:int, y:int) =
inherit Shape()
override this.Width = x
override this.Height = y
override this.Print () = printfn "I'm a Rectangle"
//test
let r = Rectangle(2,3)
printfn "The width is %i" r.Width
printfn "The area is %i" r.BoundingArea
r.Print()

Classes can have multiple constructors, mutable properties, and so on.

263

Anything C# can do...

type Circle(rad:int) =
inherit Shape()
//mutable field
let mutable radius = rad
//property overrides
override this.Width = radius * 2
override this.Height = radius * 2
//alternate constructor with default radius
new() = Circle(10)
//property with get and set
member this.Radius
with get() = radius
and set(value) = radius <- value
// test constructors
let c1 = Circle() // parameterless ctor
printfn "The width is %i" c1.Width
let c2 = Circle(2) // main ctor
printfn "The width is %i" c2.Width
// test mutable property
c2.Radius <- 3
printfn "The width is %i" c2.Width

Generics
F# supports generics and all the associated constraints.
// standard generics
type KeyValuePair<'a,'b>(key:'a, value: 'b) =
member this.Key = key
member this.Value = value
// generics with constraints
type Container<'a,'b
when 'a : equality
and 'b :> System.Collections.ICollection>
(name:'a, values:'b) =
member this.Name = name
member this.Values = values

Structs
264

Anything C# can do...

F# supports not just classes, but the .NET struct types as well, which can help to boost
performance in certain cases.

type Point2D =
struct
val X: float
val Y: float
new(x: float, y: float) = { X = x; Y = y }
end
//test
let p = Point2D() // zero initialized
let p2 = Point2D(2.0,3.0) // explicitly initialized

Exceptions
F# can create exception classes, raise them and catch them.
// create a new Exception class
exception MyError of string
try
let e = MyError("Oops!")
raise e
with
| MyError msg ->
printfn "The exception error was %s" msg
| _ ->
printfn "Some other exception"

Extension methods
Just as in C#, F# can extend existing classes with extension methods.

265

Anything C# can do...

type System.String with

member this.StartsWithA = this.StartsWith "A"
//test
let s = "Alice"
printfn "'%s' starts with an 'A' = %A" s s.StartsWithA
type System.Int32 with
member this.IsEven = this % 2 = 0
//test
let i = 20
if i.IsEven then printfn "'%i' is even" i

Parameter arrays
Just like C#'s variable length "params" keyword, this allows a variable length list of
arguments to be converted to a single array parameter.
open System
type MyConsole() =
member this.WriteLine([<ParamArray>] args: Object[]) =
for arg in args do
printfn "%A" arg
let cons = new MyConsole()
cons.WriteLine("abc", 42, 3.14, true)

Events
F# classes can have events, and the events can be triggered and responded to.

266

Anything C# can do...

type MyButton() =
let clickEvent = new Event<_>()
[<CLIEvent>]
member this.OnClick = clickEvent.Publish
member this.TestEvent(arg) =
clickEvent.Trigger(this, arg)
// test
let myButton = new MyButton()
myButton.OnClick.Add(fun (sender, arg) ->
printfn "Click event with arg=%O" arg)
myButton.TestEvent("Hello World!")

Delegates
F# can do delegates.
// delegates
type MyDelegate = delegate of int -> int
let f = MyDelegate (fun x -> x * x)
let result = f.Invoke(5)

Enums
F# supports CLI enums types, which look similar to the "union" types, but are actually
different behind the scenes.
// enums
type Color = | Red=1 | Green=2 | Blue=3
let color1 = Color.Red // simple assignment
let color2:Color = enum 2 // cast from int
// created from parsing a string
let color3 = System.Enum.Parse(typeof<Color>,"Green") :?> Color // :?> is a downcast
[<System.FlagsAttribute>]
type FileAccess = | Read=1 | Write=2 | Execute=4
let fileaccess = FileAccess.Read ||| FileAccess.Write

Working with the standard user interface

267

Anything C# can do...

Finally, F# can work with the WinForms and WPF user interface libraries, just like C#.
Here is a trivial example of opening a form and handling a click event.
open System.Windows.Forms
let form = new Form(Width= 400, Height = 300, Visible = true, Text = "Hello World")
form.TopMost <- true
form.Click.Add (fun args-> printfn "the form was clicked")
form.Show()

268

Why use F#: Conclusion

This completes the tour of F# features. I hope that the examples have given you some
appreciation of the power of F# and functional programming. If you have any comments on
the series as a whole, please leave them at the bottom of this page.
In later series I hope to go deeper into data structures, pattern matching, list processing,
asynchronous and parallel programming, and much more.
But before those, I recommend you read the "thinking functionally" series, which will help
you understand functional programming at a deeper level.

269

The "Thinking Functionally" Series

This series of posts will introduce you to the fundamentals of functional programming -- what
does it really mean to "program functionally", and how this approach differs from object
oriented or imperative programming.
Thinking Functionally: Introduction. A look at the basics of functional programming.
Mathematical functions. The impetus behind functional programming.
Function Values and Simple Values. Binding not assignment.
How types work with functions. Understanding the type notation.
Currying. Breaking multi-parameter functions into smaller one-parameter functions.
Partial application. Baking-in some of the parameters of a function.
Function associativity and composition. Building new functions from existing ones.
Defining functions. Lambdas and more.
Function signatures. A function signature can give you some idea of what it does.
Organizing functions. Nested functions and modules.
Attaching functions to types. Creating methods the F# way.
Worked example: A stack based calculator. Using combinators to build functionality.

270

Thinking Functionally: Introduction

Now that you have seen some of the power of F# in the "why use F#" series, we're going to
step back and look at the fundamentals of functional programming -- what does it really
mean to "program functionally", and how this approach is different from object oriented or
imperative programming.

Changing the way you think

It is important to understand that functional programming is not just a stylistic difference; it is
a completely different way of thinking about programming, in the way that truly objectoriented programming (in Smalltalk say) is also a different way of thinking from a traditional
imperative language such as C.
F# does allow non-functional styles, and it is tempting to retain the habits you already are
familiar with. You could just use F# in a non-functional way without really changing your
mindset, and not realize what you are missing. To get the most out of F#, and to be fluent
and comfortable with functional programming in general, it is critical that you think
functionally, not imperatively. And that is the goal of this series: to help you understand
functional programming in a deep way, and help to change the way you think.
This will be a quite abstract series, although I will use lots of short code examples to
demonstrate the points. We will cover the following points:
Mathematical functions. The first post introduces the mathematical ideas behind
functional languages, and the benefits that come from this approach.
Functions and values. The next post introduces functions and values, showing how
"values" are different from variables, and why there are similarities between function
and simple values.
Types. Then we move on to the basic types that work with functions: primitive types
such as string and int; the unit type, function types, and generic types.
Functions with multiple parameters. Next, I explain the concepts of "currying" and
"partial application". This is where your brain can start to hurt, if you are coming from an
imperative background!
Defining functions. Then some posts devoted to the many different ways to define and
combine functions.
Function signatures. Then a important post on the critical topic of function signatures:
what they mean and how to use them as an aid to understanding.
Organizing functions. Once you know how to create functions, how can you organize
them to make them available to the rest of your code?
271

Thinking Functionally: Introduction

272

Mathematical functions

Mathematical functions
The impetus behind functional programming comes from mathematics. Mathematical
functions have a number of very nice features that functional languages try to emulate in the
real world.
So first, let's start with a mathematical function that adds 1 to a number.
Add1(x) = x+1

What does this really mean? Well it seems pretty straightforward. It means that there is an
operation that starts with a number, and adds one to it.
Let's introduce some terminology:
The set of values that can be used as input to the function is called the domain. In this
case, it could be the set of real numbers, but to make life simpler for now, let's restrict it
to integers only.
The set of possible output values from the function is called the range (technically, the
image on the codomain). In this case, it is also the set of integers.
The function is said to map the domain to the range.

Here's how the definition would look in F#

let add1 x = x + 1

If you type that into the F# interactive window (don't forget the double semicolons) you will
see the result (the "signature" of the function):
val add1 : int -> int

273

Mathematical functions

Let's look at that output in detail:

The overall meaning is "the function add1 maps integers (the domain) onto integers
(the range)".
" add1 " is defined as a "val", short for "value". Hmmm? what does that mean? We'll
discuss values shortly.
The arrow notation " -> " is used to show the domain and range. In this case, the
domain is the int type, and the range is also the int type.
Also note that the type was not specified, yet the F# compiler guessed that the function was
working with ints. (Can this be tweaked? Yes, as we'll see shortly).

Key properties of mathematical functions

Mathematical functions have some properties that are very different from the kinds of
functions you are used to in procedural programming.
A function always gives the same output value for a given input value
A function has no side effects.
These properties provide some very powerful benefits, and so functional programming
languages try to enforce these properties in their design as well. Let's look at each of them
in turn.

Mathematical functions always give the same output for a

given input
In imperative programming, we think that functions "do" something or "calculate" something.
A mathematical function does not do any calculation -- it is purely a mapping from input to
output. In fact, another way to think of defining a function is simply as the set of all the
mappings. For example, in a very crude way we could define the " add1 " function (in C#) as
int add1(int input)
{
switch (input)
{
case 0: return 1;
case 1: return 2;
case 2: return 3;
case 3: return 4;
etc ad infinitum
}
}

274

Mathematical functions

Obviously, we can't have a case for every possible integer, but the principle is the same. You
can see that absolutely no calculation is being done at all, just a lookup.

Mathematical functions are free from side effects

In a mathematical function, the input and the output are logically two different things, both of
which are predefined. The function does not change the input or the output -- it just maps a
pre-existing input value from the domain to a pre-existing output value in the range.
In other words, evaluating the function cannot possibly have any effect on the input, or
anything else for that matter. Remember, evaluating the function is not actually calculating or
manipulating anything; it is just a glorified lookup.
This "immutability" of the values is subtle but very important. If I am doing mathematics, I do
not expect the numbers to change underneath me when I add them! For example, if I have:
x = 5
y = x+1

I would not expect x to be changed by the adding of one to it. I would expect to get back a
different number (y) and x would be left untouched. In the world of mathematics, the integers
already exist as an unchangeable set, and the "add1" function simply defines a relationship
between them.

The power of pure functions

The kinds of functions which have repeatable results and no side effects are called "pure
functions", and you can do some interesting things with them:
They are trivially parallelizable. I could take all the integers from 1 to 1000, say, and
given 1000 different CPUs, I could get each CPU to execute the " add1 " function for the
corresponding integer at the same time, safe in the knowledge that there was no need
for any interaction between them. No locks, mutexes, semaphores, etc., needed.
I can use a function lazily, only evaluating it when I need the output. I can be sure that
the answer will be the same whether I evaluate it now or later.
I only ever need to evaluate a function once for a certain input, and I can then cache the
result, because I know that the same input always gives the same output.
If I have a number of pure functions, I can evaluate them in any order I like. Again, it
can't make any difference to the final result.
So you can see that if we can create pure functions in a programming language, we
immediately gain a lot of powerful techniques. And indeed you can do all these things in F#:

275

Mathematical functions

You have already seen an example of parallelism in the "why use F#?" series.
Evaluating functions lazily will be discussed in the "optimization" series.
Caching the results of functions is called "memoization" and will also be discussed in
the "optimization" series.
Not caring about the order of evaluation makes concurrent programming much easier,
and doesn't introduce bugs when functions are reordered or refactored.

"Unhelpful" properties of mathematical

functions
Mathematical functions also have some properties that seem not to be very helpful when
used in programming.
The input and output values are immutable
A function always has exactly one input and one output
These properties are mirrored in the design of functional programming languages too. Let's
look at each of these in turn.
The input and output values are immutable
Immutable values seem like a nice idea in theory, but how can you actually get any work
done if you can't assign to variables in a traditional way?
I can assure you that this is not as much as a problem as you might think. As you work
through this series, you'll see how this works in practice.
Mathematical functions always have exactly one input and one output
As you can see from the diagrams, there is always exactly one input and one output for a
mathematical function. This is true for functional programming languages as well, although it
may not be obvious when you first use them.
That seems like a big annoyance. How can you do useful things without having functions
with two (or more) parameters?
Well, it turns there is a way to do it, and what's more, it is completely transparent to you in
F#. It is called "currying" and it deserves its own post, which is coming up soon.
In fact, as you will later discover, these two "unhelpful" properties will turn out to be
incredibly useful and a key part of what makes functional programming so powerful.

276

Mathematical functions

277

Function Values and Simple Values

Let's look at the simple function again
let add1 x = x + 1

What does the "x" mean here? It means:

1. Accept some value from the input domain.
2. Use the name "x" to represent that value so that we can refer to it later.
This process of using a name to represent a value is called "binding". The name "x" is
"bound" to the input value.
So if we evaluate the function with the input 5 say, what is happening is that everywhere we
see "x" in the original definition, we replace it with "5", sort of like search and replace in a
word processor.
let add1 x = x + 1
add1 5
// replace "x" with "5"
// add1 5 = 5 + 1 = 6
// result is 6

It is important to understand that this is not assignment. "x" is not a "slot" or variable that is
assigned to the value and can be assigned to another value later on. It is a onetime
association of the name "x" with the value. The value is one of the predefined integers, and
cannot change. And so, once bound, x cannot change either; once associated with a value,
always associated with a value.
This concept is a critical part of thinking functionally: there are no "variables", only values.

Function values
If you think about this a bit more, you will see that the name " add1 " itself is just a binding to
"the function that adds one to its input". The function itself is independent of the name it is
bound to.

278

Function Values and Simple Values

When you type let add1 x = x + 1 you are telling the F# compiler "every time you see the
name " add1 ", replace it with the function that adds 1 to its input". " add1 " is called a
function value.
To see that the function is independent of its name, try:
let add1 x = x + 1
let plus1 = add1
add1 5
plus1 5

You can see that " add1 " and " plus1 " are two names that refer ("bound to") to the same
function.
You can always identify a function value because its signature has the standard form domain
-> range . Here is a generic function value signature:

val functionName : domain -> range

Simple values
Imagine an operation that always returned the integer 5 and didn't have any input.

This would be a "constant" operation.

How would we write this in F#? We want to tell the F# compiler "every time you see the
name c , replace it with 5". Here's how:
let c = 5

which when evaluated, returns:

val c : int = 5

279

Function Values and Simple Values

There is no mapping arrow this time, just a single int. What's new is an equals sign with the
actual value printed after it. The F# compiler knows that this binding has a known value
which it will always return, namely the value 5.
In other words, we've just defined a constant, or in F# terms, a simple value.
You can always tell a simple value from a function value because all simple values have a
signature that looks like:
val aName: type = constant // Note that there is no arrow

Simple values vs. function values

It is important to understand that in F#, unlike languages such as C#, there is very little
difference between simple values and function values. They are both values which can be
bound to names (using the same keyword let ) and then passed around. And in fact, one
of the key aspects of thinking functionally is exactly that: functions are values that can be
passed around as inputs to other functions, as we will soon see.
Note that there is a subtle difference between a simple value and a function value. A function
always has a domain and range and must be "applied" to an argument to get a result. A
simple value does not need to be evaluated after being bound. Using the example above, if
we wanted to define a "constant function" that returns five we would have to use
let c = fun()->5
// or
let c() = 5

The signature for these functions is:

val c : unit -> int

instead of:
val c : int = 5

More on unit, function syntax and anonymous functions later.

"Values" vs. "Objects"

280

Function Values and Simple Values

In a functional programming language like F#, most things are called "values". In an objectoriented language like C#, most things are called "objects". So what is the difference
between a "value" and an "object"?
A value, as we have seen above, is just a member of a domain. The domain of ints, the
domain of strings, the domain of functions that map ints to strings, and so on. In principle,
values are immutable. And values do not have any behavior attached them.
An object, in a standard definition, is an encapsulation of a data structure with its associated
behavior (methods). In general, objects are expected to have state (that is, be mutable), and
all operations that change the internal state must be provided by the object itself (via "dot"
notation).
In F#, even the primitive values have some object-like behavior. For example, you can dot
into a string to get its length:
"abc".Length

But, in general, we will avoid using "object" for standard values in F#, reserving it to refer to
instances of true classes, or other values that expose member methods.

Naming Values
Standard naming rules are used for value and function names, basically, any alphanumeric
string, including underscores. There are a couple of extras:
You can put an apostrophe anywhere in a name, except the first character. So:
A'b'c begin' // valid names

The final tick is often used to signal some sort of "variant" version of a value:
let f = x
let f' = derivative f
let f'' = derivative f'

or define variants of existing keywords

let if' b t f = if b then t else f

You can also put double backticks around any string to make a valid identifier.

281

Function Values and Simple Values

``this is a name`` ``123`` //valid names

You might want to use the double backtick trick sometimes:

When you want to use an identifier that is the same as a keyword
let ``begin`` = "begin"

When trying to use natural language for business rules, unit tests, or BDD style
executable specifications a la Cucumber.
let `ìs first time customer?`` = true
let `àdd gift to order`` = ()
if `ìs first time customer?`` then `àdd gift to order``
// Unit test
let [<Test>] ``When input is 2 then expect square is 4``=
// code here
// BDD clause
let [<Given>] `Ì have (.*) N products in my cart`` (n:int) =
// code here

Unlike C#, the naming convention for F# is that functions and values start with lowercase
letters rather than uppercase ( camelCase rather than PascalCase ) unless designed for
exposure to other .NET languages. Types and modules use uppercase however.

282

How types work with functions

Now that we have some understanding of functions, we'll look at how types work with
functions, both as domains and ranges. This is just an overview; the series "understanding
F# types" will cover types in detail.
First, we need to understand the type notation a bit more. We've seen that the arrow
notation " -> " is used to show the domain and range. So that a function signature always
looks like:
val functionName : domain -> range

Here are some example functions:

let intToString x = sprintf "x is %i" x // format int to string
let stringToInt x = System.Int32.Parse(x)

If you evaluate that in the F# interactive window, you will see the signatures:
val intToString : int -> string
val stringToInt : string -> int

This means:
intToString has a domain of int which it maps onto the range string .
stringToInt has a domain of string which it maps onto the range int .

Primitive types
The possible primitive types are what you would expect: string, int, float, bool, char, byte,
etc., plus many more derived from the .NET type system.
Here are some more examples of functions using primitive types:
let intToFloat x = float x // "float" fn. converts ints to floats
let intToBool x = (x = 2) // true if x equals 2
let stringToString x = x + " world"

and their signatures are:

283

How types work with functions

val intToFloat : int -> float

val intToBool : int -> bool
val stringToString : string -> string

Type annotations
In the previous examples, the F# compiler correctly determined the types of the parameters
and results. But this is not always the case. If you try the following code, you will get a
compiler error:
let stringLength x = x.Length
=> error FS0072: Lookup on object of indeterminate type

The compiler does not know what type "x" is, and therefore does not know if "Length" is a
valid method. In most cases, this can be fixed by giving the F# compiler a "type annotation"
so that it knows which type to use. In the corrected version below, we indicate that the type
of "x" is a string.
let stringLength (x:string) = x.Length

The parens around the x:string param are important. If they are missing, the compiler
thinks that the return value is a string! That is, an "open" colon is used to indicate the type of
the return value, as you can see in the example below.
let stringLengthAsInt (x:string) :int = x.Length

We're indicating that the x param is a string and the return value is an int.

Function types as parameters

A function that takes other functions as parameters, or returns a function, is called a higherorder function (sometimes abbreviated as HOF). They are used as a way of abstracting out
common behavior. These kinds of functions are extremely common in F#; most of the
standard libraries use them.
Consider a function evalWith5ThenAdd2 , which takes a function as a parameter, then
evaluates the function with the value 5, and adds 2 to the result:

284

How types work with functions

let evalWith5ThenAdd2 fn = fn 5 + 2 // same as fn(5) + 2

The signature of this function looks like this:

val evalWith5ThenAdd2 : (int -> int) -> int

You can see that the domain is (int->int) and the range is int . What does that mean? It
means that the input parameter is not a simple value, but a function, and what's more is
restricted only to functions that map ints to ints . The output is not a function, just an int.
Let's try it:
let add1 x = x + 1 // define a function of type (int -> int)
evalWith5ThenAdd2 add1 // test it

gives:
val add1 : int -> int
val it : int = 8

" add1 " is a function that maps ints to ints, as we can see from its signature. So it is a valid
parameter for the evalWith5ThenAdd2 function. And the result is 8.
By the way, the special word " it " is used for the last thing that was evaluated; in this case
the result we want. It's not a keyword, just a convention.
Here's another one:
let times3 x = x * 3 // a function of type (int -> int)
evalWith5ThenAdd2 times3 // test it

gives:
val times3 : int -> int
val it : int = 17

" times3 " is also a function that maps ints to ints, as we can see from its signature. So it is
also a valid parameter for the evalWith5ThenAdd2 function. And the result is 17.
Note that the input is sensitive to the types. If our input function uses floats rather than
ints , it will not work. For example, if we have:

285

How types work with functions

let times3float x = x * 3.0 // a function of type (float->float)

evalWith5ThenAdd2 times3float

Evaluating this will give an error:

error FS0001: Type mismatch. Expecting a int -> int but
given a float -> float

meaning that the input function should have been an int->int function.

Functions as output
A function value can also be the output of a function. For example, the following function will
generate an "adder" function that adds using the input value.
let adderGenerator numberToAdd = (+) numberToAdd

The signature is:

val adderGenerator : int -> (int -> int)

which means that the generator takes an int , and creates a function (the "adder") that
maps ints to ints . Let's see how it works:
let add1 = adderGenerator 1
let add2 = adderGenerator 2

This creates two adder functions. The first generated function adds 1 to its input, and the
second adds 2. Note that the signatures are just as we would expect them to be.
val add1 : (int -> int)
val add2 : (int -> int)

And we can now use these generated functions in the normal way. They are
indistinguishable from functions defined explicitly
add1 5 // val it : int = 6
add2 5 // val it : int = 7

Using type annotations to constrain function types

286

How types work with functions

In the first example, we had the function:

let evalWith5ThenAdd2 fn = fn 5 +2
=> val evalWith5ThenAdd2 : (int -> int) -> int

In this case F# could deduce that " fn " mapped ints to ints , so its signature would be
int->int

But what is the signature of "fn" in this following case?

let evalWith5 fn = fn 5

Obviously, " fn " is some kind of function that takes an int, but what does it return? The
compiler can't tell. If you do want to specify the type of the function, you can add a type
annotation for function parameters in the same way as for a primitive type.
let evalWith5AsInt (fn:int->int) = fn 5
let evalWith5AsFloat (fn:int->float) = fn 5

Alternatively, you could also specify the return type instead.

let evalWith5AsString fn :string = fn 5

Because the main function returns a string, the " fn " function is also constrained to return a
string, so no explicit typing is required for "fn".

The "unit" type

When programming, we sometimes want a function to do something without returning a
value. Consider the function " printInt ", defined below. The function doesn't actually return
anything. It just prints a string to the console as a side effect.
let printInt x = printf "x is %i" x // print to console

So what is the signature for this function?

val printInt : int -> unit

What is this " unit "?

287

How types work with functions

Well, even if a function returns no output, it still needs a range. There are no "void" functions
in mathematics-land. Every function must have some output, because a function is a
mapping, and a mapping has to have something to map to!

So in F#, functions like this return a special range called " unit ". This range has exactly one
value in it, called " () ". You can think of unit and () as somewhat like "void" (the type)
and "null" (the value) in C#. But unlike void/null, unit is a real type and () is a real value.
To see this, evaluate:
let whatIsThis = ()

and you will see the signature:

val whatIsThis : unit = ()

Which means that the value " whatIsThis " is of type unit and has been bound to the value
()

So, going back to the signature of " printInt ", we can now understand it:
val printInt : int -> unit

This signature says: printInt has a domain of int which it maps onto nothing that we
care about.

Parameterless functions
Now that we understand unit, can we predict its appearance in other contexts? For example,
let's try to create a reusable "hello world" function. Since there is no input and no output, we
would expect it to have a signature unit -> unit . Let's see:
let printHello = printf "hello world" // print to console

The result is:

288

How types work with functions

hello world
val printHello : unit = ()

Not quite what we expected. "Hello world" is printed immediately and the result is not a
function, but a simple value of type unit. As we saw earlier, we can tell that this is a simple
value because it has a signature of the form:
val aName: type = constant

So in this case, we see that printHello is actually a simple value with the value () . It's
not a function that we can call again.
Why the difference between printInt and printHello ? In the printInt case, the value
could not be determined until we knew the value of the x parameter, so the definition was of
a function. In the printHello case, there were no parameters, so the right hand side could
be determined immediately. Which it was, returning the () value, with the side effect of
printing to the console.
We can create a true reusable function that is parameterless by forcing the definition to have
a unit argument, like this:
let printHelloFn () = printf "hello world" // print to console

The signature is now:

val printHelloFn : unit -> unit

and to call it, we have to pass the () value as a parameter, like so:
printHelloFn ()

Forcing unit types with the ignore function

In some cases the compiler requires a unit type and will complain. For example, both of the
following will be compiler errors:
do 1+1 // => FS0020: This expression should have type 'unit'
let something =
2+2 // => FS0020: This expression should have type 'unit'
"hello"

289

How types work with functions

To help in these situations, there is a special function ignore that takes anything and
returns the unit type. The correct version of this code would be:
do (1+1 |> ignore) // ok
let something =
2+2 |> ignore // ok
"hello"

Generic types
In many cases, the type of the function parameter can be any type, so we need a way to
indicate this. F# uses the .NET generic type system for this situation.
For example, the following function converts the parameter to a string and appends some
text:
let onAStick x = x.ToString() + " on a stick"

It doesn't matter what type the parameter is, as all objects understand ToString() .
The signature is:
val onAStick : 'a -> string

What is this type called 'a ? That is F#'s way of indicating a generic type that is not known
at compile time. The apostrophe in front of the "a" means that the type is generic. The
signature for the C# equivalent of this would be:
string onAStick<a>();
//or more idiomatically
string OnAStick<TObject>(); // F#'s use of 'a is like
// C#'s "TObject" convention

Note that the F# function is still strongly typed with a generic type. It does not take a
parameter of type Object . This strong typing is desirable so that when functions are
composed together, type safety is still maintained.
Here's the same function being used with an int, a float and a string

290

How types work with functions

onAStick 22
onAStick 3.14159
onAStick "hello"

If there are two generic parameters, the compiler will give them different names: 'a for the
first generic, 'b for the second generic, and so on. Here's an example:
let concatString x y = x.ToString() + y.ToString()

The type signature for this has two generics: 'a and 'b :
val concatString : 'a -> 'b -> string

On the other hand, the compiler will recognize when only one generic type is required. In the
following example, the x and y parameters must be of the same type:
let isEqual x y = (x=y)

So the function signature has the same generic type for both of them:
val isEqual : 'a -> 'a -> bool

Generic parameters are also very important when it comes to lists and more abstract
structures, and we will be seeing them a lot in upcoming examples.

Other types
The types discussed so far are just the basic types. These types can be combined in various
ways to make much more complex types. A full discussion of these types will have to wait for
another series, but meanwhile, here is a brief introduction to them so that you can recognize
them in function signatures.
The "tuple" types. These are pairs, triples, etc., of other types. For example ("hello",
1) is a tuple made from a string and an int. The comma is the distinguishing

characteristic of a tuple -- if you see a comma in F#, it is almost certainly part of a tuple!
In function signatures, tuples are written as the "multiplication" of the two types involved. So
in this case, the tuple would have type:

291

How types work with functions

string * int // ("hello", 1)

The collection types. The most common of these are lists, sequences, and arrays.
Lists and arrays are fixed size, while sequences are potentially infinite (behind the
scenes, sequences are the same as IEnumerable ). In function signatures, they have
their own keywords: " list ", " seq ", and " [] " for arrays.
int list // List type e.g. [1;2;3]
string list // List type e.g. ["a";"b";"c"]
seq<int> // Seq type e.g. seq{1..10}
int [] // Array type e.g. [|1;2;3|]

The option type. This is a simple wrapper for objects that might be missing. There are
two cases: Some and None . In function signatures, they have their own " option "
keyword:
int option // Some(1)

The discriminated union type. These are built from a set of choices of other types. We
saw some examples of this in the "why use F#?" series. In function signatures, they are
referred to by the name of the type, so there is no special keyword.
The record type. These are like structures or database rows, a list of named slots. We
saw some examples of this in the "why use F#?" series as well. In function signatures,
they are referred to by the name of the type, so again there is no special keyword.

Test your understanding of types

How well do you understand the types yet? Here are some expressions for you -- see if you
can guess their signatures. To see if you are correct, just run them in the interactive window!

292

How types work with functions

let testA = float 2

let testB x = float 2
let testC x = float 2 + x
let testD x = x.ToString().Length
let testE (x:float) = x.ToString().Length
let testF x = printfn "%s" x
let testG x = printfn "%f" x
let testH = 2 * 2 |> ignore
let testI x = 2 * 2 |> ignore
let testJ (x:int) = 2 * 2 |> ignore
let testK = "hello"
let testL() = "hello"
let testM x = x=x
let testN x = x 1 // hint: what kind of thing is x?
let testO x:string = x 1 // hint: what does :string modify?

293

Currying

Currying
After that little digression on basic types, we can turn back to functions again, and in
particular the puzzle we mentioned earlier: if a mathematical function can only have one
parameter, then how is it possible that an F# function can have more than one?
The answer is quite simple: a function with multiple parameters is rewritten as a series of
new functions, each with only one parameter. And this is done automatically by the compiler
for you. It is called "currying", after Haskell Curry, a mathematician who was an important
influence on the development of functional programming.
To see how this works in practice, let's use a very basic example that prints two numbers:
//normal version
let printTwoParameters x y =
printfn "x=%i y=%i" x y

Internally, the compiler rewrites it as something more like:

//explicitly curried version
let printTwoParameters x = // only one parameter!
let subFunction y =
printfn "x=%i y=%i" x y // new function with one param
subFunction // return the subfunction

Let's examine this in more detail:

1. Construct the function called " printTwoParameters " but with only one parameter: "x"
2. Inside that, construct a subfunction that has only one parameter: "y". Note that this inner
function uses the "x" parameter but x is not passed to it explicitly as a parameter. The
"x" parameter is in scope, so the inner function can see it and use it without needing it to
be passed in.
3. Finally, return the newly created subfunction.
4. This returned function is then later used against "y". The "x" parameter is baked into it,
so the returned function only needs the y param to finish off the function logic.
By rewriting it this way, the compiler has ensured that every function has only one
parameter, as required. So when you use " printTwoParameters ", you might think that you
are using a two parameter function, but it is actually only a one parameter function! You can
see for yourself by passing only one argument instead of two:

294

Currying

// eval with one argument

printTwoParameters 1
// get back a function!
val it : (int -> unit) = <fun:printTwoParameters@286-3>

If you evaluate it with one argument, you don't get an error, you get back a function.
So what you are really doing when you call printTwoParameters with two arguments is:
You call printTwoParameters with the first argument (x)
printTwoParameters returns a new function that has "x" baked into it.

You then call the new function with the second argument (y)
Here is an example of the step by step version, and then the normal version again.
// step by step version
let x = 6
let y = 99
let intermediateFn = printTwoParameters x // return fn with
// x "baked in"
let result = intermediateFn y
// inline version of above
let result = (printTwoParameters x) y
// normal version
let result = printTwoParameters x y

Here is another example:

295

Currying

//normal version
let addTwoParameters x y =
x + y
//explicitly curried version
let addTwoParameters x = // only one parameter!
let subFunction y =
x + y // new function with one param
subFunction // return the subfunction
// now use it step by step
let x = 6
let y = 99
let intermediateFn = addTwoParameters x // return fn with
// x "baked in"
let result = intermediateFn y
// normal version
let result = addTwoParameters x y

Again, the "two parameter function" is actually a one parameter function that returns an
intermediate function.
But wait a minute -- what about the " + " operation itself? It's a binary operation that must
take two parameters, surely? No, it is curried like every other function. There is a function
called " + " that takes one parameter and returns a new intermediate function, exactly like
addTwoParameters above.

When we write the statement x+y , the compiler reorders the code to remove the infix and
turns it into (+) x y , which is the function named + called with two parameters. Note that
the function named "+" needs to have parentheses around it to indicate that it is being used
as a normal function name rather than as an infix operator.
Finally, the two parameter function named + is treated as any other two parameter function
would be.
// using plus as a single value function
let x = 6
let y = 99
let intermediateFn = (+) x // return add with x baked in
let result = intermediateFn y
// using plus as a function with two parameters
let result = (+) x y
// normal version of plus as infix operator
let result = x + y

296

Currying

And yes, this works for all other operators and built in functions like printf.
// normal version of multiply
let result = 3 * 5
// multiply as a one parameter function
let intermediateFn = (*) 3 // return multiply with "3" baked in
let result = intermediateFn 5
// normal version of printfn
let result = printfn "x=%i y=%i" 3 5
// printfn as a one parameter function
let intermediateFn = printfn "x=%i y=%i" 3 // "3" is baked in
let result = intermediateFn 5

Signatures of curried functions

Now that we know how curried functions work, what should we expect their signatures to
look like?
Going back to the first example, " printTwoParameters ", we saw that it took one argument
and returned an intermediate function. The intermediate function also took one argument
and returned nothing (that is, unit). So the intermediate function has type int->unit . In
other words, the domain of printTwoParameters is int and the range is int->unit . Putting
this together we see that the final signature is:
val printTwoParameters : int -> (int -> unit)

If you evaluate the explicitly curried implementation, you will see the parentheses in the
signature, as written above, but if you evaluate the normal implementation, which is implicitly
curried, the parentheses are left off, like so:
val printTwoParameters : int -> int -> unit

The parentheses are optional. If you are trying to make sense of function signatures it might
be helpful to add them back in mentally.
At this point you might be wondering, what is the difference between a function that returns
an intermediate function and a regular two parameter function?
Here's a one parameter function that returns a function:

297

Currying

let add1Param x = (+) x

// signature is = int -> (int -> int)

Here's a two parameter function that returns a simple value:

let add2Params x y = (+) x y
// signature is = int -> int -> int

The signatures are slightly different, but in practical terms, there is no difference*, only that
the second function is automatically curried for you.

Functions with more than two parameters

How does currying work for functions with more than two parameters? Exactly the same
way: for each parameter except the last one, the function returns an intermediate function
with the previous parameters baked in.
Consider this contrived example. I have explicitly specified the types of the parameters, but
the function itself does nothing.
let multiParamFn (p1:int)(p2:bool)(p3:string)(p4:float)=
() //do nothing
let intermediateFn1 = multiParamFn 42
// intermediateFn1 takes a bool
// and returns a new function (string -> float -> unit)
let intermediateFn2 = intermediateFn1 false
// intermediateFn2 takes a string
// and returns a new function (float -> unit)
let intermediateFn3 = intermediateFn2 "hello"
// intermediateFn3 takes a float
// and returns a simple value (unit)
let finalResult = intermediateFn3 3.141

The signature of the overall function is:

val multiParamFn : int -> bool -> string -> float -> unit

and the signatures of the intermediate functions are:

298

Currying

val intermediateFn1 : (bool -> string -> float -> unit)

val intermediateFn2 : (string -> float -> unit)
val intermediateFn3 : (float -> unit)
val finalResult : unit = ()

A function signature can tell you how many parameters the function takes: just count the
number of arrows outside of parentheses. If the function takes or returns other function
parameters, there will be other arrows in parentheses, but these can be ignored. Here are
some examples:
int->int->int // two int parameters and returns an int
string->bool->int // first param is a string, second is a bool,
// returns an int
int->string->bool->unit // three params (int,string,bool)
// returns nothing (unit)
(int->string)->int // has only one parameter, a function
// value (from int to string)
// and returns a int
(int->string)->(int->bool) // takes a function (int to string)
// returns a function (int to bool)

Issues with multiple parameters

The logic behind currying can produce some unexpected results until you understand it.
Remember that you will not get an error if you evaluate a function with fewer arguments than
it is expecting. Instead you will get back a partially applied function. If you then go on to use
this partially applied function in a context where you expect a value, you will get obscure
error messages from the compiler.
Here's an innocuous looking function:
// create a function
let printHello() = printfn "hello"

What would you expect to happen when we call it as shown below? Will it print "hello" to the
console? Try to guess before evaluating it, and here's a hint: be sure to take a look at the
function signature.

299

Currying

// call it
printHello

It will not be called as expected. The original function expects a unit argument that was not
supplied, so you are getting a partially applied function (in this case with no arguments).
How about this? Will it compile?
let addXY x y =
printfn "x=%i y=%i" x
x + y

If you evaluate it, you will see that the compiler complains about the printfn line.
printfn "x=%i y=%i" x
//^^^^^^^^^^^^^^^^^^^^^
//warning FS0193: This expression is a function value, i.e. is missing
//arguments. Its type is ^a -> unit.

If you didn't understand currying, this message would be very cryptic! All expressions that
are evaluated standalone like this (i.e. not used as a return value or bound to something with
"let") must evaluate to the unit value. And in this case, it is does not evaluate to the unit
value, but instead evaluates to a function. This is a long winded way of saying that printfn
is missing an argument.
A common case of errors like this is when interfacing with the .NET library. For example, the
ReadLine method of a TextReader must take a unit parameter. It is often easy to forget this

and leave off the parens, in which case you do not get a compiler error immediately, but only
when you try to treat the result as a string.
let reader = new System.IO.StringReader("hello");
let line1 = reader.ReadLine // wrong but compiler doesn't
// complain
printfn "The line is %s" line1 //compiler error here!
// ==> error FS0001: This expression was expected to have
// type string but here has type unit -> string
let line2 = reader.ReadLine() //correct
printfn "The line is %s" line2 //no compiler error

In the code above, line1 is just a pointer or delegate to the Readline method, not the
string that we expected. The use of () in reader.ReadLine() actually executes the
function.

300

Currying

Too many parameters

You can get similar cryptic messages when you have too many parameters as well. Here are
some examples of passing too many parameters to printf.
printfn "hello" 42
// ==> error FS0001: This expression was expected to have
// type 'a -> 'b but here has type unit
printfn "hello %i" 42 43
// ==> Error FS0001: Type mismatch. Expecting a 'a -> 'b -> 'c
// but given a 'a -> unit
printfn "hello %i %i" 42 43 44
// ==> Error FS0001: Type mismatch. Expecting a 'a->'b->'c->'d
// but given a 'a -> 'b -> unit

For example, in the last case, the compiler is saying that it expects the format argument to
have three parameters (the signature 'a -> 'b -> 'c -> 'd has three parameters) but it is
given only two (the signature 'a -> 'b -> unit has two parameters).
In cases not using printf , passing too many parameters will often mean that you end up
with a simple value that you then try to pass a parameter to. The compiler will complain that
the simple value is not a function.
let add1 x = x + 1
let x = add1 2 3
// ==> error FS0003: This value is not a function
// and cannot be applied

If you break the call into a series of explicit intermediate functions, as we did earlier, you can
see exactly what is going wrong.
let add1 x = x + 1
let intermediateFn = add1 2 //returns a simple value
let x = intermediateFn 3 //intermediateFn is not a function!
// ==> error FS0003: This value is not a function
// and cannot be applied

301

Partial application

Partial application
In the previous post on currying, we looked at breaking multiple parameter functions into
smaller one parameter functions. It is the mathematically correct way of doing it, but that is
not the only reason it is done -- it also leads to a very powerful technique called partial
function application. This is a very widely used style in functional programming, and it is
important to understand it.
The idea of partial application is that if you fix the first N parameters of the function, you get
a function of the remaining parameters. From the discussion on currying, you can probably
see how this comes about naturally.
Here are some simple examples that demonstrate this:
// create an "adder" by partial application of add
let add42 = (+) 42 // partial application
add42 1
add42 3
// create a new list by applying the add42 function
// to each element
[1;2;3] |> List.map add42
// create a "tester" by partial application of "less than"
let twoIsLessThan = (<) 2 // partial application
twoIsLessThan 1
twoIsLessThan 3
// filter each element with the twoIsLessThan function
[1;2;3] |> List.filter twoIsLessThan
// create a "printer" by partial application of printfn
let printer = printfn "printing param=%i"
// loop over each element and call the printer function
[1;2;3] |> List.iter printer

In each case, we create a partially applied function that we can then reuse in multiple
contexts.
The partial application can just as easily involve fixing function parameters, of course. Here
are some examples:

302

Partial application

// an example using List.map

let add1 = (+) 1
let add1ToEach = List.map add1 // fix the "add1" function
// test
add1ToEach [1;2;3;4]
// an example using List.filter
let filterEvens =
List.filter (fun i -> i%2 = 0) // fix the filter function
// test
filterEvens [1;2;3;4]

The following more complex example shows how the same approach can be used to create
"plug in" behavior that is transparent.
We create a function that adds two numbers, but in addition takes a logging function
that will log the two numbers and the result.
The logging function has two parameters: (string) "name" and (generic) "value", so it
has signature string->'a->unit .
We then create various implementations of the logging function, such as a console
logger or a popup logger.
And finally we partially apply the main function to create new functions that have a
particular logger baked into them.

303

Partial application

// create an adder that supports a pluggable logging function

let adderWithPluggableLogger logger x y =
logger "x" x
logger "y" y
let result = x + y
logger "x+y" result
result
// create a logging function that writes to the console
let consoleLogger argName argValue =
printfn "%s=%A" argName argValue
//create an adder with the console logger partially applied
let addWithConsoleLogger = adderWithPluggableLogger consoleLogger
addWithConsoleLogger 1 2
addWithConsoleLogger 42 99
// create a logging function that creates popup windows
let popupLogger argName argValue =
let message = sprintf "%s=%A" argName argValue
System.Windows.Forms.MessageBox.Show(
text=message,caption="Logger")
|> ignore
//create an adder with the popup logger partially applied
let addWithPopupLogger = adderWithPluggableLogger popupLogger
addWithPopupLogger 1 2
addWithPopupLogger 42 99

These functions with the logger baked in can in turn be used like any other function. For
example, we can create a partial application to add 42, and then pass that into a list function,
just like we did for the simple " add42 " function.
// create a another adder with 42 baked in
let add42WithConsoleLogger = addWithConsoleLogger 42
[1;2;3] |> List.map add42WithConsoleLogger
[1;2;3] |> List.map add42 //compare without logger

These partially applied functions are a very useful tool. We can create library functions which
are flexible (but complicated), yet make it easy to create reusable defaults so that callers
don't have to be exposed to the complexity all the time.

Designing functions for partial application

304

Partial application

You can see that the order of the parameters can make a big difference in the ease of use
for partial application. For example, most of the functions in the List library such as
List.map and List.filter have a similar form, namely:

List-function [function parameter(s)] [list]

The list is always the last parameter. Here are some examples of the full form:
List.map (fun i -> i+1) [0;1;2;3]
List.filter (fun i -> i>1) [0;1;2;3]
List.sortBy (fun i -> -i ) [0;1;2;3]

And the same examples using partial application:

let eachAdd1 = List.map (fun i -> i+1)
eachAdd1 [0;1;2;3]
let excludeOneOrLess = List.filter (fun i -> i>1)
excludeOneOrLess [0;1;2;3]
let sortDesc = List.sortBy (fun i -> -i)
sortDesc [0;1;2;3]

If the library functions were written with the parameters in a different order, it would be much
more inconvenient to use them with partial application.
As you write your own multi-parameter functions, you might wonder what the best parameter
order is. As with all design questions, there is no "right" answer to this question, but here are
some commonly accepted guidelines:
1. Put earlier: parameters more likely to be static
2. Put last: the data structure or collection (or most varying argument)
3. For well-known operations such as "subtract", put in the expected order
Guideline 1 is straightforward. The parameters that are most likely to be "fixed" with partial
application should be first. We saw this with the logger example earlier.
Guideline 2 makes it easier to pipe a structure or collection from function to function. We
have seen this many times already with list functions.

305

Partial application

// piping using list functions

let result =
[1..10]
|> List.map (fun i -> i+1)
|> List.filter (fun i -> i>5)

Similarly, partially applied list functions are easy to compose, because the list parameter
itself can be easily elided:
let compositeOp = List.map (fun i -> i+1)
>> List.filter (fun i -> i>5)
let result = compositeOp [1..10]

Wrapping BCL functions for partial application

The .NET base class library functions are easy to access in F#, but are not really designed
for use with a functional language like F#. For example, most functions have the data
parameter first, while with F#, as we have seen, the data parameter should normally come
last.
However, it is easy enough to create wrappers for them that are more idiomatic. For
example, in the snippet below, the .NET string functions are rewritten to have the string
target be the last parameter rather than the first:
// create wrappers for .NET string functions
let replace oldStr newStr (s:string) =
s.Replace(oldValue=oldStr, newValue=newStr)
let startsWith lookFor (s:string) =
s.StartsWith(lookFor)

Once the string becomes the last parameter, we can then use them with pipes in the
expected way:
let result =
"hello"
|> replace "h" "j"
|> startsWith "j"
["the"; "quick"; "brown"; "fox"]
|> List.filter (startsWith "f")

or with function composition:

306

Partial application

let compositeOp = replace "h" "j" >> startsWith "j"

let result = compositeOp "hello"

Understanding the "pipe" function

Now that you have seen how partial application works, you should be able to understand
how the "pipe" function works.
The pipe function is defined as:
let (|>) x f = f x

All it does is allow you to put the function argument in front of the function rather than after.
That's all.
let doSomething x y z = x+y+z
doSomething 1 2 3 // all parameters after function

If the function has multiple parameters, then it appears that the input is the final parameter.
Actually what is happening is that the function is partially applied, returning a function that
has a single parameter: the input
Here's the same example rewritten to use partial application
let doSomething x y =
let intermediateFn z = x+y+z
intermediateFn // return intermediateFn
let doSomethingPartial = doSomething 1 2
doSomethingPartial 3 // only one parameter after function now
3 |> doSomethingPartial // same as above - last parameter piped in

As you have already seen, the pipe operator is extremely common in F#, and used all the
time to preserve a natural flow. Here are some more usages that you might see:
"12" |> int // parses string "12" to an int
1 |> (+) 2 |> (*) 3 // chain of arithmetic

The reverse pipe function

You might occasionally see the reverse pipe function "<|" being used.

307

Partial application

let (<|) f x = f x

It seems that this function doesn't really do anything different from normal, so why does it
exist?
The reason is that, when used in the infix style as a binary operator, it reduces the need for
parentheses and can make the code cleaner.
printf "%i" 1+2 // error
printf "%i" (1+2) // using parens
printf "%i" <| 1+2 // using reverse pipe

You can also use piping in both directions at once to get a pseudo infix notation.
let add x y = x + y
(1+2) add (3+4) // error
1+2 |> add <| 3+4 // pseudo infix

308

Function associativity and composition

Function associativity
If we have a chain of functions in a row, how are they combined?
For example, what does this mean?
let F x y z = x y z

Does it mean apply the function y to the argument z, and then take the result and use it as
an argument for x? In which case it is the same as:
let F x y z = x (y z)

Or does it mean apply the function x to the argument y, and then take the resulting function
and evaluate it with the argument z? In which case it is the same as:
let F x y z = (x y) z

The answer is the latter. Function application is left associative. That is, evaluating x y z is
the same as evaluating (x y) z . And evaluating w x y z is the same as evaluating ((w
x) y) z . This should not be a surprise. We have already seen that this is how partial

application works. If you think of x as a two parameter function, then (x y) z is the result of
partial application of the first parameter, followed by passing the z argument to the
intermediate function.
If you do want to do right association, you can use explicit parentheses, or you can use a
pipe. The following three forms are equivalent.
let F x y z = x (y z)
let F x y z = y z |> x // using forward pipe
let F x y z = x <| y z // using backward pipe

As an exercise, work out the signatures for these functions without actually evaluating them!

Function composition
309

Function associativity and composition

We've mentioned function composition a number of times in passing now, but what does it
actually mean? It can seem quite intimidating at first, but it is actually quite simple.
Say that you have a function "f" that maps from type "T1" to type "T2", and say that you also
have a function "g" that maps from type "T2" to type "T3". Then you can connect the output
of "f" to the input of "g", creating a new function that maps from type "T1" to type "T3".

Here's an example
let f (x:int) = float x * 3.0 // f is int->float
let g (x:float) = x > 4.0 // g is float->bool

We can create a new function h that takes the output of "f" and uses it as the input for "g".
let h (x:int) =
let y = f(x)
g(y) // return output of g

A much more compact way is this:

let h (x:int) = g ( f(x) ) // h is int->bool
//test
h 1
h 2

So far, so straightforward. What is interesting is that we can define a new function called
"compose" that, given functions "f" and "g", combines them in this way without even knowing
their signatures.
let compose f g x = g ( f(x) )

If you evaluate this, you will see that the compiler has correctly deduced that if " f " is a
function from generic type 'a to generic type 'b , then " g " is constrained to have generic
type 'b as an input. And the overall signature is:

310

Function associativity and composition

val compose : ('a -> 'b) -> ('b -> 'c) -> 'a -> 'c

(Note that this generic composition operation is only possible because every function has
one input and one output. This approach would not be possible in a non-functional
language.)
As we have seen, the actual definition of compose uses the " >> " symbol.
let (>>) f g x = g ( f(x) )

Given this definition, we can now use composition to build new functions from existing ones.
let add1 x = x + 1
let times2 x = x * 2
let add1Times2 x = (>>) add1 times2 x
//test
add1Times2 3

This explicit style is quite cluttered. We can do a few things to make it easier to use and
understand.
First, we can leave off the x parameter so that the composition operator returns a partial
application.
let add1Times2 = (>>) add1 times2

And now we have a binary operation, so we can put the operator in the middle.
let add1Times2 = add1 >> times2

And there you have it. Using the composition operator allows code to be cleaner and more
straightforward.
let add1 x = x + 1
let times2 x = x * 2
//old style
let add1Times2 x = times2(add1 x)
//new style
let add1Times2 = add1 >> times2

311

Function associativity and composition

Using the composition operator in practice

The composition operator (like all infix operators) has lower precedence than normal
function application. This means that the functions used in composition can have arguments
without needing to use parentheses.
For example, if the "add" and "times" functions have an extra parameter, this can be passed
in during the composition.
let add n x = x + n
let times n x = x * n
let add1Times2 = add 1 >> times 2
let add5Times3 = add 5 >> times 3
//test
add5Times3 1

As long as the inputs and outputs match, the functions involved can use any kind of value.
For example, consider the following, which performs a function twice:
let twice f = f >> f //signature is ('a -> 'a) -> ('a -> 'a)

Note that the compiler has deduced that the function f must use the same type for both input
and output.
Now consider a function like " + ". As we have seen earlier, the input is an int , but the
output is actually a partially applied function (int->int) . The output of " + " can thus be
used as the input of " twice ". So we can write something like:
let add1 = (+) 1 // signature is (int -> int)
let add1Twice = twice add1 // signature is also (int -> int)
//test
add1Twice 9

On the other hand, we can't write something like:

let addThenMultiply = (+) >> (*)

because the input to "*" must be an int value, not an int->int function (which is what the
output of addition is).

312

Function associativity and composition

But if we tweak it so that the first function has an output of just int instead, then it does
work:
let add1ThenMultiply = (+) 1 >> (*)
// (+) 1 has signature (int -> int) and output is an 'int'
//test
add1ThenMultiply 2 7

Composition can also be done backwards using the " << " operator, if needed.
let times2Add1 = add 1 << times 2
times2Add1 3

Reverse composition is mainly used to make code more English-like. For example, here is a
simple example:
let myList = []
myList |> List.isEmpty |> not // straight pipeline
myList |> (not << List.isEmpty) // using reverse composition

Composition vs. pipeline

At this point, you might be wondering what the difference is between the composition
operator and the pipeline operator, as they can seem quite similar.
First let's look again at the definition of the pipeline operator:
let (|>) x f = f x

All it does is allow you to put the function argument in front of the function rather than after.
That's all. If the function has multiple parameters, then the input would be the final
parameter. Here's the example used earlier.
let doSomething x y z = x+y+z
doSomething 1 2 3 // all parameters after function
3 |> doSomething 1 2 // last parameter piped in

Composition is not the same thing and cannot be a substitute for a pipe. In the following
case the number 3 is not even a function, so its "output" cannot be fed into doSomething :

313

Function associativity and composition

3 >> doSomething 1 2 // not allowed

// f >> g is the same as g(f(x)) so rewriting it we have:
doSomething 1 2 ( 3(x) ) // implies 3 should be a function!
// error FS0001: This expression was expected to have type 'a->'b
// but here has type int

The compiler is complaining that "3" should be some sort of function 'a->'b .
Compare this with the definition of composition, which takes 3 arguments, where the first two
must be functions.
let (>>) f g x = g ( f(x) )
let add n x = x + n
let times n x = x * n
let add1Times2 = add 1 >> times 2

Trying to use a pipe instead doesn't work. In the following example, " add 1 " is a (partial)
function of type int->int , and cannot be used as the second parameter of " times 2 ".
let add1Times2 = add 1 |> times 2 // not allowed
// x |> f is the same as f(x) so rewriting it we have:
let add1Times2 = times 2 (add 1) // add1 should be an int
// error FS0001: Type mismatch. 'int -> int' does not match 'int'

The compiler is complaining that " times 2 " should take an int->int parameter, that is, be
of type (int->int)->'a .

314

Defining functions

Defining functions
We have seen how to create typical functions using the "let" syntax, below:
let add x y = x + y

In this section, we'll look at some other ways of creating functions, and tips for defining
functions.

Anonymous functions (a.k.a. lambdas)

If you are familiar with lambdas in other languages, this will not be new to you. An
anonymous function (or "lambda expression") is defined using the form:
fun parameter1 parameter2 etc -> expression

If you are used to lambdas in C# there are a couple of differences:

the lambda must have the special keyword fun , which is not needed in the C# version
the arrow symbol is a single arrow -> rather than the double arrow ( => ) in C#.
Here is a lambda that defines addition:
let add = fun x y -> x + y

This is exactly the same as a more conventional function definition:

let add x y = x + y

Lambdas are often used when you have a short expression and you don't want to define a
function just for that expression. This is particularly common with list operations, as we have
seen already.
// with separately defined function
let add1 i = i + 1
[1..10] |> List.map add1
// inlined without separately defined function
[1..10] |> List.map (fun i -> i + 1)

315

Defining functions

Note that you must use parentheses around the lambda.

Lambdas are also used when you want to make it clear that you are returning a function
from another function. For example, the " adderGenerator " function that we talked about
earlier could be rewritten with a lambda.
// original definition
let adderGenerator x = (+) x
// definition using lambda
let adderGenerator x = fun y -> x + y

The lambda version is slightly longer, but makes it clear that an intermediate function is
being returned.
You can nest lambdas as well. Here is yet another definition of adderGenerator , this time
using lambdas only.
let adderGenerator = fun x -> (fun y -> x + y)

Can you see that all three of the following definitions are the same thing?
let adderGenerator1 x y = x + y
let adderGenerator2 x = fun y -> x + y
let adderGenerator3 = fun x -> (fun y -> x + y)

If you can't see it, then do reread the post on currying. This is important stuff to understand!

Pattern matching on parameters

When defining a function, you can pass an explicit parameter, as we have seen, but you can
also pattern match directly in the parameter section. In other words, the parameter section
can contain patterns, not just identifiers!
The following example demonstrates how to use patterns in a function definition:

316

Defining functions

type Name = {first:string; last:string} // define a new type

This kind of matching can only occur when the matching is always possible. For example,
you cannot match on union types or lists this way, because some cases might not be
matched.
let f3 (x::xs) = // use pattern matching on a list
printfn "first element is=%A" x

You will get a warning about incomplete pattern matches.

A common mistake: tuples vs. multiple

parameters
If you come from a C-like language, a tuple used as a single function parameter can look
awfully like multiple parameters. They are not the same thing at all! As I noted earlier, if you
see a comma, it is probably part of a tuple. Parameters are separated by spaces.
Here is an example of the confusion:

317

Defining functions

// a function that takes two distinct parameters

let addTwoParams x y = x + y
// a function that takes a single tuple parameter
let addTuple aTuple =
let (x,y) = aTuple
x + y
// another function that takes a single tuple parameter
// but looks like it takes two ints
let addConfusingTuple (x,y) = x + y

The first definition, " addTwoParams ", takes two parameters, separated with spaces.
The second definition, " addTuple ", takes a single parameter. It then binds "x" and "y" to
the inside of the tuple and does the addition.
The third definition, " addConfusingTuple ", takes a single parameter just like " addTuple ",
but the tricky thing is that the tuple is unpacked and bound as part of the parameter
definition using pattern matching. Behind the scenes, it is exactly the same as
" addTuple ".
Let's look at the signatures (it is always a good idea to look at the signatures if you are
unsure)
val addTwoParams : int -> int -> int // two params
val addTuple : int * int -> int // tuple->int
val addConfusingTuple : int * int -> int // tuple->int

Now let's use them:

//test
addTwoParams 1 2 // ok - uses spaces to separate args
addTwoParams (1,2) // error trying to pass a single tuple
// => error FS0001: This expression was expected to have type
// int but here has type 'a * 'b

Here we can see an error occur in the second case above.

First, the compiler treats (1,2) as a generic tuple of type ('a * 'b) , which it attempts to
pass as the first parameter to " addTwoParams ". Then it complains that the first parameter of
addTwoParams is an int , and we're trying to pass a tuple.

To make a tuple, use a comma! Here's how to do it correctly:

318

Defining functions

addTuple (1,2) // ok
addConfusingTuple (1,2) // ok
let x = (1,2)
addTuple x // ok
let y = 1,2 // it's the comma you need,
// not the parentheses!
addTuple y // ok
addConfusingTuple y // ok

Conversely, if you attempt to pass multiple arguments to a function expecting a tuple, you
will also get an obscure error.
addConfusingTuple 1 2 // error trying to pass two args
// => error FS0003: This value is not a function and
// cannot be applied

In this case, the compiler thinks that, since you are passing two arguments,
addConfusingTuple must be curryable. So then " addConfusingTuple 1 " would be a partial

application that returns another intermediate function. Trying to apply that intermediate
function with "2" gives an error, because there is no intermediate function! We saw this exact
same error in the post on currying, when we discussed the issues that can occur from
having too many parameters.

Why not use tuples as parameters?

The discussion of the issues with tuples above shows that there's another way to define
functions with more than one parameter: rather than passing them in separately, all the
parameters can be combined into a single composite data structure. In the example below,
the function takes a single parameter, which is a tuple containing three items.
let f (x,y,z) = x + y * z
// type is int * int * int -> int
// test
f (1,2,3)

Note that the function signature is different from a true three parameter function. There is
only one arrow, so only one parameter, and the stars indicate that this is a tuple of
(int*int*int) .

When would we want to use tuple parameters instead of individual ones?

319

Defining functions

When the tuples are meaningful in themselves. For example, if we are working with
three dimensional coordinates, a three-tuple might well be more convenient than three
separate dimensions.
Tuples are occasionally used to bundle data together in a single structure that should be
kept together. For example, the TryParse functions in .NET library return the result and
a Boolean as a tuple. But if you have a lot of data that is kept together as a bundle, then
you will probably want to define a record or class type to store it.

A special case: tuples and .NET library functions

One area where commas are seen a lot is when calling .NET library functions!
These all take tuple-like arguments, and so these calls look just the same as they would
from C#:
// correct
System.String.Compare("a","b")
// incorrect
System.String.Compare "a" "b"

The reason is that .NET library functions are not curried and cannot be partially applied. All
the parameters must always be passed in, and using a tuple-like approach is the obvious
way to do this.
But do note that although these calls look like tuples, they are actually a special case. Real
tuples cannot be used, so the following code is invalid:
let tuple = ("a","b")
System.String.Compare tuple // error
System.String.Compare "a","b" // error

If you do want to partially apply .NET library functions, it is normally trivial to write wrapper
functions for them, as we have seen earlier, and as shown below:

320

Defining functions

// create a wrapper function

let strCompare x y = System.String.Compare(x,y)
// partially apply it
let strCompareWithB = strCompare "B"
// use it with a higher order function
["A";"B";"C"]
|> List.map strCompareWithB

Guidelines for separate vs. grouped

parameters
The discussion on tuples leads us to a more general topic: when should function parameters
be separate and when should they be grouped?
Note that F# is different from C# in this respect. In C# all the parameters are always
provided, so the question does not even arise! In F#, due to partial application, only some
parameters might be provided, so you need to distinguish between those that are required to
be grouped together vs. those that are independent.
Here are some general guidelines of how to structure parameters when you are designing
your own functions.
In general, it is always better to use separate parameters rather than passing them as a
single structure such as a tuple or record. This allows for more flexible behavior such as
partial application.
But, when a group of parameters must all be set at once, then do use some sort of
grouping mechanism.
In other words, when designing a function, ask yourself "could I provide this parameter in
isolation?" If the answer is no, the parameters should be grouped.
Let's look at some examples:

321

Defining functions

// Pass in two numbers for addition.

// The numbers are independent, so use two parameters
let add x y = x + y
// Pass in two numbers as a geographical co-ordinate.
// The numbers are dependent, so group them into a tuple or record
let locateOnMap (xCoord,yCoord) = // do something
// Set first and last name for a customer.
// The values are dependent, so group them into a record.
type CustomerName = {First:string; Last:string}
let setCustomerName aCustomerName = // good
let setCustomerName first last = // not recommended
// Set first and last name and and pass the
// authorizing credentials as well.
// The name and credentials are independent, keep them separate
let setCustomerName myCredentials aName = //good

Finally, do be sure to order the parameters appropriately to assist with partial application
(see the guidelines in the earlier post). For example, in the last function above, why did I put
the myCredentials parameter ahead of the aName parameter?

Parameter-less functions
Sometimes we may want functions that don't take any parameters at all. For example, we
may want a "hello world" function that we can call repeatedly. As we saw in a previous
section, the naive definition will not work.
let sayHello = printfn "Hello World!" // not what we want

The fix is to add a unit parameter to the function, or use a lambda.

let sayHello() = printfn "Hello World!" // good
let sayHello = fun () -> printfn "Hello World!" // good

And then the function must always be called with a unit argument:
// call it
sayHello()

This is particularly common with the .NET libraries. Some examples are:

322

Defining functions

Console.ReadLine()
System.Environment.GetCommandLineArgs()
System.IO.Directory.GetCurrentDirectory()

Do remember to call them with the unit parameter!

Defining new operators

You can define functions named using one or more of the operator symbols (see the F#
documentation for the exact list of symbols that you can use):
// define
let (.*%) x y = x + y + 1

You must use parentheses around the symbols when defining them.
Note that for custom operators that begin with * , a space is required; otherwise the (* is
interpreted as the start of a comment:
let ( *+* ) x y = x + y + 1

Once defined, the new function can be used in the normal way, again with parens around
the symbols:
let result = (.*%) 2 3

If the function has exactly two parameters, you can use it as an infix operator without
parentheses.
let result = 2 .*% 3

You can also define prefix operators that start with ! or ~ (with some restrictions -- see
the F# documentation on operator overloading)
let (~%%) (s:string) = s.ToCharArray()
//use
let result = %% "hello"

323

Defining functions

In F# it is quite common to create your own operators, and many libraries will export
operators with names such as >=> and <*> .

Point-free style
We have already seen many examples of leaving off the last parameter of functions to
reduce clutter. This style is referred to as point-free style or tacit programming.
Here are some examples:
let add x y = x + y // explicit
let add x = (+) x // point free
let add1Times2 x = (x + 1) * 2 // explicit
let add1Times2 = (+) 1 >> (*) 2 // point free
let sum list = List.reduce (fun sum e -> sum+e) list // explicit
let sum = List.reduce (+) // point free

There are pros and cons to this style.

On the plus side, it focuses attention on the high level function composition rather than the
low level objects. For example " (+) 1 >> (*) 2 " is clearly an addition operation followed by
a multiplication. And " List.reduce (+) " makes it clear that the plus operation is key, without
needing to know about the list it is actually applied to.
Point-free helps to clarify the underlying algorithm and reveal commonalities between code - the " reduce " function used above is a good example of this -- it will be discussed in a
planned series on list processing.
On the other hand, too much point-free style can make for confusing code. Explicit
parameters can act as a form of documentation, and their names (such as "list") make it
clear what the function is acting on.
As with anything in programming, the best guideline is to use the approach that provides the
most clarity.

Combinators
The word "combinator" is used to describe functions whose result depends only on their
parameters. That means there is no dependency on the outside world, and in particular no
other functions or global value can be accessed at all.

324

Defining functions

In practice, this means that a combinator function is limited to combining its parameters in
various ways.
We have already seen some combinators already: the "pipe" operator and the "compose"
operator. If you look at their definitions, it is clear that all they do is reorder the parameters in
various ways
let (|>) x f = f x // forward pipe
let (<|) f x = f x // reverse pipe
let (>>) f g x = g (f x) // forward composition
let (<<) g f x = g (f x) // reverse composition

On the other hand, a function like "printf", although primitive, is not a combinator, because it
has a dependency on the outside world (I/O).

Combinator birds
Combinators are the basis of a whole branch of logic (naturally called "combinatory logic")
that was invented many years before computers and programming languages. Combinatory
logic has had a very large influence on functional programming.
To read more about combinators and combinatory logic, I recommend the book "To Mock a
Mockingbird" by Raymond Smullyan. In it, he describes many other combinators and
whimsically gives them names of birds. Here are some examples of some standard
combinators and their bird names:
let I x = x // identity function, or the Idiot bird
let K x y = x // the Kestrel
let M x = x >> x // the Mockingbird
let T x y = y x // the Thrush (this looks familiar!)
let Q x y z = y (x z) // the Queer bird (also familiar!)
let S x y z = x z (y z) // The Starling
// and the infamous...
let rec Y f x = f (Y f) x // Y-combinator, or Sage bird

The letter names are quite standard, so if you refer to "the K combinator", everyone will be
familiar with that terminology.
It turns out that many common programming patterns can be represented using these
standard combinators. For example, the Kestrel is a common pattern in fluent interfaces
where you do something but then return the original object. The Thrush is the pipe
operation, the Queer bird is forward composition, and the Y-combinator is famously used to
make functions recursive.

325

Defining functions

Indeed, there is a well-known theorem that states that any computable function whatsoever
can be built from just two basic combinators, the Kestrel and the Starling.

Combinator libraries
A combinator library is a code library that exports a set of combinator functions that are
designed to work together. The user of the library can then easily combine simple functions
together to make bigger and more complex functions, like building with Lego.
A well designed combinator library allows you to focus on the high level operations, and
push the low level "noise" to the background. We've already seen some examples of this
power in the examples in "why use F#" series, and the List module is full of them -- the
" fold " and " map " functions are also combinators, if you think about it.
Another advantage of combinators is that they are the safest type of function. As they have
no dependency on the outside world they cannot change if the global environment changes.
A function that reads a global value or uses a library function can break or alter between
calls if the context is different. This can never happen with combinators.
In F#, combinator libraries are available for parsing (the FParsec library), HTML
construction, testing frameworks, and more. We'll discuss and use combinators further in
later series.

Recursive functions
Often, a function will need to refer to itself in its body. The classic example is the Fibonacci
function:
let fib i =
match i with
| 1 -> 1
| 2 -> 1
| n -> fib(n-1) + fib(n-2)

Unfortunately, this will not compile:

error FS0039: The value or constructor 'fib' is not defined

You have to tell the compiler that this is a recursive function using the rec keyword.

326

Defining functions

let rec fib i =

match i with
| 1 -> 1
| 2 -> 1
| n -> fib(n-1) + fib(n-2)

Recursive functions and data structures are extremely common in functional programming,
and I hope to devote a whole later series to this topic.

327

Function signatures

Function signatures
It may not be obvious, but F# actually has two syntaxes - one for normal (value)
expressions, and one for type definitions. For example:
[1;2;3] // a normal expression
int list // a type expression
Some 1 // a normal expression
int option // a type expression
(1,"a") // a normal expression
int * string // a type expression

Type expressions have a special syntax that is different from the syntax used in normal
expressions. You have already seen many examples of this when you use the interactive
session, because the type of each expression has been printed along with its evaluation.
As you know, F# uses type inference to deduce types, so you don't often need to explicitly
specify types in your code, especially for functions. But in order to work effectively in F#, you
do need to understand the type syntax, so that you can build your own types, debug type
errors, and understand function signatures. In this post, we'll focus on its use in function
signatures.
Here are some example function signatures using the type syntax:
// expression syntax // type syntax
let add1 x = x + 1 // int -> int
let add x y = x + y // int -> int -> int
let print x = printf "%A" x // 'a -> unit
System.Console.ReadLine // unit -> string
List.sum // 'a list -> 'a
List.filter // ('a -> bool) -> 'a list -> 'a list
List.map // ('a -> 'b) -> 'a list -> 'b list

Understanding functions through their

signatures
Just by examining a function's signature, you can often get some idea of what it does. Let's
look at some examples and analyze them in turn.

328

Function signatures

// function signature 1
int -> int -> int

This function takes two int parameters and returns another, so presumably it is some sort
of mathematical function such as addition, subtraction, multiplication, or exponentiation.
// function signature 2
int -> unit

This function takes an int and returns a unit , which means that the function is doing
something important as a side-effect. Since there is no useful return value, the side effect is
probably something to do with writing to IO, such as logging, writing to a file or database, or
something similar.
// function signature 3
unit -> string

This function takes no input but returns a string , which means that the function is
conjuring up a string out of thin air! Since there is no explicit input, the function probably has
something to do with reading (from a file say) or generating (a random string, say).
// function signature 4
int -> (unit -> string)

This function takes an int input and returns a function that when called, returns strings.
Again, the function probably has something to do with reading or generating. The input
probably initializes the returned function somehow. For example, the input could be a file
handle, and the returned function something like readline() . Or the input could be a seed
for a random string generator. We can't tell exactly, but we can make some educated
guesses.
// function signature 5
'a list -> 'a

This function takes a list of some type, but returns only one of that type, which means that
the function is merging or choosing elements from the list. Examples of functions with this
signature are List.sum , List.max , List.head and so on.
// function signature 6
('a -> bool) -> 'a list -> 'a list

329

Function signatures

This function takes two parameters: the first is a function that maps something to a bool (a
predicate), and the second is a list. The return value is a list of the same type. Predicates
are used to determine whether a value meets some sort of criteria, so it looks like the
function is choosing elements from the list based on whether the predicate is true or not and
then returning a subset of the original list. A typical function with this signature is
List.filter .

// function signature 7
('a -> 'b) -> 'a list -> 'b list

This function takes two parameters: the first maps type 'a to type 'b , and the second is a
list of 'a . The return value is a list of a different type 'b . A reasonable guess is that the
function takes each of the 'a s in the list, maps them to a 'b using the function passed in
as the first parameter, and returns the new list of 'b s. And indeed, the prototypical function
with this signature is List.map .

Using function signatures to find a library method

Function signatures are an important part of searching for library functions. The F# libraries
have hundreds of functions in them and they can initially be overwhelming. Unlike an object
oriented language, you cannot simply "dot into" an object to find all the appropriate methods.
However, if you know the signature of the function you are looking for, you can often narrow
down the list of candidates quickly.
For example, let's say you have two lists and you are looking for a function to combine them
into one. What would the signature be for this function? It would take two list parameters and
return a third, all of the same type, giving the signature:
'a list -> 'a list -> 'a list

Now go to the MSDN documentation for the F# List module, and scan down the list of
functions, looking for something that matches. As it happens, there is only one function with
that signature:
append : 'T list -> 'T list -> 'T list

which is exactly the one we want!

330

Function signatures

Defining your own types for function

signatures
Sometimes you may want to create your own types to match a desired function signature.
You can do this using the "type" keyword, and define the type in the same way that a
signature is written:
type Adder = int -> int
type AdderGenerator = int -> Adder

You can then use these types to constrain function values and parameters.
For example, the second definition below will fail because of type constraints. If you remove
the type constraint (as in the third definition) there will not be any problem.
let a:AdderGenerator = fun x -> (fun y -> x + y)
let b:AdderGenerator = fun (x:float) -> (fun y -> x + y)
let c = fun (x:float) -> (fun y -> x + y)

Test your understanding of function signatures

How well do you understand function signatures? See if you can create simple functions that
have each of these signatures. Avoid using explicit type annotations!
val testA = int -> int
val testB = int -> int -> int
val testC = int -> (int -> int)
val testD = (int -> int) -> int
val testE = int -> int -> int -> int
val testF = (int -> int) -> (int -> int)
val testG = int -> (int -> int) -> int
val testH = (int -> int -> int) -> int

331

Organizing functions

Organizing functions
Now that you know how to define functions, how can you organize them?
In F#, there are three options:
functions can be nested inside other functions.
at an application level, the top level functions are grouped into "modules".
alternatively, you can also use the object-oriented approach and attach functions to
types as methods.
We'll look at the first two options in this post, and the third in the next post.

Nested Functions
In F#, you can define functions inside other functions. This is a great way to encapsulate
"helper" functions that are needed for the main function but shouldn't be exposed outside.
In the example below add is nested inside addThreeNumbers :
let addThreeNumbers x y z =
//create a nested helper function
let add n =
fun x -> x + n
// use the helper function
x |> add y |> add z
// test
addThreeNumbers 2 3 4

A nested function can access its parent function parameters directly, because they are in
scope. So, in the example below, the printError nested function does not need to have
any parameters of its own -- it can access the n and max values directly.

332

Organizing functions

let validateSize max n =

//create a nested helper function with no params
let printError() =
printfn "Oops: '%i' is bigger than max: '%i'" n max
// use the helper function
if n > max then printError()
// test
validateSize 10 9
validateSize 10 11

A very common pattern is that the main function defines a nested recursive helper function,
and then calls it with the appropriate initial values. The code below is an example of this:
let sumNumbersUpTo max =
// recursive helper function with accumulator
let rec recursiveSum n sumSoFar =
match n with
| 0 -> sumSoFar
| _ -> recursiveSum (n-1) (n+sumSoFar)
// call helper function with initial values
recursiveSum max 0
// test
sumNumbersUpTo 10

When nesting functions, do try to avoid very deeply nested functions, especially if the nested
functions directly access the variables in their parent scopes rather than having parameters
passed to them. A badly nested function will be just as confusing as the worst kind of deeply
nested imperative branching.
Here's how not to do it:

333

Organizing functions

// wtf does this function do?

let f x =
let f2 y =
let f3 z =
x * z
let f4 z =
let f5 z =
y * z
let f6 () =
y * x
f6()
f4 y
x * f2 x

Modules
A module is just a set of functions that are grouped together, typically because they work on
the same data type or types.
A module definition looks very like a function definition. It starts with the module keyword,
then an = sign, and then the contents of the module are listed. The contents of the module
must be indented, just as expressions in a function definition must be indented.
Here's a module that contains two functions:
module MathStuff =
let add x y = x + y
let subtract x y = x - y

Now if you try this in Visual Studio, and you hover over the add function, you will see that
the full name of the add function is actually MathStuff.add , just as if MathStuff was a
class and add was a method.
Actually, that's exactly what is going on. Behind the scenes, the F# compiler creates a static
class with static methods. So the C# equivalent would be:

334

Organizing functions

static class MathStuff

{
static public int add(int x, int y)
{
return x + y;
}
static public int subtract(int x, int y)
{
return x - y;
}
}

If you realize that modules are just static classes, and that functions are static methods, then
you will already have a head-start on understanding how modules work in F#, as most of the
rules that apply to static classes also apply to modules.
And, just as in C# every standalone function must be part of a class, in F# every standalone
function must be part of a module.

Accessing functions across module boundaries

If you want to access a function in another module, you can refer to it by its qualified name.
module MathStuff =
let add x y = x + y
let subtract x y = x - y
module OtherStuff =
// use a function from the MathStuff module
let add1 x = MathStuff.add x 1

You can also import all the functions in another module with the open directive, after which
you can use the short name, rather than having to specify the qualified name.
module OtherStuff =
open MathStuff // make all functions accessible
let add1 x = add x 1

The rules for using qualified names are exactly as you would expect. That is, you can always
use a fully qualified name to access a function, and you can use relative names or
unqualified names based on what other modules are in scope.

335

Organizing functions

Nested modules
Just like static classes, modules can contain child modules nested within them, as shown
below:
module MathStuff =
let add x y = x + y
let subtract x y = x - y
// nested module
module FloatLib =
let add x y :float = x + y
let subtract x y :float = x - y

And other modules can reference functions in the nested modules using either a full name or
a relative name as appropriate:
module OtherStuff =
open MathStuff
let add1 x = add x 1
// fully qualified
let add1Float x = MathStuff.FloatLib.add x 1.0
//with a relative path
let sub1Float x = FloatLib.subtract x 1.0

Top level modules

So if there can be nested child modules, that implies that, going back up the chain, there
must always be some top-level parent module. This is indeed true.
Top level modules are defined slightly differently than the modules we have seen so far.
The module MyModuleName line must be the first declaration in the file
There is no = sign
The contents of the module are not indented
In general, there must be a top level module declaration present in every .FS source file.
There some exceptions, but it is good practice anyway. The module name does not have to
be the same as the name of the file, but two files cannot share the same module name.

336

Organizing functions

For .FSX script files, the module declaration is not needed, in which case the module name
is automatically set to the filename of the script.
Here is an example of MathStuff declared as a top level module:
// top level module
module MathStuff
let add x y = x + y
let subtract x y = x - y
// nested module
module FloatLib =
let add x y :float = x + y
let subtract x y :float = x - y

Note the lack of indentation for the top level code (the contents of module MathStuff ), but
that the content of a nested module like FloatLib does still need to be indented.

doesn't get contaminated with previous evaluations

Shadowing
Here's our example module again. Notice that MathStuff has an add function and
FloatLib also has an add function.

module MathStuff =
let add x y = x + y
let subtract x y = x - y
// nested module
module FloatLib =
let add x y :float = x + y
let subtract x y :float = x - y

Now what happens if I bring both of them into scope, and then use add ?
open MathStuff
open MathStuff.FloatLib
let result = add 1 2 // Compiler error: This expression was expected to
// have type float but here has type int

What happened was that the MathStuff.FloatLib module has masked or overridden the
original MathStuff module, which has been "shadowed" by FloatLib .
As a result you now get a FS0001 compiler error because the first parameter 1 is expected
to be a float. You would have to change 1 to 1.0 to fix this.
Unfortunately, this is invisible and easy to overlook. Sometimes you can do cool tricks with
this, almost like subclassing, but more often, it can be annoying if you have functions with
the same name (such as the very common map ).
If you don't want this to happen, there is a way to stop it by using the
RequireQualifiedAccess attribute. Here's the same example where both modules are

decorated with it.

338

Organizing functions

[<RequireQualifiedAccess>]
module MathStuff =
let add x y = x + y
let subtract x y = x - y
// nested module
[<RequireQualifiedAccess>]
module FloatLib =
let add x y :float = x + y
let subtract x y :float = x - y

Now the open isn't allowed:

open MathStuff // error
open MathStuff.FloatLib // error

But we can still access the functions (without any ambiguity) via their qualified name:
let result = MathStuff.add 1 2
let result = MathStuff.FloatLib.add 1.0 2.0

Access Control
F# supports the use of standard .NET access control keywords such as public , private ,
and internal . The MSDN documentation has the complete details.
These access specifiers can be put on the top-level ("let bound") functions, values,
types and other declarations in a module. They can also be specified for the modules
themselves (you might want a private nested module, for example).
Everything is public by default (with a few exceptions) so you will need to use private
or internal if you want to protect them.
These access specifiers are just one way of doing access control in F#. Another completely
different way is to use module "signature" files, which are a bit like C header files. They
describe the content of the module in an abstract way. Signatures are very useful for doing
serious encapsulation, but that discussion will have to wait for the planned series on
encapsulation and capability based security.

Namespaces

339

Organizing functions

Namespaces in F# are similar to namespaces in C#. They can be used to organize modules
and types to avoid name collisions.
A namespace is declared with a namespace keyword, as shown below.
namespace Utilities
module MathStuff =
// functions
let add x y = x + y
let subtract x y = x - y

Because of this namespace, the fully qualified name of the MathStuff module now
becomes Utilities.MathStuff and the fully qualified name of the add function now
becomes Utilities.MathStuff.add .
With the namespace, the indentation rules apply, so that the module defined above must
have its content indented, as it it were a nested module.
You can also declare a namespace implicitly by adding dots to the module name. That is,
the code above could also be written as:
module Utilities.MathStuff
// functions
let add x y = x + y
let subtract x y = x - y

The fully qualified name of the MathStuff module is still Utilities.MathStuff , but in this
case, the module is a top-level module and the contents do not need to be indented.
Some additional things to be aware of when using namespaces:
Namespaces are optional for modules. And unlike C#, there is no default namespace
for an F# project, so a top level module without a namespace will be at the global level.
If you are planning to create reusable libraries, be sure to add some sort of namespace
to avoid naming collisions with code in other libraries.
Namespaces can directly contain type declarations, but not function declarations. As
noted earlier, all function and value declarations must be part of a module.
Finally, be aware that namespaces don't work well in scripts. For example, if you try to
to send a namespace declaration such as namespace Utilities below to the interactive
window, you will get an error.

Namespace hierarchies
340

Organizing functions

You can create a namespace hierarchy by simply separating the names with periods:
namespace Core.Utilities
module MathStuff =
let add x y = x + y

And if you want to put two namespaces in the same file, you can. Note that all namespaces
must be fully qualified -- there is no nesting.
namespace Core.Utilities
module MathStuff =
let add x y = x + y
namespace Core.Extra
module MoreMathStuff =
let add x y = x + y

One thing you can't do is have a naming collision between a namespace and a module.
namespace Core.Utilities
module MathStuff =
let add x y = x + y
namespace Core
// fully qualified name of module
// is Core.Utilities
// Collision with namespace above!
module Utilities =
let add x y = x + y

Mixing types and functions in modules

We've seen that a module typically consists of a set of related functions that act on a data
type.
In an object oriented program, the data structure and the functions that act on it would be
combined in a class. However in functional-style F#, a data structure and the functions that
act on it are combined in a module instead.
There are two common patterns for mixing types and functions together:

341

Organizing functions

having the type declared separately from the functions

having the type declared in the same module as the functions
In the first approach, the type is declared outside any module (but in a namespace) and then
the functions that work on the type are put in a module with a similar name.
// top-level module
namespace Example
// declare the type outside the module
type PersonType = {First:string; Last:string}
// declare a module for functions that work on the type
module Person =
// constructor
let create first last =
{First=first; Last=last}
// method that works on the type
let fullName {First=first; Last=last} =
first + " " + last
// test
let person = Person.create "john" "doe"
Person.fullName person |> printfn "Fullname=%s"

In the alternative approach, the type is declared inside the module and given a simple name
such as " T " or the name of the module. So the functions are accessed with names like
MyModule.Func1 and MyModule.Func2 while the type itself is accessed with a name like
MyModule.T . Here's an example:

module Customer =
// Customer.T is the primary type for this module
type T = {AccountId:int; Name:string}
// constructor
let create id name =
{T.AccountId=id; T.Name=name}
// method that works on the type
let isValid {T.AccountId=id; } =
id > 0
// test
let customer = Customer.create 42 "bob"
Customer.isValid customer |> printfn "Is valid?=%b"

342

Organizing functions

Note that in both cases, you should have a constructor function that creates new instances
of the type (a factory method, if you will), Doing this means that you will rarely have to
explicitly name the type in your client code, and therefore, you not should not care whether it
lives in the module or not!
So which approach should you choose?
The former approach is more .NET like, and much better if you want to share your
libraries with other non-F# code, as the exported class names are what you would
expect.
The latter approach is more common for those used to other functional languages. The
type inside a module compiles into nested classes, which is not so nice for interop.
For yourself, you might want to experiment with both. And in a team programming situation,
you should choose one style and be consistent.

Modules containing types only

If you have a set of types that you need to declare without any associated functions, don't
bother to use a module. You can declare types directly in a namespace and avoid nested
classes.
For example, here is how you might think to do it:
// top-level module
module Example
// declare the type inside a module
type PersonType = {First:string; Last:string}
// no functions in the module, just types...

And here is a alternative way to do it. The module keyword has simply been replaced with
namespace .

// use a namespace
namespace Example
// declare the type outside any module
type PersonType = {First:string; Last:string}

In both cases, PersonType will have the same fully qualified name.
Note that this only works with types. Functions must always live in a module.

343

Organizing functions

344

Attaching functions to types

Although we have focused on the pure functional style so far, sometimes it is convenient to
switch to an object oriented style. And one of the key features of the OO style is the ability to
attach functions to a class and "dot into" the class to get the desired behavior.
In F#, this is done using a feature called "type extensions". And any F# type, not just
classes, can have functions attached to them.
Here's an example of attaching a function to a record type.
module Person =
type T = {First:string; Last:string} with
// member defined with type declaration
member this.FullName =
this.First + " " + this.Last
// constructor
let create first last =
{First=first; Last=last}
// test
let person = Person.create "John" "Doe"
let fullname = person.FullName

The key things to note are:

The with keyword indicates the start of the list of members
The member keyword shows that this is a member function (i.e. a method)
The word this is a placeholder for the object that is being dotted into (called a "selfidentifier"). The placeholder prefixes the function name, and then the function body then
uses the same placeholder when it needs to refer to the current instance. There is no
requirement to use a particular word, just as long as it is consistent. You could use
this or self or me or any other word that commonly indicates a self reference.

You don't have to add a member at the same time that you declare the type, you can always
add it later in the same module:

345

Attaching functions to types

module Person =
type T = {First:string; Last:string} with
// member defined with type declaration
member this.FullName =
this.First + " " + this.Last
// constructor
let create first last =
{First=first; Last=last}
// another member added later
type T with
member this.SortableName =
this.Last + ", " + this.First
// test
let person = Person.create "John" "Doe"
let fullname = person.FullName
let sortableName = person.SortableName

These examples demonstrate what are called "intrinsic extensions". They are compiled into
the type itself and are always available whenever the type is used. They also show up when
you use reflection.
With intrinsic extensions, it is even possible to have a type definition that divided across
several files, as long as all the components use the same namespace and are all compiled
into the same assembly. Just as with partial classes in C#, this can be useful to separate
generated code from authored code.

Optional extensions
Another alternative is that you can add an extra member from a completely different module.
These are called "optional extensions". They are not compiled into the type itself, and
require some other module to be in scope for them to work (this behavior is just like C#
extension methods).
For example, let's say we have a Person type defined:

346

Attaching functions to types

The example below demonstrates how to add an UppercaseName extension to it in a different

module:
// in a different module
module PersonExtensions =
type Person.T with
member this.UppercaseName =
this.FullName.ToUpper()

So now let's test this extension:

let person = Person.create "John" "Doe"
let uppercaseName = person.UppercaseName

Uh-oh, we have an error. What's wrong is that the PersonExtensions is not in scope. Just as
for C#, any extensions have to be brought into scope in order to be used.
Once we do that, everything is fine:
// bring the extension into scope first!
open PersonExtensions
let person = Person.create "John" "Doe"
let uppercaseName = person.UppercaseName

Extending system types

347

Attaching functions to types

You can extend types that are in the .NET libraries as well. But be aware that when
extending a type, you must use the actual type name, not a type abbreviation.
For example, if you try to extend int , you will fail, because int is not the true name of the
type:
type int with
member this.IsEven = this % 2 = 0

You must use System.Int32 instead:

type System.Int32 with
member this.IsEven = this % 2 = 0
//test
let i = 20
if i.IsEven then printfn "'%i' is even" i

Static members
You can make the member functions static by:
adding the keyword static
dropping the this placeholder
module Person =
type T = {First:string; Last:string} with
// member defined with type declaration
member this.FullName =
this.First + " " + this.Last
// static constructor
static member Create first last =
{First=first; Last=last}
// test
let person = Person.T.Create "John" "Doe"
let fullname = person.FullName

And you can create static members for system types as well:

348

Attaching functions to types

type System.Int32 with

static member IsOdd x = x % 2 = 1
type System.Double with
static member Pi = 3.141
//test
let result = System.Int32.IsOdd 20
let pi = System.Double.Pi

Attaching existing functions

A very common pattern is to attach pre-existing standalone functions to a type. This has a
couple of benefits:
While developing, you can create standalone functions that refer to other standalone
functions. This makes programming easier because type inference works much better
with functional-style code than with OO-style ("dotting into") code.
But for certain key functions, you can attach them to the type as well. This gives clients
the choice of whether to use functional or object-oriented style.
One example of this in the F# libraries is the function that calculates a list's length. It is
available as a standalone function in the List module, but also as a method on a list
instance.
let list = [1..10]
// functional style
let len1 = List.length list
// OO style
let len2 = list.Length

In the following example, we start with a type with no members initially, then define some
functions, then finally attach the fullName function to the type.

349

Attaching functions to types

module Person =
// type with no members initially
type T = {First:string; Last:string}
// constructor
let create first last =
{First=first; Last=last}
// standalone function
let fullName {First=first; Last=last} =
first + " " + last
// attach preexisting function as a member
type T with
member this.FullName = fullName this
// test
let person = Person.create "John" "Doe"
let fullname = Person.fullName person // functional style
let fullname2 = person.FullName // OO style

The standalone fullName function has one parameter, the person. In the attached member,
the parameter comes from the this self-reference.

Attaching existing functions with multiple parameters

One nice thing is that when the previously defined function has multiple parameters, you
don't have to respecify them all when doing the attachment, as long as the this parameter
is first.
In the example below, the hasSameFirstAndLastName function has three parameters. Yet
when we attach it, we only need to specify one!

350

Attaching functions to types

module Person =
// type with no members initially
type T = {First:string; Last:string}
// constructor
let create first last =
{First=first; Last=last}
// standalone function
let hasSameFirstAndLastName (person:T) otherFirst otherLast =
person.First = otherFirst && person.Last = otherLast
// attach preexisting function as a member
type T with
member this.HasSameFirstAndLastName = hasSameFirstAndLastName this
// test
let person = Person.create "John" "Doe"
let result1 = Person.hasSameFirstAndLastName person "bob" "smith" // functional style
let result2 = person.HasSameFirstAndLastName "bob" "smith" // OO style

Why does this work? Hint: think about currying and partial application!

Tuple-form methods
When we start having methods with more than one parameter, we have to make a decision:
we could use the standard (curried) form, where parameters are separated with spaces,
and partial application is supported.
we could pass in all the parameters at once, comma-separated, in a single tuple.
The "curried" form is more functional, and the "tuple" form is more object-oriented.
The tuple form is also how F# interacts with the standard .NET libraries, so let's examine this
approach in more detail.
As a testbed, here is a Product type with two methods, each implemented using one of the
approaches. The CurriedTotal and TupleTotal methods each do the same thing: work out
the total price for a given quantity and discount.

351

Attaching functions to types

type Product = {SKU:string; Price: float} with

// curried style
member this.CurriedTotal qty discount =
(this.Price * float qty) - discount
// tuple style
member this.TupleTotal(qty,discount) =
(this.Price * float qty) - discount

And here's some test code:

let product = {SKU="ABC"; Price=2.0}
let total1 = product.CurriedTotal 10 1.0
let total2 = product.TupleTotal(10,1.0)

No difference so far.
We know that curried version can be partially applied:
let totalFor10 = product.CurriedTotal 10
let discounts = [1.0..5.0]
let totalForDifferentDiscounts
= discounts |> List.map totalFor10

But the tuple approach can do a few things that that the curried one can't, namely:
Named parameters
Optional parameters
Overloading

Named parameters with tuple-style parameters

The tuple-style approach supports named parameters:
let product = {SKU="ABC"; Price=2.0}
let total3 = product.TupleTotal(qty=10,discount=1.0)
let total4 = product.TupleTotal(discount=1.0, qty=10)

As you can see, when names are used, the parameter order can be changed.
Note: if some parameters are named and some are not, the named ones must always be
last.

352

Attaching functions to types

Optional parameters with tuple-style parameters

For tuple-style methods, you can specify an optional parameter by prefixing the parameter
name with a question mark.
If the parameter is set, it comes through as Some value
If the parameter is not set, it comes through as None
Here's an example:
type Product = {SKU:string; Price: float} with
// optional discount
member this.TupleTotal2(qty,?discount) =
let extPrice = this.Price * float qty
match discount with
| None -> extPrice
| Some discount -> extPrice - discount

And here's a test:

let product = {SKU="ABC"; Price=2.0}
// discount not specified
let total1 = product.TupleTotal2(10)
// discount specified
let total2 = product.TupleTotal2(10,1.0)

This explicit matching of the None and Some can be tedious, and there is a slightly more
elegant solution for handling optional parameters.
There is a function defaultArg which takes the parameter as the first argument and a
default for the second argument. If the parameter is set, the value is returned. And if not, the
default value is returned.
Let's see the same code rewritten to use defaultArg
type Product = {SKU:string; Price: float} with
// optional discount
member this.TupleTotal2(qty,?discount) =
let extPrice = this.Price * float qty
let discount = defaultArg discount 0.0
//return
extPrice - discount

353

Attaching functions to types

Method overloading
In C#, you can have multiple methods with the same name that differ only in their function
signature (e.g. different parameter types and/or number of parameters)
In the pure functional model, that does not make sense -- a function works with a particular
domain type and a particular range type. The same function cannot work with different
domains and ranges.
However, F# does support method overloading, but only for methods (that is functions
attached to types) and of these, only those using tuple-style parameter passing.
Here's an example, with yet another variant on the TupleTotal method!
type Product = {SKU:string; Price: float} with
// no discount
member this.TupleTotal3(qty) =
printfn "using non-discount method"
this.Price * float qty
// with discount
member this.TupleTotal3(qty, discount) =
printfn "using discount method"
(this.Price * float qty) - discount

Normally, the F# compiler would complain that there are two methods with the same name,
but in this case, because they are tuple based and because their signatures are different, it
is acceptable. (To make it obvious which one is being called, I have added a small
debugging message.)
And here's a test:
let product = {SKU="ABC"; Price=2.0}
// discount not specified
let total1 = product.TupleTotal3(10)
// discount specified
let total2 = product.TupleTotal3(10,1.0)

Hey! Not so fast... The downsides of using

methods

354

Attaching functions to types

If you are coming from an object-oriented background, you might be tempted to use
methods everywhere, because that is what you are familiar with. But be aware that there
some major downsides to using methods as well:
Methods don't play well with type inference
Methods don't play well with higher order functions
In fact, by overusing methods you would be needlessly bypassing the most powerful and
useful aspects of programming in F#.
Let's see what I mean.

Methods don't play well with type inference

Let's go back to our Person example again, the one that had the same logic implemented
both as a standalone function and as a method:
module Person =
// type with no members initially
type T = {First:string; Last:string}
// constructor
let create first last =
{First=first; Last=last}
// standalone function
let fullName {First=first; Last=last} =
first + " " + last
// function as a member
type T with
member this.FullName = fullName this

Now let's see how well each one works with type inference. Say that I want to print the full
name of a person, so I will define a function printFullName that takes a person as a
parameter.
Here's the code using the module level standalone function.
open Person
// using standalone function
let printFullName person =
printfn "Name is %s" (fullName person)
// type inference worked:
// val printFullName : Person.T -> unit

355

Attaching functions to types

This compiles without problems, and the type inference has correctly deduced that
parameter was a person
Now let's try the "dotted" version:
open Person
// using method with "dotting into"
let printFullName2 person =
printfn "Name is %s" (person.FullName)

This does not compile at all, because the type inference does not have enough information
to deduce the parameter. Any object might implement .FullName -- there is just not enough
to go on.
Yes, we could annotate the function with the parameter type, but that defeats the whole
purpose of type inference.

Methods don't play well with higher order functions

A similar problem happens with higher order functions. For example, let's say that, given a
list of people, we want to get all their full names.
With a standalone function, this is trivial:
open Person
let list = [
Person.create "Andy" "Anderson";
Person.create "John" "Johnson";
Person.create "Jack" "Jackson"]
//get all the full names at once
list |> List.map fullName

With object methods, we have to create special lambdas everywhere:

open Person
let list = [
Person.create "Andy" "Anderson";
Person.create "John" "Johnson";
Person.create "Jack" "Jackson"]
//get all the full names at once
list |> List.map (fun p -> p.FullName)

356

Attaching functions to types

And this is just a simple example. Object methods don't compose well, are hard to pipe, and
so on.
So, a plea for those of you new to functionally programming. Don't use methods at all if you
can, especially when you are learning. They are a crutch that will stop you getting the full
benefit from functional programming.

357

Worked example: A stack based calculator

In this post, we'll implement a simple stack based calculator (also known as "reverse Polish"
style). The implementation is almost entirely done with functions, with only one special type
and no pattern matching at all, so it is a great testing ground for the concepts introduced in
this series.
If you are not familiar with a stack based calculator, it works as follows: numbers are pushed
on a stack, and operations such as addition and multiplication pop numbers off the stack and
push the result back on.
Here is a diagram showing a simple calculation using a stack:

The first steps to designing a system like this is to think about how it would be used.
Following a Forth like syntax, we will give each action a label, so that the example above
might want to be written something like:
EMPTY ONE THREE ADD TWO MUL SHOW

We might not be able to get this exact syntax, but let's see how close we can get.

The Stack data type

First we need to define the data structure for a stack. To keep things simple, we'll just use a
list of floats.
type Stack = float list

But, hold on, let's wrap it in a single case union type to make it more descriptive, like this:
type Stack = StackContents of float list

358

Worked example: A stack based calculator

For more details on why this is nicer, read the discussion of single case union types in this
post.
Now, to create a new stack, we use StackContents as a constructor:
let newStack = StackContents [1.0;2.0;3.0]

And to extract the contents of an existing Stack, we pattern match with StackContents :
let (StackContents contents) = newStack
// "contents" value set to
// float list = [1.0; 2.0; 3.0]

The Push function

Next we need a way to push numbers on to the stack. This will be simply be prepending the
new value at the front of the list using the " :: " operator.
Here is our push function:
let push x aStack =
let (StackContents contents) = aStack
let newContents = x::contents
StackContents newContents

This basic function has a number of things worth discussing.

First, note that the list structure is immutable, so the function must accept an existing stack
and return a new stack. It cannot just alter the existing stack. In fact, all of the functions in
this example will have a similar format like this:
Input: a Stack plus other parameters
Output: a new Stack

Next, what should the order of the parameters be? Should the stack parameter come first or
last? If you remember the discussion of designing functions for partial application, you will
remember that the most changeable thing should come last. You'll see shortly that this
guideline will be born out.
Finally, the function can be made more concise by using pattern matching in the function
parameter itself, rather than using a let in the body of the function.

359

Worked example: A stack based calculator

Here is the rewritten version:

let push x (StackContents contents) =
StackContents (x::contents)

Much nicer!
And by the way, look at the nice signature it has:
val push : float -> Stack -> Stack

As we know from a previous post, the signature tells you a lot about the function. In this
case, I could probably guess what it did from the signature alone, even without knowing that
the name of the function was "push". This is one of the reasons why it is a good idea to have
explicit type names. If the stack type had just been a list of floats, it wouldn't have been as
self-documenting.
Anyway, now let's test it:
let emptyStack = StackContents []
let stackWith1 = push 1.0 emptyStack
let stackWith2 = push 2.0 stackWith1

Works great!

Building on top of "push"

With this simple function in place, we can easily define an operation that pushes a particular
number onto the stack.
let ONE stack = push 1.0 stack
let TWO stack = push 2.0 stack

But wait a minute! Can you see that the stack parameter is used on both sides? In fact, we
don't need to mention it at all. Instead we can skip the stack parameter and write the
functions using partial application as follows:

360

Worked example: A stack based calculator

let ONE = push 1.0

let TWO = push 2.0
let THREE = push 3.0
let FOUR = push 4.0
let FIVE = push 5.0

Now you can see that if the parameters for push were in a different order, we wouldn't have
been able to do this.
While we're at it, let's define a function that creates an empty stack as well:
let EMPTY = StackContents []

Let's test all of these now:

let stackWith1 = ONE EMPTY
let stackWith2 = TWO stackWith1
let stackWith3 = THREE stackWith2

These intermediate stacks are annoying ? can we get rid of them? Yes! Note that these
functions ONE, TWO, THREE all have the same signature:
Stack -> Stack

This means that they can be chained together nicely! The output of one can be fed into the
input of the next, as shown below:
let result123 = EMPTY |> ONE |> TWO |> THREE
let result312 = EMPTY |> THREE |> ONE |> TWO

Popping the stack

That takes care of pushing onto the stack ? what about a pop function next?
When we pop the stack, we will return the top of the stack, obviously, but is that all?
In an object-oriented style, the answer is yes. In an OO approach, we would mutate the
stack itself behind the scenes, so that the top element was removed.
But in a functional style, the stack is immutable. The only way to remove the top element is
to create a new stack with the element removed. In order for the caller to have access to this
new diminished stack, it needs to be returned along with the top element itself.

361

Worked example: A stack based calculator

In other words, the pop function will have to return two values, the top plus the new stack.
The easiest way to do this in F# is just to use a tuple.
Here's the implementation:
/// Pop a value from the stack and return it
/// and the new stack as a tuple
let pop (StackContents contents) =
match contents with
| top::rest ->
let newStack = StackContents rest
(top,newStack)

This function is also very straightforward.

As before, we are extracting the contents directly in the parameter.
We then use a match..with expression to test the contents.
Next, we separate the top element from the rest, create a new stack from the remaining
elements and finally return the pair as a tuple.
Try the code above and see what happens. You will get a compiler error! The compiler has
caught a case we have overlooked -- what happens if the stack is empty?
So now we have to decide how to handle this.
Option 1: Return a special "Success" or "Error" state, as we did in a post from the "why
use F#?" series.
Option 2: Throw an exception.
Generally, I prefer to use error cases, but in this case, we'll use an exception. So here's the
pop code changed to handle the empty case:

/// Pop a value from the stack and return it

/// and the new stack as a tuple
let pop (StackContents contents) =
match contents with
| top::rest ->
let newStack = StackContents rest
(top,newStack)
| [] ->
failwith "Stack underflow"

Now let's test it:

362

Worked example: A stack based calculator

let initialStack = EMPTY |> ONE |> TWO

let popped1, poppedStack = pop initialStack
let popped2, poppedStack2 = pop poppedStack

and to test the underflow:

let _ = pop EMPTY

Writing the math functions

Now with both push and pop in place, we can work on the "add" and "multiply" functions:
let ADD stack =
let x,s = pop stack //pop the top of the stack
let y,s2 = pop s //pop the result stack
let result = x + y //do the math
push result s2 //push back on the doubly-popped stack
let MUL stack =
let x,s = pop stack //pop the top of the stack
let y,s2 = pop s //pop the result stack
let result = x * y //do the math
push result s2 //push back on the doubly-popped stack

Test these interactively:

let add1and2 = EMPTY |> ONE |> TWO |> ADD
let add2and3 = EMPTY |> TWO |> THREE |> ADD
let mult2and3 = EMPTY |> TWO |> THREE |> MUL

It works!

Time to refactor...
It is obvious that there is significant duplicate code between these two functions. How can
we refactor?
Both functions pop two values from the stack, apply some sort of binary function, and then
push the result back on the stack. This leads us to refactor out the common code into a
"binary" function that takes a two parameter math function as a parameter:

363

Worked example: A stack based calculator

let binary mathFn stack =

// pop the top of the stack
let y,stack' = pop stack
// pop the top of the stack again
let x,stack'' = pop stack'
// do the math
let z = mathFn x y
// push the result value back on the doubly-popped stack
push z stack''

Note that in this implementation, I've switched to using ticks to represent changed states of
the "same" object, rather than numeric suffixes. Numeric suffixes can easily get quite
confusing.
Question: why are the parameters in the order they are, instead of mathFn being after
stack ?

Now that we have binary , we can define ADD and friends more simply:
Here's a first attempt at ADD using the new binary helper:
let ADD aStack = binary (fun x y -> x + y) aStack

But we can eliminate the lambda, as it is exactly the definition of the built-in + function!
Which gives us:
let ADD aStack = binary (+) aStack

And again, we can use partial application to hide the stack parameter. Here's the final
definition:
let ADD = binary (+)

And here's the definition of some other math functions:

let SUB = binary (-)
let MUL = binary (*)
let DIV = binary (../)

Let's test interactively again.

364

Worked example: A stack based calculator

let div2by3 = EMPTY |> THREE|> TWO |> DIV

let sub2from5 = EMPTY |> TWO |> FIVE |> SUB
let add1and2thenSub3 = EMPTY |> ONE |> TWO |> ADD |> THREE |> SUB

In a similar fashion, we can create a helper function for unary functions

let unary f stack =
let x,stack' = pop stack //pop the top of the stack
push (f x) stack' //push the function value on the stack

And then define some unary functions:

let NEG = unary (fun x -> -x)
let SQUARE = unary (fun x -> x * x)

Test interactively again:

let neg3 = EMPTY |> THREE|> NEG
let square2 = EMPTY |> TWO |> SQUARE

Putting it all together

In the original requirements, we mentioned that we wanted to be able to show the results, so
let's define a SHOW function.
let SHOW stack =
let x,_ = pop stack
printfn "The answer is %f" x
stack // keep going with same stack

Note that in this case, we pop the original stack but ignore the diminished version. The final
result of the function is the original stack, as if it had never been popped.
So now finally, we can write the code example from the original requirements
EMPTY |> ONE |> THREE |> ADD |> TWO |> MUL |> SHOW

Going further
This is fun -- what else can we do?

365

Worked example: A stack based calculator

Well, we can define a few more core helper functions:

/// Duplicate the top value on the stack
let DUP stack =
// get the top of the stack
let x,_ = pop stack
// push it onto the stack again
push x stack
/// Swap the top two values
let SWAP stack =
let x,s = pop stack
let y,s' = pop s
push y (push x s')
/// Make an obvious starting point
let START = EMPTY

And with these additional functions in place, we can write some nice examples:
START
|> ONE |> TWO |> SHOW
START
|> ONE |> TWO |> ADD |> SHOW
|> THREE |> ADD |> SHOW
START
|> THREE |> DUP |> DUP |> MUL |> MUL // 27
START
|> ONE |> TWO |> ADD |> SHOW // 3
|> THREE |> MUL |> SHOW // 9
|> TWO |> SWAP |> DIV |> SHOW // 9 div 2 = 4.5

Using composition instead of piping

But that's not all. In fact, there is another very interesting way to think about these functions.
As I pointed out earlier, they all have an identical signature:
Stack -> Stack

So, because the input and output types are the same, these functions can be composed
using the composition operator >> , not just chained together with pipes.

366

Worked example: A stack based calculator

Here are some examples:

In each of these cases, a new function is defined by composing other functions together to
make a new one. This is a good example of the "combinator" approach to building up
functionality.

Pipes vs composition
We have now seen two different ways that this stack based model can be used; by piping or
by composition. So what is the difference? And why would we prefer one way over another?
The difference is that piping is, in a sense, a "realtime transformation" operation. When you
use piping you are actually doing the operations right now, passing a particular stack around.
On the other hand, composition is a kind of "plan" for what you want to do, building an
overall function from a set of parts, but not actually running it yet.

367

Worked example: A stack based calculator

So for example, I can create a "plan" for how to square a number by combining smaller
operations:
let COMPOSED_SQUARE = DUP >> MUL

I cannot do the equivalent with the piping approach.

let PIPED_SQUARE = DUP |> MUL

This causes a compilation error. I have to have some sort of concrete stack instance to
make it work:
let stackWith2 = EMPTY |> TWO
let twoSquared = stackWith2 |> DUP |> MUL

And even then, I only get the answer for this particular input, not a plan for all possible
inputs, as in the COMPOSED_SQUARE example.
The other way to create a "plan" is to explicitly pass in a lambda to a more primitive function,
as we saw near the beginning:
let LAMBDA_SQUARE = unary (fun x -> x * x)

This is much more explicit (and is likely to be faster) but loses all the benefits and clarity of
the composition approach.
So, in general, go for the composition approach if you can!

The complete code

Here's the complete code for all the examples so far.
// ==============================================
// Types
// ==============================================
type Stack = StackContents of float list
// ==============================================
// Stack primitives
// ==============================================
/// Push a value on the stack

368

Worked example: A stack based calculator

let push x (StackContents contents) =

StackContents (x::contents)
/// Pop a value from the stack and return it
/// and the new stack as a tuple
let pop (StackContents contents) =
match contents with
| top::rest ->
let newStack = StackContents rest
(top,newStack)
| [] ->
failwith "Stack underflow"
// ==============================================
// Operator core
// ==============================================
// pop the top two elements
// do a binary operation on them
// push the result
let binary mathFn stack =
let y,stack' = pop stack
let x,stack'' = pop stack'
let z = mathFn x y
push z stack''
// pop the top element
// do a unary operation on it
// push the result
let unary f stack =
let x,stack' = pop stack
push (f x) stack'
// ==============================================
// Other core
// ==============================================
/// Pop and show the top value on the stack
let SHOW stack =
let x,_ = pop stack
printfn "The answer is %f" x
stack // keep going with same stack
/// Duplicate the top value on the stack
let DUP stack =
let x,s = pop stack
push x (push x s)
/// Swap the top two values
let SWAP stack =
let x,s = pop stack
let y,s' = pop s
push y (push x s')

369

Worked example: A stack based calculator

/// Drop the top value on the stack

let DROP stack =
let _,s = pop stack //pop the top of the stack
s //return the rest
// ==============================================
// Words based on primitives
// ==============================================
// Constants
// ------------------------------let EMPTY = StackContents []
let START = EMPTY

// Numbers
// ------------------------------let ONE = push 1.0
let TWO = push 2.0
let THREE = push 3.0
let FOUR = push 4.0
let FIVE = push 5.0
// Math functions
// ------------------------------let ADD = binary (+)
let SUB = binary (-)
let MUL = binary (*)
let DIV = binary (../)
let NEG = unary (fun x -> -x)

// ==============================================
// Words based on composition
// ==============================================
let SQUARE =
DUP >> MUL
let CUBE =
DUP >> DUP >> MUL >> MUL
let SUM_NUMBERS_UPTO =
DUP // n
>> ONE >> ADD // n+1
>> MUL // n(n+1)
>> TWO >> SWAP >> DIV // n(n+1) / 2

Summary
370

Worked example: A stack based calculator

So there we have it, a simple stack based calculator. We've seen how we can start with a
few primitive operations ( push , pop , binary , unary ) and from them, build up a whole
domain specific language that is both easy to implement and easy to use.
As you might guess, this example is based heavily on the Forth language. I highly
recommend the free book "Thinking Forth", which is not just about the Forth language, but
about (non object-oriented!) problem decomposition techniques which are equally applicable
to functional programming.
I got the idea for this post from a great blog by Ashley Feniello. If you want to go deeper into
emulating a stack based language in F#, start there. Have fun!

371

The "Expressions and syntax" Series

In this series of posts we'll look at how functions and values are combined into expressions,
and the different kinds of expressions that are available in F#.
Expressions and syntax: Introduction. How to code in F#.
Expressions vs. statements. Why expressions are safer and make better building
blocks.
Overview of F# expressions. Control flows, lets, dos, and more.
Binding with let, use, and do. How to use them.
F# syntax: indentation and verbosity. Understanding the offside rule.
Parameter and value naming conventions. a, f, x and friends.
Control flow expressions. And how to avoid using them.
Exceptions. Syntax for throwing and catching.
Match expressions. The workhorse of F#.
Formatted text using printf. Tips and techniques for printing and logging.
Worked example: Parsing command line arguments. Pattern matching in practice.
Worked example: Roman numerals. More pattern matching in practice.

372

Expressions and syntax: Introduction

NOTE: Before reading this series, I suggest that you read the "thinking functionally" series
as a prerequisite.
In this series we'll look at how functions and values are combined into expressions, and the
different kinds of expressions that are available in F#.
We'll also look at some other basic topics, such as let bindings, F# syntax, pattern
matching, and outputting text with printf .
This series is not meant to be exhaustive or definitive. Much of the syntax and usage of F#
should be obvious from the examples, and the MSDN documentation has all the details if
you need them. Rather we will just focus on explaining some of the essential areas that
might be confusing.
So, we'll start with some general tips, talk about how let bindings work, and explain the
indentation rules.
After that, the next few posts will cover match..with expressions, the imperative control flow
expressions, and exception expressions. Computation expressions and object-oriented
expressions will be left to later series.
Finally, we'll finish with some worked examples that use pattern matching as an integral part
of their design.

373

Expressions vs. statements

In programming language terminology, an "expression" is a combination of values and
functions that are combined and interpreted by the compiler to create a new value, as
opposed to a "statement" which is just a standalone unit of execution and doesn't return
anything. One way to think of this is that the purpose of an expression is to create a value
(with some possible side-effects), while the sole purpose of a statement is to have sideeffects.
C# and most imperative languages make a distinction between expressions and statements
and have rules about where each kind can be used. But as should be apparent, a truly pure
functional language cannot support statements at all, because in a truly pure language,
there would be no side-effects.
Even though F# is not pure, it does follow the same principle. In F# everything is an
expression. Not just values and functions, but also control flows (such as if-then-else and
loops), pattern matching, and so on.
There are some subtle benefits to using expressions over statements. First, unlike
statements, smaller expressions can be combined (or "composed") into larger expressions.
So if everything is an expression, then everything is also composable.
Second, a series of statements always implies a specific order of evaluation, which means
that a statement cannot be understood without looking at prior statements. But with pure
expressions, the subexpressions do not have any implied order of execution or
dependencies.
So in the expression a+b , if both the ' a ' and ' b ' parts are pure, then the ' a ' part can be
isolated, understood, tested and evaluated on its own, as can the ' b ' part.
This "isolatibility" of expressions is another beneficial aspect of functional programming.
Note that the F# interactive window also relies on everything being an expression. It
would be much harder to use a C# interactive window.

Expressions are safer and more compact

Using expressions consistently leads to code that is both safer and more compact. Let's see
what I mean by this.

374

Expressions vs. statements

First, let's look at a statement based approach. Statements don't return values, so you have
to use temporary variables that are assigned to from within statement bodies. Here are
some examples using a C-like language (OK, C#) rather than F#:
public void IfThenElseStatement(bool aBool)
{
int result; //what is the value of result before it is used?
if (aBool)
{
result = 42; //what is the result in the 'else' case?
}
Console.WriteLine("result={0}", result);
}

Because the "if-then" is a statement, the result variable must be defined outside the
statement and but assigned to inside the statement, which leads to some issues:
The result variable has to be set up outside the statement itself. What initial value
should it be set to?
What if I forget to assign to the result variable in the if statement? The purpose of
the "if "statement is purely to have side effects (the assignment to the variables). This
means that the statements are potentially buggy, because it would be easy to forget to
do an assignment in one branch. And because the assignment was just a side effect,
the compiler could not offer any warning. Since the result variable has already been
defined in scope, I could easily use it, unaware that it was invalid.
What is the value of the result variable in the "else" case? In this case, I haven't
specified a value. Did I forget? Is this a potential bug?
Finally, the reliance on side-effects to get things done means that the statements are
not easily usable in another context (for example, extracted for refactoring, or
parallelizing) because they have a dependency on a variable that is not part of the
statement itself.
Note: the code above will not compile in C# because the compiler will complain if you use an
unassigned local variable like this. But having to define some default value for result
before it is even used is still a problem.
For comparison, here is the same code, rewritten in an expression-oriented style:
public void IfThenElseExpression(bool aBool)
{
int result = aBool ? 42 : 0;
Console.WriteLine("result={0}", result);
}

375

Expressions vs. statements

In the expression-oriented version, none of the earlier issues even apply!

The result variable is declared at the same time that it is assigned. No variables have
to be set up "outside" the expression and there is no worry about what initial value they
should be set to.
The "else" is explicitly handled. There is no chance of forgetting to do an assignment in
one of the branches.
And I cannot possibly forget to assign to result , because then the variable would not
even exist!
In F#, the two examples would be written as:
let IfThenElseStatement aBool =
let mutable result = 0 // mutable keyword required
if (aBool) then result <- 42
printfn "result=%i" result

The " mutable " keyword is considered a code smell in F#, and is discouraged except in
certain special cases. It should be avoided at all cost while you are learning!
In the expression based version, the mutable variable has been eliminated and there is no
reassignment anywhere.
let IfThenElseExpression aBool =
let result = if aBool then 42 else 0
// note that the else case must be specified
printfn "result=%i" result

Once we have the if statement converted into an expression, it is now trivial to refactor it
and move the entire subexpression to a different context without introducing errors.
Here's the refactored version in C#:
public int StandaloneSubexpression(bool aBool)
{
return aBool ? 42 : 0;
}
public void IfThenElseExpressionRefactored(bool aBool)
{
int result = StandaloneSubexpression(aBool);
Console.WriteLine("result={0}", result);
}

And in F#:

376

Expressions vs. statements

let StandaloneSubexpression aBool =

if aBool then 42 else 0
let IfThenElseExpressionRefactored aBool =
let result = StandaloneSubexpression aBool
printfn "result=%i" result

Statements vs. expressions for loops

Going back to C# again, here is a similar example of statements vs. expressions using a
loop statement
public void LoopStatement()
{
int i; //what is the value of i before it is used?
int length;
var array = new int[] { 1, 2, 3 };
int sum; //what is the value of sum if the array is empty?
length = array.Length; //what if I forget to assign to length?
for (i = 0; i < length; i++)
{
sum += array[i];
}
Console.WriteLine("sum={0}", sum);
}

I've used an old-style "for" statement, where the index variables are declared outside the
loop. Many of the issues discussed earlier apply to the loop index " i " and the max value
" length ", such as: can they be used outside the loop? And what happens if they are not
assigned to?
A more modern version of a for-loop addresses these issues by declaring and assigning the
loop variables in the "for" loop itself, and by requiring the " sum " variable to be initialized:

377

Expressions vs. statements

public void LoopStatementBetter()

{
var array = new int[] { 1, 2, 3 };
int sum = 0; // initialization is required
for (var i = 0; i < array.Length; i++)
{
sum += array[i];
}
Console.WriteLine("sum={0}", sum);
}

This more modern version follows the general principle of combining the declaration of a
local variable with its first assignment.
But of course, we can keep improving by using a foreach loop instead of a for loop:
public void LoopStatementForEach()
{
var array = new int[] { 1, 2, 3 };
int sum = 0; // initialization is required
foreach (var i in array)
{
sum += i;
}
Console.WriteLine("sum={0}", sum);
}

Each time, not only are we condensing the code, but we are reducing the likelihood of
errors.
But taking that principle to its logical conclusion leads to a completely expression based
approach! Here's how it might be done using LINQ:
public void LoopExpression()
{
var array = new int[] { 1, 2, 3 };
var sum = array.Aggregate(0, (sumSoFar, i) => sumSoFar + i);
Console.WriteLine("sum={0}", sum);
}

378

Expressions vs. statements

Note that I could have used LINQ's built-in "sum" function, but I used Aggregate in order to
show how the sum logic embedded in a statement can be converted into a lambda and used
as part of an expression.
In the next post, we'll look at the various kinds of expressions in F#.

379

Overview of F# expressions

Overview of F# expressions
In this post we'll look at the different kinds of expressions that are available in F# and some
general tips for using them.

Is everything really an expression?

You might be wondering how "everything is an expression" actually works in practice.
Let's start with some basic expression examples that should be familiar:
1 // literal
[1;2;3] // list expression
-2 // prefix operator
2 + 2 // infix operator
"string".Length // dot lookup
printf "hello" // function application

No problems there. Those are obviously expressions.

But here are some more complex things which are also expressions. That is, each of these
returns a value that can be used for something else.
fun () -> 1 // lambda expression
match 1 with // match expression
| 1 -> "a"
| _ -> "b"
if true then "a" else "b" // if-then-else
for i in [1..10] // for loop
do printf "%i" i
try // exception handling
let result = 1 / 0
printfn "%i" result
with
| e ->
printfn "%s" e.Message

let n=1 in n+2 // let expression

380

Overview of F# expressions

In other languages, these might be statements, but in F# they really do return values, as you
can see by binding a value to the result:
let x1 = fun () -> 1
let x2 = match 1 with
| 1 -> "a"
| _ -> "b"
let x3 = if true then "a" else "b"
let x4 = for i in [1..10]
do printf "%i" i
let x5 = try
let result = 1 / 0
printfn "%i" result
with
| e ->
printfn "%s" e.Message

let x6 = let n=1 in n+2

What kinds of expressions are there?

There are lots of diffent kinds of expressions in F#, about 50 currently. Most of them are
trivial and obvious, such as literals, operators, function application, "dotting into", and so on.
The more interesting and high-level ones can be grouped as follows:
Lambda expressions
"Control flow" expressions, including:
The match expression (with the match..with syntax)
Expressions related to imperative control flow, such as if-then-else, loops
Exception-related expressions
"let" and "use" expressions
Computation expressions such as async {..}
Expressions related to object-oriented code, including casts, interfaces, etc
We have already discussed lambdas in the "thinking functionally" series, and as noted
earlier, computation expressions and object-oriented expressions will be left to later series.
So, in upcoming posts in this series, we will focus on "control flow" expressions and "let"
expressions.

381

Overview of F# expressions

"Control flow" expressions

In imperative languages, control flow expressions like if-then-else, for-in-do, and match-with
are normally implemented as statements with side-effects, In F#, they are all implemented
as just another type of expression.
In fact, it is not even helpful to think of "control flow" in a functional language; the concept
doesn't really exist. Better to just think of the program as a giant expression containing subexpressions, some of which are evaluated and some of which are not. If you can get your
head around this way of thinking, you have a good start on thinking functionally.
There will be some upcoming posts on these different types of control flow expressions:
The match expression
Imperative control flow: if-then-else and for loops
Exceptions

"let" bindings as expressions

What about let x=something ? In the examples above we saw:
let x5 = let n=1 in n+2

How can " let " be an expression? The reason will be discussed in the next post on "let",
"use" and "do".

General tips for using expressions

But before we cover the important expression types in details, here are some tips for using
expressions in general.

Multiple expressions on one line

Normally, each expression is put on a new line. But you can use a semicolon to separate
expressions on one line if you need to. Along with its use as a separator for list and record
elements, this is one of the few times where a semicolon is used in F#.
let f x = // one expression per line
printfn "x=%i" x
x + 1
let f x = printfn "x=%i" x; x + 1 // all on same line with ";"

382

Overview of F# expressions

The rule about requiring unit values until the last expression still applies, of course:
let x = 1;2 // error: "1;" should be a unit expression
let x = ignore 1;2 // ok
let x = printf "hello";2 // ok

Understanding expression evaluation order

In F#, expressions are evaluated from the "inside out" -- that is, as soon as a complete
subexpression is "seen", it is evaluated.
Have a look at the following code and try to guess what will happen, then evaluate the code
and see.
// create a clone of if-then-else
let test b t f = if b then t else f
// call it with two different choices
test true (printfn "true") (printfn "false")

What happens is that both "true" and "false" are printed, even though the test function will
never actually evaluate the "else" branch. Why? Because the (printfn "false") expression
is evaluated immediately, regardless of how the test function will be using it.
This style of evaluation is called "eager". It has the advantage that it is easy to understand,
but it does mean that it can be inefficient on occasion.
The alternative style of evaluation is called "lazy", whereby expressions are only evaluated
when they are needed. The Haskell language follows this approach, so a similar example in
Haskell would only print "true".
In F#, there are a number of techniques to force expressions not to be evaluated
immediately. The simplest it to wrap it in a function that only gets evaluated on demand:
// create a clone of if-then-else that accepts functions rather than simple values
let test b t f = if b then t() else f()
// call it with two different functions
test true (fun () -> printfn "true") (fun () -> printfn "false")

The problem with this is that now the "true" function might be evaluated twice by mistake,
when we only wanted to evaluate it once!

383

Overview of F# expressions

So, the preferred way for expressions not to be evaluated immediately is to use the Lazy<>
wrapper.
// create a clone of if-then-else with no restrictions...
let test b t f = if b then t else f
// ...but call it with lazy values
let f = test true (lazy (printfn "true")) (lazy (printfn "false"))

The final result value f is also a lazy value, and can be passed around without being
evaluated until you are finally ready to get the result.
f.Force() // use Force() to force the evaluation of a lazy value

If you never need the result, and never call Force() , then the wrapped value will never be
evaluated.
There will much more on laziness in an upcoming series on performance.

384

Binding with let, use, and do

As we've have already seen, there are no "variables" in F#. Instead there are values.
And we have also seen that keywords such as let , use , and do act as bindings -associating an identifier with a value or function expression.
In this post we'll look at these bindings in more detail.

"let" bindings
The let binding is straightforward, it has the general form:
let aName = someExpression

But there are two uses of let that are subtly different. One is to define a named expression
at a the top level of a module*, and the other is to define a local name used in the context of
some expression. This is somewhat analogous to the difference between "top level" method
names and "local" variable names in C#.
* and in a later series, when we talk about OO features, classes can have top level let
bindings too.
Here's an example of both types:
module MyModule =
let topLevelName =
let nestedName1 = someExpression
let nestedName2 = someOtherExpression
finalExpression

The top level name is a definition, which is part of the module, and you can access it with a
fully qualified name such as MyModule.topLevelName . It's the equivalent of a class method, in
a sense.
But the nested names are completely inaccessible to anyone -- they are only valid within the
context of the top level name binding.

Patterns in "let" bindings

385

Binding with let, use, and do

We have already seen examples of how bindings can use patterns directly
let a,b = 1,2
type Person = {First:string; Last:string}
let alice = {First="Alice"; Last="Doe"}
let {First=first} = alice

And in function definitions the binding includes parameters as well:

// pattern match the parameters
let add (x,y) = x + y
// test
let aTuple = (1,2)
add aTuple

The details of the various pattern bindings depends on the type being bound, and will be
discussed further in later posts on pattern matching.

Nested "let" bindings as expressions

We have emphasized that an expression is composed from smaller expressions. But what
about a nested let ?
let nestedName = someExpression

How can " let " be an expression? What does it return?

The answer that a nested "let" can never be used in isolation -- it must always be part of a
larger code block, so that it can be interpreted as:
let nestedName = [some expression] in [some other expression involving nestedName]

That is, every time you see the symbol "nestedName" in the second expression (called the
body expression), substitute it with the first expression.
So for example, the expression:

386

Binding with let, use, and do

// standard syntax
let f () =
let x = 1
let y = 2
x + y // the result

really means:
// syntax using "in" keyword
let f () =
let x = 1 in // the "in" keyword is available in F#
let y = 2 in
x + y // the result

When the substitutions are performed, the last line becomes:

(definition of x) + (definition of y)
// or
(1) + (2)

In a sense, the nested names are just "macros" or "placeholders" that disappear when the
expression is compiled. And therefore you should be able to see that the nested let s have
no effect on the expression as whole. So, for example, the type of an expression containing
nested let s is just the type of the final body expression.
If you understand how nested let bindings work, then certain errors become
understandable. For example, if there is nothing for a nested "let" to be "in", the entire
expression is not complete. In the example below, there is nothing following the let line,
which is an error:
let f () =
let x = 1
// error FS0588: Block following this 'let' is unfinished.
// Expect an expression.

And you cannot have multiple expression results, because you cannot have multiple body
expressions. Anything evaluated before the final body expression must be a " do "
expression (see below), and return unit .

387

Binding with let, use, and do

let f () =
2 + 2 // warning FS0020: This expression should
// have type 'unit'
let x = 1
x + 1 // this is the final result

In a case like this, you must pipe the results into "ignore".
let f () =
2 + 2 |> ignore
let x = 1
x + 1 // this is the final result

"use" bindings
The use keyword serves the same purpose as let -- it binds the result of an expression
to a named value.
The key difference is that is also automatically disposes the value when it goes out of scope.
Obviously, this means that use only applies in nested situations. You cannot have a top
level use and the compiler will warn you if you try.
module A =
use f () = // Error
let x = 1
x + 1

To see how a proper use binding works, first let's create a helper function that creates an
IDisposable on the fly.

// create a new object that implements IDisposable

let makeResource name =
{ new System.IDisposable
with member this.Dispose() = printfn "%s disposed" name }

Now let's test it with a nested use binding:

388

Binding with let, use, and do

let exampleUseBinding name =

use myResource = makeResource name
printfn "done"
//test
exampleUseBinding "hello"

We can see that "done" is printed, and then immediately after that, myResource goes out of
scope, its Dispose is called, and "hello disposed" is also printed.
On the other hand, if we test it using the regular let binding, we don't get the same effect.
let exampleLetBinding name =
let myResource = makeResource name
printfn "done"
//test
exampleLetBinding "hello"

In this case, we see that "done" is printed, but Dispose is never called.

"Use" only works with IDisposables

Note that "use" bindings only work with types that implement IDisposable , and the compiler
will complain otherwise:
let exampleUseBinding2 name =
use s = "hello" // Error: The type 'string' is
// not compatible with the type 'IDisposable'
printfn "done"

Don't return "use'd" values

It is important to realize that the value is disposed as soon as it goes out of scope in the
expression where it was declared. If you attempt to return the value for use by another
function, the return value will be invalid.
The following example shows how not to do it:

389

Binding with let, use, and do

let returnInvalidResource name =

use myResource = makeResource name
myResource // don't do this!
// test
let resource = returnInvalidResource "hello"

If you need to work with a disposable "outside" the function that created it, probably the best
way is to use a callback.
The function then would work as follows:
create the disposable.
evaluate the callback with the disposable
call Dispose on the disposable
Here's an example:
let usingResource name callback =
use myResource = makeResource name
callback myResource
printfn "done"
let callback aResource = printfn "Resource is %A" aResource
do usingResource "hello" callback

This approach guarantees that the same function that creates the disposable also disposes
of it and there is no chance of a leak.
Another possible way is to not use a use binding on creation, but use a let binding
instead, and make the caller responsible for disposing.
Here's an example:
let returnValidResource name =
// "let" binding here instead of "use"
let myResource = makeResource name
myResource // still valid
let testValidResource =
// "use" binding here instead of "let"
use resource = returnValidResource "hello"
printfn "done"

Personally, I don't like this approach, because it is not symmetrical and separates the create
from the dispose, which could lead to resource leaks.

390

Binding with let, use, and do

The "using" function

The preferred approach to sharing a disposable, shown above, used a callback function.
There is a built-in using function that works in the same way. It takes two parameters:
the first is an expression that creates the resource
the second is a function that uses the resource, taking it as a parameter
Here's our earlier example rewritten with the using function:
let callback aResource = printfn "Resource is %A" aResource
using (makeResource "hello") callback

In practice, the using function is not used that often, because it is so easy to make your
own custom version of it, as we saw earlier.

Misusing "use"
One trick in F# is to appropriate the use keyword to do any kind of "stop" or "revert"
functionality automatically.
The way to do this is:
Create an extension method for some type
In that method, start the behavior you want but then return an IDisposable that stops
the behavior.
For example, here is an extension method that starts a timer and then returns an
IDisposable that stops it.

391

Binding with let, use, and do

module TimerExtensions =
type System.Timers.Timer with
static member StartWithDisposable interval handler =
// create the timer
let timer = new System.Timers.Timer(interval)
// add the handler and start it
do timer.Elapsed.Add handler
timer.Start()
// return an IDisposable that calls "Stop"
{ new System.IDisposable with
member disp.Dispose() =
do timer.Stop()
do printfn "Timer stopped"
}

So now in the calling code, we create the timer and bind it with use . When the timer value
goes out of scope, it will stop automatically!
open TimerExtensions
let testTimerWithDisposable =
let handler = (fun _ -> printfn "elapsed")
use timer = System.Timers.Timer.StartWithDisposable 100.0 handler
System.Threading.Thread.Sleep 500

This same approach can be used for other common pairs of operations, such as:
opening/connecting and then closing/disconnecting a resource (which is what
IDisposable is supposed to be used for anyway, but your target type might not have

implemented it)
registering and then deregistering an event handler (instead of using WeakReference )
in a UI, showing a splash screen at the start of a block of code, and then automatically
closing it at the end of the block
I wouldn't recommend this approach generally, because it does hide what is going on, but on
occasion it can be quite useful.

"do" bindings
Sometimes we might want to execute code independently of a function or value definition.
This can be useful in module initialization, class initialization and so on.

392

Binding with let, use, and do

That is, rather than having " let x = do something " we just the " do something " on its own.
This is analogous to a statement in an imperative language.
You can do this by prefixing the code with " do ":
do printf "logging"

In many situations, the do keyword can be omitted:

printf "logging"

But in both cases, the expression must return unit. If it does not, you will get a compiler error.
do 1 + 1 // warning: This expression is a function

As always, you can force a non-unit result to be discarded by piping the results into
" ignore ".
do ( 1+1 |> ignore )

You will also see the " do " keyword used in loops in the same way.
Note that although you can sometimes omit it, it is considered good practice to always have
an explicit " do ", as it acts as documentation that you do not want a result, only the sideeffects.

"do" for module initialization

Just like let , do can be used both in a nested context, and at the top level in a module or
class.
When used at the module level, the do expression is evaluated once only, when the
module is first loaded.
module A =
module B =
do printfn "Module B initialized"
module C =
do printfn "Module C initialized"
do printfn "Module A initialized"

393

Binding with let, use, and do

This is somewhat analogous to a static class constructor in C#, except that if there are
multiple modules, the order of initialization is fixed and they are initialized in order of
declaration.

let! and use! and do!

When you see let! , use! and do! (that is, with exclamation marks) and they are part of
a curly brace {..} block, then they are being used as part of a "computation expression".
The exact meaning of let! , use! and do! in this context depends on the computation
expression itself. Understanding computation expressions in general will have to wait for a
later series.
The most common type of computation expression you will run into are asynchronous
workflows, indicated by a async{..} block. In this context, it means they are being used to
wait for an async operation to finish, and only then bind to the result value.
Here are some examples we saw earlier in a post from the "why use F#?" series:

394

Binding with let, use, and do

//This simple workflow just sleeps for 2 seconds.

open System
let sleepWorkflow = async{
printfn "Starting sleep workflow at %O" DateTime.Now.TimeOfDay
// do! means to wait as well
do! Async.Sleep 2000
printfn "Finished sleep workflow at %O" DateTime.Now.TimeOfDay
}
//test
Async.RunSynchronously sleepWorkflow

// Workflows with other async workflows nested inside them.

/// Within the braces, the nested workflows can be blocked on by using the let! or use
! syntax.
let nestedWorkflow = async{
printfn "Starting parent"
// let! means wait and then bind to the childWorkflow value
let! childWorkflow = Async.StartChild sleepWorkflow
// give the child a chance and then keep working
do! Async.Sleep 100
printfn "Doing something useful while waiting "
// block on the child
let! result = childWorkflow
// done
printfn "Finished parent"
}
// run the whole workflow
Async.RunSynchronously nestedWorkflow

Attributes on let and do bindings

If they are at the top-level in a module, let and do bindings can have attributes. F#
attributes use the syntax [<MyAttribute>] .
Here are some examples in C# and then the same code in F#:

395

Binding with let, use, and do

class AttributeTest
{
[Obsolete]
public static int MyObsoleteFunction(int x, int y)
{
return x + y;
}
[CLSCompliant(false)]
public static void NonCompliant()
{
}
}

module AttributeTest =
[<Obsolete>]
let myObsoleteFunction x y = x + y
[<CLSCompliant(false)>]
let nonCompliant () = ()

Let's have a brief look at three attribute examples:

The EntryPoint attribute used to indicate the "main" function.
The various AssemblyInfo attributes.
The DllImport attribute for interacting with unmanaged code.

The EntryPoint attribute

The special EntryPoint attribute is used to mark the entry point of a standalone app, just as
in C#, the static void Main method is.
Here's the familiar C# version:
class Program
{
static int Main(string[] args)
{
foreach (var arg in args)
{
Console.WriteLine(arg);
}
//same as Environment.Exit(code)
return 0;
}
}

396

Binding with let, use, and do

And here's the F# equivalent:

module Program
[<EntryPoint>]
let main args =
args |> Array.iter printfn "%A"
0 // return is required!

Just as in C#, the args are an array of strings. But unlike C#, where the static Main method
can be void , the F# function must return an int.
Also, a big gotcha is that the function that has this attribute must be the very last function in
the last file in the project! Otherwise you get this error:
error FS0191: A function labelled with the 'EntryPointAttribute' atribute must be the
last declaration in the last file in the compilation sequence

Why is the F# compiler so fussy? In C#, the class can go anywhere.

One analogy that might help is this: in some sense, the whole application is a single huge
expression bound to main , where main is an expression that contains subexpressions that
contain other subexpressions.
[<EntryPoint>]
let main args =
the entire application as a set of subexpressions

Now in F# projects, there are no forward references allowed. That is, expressions that refer
to other expressions must be declared after them. And so logically, the highest, most toplevel function of them all, main , must come last of all.

The AssemblyInfo attributes

In a C# project, there is an AssemblyInfo.cs file that contains all the assembly level
attributes.
In F#, the equivalent way to do this is with a dummy module which contains a do
expression annotated with these attributes.

397

Binding with let, use, and do

open System.Reflection
module AssemblyInfo =
[<assembly: AssemblyTitle("MyAssembly")>]
[<assembly: AssemblyVersion("1.2.0.0")>]
[<assembly: AssemblyFileVersion("1.2.3.4152")>]
do () // do nothing -- just a placeholder for the attribute

The DllImport attribute

Another occasionally useful attribute is the DllImport attribute. Here's a C# example.
using System.Runtime.InteropServices;
[TestFixture]
public class TestDllImport
{
[DllImport("shlwapi", CharSet = CharSet.Auto, EntryPoint = "PathCanonicalize", Set
LastError = true)]
private static extern bool PathCanonicalize(StringBuilder lpszDst, string lpszSrc)
;
[Test]
public void TestPathCanonicalize()
{
var input = @"A:\name_1\.\name_2\..\name_3";
var expected = @"A:\name_1\name_3";
var builder = new StringBuilder(260);
PathCanonicalize(builder, input);
var actual = builder.ToString();
Assert.AreEqual(expected,actual);
}
}

It works the same way in F# as in C#. One thing to note is that the extern declaration ...
puts the types before the parameters, C-style.

398

Binding with let, use, and do

open System.Runtime.InteropServices
open System.Text
[<DllImport("shlwapi", CharSet = CharSet.Ansi, EntryPoint = "PathCanonicalize", SetLas
tError = true)>]
extern bool PathCanonicalize(StringBuilder lpszDst, string lpszSrc)
let TestPathCanonicalize() =
let input = @"A:\name_1\.\name_2\..\name_3"
let expected = @"A:\name_1\name_3"
let builder = new StringBuilder(260)
let success = PathCanonicalize(builder, input)
let actual = builder.ToString()
printfn "actual=%s success=%b" actual (expected = actual)
// test
TestPathCanonicalize()

Interop with unmanaged code is a big topic which will need its own series.

399

F# syntax: indentation and verbosity

The syntax for F# is mostly straightforward. But there are a few rules that you should
understand if you want to avoid common indentation errors. If you are familiar with a
language like Python that also is whitespace sensitive, be aware that that the rules for
indentation in F# are subtly different.

Indentation and the "offside" rule

In soccer, the offside rule says that in some situations, a player cannot be "ahead" of the ball
when they should be behind or level with it. The "offside line" is the line the player must not
cross. F# uses the same term to describe the line at which indentation must start. As with
soccer, the trick to avoiding a penalty is to know where the line is and not get ahead of it.
Generally, once an offside line has been set, all the expressions must align with the line.
//character columns
//3456789
let f =
let x=1 // offside line is at column 3
let y=1 // this line must start at column 3
x+y // this line must start at column 3
let f =
let x=1 // offside line is at column 3
x+1 // oops! don't start at column 4
// error FS0010: Unexpected identifier in binding
let f =
let x=1 // offside line is at column 3
x+1 // offside! You are ahead of the ball!
// error FS0588: Block following this
// 'let' is unfinished

Various tokens can trigger new offside lines to be created. For example, when the F# sees
the " = " used in a let expression, a new offside line is created at the position of the very
next symbol or word encountered.

400

F# syntax: indentation and verbosity

//character columns
//34567890123456789
let f = let x=1 // line is now at column 11 (start of "let x=")
x+1 // must start at column 11 from now on
// | // offside line at col 11
let f = let x=1 // line is now at column 11 (start of "let x=")
x+1 // offside!

// | // offside line at col 4

let f =
let x=1 // first word after = sign defines the line
// offside line is now at column 4
x+1 // must start at column 4 from now on

Other tokens have the same behavior, including parentheses, " then ", " else ", " try ",
" finally " and " do ", and " -> " in match clauses.
//character columns
//34567890123456789
let f =
let g = (
1+2) // first char after "(" defines
// a new line at col 5
g
let f =
if true then
1+2 // first char after "then" defines
// a new line at col 5
let f =
match 1 with
| 1 ->
1+2 // first char after match "->" defines
// a new line at col 8

The offside lines can be nested, and are pushed and popped as you would expect:

401

F# syntax: indentation and verbosity

//character columns
//34567890123456789
let f =
let g = let x = 1 // first word after "let g ="
// defines a new offside line at col 12
x + 1 // "x" must align at col 12
// pop the offside line stack now
g + 1 // back to previous line. "g" must align
// at col 4

New offside lines can never go forward further than the previous line on the stack:
let f =
let g = ( // let defines a new line at col 4
1+2) // oops! Cant define new line less than 4
g

Special cases
There are number of special cases which have been created to make code formatting more
flexible. Many of them will seem natural, such as aligning the start of each part of an ifthen-else expression or a try-catch expression. There are some non-obvious ones,

however.
Infix operators such as "+", "|>" and ">>" are allowed to be outside the line by their length
plus one space:
//character columns
//34567890123456789
let x = 1 // defines a new line at col 10
+ 2 // "+" allowed to be outside the line
+ 3
let f g h = g // defines a new line at col 15
>> h // ">>" allowed to be outside the line

If an infix operator starts a line, that line does not have to be strict about the alignment:
let x = 1 // defines a new line at col 10
+ 2 // infix operators that start a line don't count
* 3 // starts with "*" so doesn't need to align
- 4 // starts with "-" so doesn't need to align

If a " fun " keyword starts an expression, the "fun" does not start a new offside line:

402

F# syntax: indentation and verbosity

//character columns
//34567890123456789
let f = fun x -> // "fun" should define a new line at col 9
let y = 1 // but doesn't. The real line starts here.
x + y

Finding out more

There are many more details as to how indentation works, but the examples above should
cover most of the common cases. If you want to know more, the complete language spec for
F# is available from Microsoft as a downloadable PDF, and is well worth reading.

"Verbose" syntax
By default, F# uses indentation to indicate block structure -- this is called "light" syntax.
There is an alternative syntax that does not use indentation; it is called "verbose" syntax.
With verbose syntax, you are not required to use indentation, and whitespace is not
significant, but the downside is that you are required to use many more keywords, including
things like:
" in " keywords after every "let" and "do" binding
" begin "/" end " keywords for code blocks such as if-then-else
" done " keywords at the end of loops
keywords at the beginning and end of type definitions
Here is an example of verbose syntax with wacky indentation that would not otherwise be
acceptable:
#indent "off"
let f =
let x = 1 in
if x=2 then
begin "a" end else begin
"b"
end
#indent "on"

Verbose syntax is always available, even in "light" mode, and is occasionally useful. For
example, when you want to embed "let" into a one line expression:

403

F# syntax: indentation and verbosity

let x = let y = 1 in let z = 2 in y + z

Other cases when you might want to use verbose syntax are:
when outputting generated code
to be compatible with OCaml
if you are visually impaired or blind and use a screen reader
or just to gain some insight into the abstract syntax tree used by the F# parser
Other than these cases, verbose syntax is rarely used in practice.

404

Parameter and value naming conventions

If you are coming to F# from an imperative language such as C#, then you might find a lot of
the names shorter and more cryptic than you are used to.
In C# and Java, the best practice is to have long descriptive identifiers. In functional
languages, the function names themselves can be descriptive, but the local identifiers inside
a function tend to be quite short, and piping and composition is used a lot to get everything
on a minimal number of lines.
For example, here is a crude implementation of a prime number sieve with very descriptive
names for the local values.
let primesUpTo n =
// create a recursive intermediate function
let rec sieve listOfNumbers =
match listOfNumbers with
| [] -> []
| primeP::sievedNumbersBiggerThanP->
let sievedNumbersNotDivisibleByP =
sievedNumbersBiggerThanP
|> List.filter (fun i-> i % primeP > 0)
//recursive part
let newPrimes = sieve sievedNumbersNotDivisibleByP
primeP :: newPrimes
// use the sieve
let listOfNumbers = [2..n]
sieve listOfNumbers // return
//test
primesUpTo 100

Here is the same implementation, with terser, idiomatic names and more compact code:
let primesUpTo n =
let rec sieve l =
match l with
| [] -> []
| p::xs ->
p :: sieve [for x in xs do if (x % p) > 0 then yield x]
[2..n] |> sieve

The cryptic names are not always better, of course, but if the function is kept to a few lines
and the operations used are standard, then this is a fairly common idiom.

405

Parameter and value naming conventions

The common naming conventions are as follows:

"a", "b", "c" etc., are types
"f", "g", "h" etc., are functions
"x", "y", "z" etc., are arguments to the functions
Lists are indicated by adding an "s" suffix, so that " xs " is a list of x 's, " fs " is a list of
functions, and so on. It is extremely common to see " x::xs " meaning the head (first
element) and tail (the remaining elements) of a list.
"" is used whenever you don't care about the value. So "`x:: " means that you don't care
about the rest of the list, and " let f _ = something " means you don't care about the
argument to f`.

Another reason for the short names is that often, they cannot be assigned to anything
meaningful. For example, the definition of the pipe operator is:
let (|>) x f = f x

We don't know what f and x are going to be, f could be any function and x could be
any value. Making this explicit does not make the code any more understandable.
let (|>) aValue aFunction = aFunction aValue // any better?

The style used on this site

On this site I will use both styles. For the introductory series, when most of the concepts are
new, I will use a very descriptive style, with intermediate values and long names. But in more
advanced series, the style will become terser.

406

Control flow expressions

In this post, we'll look at the control flow expressions, namely:
if-then-else
for x in collection (which is the same as foreach in C#)
for x = start to end
while-do
These control flow expressions are no doubt very familiar to you. But they are very
"imperative" rather than functional.
So I would strongly recommend that you do not use them if at all possible, especially when
you are learning to think functionally. If you do use them as a crutch, you will find it much
harder to break away from imperative thinking.
To help you do this, I will start each section with examples of how to avoid using them by
using more idiomatic constructs instead. If you do need to use them, there are some
"gotchas" that you need to be aware of.

If-then-else
How to avoid using if-then-else
The best way to avoid if-then-else is to use "match" instead. You can match on a
boolean, which is similar to the classic then/else branches. But much, much better, is to
avoid the equality test and actually match on the thing itself, as shown in the last
implementation below.

407

Control flow expressions

// bad
let f x =
if x = 1
then "a"
else "b"
// not much better
let f x =
match x=1 with
| true -> "a"
| false -> "b"
// best
let f x =
match x with
| 1 -> "a"
| _ -> "b"

Part of the reason why direct matching is better is that the equality test throws away useful
information that you often need to retrieve again.
This is demonstrated by the next scenario, where we want to get the first element of a list in
order to print it. Obviously, we must be careful not to attempt this for an empty list.
The first implementation does a test for empty and then a second operation to get the first
element. A much better approach is to match and extract the element in one single step, as
shown in the second implementation.
// bad
let f list =
if List.isEmpty list
then printfn "is empty"
else printfn "first element is %s" (List.head list)
// much better
let f list =
match list with
| [] -> printfn "is empty"
| x::_ -> printfn "first element is %s" x

The second implementation is not only easier to understand, it is more efficient.

If the boolean test is complicated, it can still be done with match by using extra " when "
clauses (called "guards"). Compare the first and second implementations below to see the
difference.

408

Control flow expressions

// bad
let f list =
if List.isEmpty list
then printfn "is empty"
elif (List.head list) > 0
then printfn "first element is > 0"
else printfn "first element is <= 0"
// much better
let f list =
match list with
| [] -> printfn "is empty"
| x::_ when x > 0 -> printfn "first element is > 0"
| x::_ -> printfn "first element is <= 0"

Again, the second implementation is easier to understand and also more efficient.
The moral of the tale is: if you find yourself using if-then-else or matching on booleans,
consider refactoring your code.

How to use if-then-else

If you do need to use if-then-else, be aware that even though the syntax looks familiar, there
is a catch that you must be aware of: " if-then-else " is an expression, not a statement, and
as with every expression in F#, it must return a value of a particular type.
Here are two examples where the return type is a string.
let v = if true then "a" else "b" // value : string
let f x = if x then "a" else "b" // function : bool->string

But as a consequence, both branches must return the same type! If this is not true, then the
expression as a whole cannot return a consistent type and the compiler will complain.
Here is an example of different types in each branch:
let v = if true then "a" else 2
// error FS0001: This expression was expected to have
// type string but here has type int

The "else" clause is optional, but if it is absent, the "else" clause is assumed to return unit,
which means that the "then" clause must also return unit. You will get a complaint from the
compiler if you make this mistake.

409

Control flow expressions

let v = if true then "a"

// error FS0001: This expression was expected to have type unit
// but here has type string

If the "then" clause returns unit, then the compiler will be happy.
let v2 = if true then printfn "a" // OK as printfn returns unit

Note that there is no way to return early in a branch. The return value is the entire
expression. In other words, the if-then-else expression is more closely related to the C#
ternary if operator (?:) than to the C# if-then-else statement.

if-then-else for one liners

One of the places where if-then-else can be genuinely useful is to create simple one-liners
for passing into other functions.
let posNeg x = if x > 0 then "+" elif x < 0 then "-" else "0"
[-5..5] |> List.map posNeg

Returning functions
Don't forget that an if-then-else expression can return any value, including function values.
For example:
let greetings =
if (System.DateTime.Now.Hour < 12)
then (fun name -> "good morning, " + name)
else (fun name -> "good day, " + name)
//test
greetings "Alice"

Of course, both functions must have the same type, meaning that they must have the same
function signature.

Loops
How to avoid using loops

410

Control flow expressions

The best way to avoid loops is to use the built in list and sequence functions instead. Almost
anything you want to do can be done without using explicit loops. And often, as a side
benefit, you can avoid mutable values as well. Here are some examples to start with, and for
more details please read the upcoming series devoted to list and sequence operations.
Example: Printing something 10 times:
// bad
for i = 1 to 10 do
printf "%i" i
// much better
[1..10] |> List.iter (printf "%i")

Example: Summing a list:

// bad
let sum list =
let mutable total = 0 // uh-oh -- mutable value
for e in list do
total <- total + e // update the mutable value
total // return the total
// much better
let sum list = List.reduce (+) list
//test
sum [1..10]

Example: Generating and printing a sequence of random numbers:

411

Control flow expressions

// bad
let printRandomNumbersUntilMatched matchValue maxValue =
let mutable continueLooping = true // another mutable value
let randomNumberGenerator = new System.Random()
while continueLooping do
// Generate a random number between 1 and maxValue.
let rand = randomNumberGenerator.Next(maxValue)
printf "%d " rand
if rand = matchValue then
printfn "\nFound a %d!" matchValue
continueLooping <- false
// much better
let printRandomNumbersUntilMatched matchValue maxValue =
let randomNumberGenerator = new System.Random()
let sequenceGenerator _ = randomNumberGenerator.Next(maxValue)
let isNotMatch = (<>) matchValue
//create and process the sequence of rands
Seq.initInfinite sequenceGenerator
|> Seq.takeWhile isNotMatch
|> Seq.iter (printf "%d ")
// done
printfn "\nFound a %d!" matchValue
//test
printRandomNumbersUntilMatched 10 20

As with if-then-else, there is a moral; if you find yourself using loops and mutables, please
consider refactoring your code to avoid them.

The three types of loops

If you want to use loops, then there are three types of loop expressions to choose from,
which are similar to those in C#.
for-in-do . This has the form for x in enumerable do something . It is the same as the
foreach loop in C#, and is the form most commonly seen in F#.
for-to-do . This has the form for x = start to finish do something . It is the same as

the standard for (i=start; i<end; i++) loops in C#.

while-do . This has the form while test do something . It is the same as the while

loop in C#. Note that there is no do-while equivalent in F#.

I won't go into any more detail than this, as the usage is straightforward. If you have trouble,
check the MSDN documentation.

412

Control flow expressions

How to use loops

As with if-then-else expressions, the loop expressions look familiar, but there are some
catches again.
All looping expressions always return unit for the whole expression, so there is no way
to return a value from inside a loop.
As with all "do" bindings, the expression inside the loop must return unit as well.
There is no equivalent of "break" and "continue" (this can generally done better using
sequences anyway)
Here's an example of the unit constraint. The expression in the loop should be unit, not int,
so the compiler will complain.
let f =
for i in [1..10] do
i + i // warning: This expression should have type 'unit'
// version 2
let f =
for i in [1..10] do
i + i |> ignore // fixed

Loops for one liners

One of the places where loops are used in practice is as list and sequence generators.
let myList = [for x in 0..100 do if x*x < 100 then yield x ]

Summary
I'll repeat what I said at the top of the post: do avoid using imperative control flow when you
are learning to think functionally. And understand the exceptions that prove the rule; the oneliners whose use is acceptable.

413

Exceptions

Exceptions
Just like other .NET languages, F# supports throwing and catching exceptions. As with the
control flow expressions, the syntax will feel familiar, but again there are a few catches that
you should know about.

Defining your own exceptions

When raising/throwing exceptions, you can use the standard system ones such as
InvalidOperationException , or you can define your own exception types using the simple

syntax shown below, where the "content" of the exception is any F# type:
exception MyFSharpError1 of string
exception MyFSharpError2 of string * int

That's it! Defining new exception classes is a lot easier than in C#!

Throwing exceptions
There are three basic ways to throw an exception
Using one of the built in functions, such as "invalidArg"
Using one of the standard .NET exception classes
Using your own custom exception types

Throwing exceptions, method 1: using one of the built in

functions
There are four useful exception keywords built into F#:
failwith throws a generic System.Exception
invalidArg throws an ArgumentException
nullArg throws a NullArgumentException
invalidOp throws an InvalidOperationException

These four probably cover most of the exceptions you would regularly throw. Here is how
they are used:

414

Exceptions

// throws a generic System.Exception

let f x =
if x then "ok"
else failwith "message"
// throws an ArgumentException
let f x =
if x then "ok"
else invalidArg "paramName" "message"
// throws a NullArgumentException
let f x =
if x then "ok"
else nullArg "paramName" "message"
// throws an InvalidOperationException
let f x =
if x then "ok"
else invalidOp "message"

By the way, there's a very useful variant of failwith called failwithf that includes
printf style formatting, so that you can make custom messages easily:

open System
let f x =
if x = "bad" then
failwithf "Operation '%s' failed at time %O" x DateTime.Now
else
printfn "Operation '%s' succeeded at time %O" x DateTime.Now
// test
f "good"
f "bad"

Throwing exceptions, method 2: using one of the standard

.NET exception classes
You can raise any .NET exception explicitly:
// you control the exception type
let f x =
if x then "ok"
else raise (new InvalidOperationException("message"))

Throwing exceptions, method 3: using your own F#

exception types
415

Exceptions

Finally, you can use your own types, as defined earlier.

// using your own F# exception types
let f x =
if x then "ok"
else raise (MyFSharpError1 "message")

And that's pretty much it for throwing exceptions.

What effect does raising an exception have on

the function type?
We said earlier that both branches of an if-then-else expression must return the same type.
But how can raising an exception work with this constraint?
The answer is that any code that raises exceptions is ignored for the purposes of
determining expression types. This means that the function signature will be based on the
normal case only, not the exception case.
For example, in the code below, the exceptions are ignored, and the overall function has
signature bool->int , as you would expect.
let f x =
if x then 42
elif true then failwith "message"
else invalidArg "paramName" "message"

Question: what do you think the function signature will be if both branches raise exceptions?
let f x =
if x then failwith "error in true branch"
else failwith "error in false branch"

Try it and see!

Catching exceptions
Exceptions are caught using a try-catch block, as in other languages. F# calls it try-with
instead, and testing for each type of exception uses the standard pattern matching syntax.

416

Exceptions

try
failwith "fail"
with
| Failure msg -> "caught: " + msg
| MyFSharpError1 msg -> " MyFSharpError1: " + msg
| :? System.InvalidOperationException as ex -> "unexpected"

If the exception to catch was thrown with failwith (e.g. a System.Exception) or a custom
F# exception, you can match using the simple tag approach shown above.
On the other hand, to catch a specific .NET exception class, you have to match using the
more complicated syntax:
:? (exception class) as ex

Again, as with if-then-else and the loops, the try-with block is an expression that returns a
value. This means that all branches of the try-with expression must return the same type.
Consider this example:
let divide x y=
try
(x+1) / y // error here -- see below
with
| :? System.DivideByZeroException as ex ->
printfn "%s" ex.Message

When we try to evaluate it, we get an error:

error FS0043: The type 'unit' does not match the type 'int'

The reason is that the " with " branch is of type unit , while the " try " branch is of type
int . So the two branches are of incompatible types.

To fix this, we need to make the " with " branch also return type int . We can do this easily
using the semicolon trick to chain expressions on one line.

417

Exceptions

let divide x y=
try
(x+1) / y
with
| :? System.DivideByZeroException as ex ->
printfn "%s" ex.Message; 0 // added 0 here!
//test
divide 1 1
divide 1 0

Now that the try-with expression has a defined type, the whole function can be assigned a
type, namely int -> int -> int , as expected.
As before, if any branch throws an exception, it doesn't count when types are being
determined.

Rethrowing exceptions
If needed, you can call the " reraise() " function in a catch handler to propagate the same
exception up the call chain. This is the same as the C# throw keyword.
let divide x y=
try
(x+1) / y
with
| :? System.DivideByZeroException as ex ->
printfn "%s" ex.Message
reraise()
//test
divide 1 1
divide 1 0

Try-finally
Another familiar expression is try-finally . As you might expect, the "finally" clause will be
called no matter what.
let f x =
try
if x then "ok" else failwith "fail"
finally
printf "this will always be printed"

418

Exceptions

The return type of the try-finally expression as a whole is always the same as return type of
the "try" clause on its own. The "finally" clause has no effect on the type of the expression as
a whole. So in the above example, the whole expression has type string .
The "finally" clause must always return unit, so any non-unit values will be flagged by the
compiler.
let f x =
try
if x then "ok" else failwith "fail"
finally
1+1 // This expression should have type 'unit

Combining try-with and try-finally

The try-with and the try-finally expressions are distinct and cannot be combined directly into
a single expression. Instead, you will have to nest them as circumstances require.
let divide x y=
try
try
(x+1) / y
finally
printf "this will always be printed"
with
| :? System.DivideByZeroException as ex ->
printfn "%s" ex.Message; 0

Should functions throw exceptions or return

error structures?
When you are designing a function, should you throw exceptions, or return structures which
encode the error? This section will discuss two different approaches.

The pair of functions approach

One approach is to provide two functions: one which assumes everything works and throws
an exception otherwise and a second "tryXXX" function that returns a missing value if
something goes wrong.
For example, we might want to design two distinct library functions for division, one that
doesn't handle exceptions and one that does:
419

Exceptions

// library function that doesn't handle exceptions

let divideExn x y = x / y
// library function that converts exceptions to None
let tryDivide x y =
try
Some (x / y)
with
| :? System.DivideByZeroException -> None // return missing

Note the use of Some and None Option types in the tryDivide code to signal to the client
whether the value is valid.
With the first function, the client code must handle the exception explicitly.
// client code must handle exceptions explicitly
try
let n = divideExn 1 0
printfn "result is %i" n
with
| :? System.DivideByZeroException as ex -> printfn "divide by zero"

Note that there is no constraint that forces the client to do this, so this approach can be a
source of errors.
With the second function the client code is simpler, and the client is constrained to handle
both the normal case and the error case.
// client code must test both cases
match tryDivide 1 0 with
| Some n -> printfn "result is %i" n
| None -> printfn "divide by zero"

This "normal vs. try" approach is very common in the .NET BCL, and also occurs in a few
cases in the F# libraries too. For example, in the List module:
List.find will throw a KeyNotFoundException if the key is not found

But List.tryFind will return an Option type, with None if the key is not found
If you are going to use this approach, do have a naming convention. For example:
"doSomethingExn" for functions that expect clients to catch exceptions.
"tryDoSomething" for functions that handle normal exceptions for you.
Note that I prefer to have an "Exn" suffix on "doSomething" rather than no suffix at all. It
makes it clear that you expect clients to catch exceptions even in normal cases.

420

Exceptions

The overall problem with this approach is that you have to do extra work to create pairs of
functions, and you reduce the safety of the system by relying on the client to catch
exceptions if they use the unsafe version of the function.

The error-code-based approach

"Writing good error-code-based code is hard, but writing good exception-based code is
really hard." Raymond Chen
In the functional world, returning error codes (or rather error types) is generally preferred to
throwing exceptions, and so a standard hybrid approach is to encode the common cases
(the ones that you would expect a user to care about) into a error type, but leave the very
unusual exceptions alone.
Often, the simplest approach is just to use the option type: Some for success and None for
errors. If the error case is obvious, as in tryDivide or tryParse , there is no need to be
explicit with more detailed error cases.
But sometimes there is more than one possible error, and each should be handled
differently. In this case, a union type with a case for each error is useful.
In the following example, we want to execute a SqlCommand. Three very common error
cases are login errors, constraint errors and foreign key errors, so we build them into the
result structure. All other errors are raised as exceptions.

421

Exceptions

open System.Data.SqlClient
type NonQueryResult =
| Success of int
| LoginError of SqlException
| ConstraintError of SqlException
| ForeignKeyError of SqlException
let executeNonQuery (sqlCommmand:SqlCommand) =
try
use sqlConnection = new SqlConnection("myconnection")
sqlCommmand.Connection <- sqlConnection
let result = sqlCommmand.ExecuteNonQuery()
Success result
with
| :?SqlException as ex -> // if a SqlException
match ex.Number with
| 18456 -> // login Failed
LoginError ex
| 2601 | 2627 -> // handle constraint error
ConstraintError ex
| 547 -> // handle FK error
ForeignKeyError ex
| _ -> // don't handle any other cases
reraise()
// all non SqlExceptions are thrown normally

The client is then forced to handle the common cases, while uncommon exceptions will be
caught by a handler higher up the call chain.
let myCmd = new SqlCommand("DELETE Product WHERE ProductId=1")
let result = executeNonQuery myCmd
match result with
| Success n -> printfn "success"
| LoginError ex -> printfn "LoginError: %s" ex.Message
| ConstraintError ex -> printfn "ConstraintError: %s" ex.Message
| ForeignKeyError ex -> printfn "ForeignKeyError: %s" ex.Message

Unlike a traditional error code approach, the caller of the function does not have to handle
any errors immediately, and can simply pass the structure around until it gets to someone
who knows how to handle it, as shown below:

422

Exceptions

let lowLevelFunction commandString =

let myCmd = new SqlCommand(commandString)
executeNonQuery myCmd //returns result
let deleteProduct id =
let commandString = sprintf "DELETE Product WHERE ProductId=%i" id
lowLevelFunction commandString //returns without handling errors
let presentationLayerFunction =
let result = deleteProduct 1
match result with
| Success n -> printfn "success"
| errorCase -> printfn "error %A" errorCase

On the other hand, unlike C#, the result of a expression cannot be accidentally thrown away.
So if a function returns an error result, the caller must handle it (unless it really wants to be
badly behaved and send it to ignore )
let presentationLayerFunction =
do deleteProduct 1 // error: throwing away a result code!

423

Match expressions

Match expressions
Pattern matching is ubiquitous in F#. It is used for binding values to expressions with let ,
and in function parameters, and for branching using the match..with syntax.
We have briefly covered binding values to expressions in a post in the "why use F#?" series,
and it will be covered many times as we investigate types.
So in this post, we'll cover the match..with syntax and its use for control flow.

What is a match expression?

We have already seen match..with expressions a number of times. And we know that it
has the form:
match [something] with
| pattern1 -> expression1
| pattern2 -> expression2
| pattern3 -> expression3

If you squint at it just right, it looks a bit like a series of lambda expressions:
match [something] with
| lambda-expression-1
| lambda-expression-2
| lambda-expression-3

Where each lambda expression has exactly one parameter:

param -> expression

So one way of thinking about match..with is that it is a choice between a set of lambda
expressions. But how to make the choice?
This is where the patterns come in. The choice is made based on whether the "match with"
value can be matched with the parameter of the lambda expression. The first lambda whose
parameter can be made to match the input value "wins"!
So for example, if the param is the wildcard _ , it will always match, and if first, always win.

424

Match expressions

_ -> expression

Order is important!
Looking at the following example:
let x =
match 1 with
| 1 -> "a"
| 2 -> "b"
| _ -> "z"

We can see that there are three lambda expressions to match, in this order:
fun 1 -> "a"
fun 2 -> "b"
fun _ -> "z"

So, the 1 pattern gets tried first, then then the 2 pattern, and finally, the _ pattern.
On the other hand, if we changed the order to put the wildcard first, it would be tried first and
always win immediately:
let x =
match 1 with
| _ -> "z"
| 1 -> "a"
| 2 -> "b"

In this case, the F# compiler helpfully warns us that the other rules will never be matched.
So this is one major difference between a " switch " or " case " statement compared with a
match..with . In a match..with , the order is important.

Formatting a match expression

Since F# is sensitive to indentation, you might be wondering how best to format this
expression, as there are quite a few moving parts.
The post on F# syntax gives an overview of how alignment works, but for match..with
expressions, here are some specific guidelines.

425

Match expressions

Guideline 1: The alignment of the | expression clauses should be directly under the
match

This guideline is straightforward.

let f x = match x with
// aligned
| 1 -> "pattern 1"
// aligned
| 2 -> "pattern 2"
// aligned
| _ -> "anything"

Guideline 2: The match..with should be on a new line

The match..with can be on the same line or a new line, but using a new line keeps the
indenting consistent, independent of the lengths of the names:
// ugly alignment!
let myVeryLongNameForAFunction myParameter = match myParameter with
| 1 -> "something"
| _ -> "anything"
// much better
let myVeryLongNameForAFunction myParameter =
match myParameter with
| 1 -> "something"
| _ -> "anything"

Guideline 3: The expression after the arrow -> should be on a new line
Again, the result expression can be on the same line as the arrow, but using a new line
again keeps the indenting consistent and helps to separate the match pattern from the result
expression.
let f x =
match x with
| "a very long pattern that breaks up the flow" -> "something"
| _ -> "anything"
let f x =
match x with
| "a very long pattern that breaks up the flow" ->
"something"
| _ ->
"anything"

426

Match expressions

Of course, when all the patterns are very compact, a common sense exception can be
made:
let f list =
match list with
| [] -> "something"
| x::xs -> "something else"

match..with is an expression
It is important to realize that match..with is not really a "control flow" construct. The
"control" does not "flow" down the branches, but instead, the whole thing is an expression
that gets evaluated at some point, just like any other expression. The end result in practice
might be the same, but it is a conceptual difference that can be important.
One consequence of it being an expression is that all branches must evaluate to the same
type -- we have already seen this same behavior with if-then-else expressions and for loops.
let x =
match 1 with
| 1 -> 42
| 2 -> true // error wrong type
| _ -> "hello" // error wrong type

You cannot mix and match the types in the expression.

You can use match expressions anywhere

Since they are normal expressions, match expressions can appear anywhere an expression
can be used.
For example, here's a nested match expression:
// nested match..withs are ok
let f aValue =
match aValue with
| x ->
match x with
| _ -> "something"

And here's a match expression embedded in a lambda:

427

Match expressions

[2..10]
|> List.map (fun i ->
match i with
| 2 | 3 | 5 | 7 -> sprintf "%i is prime" i
| _ -> sprintf "%i is not prime" i
)

Exhaustive matching
Another consequence of being an expression is that there must always be some branch that
matches. The expression as a whole must evaluate to something!
That is, the valuable concept of "exhaustive matching" comes from the "everything-is-anexpression" nature of F#. In a statement oriented language, there would be no requirement
for this to happen.
Here's an example of an incomplete match:
let x =
match 42 with
| 1 -> "a"
| 2 -> "b"

The compiler will warn you if it thinks there is a missing branch. And if you deliberately
ignore the warning, then you will get a nasty runtime error ( MatchFailureException ) when
none of the patterns match.

Exhaustive matching is not perfect

The algorithm for checking that all possible matches are listed is good but not always
perfect. Occasionally it will complain that you have not matched every possible case, when
you know that you have. In this case, you may need to add an extra case just to keep the
compiler happy.

Using (and avoiding) the wildcard match

One way to guarantee that you always match all cases is to put the wildcard parameter as
the last match:

428

Match expressions

let x =
match 42 with
| 1 -> "a"
| 2 -> "b"
| _ -> "z"

You see this pattern frequently, and I have used it a lot in these examples. It's the equivalent
of having a catch-all default in a switch statement.
But if you want to get the full benefits of exhaustive pattern matching, I would encourage you
not to use wildcards, and try to match all the cases explicitly if you can. This is particularly
true if you are matching on the cases of a union type:
type Choices = A | B | C
let x =
match A with
| A -> "a"
| B -> "b"
| C -> "c"
//NO default match

By being always explicit in this way, you can trap any error caused by adding a new case to
the union. If you had a wildcard match, you would never know.
If you can't have every case be explicit, you might try to document your boundary conditions
as much as possible, and assert an runtime error for the wildcard case.
let x =
match -1 with
| 1 -> "a"
| 2 -> "b"
| i when i >= 0 && i<=100 -> "ok"
// the last case will always match
| x -> failwithf "%i is out of range" x

Types of patterns
There are lots of different ways of matching patterns, which we'll look at next.
For more details on the various patterns, see the MSDN documentation.

Binding to values
The most basic pattern is to bind to a value as part of the match:

429

Match expressions

let y =
match (1,0) with
// binding to a named value
| (1,x) -> printfn "x=%A" x

By the way, I have deliberately left this pattern (and others in this post) as incomplete. As an
exercise, make them complete without using the wildcard.
It is important to note that the values that are bound must be distinct for each pattern. So
you can't do something like this:
let elementsAreEqual aTuple =
match aTuple with
| (x,x) ->
printfn "both parts are the same"
| (_,_) ->
printfn "both parts are different"

Instead, you have to do something like this:

let elementsAreEqual aTuple =
match aTuple with
| (x,y) ->
if (x=y) then printfn "both parts are the same"
else printfn "both parts are different"

This second option can also be rewritten using "guards" ( when clauses) instead. Guards will
be discussed shortly.

AND and OR
You can combine multiple patterns on one line, with OR logic and AND logic:
let y =
match (1,0) with
// OR -- same as multiple cases on one line
| (2,x) | (3,x) | (4,x) -> printfn "x=%A" x
// AND -- must match both patterns at once
// Note only a single "&" is used
| (2,x) & (_,1) -> printfn "x=%A" x

The OR logic is particularly common when matching a large number of union cases:

430

Match expressions

type Choices = A | B | C | D
let x =
match A with
| A | B | C -> "a or b or c"
| D -> "d"

Matching on lists
Lists can be matched explicitly in the form [x;y;z] or in the "cons" form head::tail :
let y =
match [1;2;3] with
// binding to explicit positions
// square brackets used!
| [1;x;y] -> printfn "x=%A y=%A" x y
// binding to head::tail.
// no square brackets used!
| 1::tail -> printfn "tail=%A" tail
// empty list
| [] -> printfn "empty"

A similar syntax is available for matching arrays exactly [|x;y;z|] .

It is important to understand that sequences (aka IEnumerables ) can not be matched on this
way directly, because they are "lazy" and meant to be accessed one element at a time. Lists
and arrays, on the other hand, are fully available to be matched on.
Of these patterns, the most common one is the "cons" pattern, often used in conjunction with
recursion to loop through the elements of the list.
Here are some examples of looping through lists using recursion:

431

Match expressions

// loop through a list and print the values

let rec loopAndPrint aList =
match aList with
// empty list means we're done.
| [] ->
printfn "empty"
// binding to head::tail.
| x::xs ->
printfn "element=%A," x
// do all over again with the
// rest of the list
loopAndPrint xs
//test
loopAndPrint [1..5]
// -----------------------// loop through a list and sum the values
let rec loopAndSum aList sumSoFar =
match aList with
// empty list means we're done.
| [] ->
sumSoFar
// binding to head::tail.
| x::xs ->
let newSumSoFar = sumSoFar + x
// do all over again with the
// rest of the list and the new sum
loopAndSum xs newSumSoFar
//test
loopAndSum [1..5] 0

The second example shows how we can carry state from one iteration of the loop to the next
using a special "accumulator" parameter (called sumSoFar in this example). This is a very
common pattern.

Matching on tuples, records and unions

Pattern matching is available for all the built-in F# types. More details in the series on types.

432

Match expressions

// ----------------------// Tuple pattern matching

let aTuple = (1,2)
match aTuple with
| (1,_) -> printfn "first part is 1"
| (_,2) -> printfn "second part is 2"

// ----------------------// Record pattern matching

type Person = {First:string; Last:string}
let person = {First="john"; Last="doe"}
match person with
| {First="john"} -> printfn "Matched John"
| _ -> printfn "Not John"
// ----------------------// Union pattern matching
type IntOrBool= I of int | B of bool
let intOrBool = I 42
match intOrBool with
| I i -> printfn "Int=%i" i
| B b -> printfn "Bool=%b" b

Matching the whole and the part with the "as" keyword
Sometimes you want to match the individual components of the value and also the whole
thing. You can use the as keyword for this.
let y =
match (1,0) with
// binding to three values
| (x,y) as t ->
printfn "x=%A and y=%A" x y
printfn "The whole tuple is %A" t

Matching on subtypes
You can match on subtypes, using the :? operator, which gives you a crude polymorphism:

433

Match expressions

let x = new Object()

let y =
match x with
| :? System.Int32 ->
printfn "matched an int"
| :? System.DateTime ->
printfn "matched a datetime"
| _ ->
printfn "another type"

This only works to find subclasses of a parent class (in this case, Object). The overall type of
the expression has the parent class as input.
Note that in some cases, you may need to "box" the value.
let detectType v =
match v with
| :? int -> printfn "this is an int"
| _ -> printfn "something else"
// error FS0008: This runtime coercion or type test from type 'a to int
// involves an indeterminate type based on information prior to this program point.
// Runtime type tests are not allowed on some types. Further type annotations are need
ed.

The message tells you the problem: "runtime type tests are not allowed on some types". The
answer is to "box" the value which forces it into a reference type, and then you can type
check it:
let detectTypeBoxed v =
match box v with // used "box v"
| :? int -> printfn "this is an int"
| _ -> printfn "something else"
//test
detectTypeBoxed 1
detectTypeBoxed 3.14

In my opinion, matching and dispatching on types is a code smell, just as it is in objectoriented programming. It is occasionally necessary, but used carelessly is an indication of
poor design.
In a good object oriented design, the correct approach would be to use polymorphism to
replace the subtype tests, along with techniques such as double dispatch. So if you are
doing this kind of OO in F#, you should probably use those same techniques.

434

Match expressions

Matching on multiple values

All the patterns we've looked at so far do pattern matching on a single value. How can you
do it for two or more?
The short answer is: you can't. Matches are only allowed on single values.
But wait a minute -- could we combine two values into a single tuple on the fly and match on
that? Yes, we can!
let matchOnTwoParameters x y =
match (x,y) with
| (1,y) ->
printfn "x=1 and y=%A" y
| (x,1) ->
printfn "x=%A and y=1" x

And indeed, this trick will work whenever you want to match on a set of values -- just group
them all into a single tuple.
let matchOnTwoTuples x y =
match (x,y) with
| (1,_),(1,_) -> "both start with 1"
| (_,2),(_,2) -> "both end with 2"
| _ -> "something else"
// test
matchOnTwoTuples (1,3) (1,2)
matchOnTwoTuples (3,2) (1,2)

Guards, or the "when" clause

Sometimes pattern matching is just not enough, as we saw in this example:
let elementsAreEqual aTuple =
match aTuple with
| (x,y) ->
if (x=y) then printfn "both parts are the same"
else printfn "both parts are different"

Pattern matching is based on patterns only -- it can't use functions or other kinds of
conditional tests.

435

Match expressions

But there is a way to do the equality test as part of the pattern match -- using an additional
when clause to the left of the function arrow. These clauses are known as "guards".

Here's the same logic written using a guard instead:

let elementsAreEqual aTuple =
match aTuple with
| (x,y) when x=y ->
printfn "both parts are the same"
| _ ->
printfn "both parts are different"

This is nicer, because we have integrated the test into the pattern proper, rather than using a
test after the match has been done.
Guards can be used for all sorts of things that pure patterns can't be used for, such as:
comparing the bound values
testing object properties
doing other kinds of matching, such as regular expressions
conditionals derived from functions
Let's look at some examples of these:
// -------------------------------// comparing values in a when clause
let makeOrdered aTuple =
match aTuple with
// swap if x is bigger than y
| (x,y) when x > y -> (y,x)
// otherwise leave alone
| _ -> aTuple
//test
makeOrdered (1,2)
makeOrdered (2,1)
// -------------------------------// testing properties in a when clause
let isAM aDate =
match aDate:System.DateTime with
| x when x.Hour <= 12->
printfn "AM"
// otherwise leave alone
| _ ->
printfn "PM"

436

Match expressions

//test
isAM System.DateTime.Now
// -------------------------------// pattern matching using regular expressions
open System.Text.RegularExpressions
let classifyString aString =
match aString with
| x when Regex.Match(x,@".+@.+").Success->
printfn "%s is an email" aString
// otherwise leave alone
| _ ->
printfn "%s is something else" aString

//test
classifyString "alice@example.com"
classifyString "google.com"
// -------------------------------// pattern matching using arbitrary conditionals
let fizzBuzz x =
match x with
| i when i % 15 = 0 ->
printfn "fizzbuzz"
| i when i % 3 = 0 ->
printfn "fizz"
| i when i % 5 = 0 ->
printfn "buzz"
| i ->
printfn "%i" i
//test
[1..30] |> List.iter fizzBuzz

Using active patterns instead of guards

Guards are great for one-off matches. But if there are certain guards that you use over and
over, consider using active patterns instead.
For example, the email example above could be rewritten as follows:

437

Match expressions

open System.Text.RegularExpressions
// create an active pattern to match an email address
let (|EmailAddress|_|) input =
let m = Regex.Match(input,@".+@.+")
if (m.Success) then Some input else None
// use the active pattern in the match
let classifyString aString =
match aString with
| EmailAddress x ->
printfn "%s is an email" x
// otherwise leave alone
| _ ->
printfn "%s is something else" aString
//test
classifyString "alice@example.com"
classifyString "google.com"

You can see other examples of active patterns in a previous post.

The "function" keyword

In the examples so far, we've seen a lot of this:
let f aValue =
match aValue with
| _ -> "something"

In the special case of function definitions we can simplify this dramatically by using the
function keyword.

let f =
function
| _ -> "something"

As you can see, the aValue parameter has completely disappeared, along with the
match..with .

This keyword is not the same as the fun keyword for standard lambdas, rather it combines
fun and match..with in a single step.

438

Match expressions

The function keyword works anywhere a function definition or lambda can be used, such
as nested matches:
// using match..with
let f aValue =
match aValue with
| x ->
match x with
| _ -> "something"
// using function keyword
let f =
function
| x ->
function
| _ -> "something"

or lambdas passed to a higher order function:

// using match..with
[2..10] |> List.map (fun i ->
match i with
| 2 | 3 | 5 | 7 -> sprintf "%i is prime" i
| _ -> sprintf "%i is not prime" i
)
// using function keyword
[2..10] |> List.map (function
| 2 | 3 | 5 | 7 -> sprintf "prime"
| _ -> sprintf "not prime"
)

A minor drawback of function compared with match..with is that you can't see the
original input value and have to rely on value bindings in the pattern.

Exception handling with try..with

In the previous post, we looked at catching exceptions with the try..with expression.
try
failwith "fail"
with
| Failure msg -> "caught: " + msg
| :? System.InvalidOperationException as ex -> "unexpected"

439

Match expressions

The try..with expression implements pattern matching in the same way as match..with .
So in the above example we see the use of matching on a custom pattern
| Failure msg is an example of matching on (what looks like) an active pattern
| :? System.InvalidOperationException as ex is an example of matching on the subtype

(with the use of as as well).

Because the try..with expression implements full pattern matching, we can also use
guards as well, if needed to add extra conditional logic:
let debugMode = false
try
failwith "fail"
with
| Failure msg when debugMode ->
reraise()
| Failure msg when not debugMode ->
printfn "silently logged in production: %s" msg

Wrapping match expressions with functions

Match expressions are very useful, but can lead to complex code if not used carefully.
The main problem is that match expressions doesn't compose very well. That is, it is hard to
chain match..with expressions and build simple ones into complex ones.
The best way of avoiding this is to wrap match..with expressions into functions, which can
then be composed nicely.
Here's a simple example. The match x with 42 is wrapped in a isAnswerToEverything
function.
let times6 x = x * 6
let isAnswerToEverything x =
match x with
| 42 -> (x,true)
| _ -> (x,false)
// the function can be used for chaining or composition
[1..10] |> List.map (times6 >> isAnswerToEverything)

Library functions to replace explicit matching

440

Match expressions

Most built-in F# types have such functions already available.

For example, instead of using recursion to loop through lists, you should try to use the
functions in the List module, which will do almost everything you need.
In particular, the function we wrote earlier:
let rec loopAndSum aList sumSoFar =
match aList with
| [] ->
sumSoFar
| x::xs ->
let newSumSoFar = sumSoFar + x
loopAndSum xs newSumSoFar

can be rewritten using the List module in at least three different ways!
// simplest
let loopAndSum1 aList = List.sum aList
[1..10] |> loopAndSum1
// reduce is very powerful
let loopAndSum2 aList = List.reduce (+) aList
[1..10] |> loopAndSum2
// fold is most powerful of all
let loopAndSum3 aList = List.fold (fun sum i -> sum+i) 0 aList
[1..10] |> loopAndSum3

Similarly, the Option type (discussed at length in this post) has an associated Option
module with many useful functions.
For example, a function that does a match on Some vs None can be replaced with
Option.map :

// unnecessary to implement this explicitly

let addOneIfValid optionalInt =
match optionalInt with
| Some i -> Some (i + 1)
| None -> None
Some 42 |> addOneIfValid
// much easier to use the built in function
let addOneIfValid2 optionalInt =
optionalInt |> Option.map (fun i->i+1)
Some 42 |> addOneIfValid2

441

Match expressions

Creating "fold" functions to hide matching logic

Finally, if you create your own types which need to be frequently matched, it is good practice
to create a corresponding generic "fold" function that wraps it nicely.
For example, here is a type for defining temperature.
type TemperatureType = F of float | C of float

Chances are, we will matching these cases a lot, so let's create a generic function that will
do the matching for us.
module Temperature =
let fold fahrenheitFunction celsiusFunction aTemp =
match aTemp with
| F f -> fahrenheitFunction f
| C c -> celsiusFunction c

All fold functions follow this same general pattern:

there is one function for each case in the union structure (or clause in the match
pattern)
finally, the actual value to match on comes last. (Why? See the post on "designing
functions for partial application")
Now we have our fold function, we can use it in different contexts.
Let's start by testing for a fever. We need a function for testing degrees F for fever and
another one for testing degrees C for fever.
And then we combine them both using the fold function.
let fFever tempF =
if tempF > 100.0 then "Fever!" else "OK"
let cFever tempC =
if tempC > 38.0 then "Fever!" else "OK"
// combine using the fold
let isFever aTemp = Temperature.fold fFever cFever aTemp

And now we can test.

442

Match expressions

let normalTemp = C 37.0

let result1 = isFever normalTemp
let highTemp = F 103.1
let result2 = isFever highTemp

For a completely different use, let's write a temperature conversion utility.

Again we start by writing the functions for each case, and then combine them.
let fConversion tempF =
let convertedValue = (tempF - 32.0) / 1.8
TemperatureType.C convertedValue //wrapped in type
let cConversion tempC =
let convertedValue = (tempC * 1.8) + 32.0
TemperatureType.F convertedValue //wrapped in type
// combine using the fold
let convert aTemp = Temperature.fold fConversion cConversion aTemp

Note that the conversion functions wrap the converted values in a new TemperatureType , so
the convert function has the signature:
val convert : TemperatureType -> TemperatureType

And now we can test.

let c20 = C 20.0
let resultInF = convert c20
let f75 = F 75.0
let resultInC = convert f75

We can even call convert twice in a row, and we should get back the same temperature that
we started with!
let resultInC = C 20.0 |> convert |> convert

There will be much more discussion on folds in the upcoming series on recursion and
recursive types.

443

Match expressions

444

Formatted text using printf

In this post, we'll take a small detour and look at how to create formatted text. The printing
and formatting functions are technically library functions, but in practice they as used as if
they were part of the core language.
F# supports two distinct styles of formatting text:
The standard .NET technique of "composite formatting" as seen in String.Format ,
Console.WriteLine and other places.

The C-style technique of using printf and the associated family of functions such as
printfn , sprintf and so on.

String.Format vs printf
The composite formatting technique is available in all .NET languages, and you are probably
familiar with it from C#.
Console.WriteLine("A string: {0}. An int: {1}. A float: {2}. A bool: {3}","hello",42,3
.14,true)

The printf technique, on the other hand, is based on the C-style format strings:
printfn "A string: %s. An int: %i. A float: %f. A bool: %b" "hello" 42 3.14 true

As you have seen, the printf technique is very common in F#, while String.Format ,
Console.Write and so on, are rarely used.

Why is printf preferred and considered idiomatic for F#? The reasons are:
It is statically type checked.
It is a well-behaved F# function and so supports partial application, etc.
It supports native F# types.

printf is statically type checked

Unlike String.Format , printf is statically type checked, both for the types of the
parameters, and the number.
For example, here are two snippets using printf that will fail to compile:

445

Formatted text using printf

// wrong parameter type

printfn "A string: %s" 42
// wrong number of parameters
printfn "A string: %s" "Hello" 42

The equivalent code using composite formatting will compile fine but either work incorrectly
but silently, or give a runtime error:
// wrong parameter type
Console.WriteLine("A string: {0}", 42) //works!
// wrong number of parameters
Console.WriteLine("A string: {0}","Hello",42) //works!
Console.WriteLine("A string: {0}. An int: {1}","Hello") //FormatException

printf supports partial application

The .NET formatting functions require all parameters to be passed in at the same time.
But printf is a standard, well-behaved F# function, and so supports partial application.
Here are some examples:
// partial application - explicit parameters
let printStringAndInt s i = printfn "A string: %s. An int: %i" s i
let printHelloAndInt i = printStringAndInt "Hello" i
do printHelloAndInt 42
// partial application - point free style
let printInt = printfn "An int: %i"
do printInt 42

And of course, printf can be used for function parameters anywhere a standard function
can be used.
let doSomething printerFn x y =
let result = x + y
printerFn "result is" result
let callback = printfn "%s %i"
do doSomething callback 3 4

This also includes the higher order functions for lists, etc:

446

Formatted text using printf

[1..5] |> List.map (sprintf "i=%i")

printf supports native F# types

For non-primitive types, the .NET formatting functions only support using ToString() , but
printf supports native F# types using the %A specifier:

// tuple printing
let t = (1,2)
Console.WriteLine("A tuple: {0}", t)
printfn "A tuple: %A" t
// record printing
type Person = {First:string; Last:string}
let johnDoe = {First="John"; Last="Doe"}
Console.WriteLine("A record: {0}", johnDoe )
printfn "A record: %A" johnDoe
// union types printing
type Temperature = F of int | C of int
let freezing = F 32
Console.WriteLine("A union: {0}", freezing )
printfn "A union: %A" freezing

As you can see, tuple types have a nice ToString() but other user defined types don't, so if
you want to use them with the .NET formatting functions, you will have to override the
ToString() method explicitly.

printf gotchas
There are a couple of "gotchas" to be aware of when using printf .
First, if there are too few parameters, rather than too many, the compiler will not complain
immediately, but might give cryptic errors later.
// too few parameters
printfn "A string: %s An int: %i" "Hello"

The reason, of course, is that this is not an error at all; printf is just being partially applied!
See the discussion of partial application if you are not clear of why this happens.
Another issue is that the "format strings" are not actually strings.

447

Formatted text using printf

In the .NET formatting model, the formatting strings are normal strings, so you can pass
them around, store them in resource files, and so on. Which means that the following code
works fine:
let netFormatString = "A string: {0}"
Console.WriteLine(netFormatString, "hello")

On the other hand, the "format strings" that are the first argument to printf are not really
strings at all, but something called a TextWriterFormat . Which means that the following
code does not work:
let fsharpFormatString = "A string: %s"
printfn fsharpFormatString "Hello"

The compiler does some magic behind the scenes to convert the string constant "A string:
%s" into the appropriate TextWriterFormat. The TextWriterFormat is the key component that

"knows" the type of the format string, such as string->unit or string->int->unit , which in
turn allows printf to be typesafe.
If you want to emulate the compiler, you can create your own TextWriterFormat value from a
string using the Printf.TextWriterFormat type in the Microsoft.FSharp.Core.Printf module.
If the format string is "inline", the compiler can deduce the type for you during binding:
let format:Printf.TextWriterFormat<_> = "A string: %s"
printfn format "Hello"

But if the format string is truly dynamic (e.g. stored in a resource or created on the fly), the
compiler cannot deduce the type for you, and you must explicitly provide it with the
constructor.
In the example below, my first format string has a single string parameter and returns a unit,
so I have to specify string->unit as the format type. And in the second case, I have to
specify string->int->unit as the format type.
let formatAString = "A string: %s"
let formatAStringAndInt = "A string: %s. An int: %i"
//convert to TextWriterFormat
let twFormat1 = Printf.TextWriterFormat<string->unit>(formatAString)
printfn twFormat1 "Hello"
let twFormat2 = Printf.TextWriterFormat<string->int->unit>(formatAStringAndInt)
printfn twFormat2 "Hello" 42

448

Formatted text using printf

I won't go into detail on exactly how printf and TextWriterFormat` work together right now -just be aware that is not just a matter of simple format strings being passed around.
Finally, it's worth noting that printf and family are not thread-safe, while Console.Write
and family are.

How to specify a format

The "%" format specifications are quite similar to those used in C, but with some special
customizations for F#.
As with C, the characters immediately following the % have a specific meaning, as shown
below.
%[flags][width][.precision]specifier

We'll discuss each of these attributes in more detail below.

Formatting for dummies

The most commonly used format specifiers are:
%s for strings
%b for bools
%i for ints
%f for floats
%A for pretty-printing tuples, records and union types
%O for other objects, using ToString()

These six will probably meet most of your basic needs.

Escaping %
The % character on its own will cause an error. To escape it, just double it up:
printfn "unescaped: %" // error
printfn "escape: %%"

Controlling width and alignment

449

Formatted text using printf

When formatting fixed width columns and tables, you need to have control of the alignment
and width.
You can do that with the "width" and "flags" options.
%5s , %5i . A number sets the width of the value
%*s , %*i . A star sets the width of the value dynamically (from an extra parameter just

before the param to format)

%-s , %-i . A hyphen left justifies the value.

Here are some examples of these in use:

let rows = [ (1,"a"); (-22,"bb"); (333,"ccc"); (-4444,"dddd") ]
// no alignment
for (i,s) in rows do
printfn "|%i|%s|" i s
// with alignment
for (i,s) in rows do
printfn "|%5i|%5s|" i s
// with left alignment for column 2
for (i,s) in rows do
printfn "|%5i|%-5s|" i s
// with dynamic column width=20 for column 1
for (i,s) in rows do
printfn "|%*i|%-5s|" 20 i s
// with dynamic column width for column 1 and column 2
for (i,s) in rows do
printfn "|%*i|%-*s|" 20 i 10 s

Formatting integers
There are some special options for basic integer types:
%i or %d for signed ints
%u for unsigned ints
%x and %X for lowercase and uppercase hex
%o for octal

Here are some examples:

450

Formatted text using printf

printfn "signed8: %i unsigned8: %u" -1y -1y

printfn "signed16: %i unsigned16: %u" -1s -1s
printfn "signed32: %i unsigned32: %u" -1 -1
printfn "signed64: %i unsigned64: %u" -1L -1L
printfn "uppercase hex: %X lowercase hex: %x octal: %o" 255 255 255
printfn "byte: %i " 'A'B

The specifiers do not enforce any type safety within the integer types. As you can see from
the examples above, you can pass a signed int to an unsigned specifier without problems.
What is different is how it is formatted. The unsigned specifiers treat the int as unsigned no
matter how it is actually typed.
Note that BigInteger is not a basic integer type, so you must format it with %A or %O .
printfn "bigInt: %i " 123456789I // Error
printfn "bigInt: %A " 123456789I // OK

You can control the formatting of signs and zero padding using the flags:
%0i pads with zeros
%+i shows a plus sign
% i shows a blank in place of a plus sign

Here are some examples:

451

Formatted text using printf

let rows = [ (1,"a"); (-22,"bb"); (333,"ccc"); (-4444,"dddd") ]

// with alignment
for (i,s) in rows do
printfn "|%5i|%5s|" i s
// with plus signs
for (i,s) in rows do
printfn "|%+5i|%5s|" i s
// with zero pad
for (i,s) in rows do
printfn "|%0+5i|%5s|" i s
// with left align
for (i,s) in rows do
printfn "|%-5i|%5s|" i s
// with left align and plus
for (i,s) in rows do
printfn "|%+-5i|%5s|" i s
// with left align and space instead of plus
for (i,s) in rows do
printfn "|% -5i|%5s|" i s

Formatting floats and decimals

For floating point types, there are also some special options:
%f for standard format
%e or %E for exponential format
%g or %G for the more compact of f and e .
%M for decimals

Here are some examples:

let pi = 3.14
printfn "float: %f exponent: %e compact: %g" pi pi pi
let petabyte = pown 2.0 50
printfn "float: %f exponent: %e compact: %g" petabyte petabyte petabyte

The decimal type can be used with the floating point specifiers, but you might lose some
precision. The %M specifier can be used to ensure that no precision is lost. You can see the
difference with this example:

452

Formatted text using printf

let largeM = 123456789.123456789M // a decimal

printfn "float: %f decimal: %M" largeM largeM

You can control the precision of floats using a precision specification, such as %.2f and
%.4f . For the %f and %e specifiers, the precision affects the number of digits after the

decimal point, while for %g it is the number of digits in total. Here's an example:
printfn "2 digits precision: %.2f. 4 digits precision: %.4f." 123.456789 123.456789
// output => 2 digits precision: 123.46. 4 digits precision: 123.4568.
printfn "2 digits precision: %.2e. 4 digits precision: %.4e." 123.456789 123.456789
// output => 2 digits precision: 1.23e+002. 4 digits precision: 1.2346e+002.
printfn "2 digits precision: %.2g. 4 digits precision: %.4g." 123.456789 123.456789
// output => 2 digits precision: 1.2e+02. 4 digits precision: 123.5.

The alignment and width flags work for floats and decimals as well.
printfn "|%f|" pi // normal
printfn "|%10f|" pi // width
printfn "|%010f|" pi // zero-pad
printfn "|%-10f|" pi // left aligned
printfn "|%0-10f|" pi // left zero-pad

Custom formatting functions

There are two special format specifiers that allow to you pass in a function rather than just a
simple value.
%t expects a function that outputs some text with no input
%a expects a function that outputs some text from a given input

Here's an example of using %t :

open System.IO
//define the function
let printHello (tw:TextWriter) = tw.Write("hello")
//test it
printfn "custom function: %t" printHello

Obviously, since the callback function takes no parameters, it will probably be a closure that
does reference some other value. Here's an example that prints random numbers:

453

Formatted text using printf

open System
open System.IO
//define the function using a closure
let printRand =
let rand = new Random()
// return the actual printing function
fun (tw:TextWriter) -> tw.Write(rand.Next(1,100))
//test it
for i in [1..5] do
printfn "rand = %t" printRand

For the %a specifier, the callback function takes an extra parameter. That is, when using
the %a specifier, you must pass in both a function and a value to format.
Here's an example of custom formatting a tuple:
open System
open System.IO
//define the callback function
//note that the data parameter comes after the TextWriter
let printLatLong (tw:TextWriter) (lat,long) =
tw.Write("lat:{0} long:{1}", lat, long)
// test it
let latLongs = [ (1,2); (3,4); (5,6)]
for latLong in latLongs do
// function and value both passed in to printfn
printfn "latLong = %a" printLatLong latLong

Date formatting
There are no special format specifiers for dates in F#.
If you want to format dates, you have a couple of options:
Use ToString to convert the date into a string, and then use the %s specifier
Use a custom callback function with the %a specifier as described above
Here are the two approaches in use:

454

Formatted text using printf

// function to format a date

let yymmdd1 (date:DateTime) = date.ToString("yy.MM.dd")
// function to format a date onto a TextWriter
let yymmdd2 (tw:TextWriter) (date:DateTime) = tw.Write("{0:yy.MM.dd}", date)
// test it
for i in [1..5] do
let date = DateTime.Now.AddDays(float i)
// using %s
printfn "using ToString = %s" (yymmdd1 date)
// using %a
printfn "using a callback = %a" yymmdd2 date

Which approach is better?

The ToString with %s is easier to test and use, but it will be less efficient than writing
directly to a TextWriter.

The printf family of functions

There are a number of variants of printf functions. Here is a quick guide:
F# function

C# equivalent

Comment

printf and printfn

Console.Write and
Console.WriteLine

Functions starting with "print"

write to standard out.

eprintf and
eprintfn

Console.Error.Write and
Console.Error.WriteLine

Functions starting with "eprint"

write to standard error.

fprintf and
fprintfn

TextWriter.Write and
TextWriter.WriteLine

Functions starting with "fprint"

write to a TextWriter.

sprintf

String.Format

Functions starting with "sprint"

return a string.

bprintf

StringBuilder.AppendFormat

Functions starting with "bprint"

write to a StringBuilder.

No equivalent

Functions that accept a

continuation. See next section
for a discussion.

kprintf , kfprintf ,
ksprintf and
kbprintf

All of these except bprintf and the kXXX family are automatically available (via
Microsoft.FSharp.Core.ExtraTopLevelOperators). But if you need to access them using a
module, they are in the Printf module.

455

Formatted text using printf

The usage of these should be obvious (except for the kXXX family, of which more below).
A particularly useful technique is to use partial application to "bake in" a TextWriter or
StringBuilder.
Here is an example using a StringBuilder:
let printToSb s i =
let sb = new System.Text.StringBuilder()
// use partial application to fix the StringBuilder
let myPrint format = Printf.bprintf sb format
do myPrint "A string: %s. " s
do myPrint "An int: %i" i
//get the result
sb.ToString()
// test
printToSb "hello" 42

And here is an example using a TextWriter:

open System
open System.IO
let printToFile filename s i =
let myDocsPath = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)
let fullPath = Path.Combine(myDocsPath, filename)
use sw = new StreamWriter(path=fullPath)
// use partial application to fix the TextWriter
let myPrint format = fprintf sw format
do myPrint "A string: %s. " s
do myPrint "An int: %i" i
//get the result
sw.Close()
// test
printToFile "myfile.txt" "hello" 42

More on partially applying printf

Note that in both cases above, we had to pass a format parameter when creating the partial
application.

456

Formatted text using printf

That is, we had to do:

let myPrint format = fprintf sw format

rather than the point-free version:

let myPrint = fprintf sw

This stops the compiler complaining about an incorrect type. The reason why is nonobvious. We briefly mentioned the TextWriterFormat above as the first parameter to
printf . It turns out that printf is not actually a particular function, like String.Format ,

but rather a generic function that has to be parameterized with a TextWriterFormat (or the
similar StringFormat) in order to become "real".
So, to be safe, it is best to always pair a printf with a format parameter, rather than being
overly aggressive with the partial application.

The kprintf functions

The four kXXX functions are similar to their cousins, except that they take an extra
parameter -- a continuation. That is, a function to be called immediately after the formatting
has been done.
Here's a simple snippet:
let doAfter s =
printfn "Done"
// return the result
s
let result = Printf.ksprintf doAfter "%s" "Hello"

Why would you want this? A number of reasons:

You can pass the result to another function that does something useful, such as a
logging framework
You can do things such as flushing the TextWriter
You can raise an event
Let's look at a sample that uses a external logging framework plus custom events.
First, let's create a simple logging class along the lines of log4net or
System.Diagnostics.Trace. In practice, this would be replaced by a real third-party library.

457

Formatted text using printf

open System
open System.IO
// a logging library such as log4net
// or System.Diagnostics.Trace
type Logger(name) =
let currentTime (tw:TextWriter) =
tw.Write("{0:s}",DateTime.Now)
let logEvent level msg =
printfn "%t %s [%s] %s" currentTime level name msg
member this.LogInfo msg =
logEvent "INFO" msg
member this.LogError msg =
logEvent "ERROR" msg
static member CreateLogger name =
new Logger(name)

Next in my application code, I do the following:

Create an instance of the logging framework. I've hard-coded the factory method here,
but you could also use an IoC container.
Create helper functions called logInfo and logError that call the logging framework,
and in the case of logError , show a popup message as well.

458

Formatted text using printf

// my application code
module MyApplication =
let logger = Logger.CreateLogger("MyApp")
// create a logInfo using the Logger class
let logInfo format =
let doAfter s =
logger.LogInfo(s)
Printf.ksprintf doAfter format
// create a logError using the Logger class
let logError format =
let doAfter s =
logger.LogError(s)
System.Windows.Forms.MessageBox.Show(s) |> ignore
Printf.ksprintf doAfter format
// function to exercise the logging
let test() =
do logInfo "Message #%i" 1
do logInfo "Message #%i" 2
do logError "Oops! an error occurred in my app"

Finally, when we run the test function, we should get the message written to the console,
and also see the popup message:
MyApplication.test()

You could also create an object-oriented version of the helper methods by creating a
"FormattingLogger" wrapper class around the logging library, as shown below.

459

Formatted text using printf

type FormattingLogger(name) =
let logger = Logger.CreateLogger(name)
// create a logInfo using the Logger class
member this.logInfo format =
let doAfter s =
logger.LogInfo(s)
Printf.ksprintf doAfter format
// create a logError using the Logger class
member this.logError format =
let doAfter s =
logger.LogError(s)
System.Windows.Forms.MessageBox.Show(s) |> ignore
Printf.ksprintf doAfter format
static member createLogger name =
new FormattingLogger(name)
// my application code
module MyApplication2 =
let logger = FormattingLogger.createLogger("MyApp2")
let test() =
do logger.logInfo "Message #%i" 1
do logger.logInfo "Message #%i" 2
do logger.logError "Oops! an error occurred in app 2"
// test
MyApplication2.test()

The object-oriented approach, although more familiar, is not automatically better! The pros
and cons of OO methods vs. pure functions are discussed here.

460

Worked example: Parsing command line arguments

Worked example: Parsing command line

arguments
Now that we've seen how the match expression works, let's look at some examples in
practice. But first, a word about the design approach.

Application design in F#
We've seen that a generic function takes input and emits output. But in a sense, that
approach applies at any level of functional code, even at the top level.
In fact, we can say that a functional application takes input, transforms it, and emits output:

Now ideally, the transformations work within the pure type-safe world that we create to
model the domain, but unfortunately, the real world is untyped! That is, the input is likely to
be simple strings or bytes, and the output also.
How can we work with this? The obvious solution is to have a separate stage to convert the
input to our pure internal model, and then another separate stage to convert from the
internal model to the output.

In this way, we can hide the messiness of the real world from the core of the application.
This "keep your model pure" approach is similar to the "Hexagonal Architecture" concept in
the large, or the MVC pattern in the small.
In this post and the next, we'll see some simple examples of this.

Example: parsing a command line

461

Worked example: Parsing command line arguments

We talked about the match expression in general in the previous post, so let's look at a real
example where it is useful, namely parsing a command line.
We'll design and implement two slightly different versions, one with a basic internal model,
and second one with some improvements.

Requirements
Let's say that we have three commandline options: "verbose", "subdirectories", and
"orderby". "Verbose" and "subdirectories" are flags, while "orderby" has two choices: "by
size" and "by name".
So the command line params would look like
MYAPP [/V] [/S] [/O order]
/V verbose
/S include subdirectories
/O order by. Parameter is one of
N - order by name.
S - order by size

First version
Following the design rule above, we can see that:
the input will be an array (or list) of strings, one for each argument.
the internal model will be a set of types that model the (tiny) domain.
the output is out of scope in this example.
So we'll start by first creating the internal model of the parameters, and then look at how we
can parse the input into types used in the internal model.
Here's a first stab at the model:
// constants used later
let OrderByName = "N"
let OrderBySize = "S"
// set up a type to represent the options
type CommandLineOptions = {
verbose: bool;
subdirectories: bool;
orderby: string;
}

462

Worked example: Parsing command line arguments

Ok, that looks alright. Now let's parse the arguments.

The parsing logic is very similar to the loopAndSum example in the previous post.
We create a recursive loop on the list of arguments.
Each time through the loop, we parse one argument.
The options parsed so far are passed into each loop as a parameter (the "accumulator"
pattern).
let rec parseCommandLine args optionsSoFar =
match args with
// empty list means we're done.
| [] ->
optionsSoFar
// match verbose flag
| "/v"::xs ->
let newOptionsSoFar = { optionsSoFar with verbose=true}
parseCommandLine xs newOptionsSoFar
// match subdirectories flag
| "/s"::xs ->
let newOptionsSoFar = { optionsSoFar with subdirectories=true}
parseCommandLine xs newOptionsSoFar
// match orderBy by flag
| "/o"::xs ->
//start a submatch on the next arg
match xs with
| "S"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderBySize}
parseCommandLine xss newOptionsSoFar
| "N"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderByName}
parseCommandLine xss newOptionsSoFar
// handle unrecognized option and keep looping
| _ ->
eprintfn "OrderBy needs a second argument"
parseCommandLine xs optionsSoFar
// handle unrecognized option and keep looping
| x::xs ->
eprintfn "Option '%s' is unrecognized" x
parseCommandLine xs optionsSoFar

This code is straightforward, I hope.

463

Worked example: Parsing command line arguments

Each match consist of a option::restOfList pattern. If the option is matched, a new

optionsSoFar value is created and the loop repeats with the remaining list, until the list

becomes empty, at which point we can exit the loop and return the optionsSoFar value as
the final result.
There are two special cases:
Matching the "orderBy" option creates a submatch pattern that looks at the first item in
the rest of the list and if not found, complains about a missing second parameter.
The very last match on the main match..with is not a wildcard, but a "bind to value".
Just like a wildcard, this will always succeed, but because we havd bound to the value,
it allows us to print the offending unmatched argument.
Note that for printing errors, we use eprintf rather than printf . This will write to
STDERR rather than STDOUT.
So now let's test this:
parseCommandLine ["/v"; "/s"]

Oops! That didn't work -- we need to pass in an initial optionsSoFar argument! Lets try
again:
// define the defaults to pass in
let defaultOptions = {
verbose = false;
subdirectories = false;
orderby = ByName
}
// test it
parseCommandLine ["/v"] defaultOptions
parseCommandLine ["/v"; "/s"] defaultOptions
parseCommandLine ["/o"; "S"] defaultOptions

Check that the output is what you would expect.

And we should also check the error cases:
parseCommandLine ["/v"; "xyz"] defaultOptions
parseCommandLine ["/o"; "xyz"] defaultOptions

You should see the error messages in these cases now.

Before we finish this implementation, let's fix something annoying. We are passing in these
default options every time -- can we get rid of them?

464

Worked example: Parsing command line arguments

This is a very common situation: you have a recursive function that takes a "accumulator"
parameter, but you don't want to be passing initial values all the time.
The answer is simple: just create another function that calls the recursive function with the
defaults.
Normally, this second one is the "public" one and the recursive one is hidden, so we will
rewrite the code as follows:
Rename parseCommandLine to parseCommandLineRec . There are other naming
conventions you could use as well, such as parseCommandLine' with a tick mark, or
innerParseCommandLine .

Create a new parseCommandLine that calls parseCommandLineRec with the defaults

// create the "helper" recursive function
let rec parseCommandLineRec args optionsSoFar =
// implementation as above
// create the "public" parse function
let parseCommandLine args =
// create the defaults
let defaultOptions = {
verbose = false;
subdirectories = false;
orderby = OrderByName
}
// call the recursive one with the initial options
parseCommandLineRec args defaultOptions

In this case the helper function can stand on its own. But if you really want to hide it, you can
put it as a nested subfunction in the defintion of parseCommandLine itself.
// create the "public" parse function
let parseCommandLine args =
// create the defaults
let defaultOptions =
// implementation as above
// inner recursive function
let rec parseCommandLineRec args optionsSoFar =
// implementation as above
// call the recursive one with the initial options
parseCommandLineRec args defaultOptions

465

Worked example: Parsing command line arguments

In this case, I think it would just make things more complicated, so I have kept them
separate.
So, here is all the code at once, wrapped in a module:
module CommandLineV1 =
// constants used later
let OrderByName = "N"
let OrderBySize = "S"
// set up a type to represent the options
type CommandLineOptions = {
verbose: bool;
subdirectories: bool;
orderby: string;
}
// create the "helper" recursive function
let rec parseCommandLineRec args optionsSoFar =
match args with
// empty list means we're done.
| [] ->
optionsSoFar
// match verbose flag
| "/v"::xs ->
let newOptionsSoFar = { optionsSoFar with verbose=true}
parseCommandLineRec xs newOptionsSoFar
// match subdirectories flag
| "/s"::xs ->
let newOptionsSoFar = { optionsSoFar with subdirectories=true}
parseCommandLineRec xs newOptionsSoFar
// match orderBy by flag
| "/o"::xs ->
//start a submatch on the next arg
match xs with
| "S"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderBySize}
parseCommandLineRec xss newOptionsSoFar
| "N"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderByName}
parseCommandLineRec xss newOptionsSoFar
// handle unrecognized option and keep looping
| _ ->
eprintfn "OrderBy needs a second argument"
parseCommandLineRec xs optionsSoFar

466

Worked example: Parsing command line arguments

// handle unrecognized option and keep looping

| x::xs ->
eprintfn "Option '%s' is unrecognized" x
parseCommandLineRec xs optionsSoFar
// create the "public" parse function
let parseCommandLine args =
// create the defaults
let defaultOptions = {
verbose = false;
subdirectories = false;
orderby = OrderByName
}
// call the recursive one with the initial options
parseCommandLineRec args defaultOptions

// happy path
CommandLineV1.parseCommandLine ["/v"]
CommandLineV1.parseCommandLine ["/v"; "/s"]
CommandLineV1.parseCommandLine ["/o"; "S"]
// error handling
CommandLineV1.parseCommandLine ["/v"; "xyz"]
CommandLineV1.parseCommandLine ["/o"; "xyz"]

Second version
In our initial model we used bool and string to represent the possible values.
type CommandLineOptions = {
verbose: bool;
subdirectories: bool;
orderby: string;
}

There are two problems with this:

It doesn't really represent the domain. For example, can orderby really be any
string? Would my code break if I set it to "ABC"?
The values are not self documenting. For example, the verbose value is a bool. We
only know that the bool represents the "verbose" option because of the context (the field
named verbose ) it is found in. If we passed that bool around, and took it out of context,
we would not know what it represented. I'm sure we have all seen C# functions with
many boolean parameters like this:

467

Worked example: Parsing command line arguments

myObject.SetUpComplicatedOptions(true,false,true,false,false);

Because the bool doesn't represent anything at the domain level, it is very easy to make
mistakes.
The solution to both these problems is to be as specific as possible when defining the
domain, typically by creating lots of very specific types.
So here's a new version of CommandLineOptions :
type OrderByOption = OrderBySize | OrderByName
type SubdirectoryOption = IncludeSubdirectories | ExcludeSubdirectories
type VerboseOption = VerboseOutput | TerseOutput
type CommandLineOptions = {
verbose: VerboseOption;
subdirectories: SubdirectoryOption;
orderby: OrderByOption
}

A couple of things to notice:

There are no bools or strings anywhere.
The names are quite explicit. This acts as documentation when a value is taken in
isolation, but also means that the name is unique, which helps type inference, which in
turn helps you avoid explicit type annotations.
Once we have made the changes to the domain, it is easy to fix up the parsing logic.
So, here is all the revised code, wrapped in a "v2" module:
module CommandLineV2 =
type OrderByOption = OrderBySize | OrderByName
type SubdirectoryOption = IncludeSubdirectories | ExcludeSubdirectories
type VerboseOption = VerboseOutput | TerseOutput
type CommandLineOptions = {
verbose: VerboseOption;
subdirectories: SubdirectoryOption;
orderby: OrderByOption
}
// create the "helper" recursive function
let rec parseCommandLineRec args optionsSoFar =
match args with
// empty list means we're done.
| [] ->

468

Worked example: Parsing command line arguments

optionsSoFar
// match verbose flag
| "/v"::xs ->
let newOptionsSoFar = { optionsSoFar with verbose=VerboseOutput}
parseCommandLineRec xs newOptionsSoFar
// match subdirectories flag
| "/s"::xs ->
let newOptionsSoFar = { optionsSoFar with subdirectories=IncludeSubdirecto
ries}
parseCommandLineRec xs newOptionsSoFar
// match sort order flag
| "/o"::xs ->
//start a submatch on the next arg
match xs with
| "S"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderBySize}
parseCommandLineRec xss newOptionsSoFar
| "N"::xss ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderByName}
parseCommandLineRec xss newOptionsSoFar
// handle unrecognized option and keep looping
| _ ->
printfn "OrderBy needs a second argument"
parseCommandLineRec xs optionsSoFar
// handle unrecognized option and keep looping
| x::xs ->
printfn "Option '%s' is unrecognized" x
parseCommandLineRec xs optionsSoFar
// create the "public" parse function
let parseCommandLine args =
// create the defaults
let defaultOptions = {
verbose = TerseOutput;
subdirectories = ExcludeSubdirectories;
orderby = OrderByName
}
// call the recursive one with the initial options
parseCommandLineRec args defaultOptions
// ==============================
// tests
// happy path
CommandLineV2.parseCommandLine ["/v"]
CommandLineV2.parseCommandLine ["/v"; "/s"]
CommandLineV2.parseCommandLine ["/o"; "S"]

469

Worked example: Parsing command line arguments

// error handling
CommandLineV2.parseCommandLine ["/v"; "xyz"]
CommandLineV2.parseCommandLine ["/o"; "xyz"]

Using fold instead of recursion?

We said in the previous post that it is good to avoid recursion where possible and use the
built in functions in the List module like map and fold .
So can we take this advice here and fix up this code to do this?
Unfortunately, not easily. The problem is that the list functions generally work on one
element at a time, while the "orderby" option requires a "lookahead" argument as well.
To make this work with something like fold , we need to create a "parse mode" flag to
indicate whether we are in lookahead mode or not. This is possible, but I think it just adds
extra complexity compared to the straightforward recursive version above.
And in a real-world situation, anything more complicated than this would be a signal that you
need to switch to a proper parsing system such as FParsec.
However, just to show you it can be done with fold :
module CommandLineV3 =
type OrderByOption = OrderBySize | OrderByName
type SubdirectoryOption = IncludeSubdirectories | ExcludeSubdirectories
type VerboseOption = VerboseOutput | TerseOutput
type CommandLineOptions = {
verbose: VerboseOption;
subdirectories: SubdirectoryOption;
orderby: OrderByOption
}
type ParseMode = TopLevel | OrderBy
type FoldState = {
options: CommandLineOptions ;
parseMode: ParseMode;
}
// parse the top-level arguments
// return a new FoldState
let parseTopLevel arg optionsSoFar =
match arg with
// match verbose flag

470

Worked example: Parsing command line arguments

| "/v" ->
let newOptionsSoFar = {optionsSoFar with verbose=VerboseOutput}
{options=newOptionsSoFar; parseMode=TopLevel}
// match subdirectories flag
| "/s"->
let newOptionsSoFar = { optionsSoFar with subdirectories=IncludeSubdirecto
ries}
{options=newOptionsSoFar; parseMode=TopLevel}
// match sort order flag
| "/o" ->
{options=optionsSoFar; parseMode=OrderBy}
// handle unrecognized option and keep looping
| x ->
printfn "Option '%s' is unrecognized" x
{options=optionsSoFar; parseMode=TopLevel}
// parse the orderBy arguments
// return a new FoldState
let parseOrderBy arg optionsSoFar =
match arg with
| "S" ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderBySize}
{options=newOptionsSoFar; parseMode=TopLevel}
| "N" ->
let newOptionsSoFar = { optionsSoFar with orderby=OrderByName}
{options=newOptionsSoFar; parseMode=TopLevel}
// handle unrecognized option and keep looping
| _ ->
printfn "OrderBy needs a second argument"
{options=optionsSoFar; parseMode=TopLevel}
// create a helper fold function
let foldFunction state element =
match state with
| {options=optionsSoFar; parseMode=TopLevel} ->
// return new state
parseTopLevel element optionsSoFar
| {options=optionsSoFar; parseMode=OrderBy} ->
// return new state
parseOrderBy element optionsSoFar
// create the "public" parse function
let parseCommandLine args =
let defaultOptions = {
verbose = TerseOutput;
subdirectories = ExcludeSubdirectories;
orderby = OrderByName
}

471

Worked example: Parsing command line arguments

let initialFoldState =
{options=defaultOptions; parseMode=TopLevel}
// call fold with the initial state
args |> List.fold foldFunction initialFoldState
// ==============================
// tests
// happy path
CommandLineV3.parseCommandLine ["/v"]
CommandLineV3.parseCommandLine ["/v"; "/s"]
CommandLineV3.parseCommandLine ["/o"; "S"]
// error handling
CommandLineV3.parseCommandLine ["/v"; "xyz"]
CommandLineV3.parseCommandLine ["/o"; "xyz"]

By the way, can you see a subtle change of behavior in this version?
In the previous versions, if there was no parameter to the "orderBy" option, the recursive
loop would still parse it next time. But in the 'fold' version, this token is swallowed and lost.
To see this, compare the two implementations:
// verbose set
CommandLineV2.parseCommandLine ["/o"; "/v"]
// verbose not set!
CommandLineV3.parseCommandLine ["/o"; "/v"]

To fix this would be even more work. Again this argues for the second implementation as the
easiest to debug and maintain.

Summary
In this post we've seen how to apply pattern matching to a real-world example.
More importantly, we've seen how easy it is to create a properly designed internal model for
even the smallest domain. And that this internal model provides more type safety and
documentation than using primitive types such as string and bool.
In the next example, we'll do even more pattern matching!

472

Worked example: Parsing command line arguments

473

Worked example: Roman numerals

Last time we looked at parsing a command line. This time we'll we'll look at another pattern
matching example, this time using Roman numerals.
As before, we will try to have a "pure" internal model with separate stages to convert the
input to the internal model, and then another separate stage to convert from the internal
model to the output.

Requirements
Let's start with the requirements:
1) Accept a string of letters like "MMMXCLXXIV" as a string and convert it to an integ
er.
The conversions are: I=1; V=5; X=10; L=50; C=100; D=500; and M=1000;
If a lower letter comes before a higher one, the value of the higher is reduced accord
ingly, so
IV=4; IX=9; XC=90; and so on.
2) As an additional step, validate the string of letters to see if it is a valid numbe
r. For example: "IIVVMM" is a not a valid roman numeral.

First version
As before we'll start by first creating the internal model, and then look at how we can parse
the input into the internal model.
Here's a first stab at the model. We'll treat a RomanNumeral as a list of RomanDigits .
type RomanDigit = int
type RomanNumeral = RomanDigit list

No, stop right there! A RomanDigit is not just any digit, it has to be taken from a limited set.

474

Worked example: Roman numerals

Also RomanNumeral should not just be a type alias for a list of digits. It would be better if it
was its own special type as well. We can do this by creating a single case union type.
Here's a much better version:
type RomanDigit = I | V | X | L | C | D | M
type RomanNumeral = RomanNumeral of RomanDigit list

Output: Converting a numeral to an int

Now let's do the output logic, converting a Roman numeral to an int.
The digit conversion is easy:
/// Converts a single RomanDigit to an integer
let digitToInt =
function
| I -> 1
| V -> 5
| X -> 10
| L -> 50
| C -> 100
| D -> 500
| M -> 1000
// tests
I |> digitToInt
V |> digitToInt
M |> digitToInt

Note that we're using the function keyword instead of the match..with expression.
To convert a list of digits, we'll use a recursive loop again. There is a special case when we
need to look ahead to the next digit, and if it is larger than the current one, then use their
difference.

475

Worked example: Roman numerals

let rec digitsToInt =

function
// empty is 0
| [] -> 0
// special case when a smaller comes before larger
// convert both digits and add the difference to the sum
// Example: "IV" and "CM"
| smaller::larger::ns when smaller < larger ->
(digitToInt larger - digitToInt smaller) + digitsToInt ns
// otherwise convert the digit and add to the sum
| digit::ns ->
digitToInt digit + digitsToInt ns
// tests
[I;I;I;I] |> digitsToInt
[I;V] |> digitsToInt
[V;I] |> digitsToInt
[I;X] |> digitsToInt
[M;C;M;L;X;X;I;X] |> digitsToInt // 1979
[M;C;M;X;L;I;V] |> digitsToInt // 1944

Note that the "less than" operation did not have to be defined. The types are automatically
sorted by their order of declaration.
Finally, we can convert the RomanNumeral type itself by unpacking the contents into a list and
calling digitsToInt .
/// converts a RomanNumeral to an integer
let toInt (RomanNumeral digits) = digitsToInt digits
// test
let x = RomanNumeral [I;I;I;I]
x |> toInt
let x = RomanNumeral [M;C;M;L;X;X;I;X]
x |> toInt

That takes care of the output.

Input: Converting a string to an Roman Numeral

Now let's do the input logic, converting a string to our internal model.
First, let's handle a single character conversion. It seems straightforward.

476

Worked example: Roman numerals

let charToRomanDigit =
function
| 'I' -> I
| 'V' -> V
| 'X' -> X
| 'L' -> L
| 'C' -> C
| 'D' -> D
| 'M' -> M

The compiler doesn't like that! What happens if we get some other character?
This is a great example of how exhaustive pattern matching can force you to think about
missing requirements.
So, what should be done for bad input. How about printing an error message?
Let's try again and add a case to handle all other characters:
let charToRomanDigit =
function
| 'I' -> I
| 'V' -> V
| 'X' -> X
| 'L' -> L
| 'C' -> C
| 'D' -> D
| 'M' -> M
| ch -> eprintf "%c is not a valid character" ch

The compiler doesn't like that either! The normal cases return a valid RomanDigit but the
error case returns unit . As we saw in the earlier post, every branch must return the same
type.
How can we fix this? We could throw an exception, but that seems a bit excessive. If we
think about it some more, there's no way that charToRomanDigit can always return a valid
RomanDigit . Sometimes it can, and sometimes it can't. In other words, we need to use

something like an option type here.

But on further consideration, we might also want the caller to know what the bad char was.
So we need to create our own little variant on the option type to hold both cases.
Here's the fixed up version:

477

Worked example: Roman numerals

Note that I have removed the error message. Since the bad char is being returned, the caller
can print its own message for the BadChar case.
Next, we should check the function signature to make sure it is what we expect:
charToRomanDigit : char -> ParsedChar

That looks good.

Now, how can we convert a string into these digits? We convert the string to a char array,
convert that into a list, and then do a final conversion using charToRomanDigit .
let toRomanDigitList s =
s.ToCharArray() // error FS0072
|> List.ofArray
|> List.map charToRomanDigit

But the compiler complains again with "FS0072: Lookup on object of indeterminate type",
That typically happens when you use a method rather than a function. Any object could
implement .ToCharArray() so the type inference cannot tell what type is meant.
In this case, the solution is just to use an explicit type annotation on the parameter -- our first
so far!
let toRomanDigitList (s:string) =
s.ToCharArray()
|> List.ofArray
|> List.map charToRomanDigit

But look at the signature:

478

Worked example: Roman numerals

toRomanDigitList : string -> ParsedChar list

It still has the pesky ParsedChar in it rather than RomanDigits . How do we want to proceed?
Answer, let's pass the buck again and let someone else deal with it!
"Passing the buck" in this case is actually a good design principle. This function doesn't
know what its clients might want to do -- some might want to ignore errors, while others
might want to fail fast. So just pass back the information and let them decide.
In this case, the client is the top level function that creates a RomanNumeral type. Here's our
first attempt:
// convert a string to a RomanNumeral
let toRomanNumeral s =
toRomanDigitList s
|> RomanNumeral

The compiler is not happy -- the RomanNumeral constructor requires a list of RomanDigits ,
but the toRomanDigitList is giving us a list of ParsedChars instead.
Now we finally do have to commit to an error handling policy. Let's choose to ignore bad
chars, but print out errors when they occur. We'll use the List.choose function for this. It's
similar to List.map , but in addition has a filter built into it. Elements that are valid ( Some
something ) are returned, but elements that are None are filtered out.

Our choose function should thus do the following:

For valid digits return Some digit
For the invalid BadChars , print the error message and return None .
If we do this, the output of List.choose will be a list of RomanDigits , exactly as needed as
the input to the RomanNumeral constructor.
Here is everything put together:

479

Worked example: Roman numerals

/// Convert a string to a RomanNumeral

/// Does not validate the input.E.g. "IVIV" would be valid
let toRomanNumeral s =
toRomanDigitList s
|> List.choose (
function
| Digit digit ->
Some digit
| BadChar ch ->
eprintfn "%c is not a valid character" ch
None
)
|> RomanNumeral

Ok, everything is good so far. Let's move on to validation.

Validation rules
The validation rules were not listed in the requirements, so let's put down our best guess
based on what we know about Roman numerals:
Five in a row of any digit is not allowed
Some digits are allowed in runs of up to 4. They are I,X,C, and M. The others (V,L,D)
can only appear singly.
Some lower digits can come before a higher digit, but only if they appear singly. E.g.
"IX" is ok but "IIIX" is not.
But this is only for pairs of digits. Three ascending numbers in a row is invalid. E.g. "IX"
is ok but "IXC" is not.
A single digit with no runs is always allowed

480

Worked example: Roman numerals

We can convert these requirements into a pattern matching function as follows:

let runsAllowed =
function
| I | X | C | M -> true
| V | L | D -> false
let noRunsAllowed = runsAllowed >> not
// check for validity
let rec isValidDigitList digitList =
match digitList with
// empty list is valid
| [] -> true
// A run of 5 or more anything is invalid
// Example: XXXXX
| d1::d2::d3::d4::d5::_
when d1=d2 && d1=d3 && d1=d4 && d1=d5 ->
false
// 2 or more non-runnable digits is invalid
// Example: VV
| d1::d2::_
when d1=d2 && noRunsAllowed d1 ->
false
// runs of 2,3,4 in the middle are invalid if next digit is higher
// Example: IIIX
| d1::d2::d3::d4::higher::ds
when d1=d2 && d1=d3 && d1=d4
&& runsAllowed d1 // not really needed because of the order of matching
&& higher > d1 ->
false
| d1::d2::d3::higher::ds
when d1=d2 && d1=d3
&& runsAllowed d1
&& higher > d1 ->
false
| d1::d2::higher::ds
when d1=d2
&& runsAllowed d1
&& higher > d1 ->
false
// three ascending numbers in a row is invalid
// Example: IVX
| d1::d2::d3::_ when d1<d2 && d2<= d3 ->
false

481

Worked example: Roman numerals

// A single digit with no runs is always allowed

| _::ds ->
// check the remainder of the list
isValidDigitList ds

Again, note that "equality" and "less than" did not need to be defined.
And let's test the validation:
// test valid
let validList = [
[I;I;I;I]
[I;V]
[I;X]
[I;X;V]
[V;X]
[X;I;V]
[X;I;X]
[X;X;I;I]
]
let testValid = validList |> List.map isValidDigitList
let invalidList = [
// Five in a row of any digit is not allowed
[I;I;I;I;I]
// Two in a row for V,L, D is not allowed
[V;V]
[L;L]
[D;D]
// runs of 2,3,4 in the middle are invalid if next digit is higher
[I;I;V]
[X;X;X;M]
[C;C;C;C;D]
// three ascending numbers in a row is invalid
[I;V;X]
[X;L;D]
]
let testInvalid = invalidList |> List.map isValidDigitList

Finally, we add a top level function to test validity of the RomanNumeral type itself.

482

Worked example: Roman numerals

// top level check for validity

let isValid (RomanNumeral digitList) =
isValidDigitList digitList

// test good cases

The entire code for the first version

Here's all the code in one module:
module RomanNumeralsV1 =
// ==========================================
// Types
// ==========================================
type RomanDigit = I | V | X | L | C | D | M
type RomanNumeral = RomanNumeral of RomanDigit list
// ==========================================
// Output logic
// ==========================================
/// Converts a single RomanDigit to an integer
let digitToInt =
function
| I -> 1
| V -> 5
| X -> 10
| L -> 50

483

Worked example: Roman numerals

| C -> 100
| D -> 500
| M -> 1000
/// converts a list of digits to an integer
let rec digitsToInt =
function
// empty is 0
| [] -> 0
// special case when a smaller comes before larger
// convert both digits and add the difference to the sum
// Example: "IV" and "CM"
| smaller::larger::ns when smaller < larger ->
(digitToInt larger - digitToInt smaller) + digitsToInt ns
// otherwise convert the digit and add to the sum
| digit::ns ->
digitToInt digit + digitsToInt ns
/// converts a RomanNumeral to an integer
let toInt (RomanNumeral digits) = digitsToInt digits
// ==========================================
// Input logic
// ==========================================
type ParsedChar =
| Digit of RomanDigit
| BadChar of char
let charToRomanDigit =
function
| 'I' -> Digit I
| 'V' -> Digit V
| 'X' -> Digit X
| 'L' -> Digit L
| 'C' -> Digit C
| 'D' -> Digit D
| 'M' -> Digit M
| ch -> BadChar ch
let toRomanDigitList (s:string) =
s.ToCharArray()
|> List.ofArray
|> List.map charToRomanDigit
/// Convert a string to a RomanNumeral
/// Does not validate the input.E.g. "IVIV" would be valid
let toRomanNumeral s =
toRomanDigitList s
|> List.choose (

484

Worked example: Roman numerals

function
| Digit digit ->
Some digit
| BadChar ch ->
eprintfn "%c is not a valid character" ch
None
)
|> RomanNumeral
// ==========================================
// Validation logic
// ==========================================
let runsAllowed =
function
| I | X | C | M -> true
| V | L | D -> false
let noRunsAllowed = runsAllowed >> not
// check for validity
let rec isValidDigitList digitList =
match digitList with
// empty list is valid
| [] -> true
// A run of 5 or more anything is invalid
// Example: XXXXX
| d1::d2::d3::d4::d5::_
when d1=d2 && d1=d3 && d1=d4 && d1=d5 ->
false
// 2 or more non-runnable digits is invalid
// Example: VV
| d1::d2::_
when d1=d2 && noRunsAllowed d1 ->
false
// runs of 2,3,4 in the middle are invalid if next digit is higher
// Example: IIIX
| d1::d2::d3::d4::higher::ds
when d1=d2 && d1=d3 && d1=d4
&& runsAllowed d1 // not really needed because of the order of matching
&& higher > d1 ->
false
| d1::d2::d3::higher::ds
when d1=d2 && d1=d3
&& runsAllowed d1
&& higher > d1 ->
false

485

Worked example: Roman numerals

| d1::d2::higher::ds
when d1=d2
&& runsAllowed d1
&& higher > d1 ->
false
// three ascending numbers in a row is invalid
// Example: IVX
| d1::d2::d3::_ when d1<d2 && d2<= d3 ->
false
// A single digit with no runs is always allowed
| _::ds ->
// check the remainder of the list
isValidDigitList ds
// top level check for validity
let isValid (RomanNumeral digitList) =
isValidDigitList digitList

Second version
The code works, but there is something that's bugging me about it. The validation logic
seems very complicated. Surely the Romans didn't have to think about all of this?
And also, I can think of examples that should fail validation, but pass, such as "VIV":
"VIV" |> toRomanNumeral |> isValid

We could try to tighten up our validation rules, but let's try another tack. Complicated logic is
often a sign that you don't quite understand the domain properly.
In other words -- could we change the internal model to make everything simpler?
What about if we stopped trying to map letters to digits, and created a domain that mapped
how the Romans thought it. In this model "I", "II", "III", "IV" and so on would each be a
separate digit.
Let's run with it and see what happens.
Here's the new types for the domain. I now have a digit type for every possible digit. The
RomanNumeral type stays the same.

486

Worked example: Roman numerals

type RomanDigit =
| I | II | III | IIII
| IV | V
| IX | X | XX | XXX | XXXX
| XL | L
| XC | C | CC | CCC | CCCC
| CD | D
| CM | M | MM | MMM | MMMM
type RomanNumeral = RomanNumeral of RomanDigit list

Output: second version

Next, converting a single RomanDigit to an integer is the same as before, but with more
cases:
/// Converts a single RomanDigit to an integer
let digitToInt =
function
| I -> 1 | II -> 2 | III -> 3 | IIII -> 4
| IV -> 4 | V -> 5
| IX -> 9 | X -> 10 | XX -> 20 | XXX -> 30 | XXXX -> 40
| XL -> 40 | L -> 50
| XC -> 90 | C -> 100 | CC -> 200 | CCC -> 300 | CCCC -> 400
| CD -> 400 | D -> 500
| CM -> 900 | M -> 1000 | MM -> 2000 | MMM -> 3000 | MMMM -> 4000
// tests
I |> digitToInt
III |> digitToInt
V |> digitToInt
CM |> digitToInt

Finally, the top level function is identical:

487

Worked example: Roman numerals

/// converts a RomanNumeral to an integer

let toInt (RomanNumeral digits) = digitsToInt digits
// test
let x = RomanNumeral [M;CM;LX;X;IX]
x |> toInt

Input: second version

For the input parsing, we'll keep the ParsedChar type. But this time, we have to match 1,2,3,
or 4 chars at a time. That means we can't just pull off one character like we did in the first
version -- we have to match in the main loop. This means the loop now has to be recursive.
Also, we want to convert IIII into a single IIII digit rather than 4 separate I digits, so we
put the longest matches at the front.
type ParsedChar =
| Digit of RomanDigit
| BadChar of char
let rec toRomanDigitListRec charList =
match charList with
// match the longest patterns first
// 4 letter matches
| 'I'::'I'::'I'::'I'::ns ->
Digit IIII :: (toRomanDigitListRec ns)
| 'X'::'X'::'X'::'X'::ns ->
Digit XXXX :: (toRomanDigitListRec ns)
| 'C'::'C'::'C'::'C'::ns ->
Digit CCCC :: (toRomanDigitListRec ns)
| 'M'::'M'::'M'::'M'::ns ->
Digit MMMM :: (toRomanDigitListRec ns)
// 3 letter matches
| 'I'::'I'::'I'::ns ->
Digit III :: (toRomanDigitListRec ns)
| 'X'::'X'::'X'::ns ->
Digit XXX :: (toRomanDigitListRec ns)
| 'C'::'C'::'C'::ns ->
Digit CCC :: (toRomanDigitListRec ns)
| 'M'::'M'::'M'::ns ->
Digit MMM :: (toRomanDigitListRec ns)
// 2 letter matches
| 'I'::'I'::ns ->
Digit II :: (toRomanDigitListRec ns)
| 'X'::'X'::ns ->
Digit XX :: (toRomanDigitListRec ns)

488

Worked example: Roman numerals

| 'C'::'C'::ns ->
Digit CC :: (toRomanDigitListRec ns)
| 'M'::'M'::ns ->
Digit MM :: (toRomanDigitListRec ns)
| 'I'::'V'::ns ->
Digit IV :: (toRomanDigitListRec ns)
| 'I'::'X'::ns ->
Digit IX :: (toRomanDigitListRec ns)
| 'X'::'L'::ns ->
Digit XL :: (toRomanDigitListRec ns)
| 'X'::'C'::ns ->
Digit XC :: (toRomanDigitListRec ns)
| 'C'::'D'::ns ->
Digit CD :: (toRomanDigitListRec ns)
| 'C'::'M'::ns ->
Digit CM :: (toRomanDigitListRec ns)
// 1 letter matches
| 'I'::ns ->
Digit I :: (toRomanDigitListRec ns)
| 'V'::ns ->
Digit V :: (toRomanDigitListRec ns)
| 'X'::ns ->
Digit X :: (toRomanDigitListRec ns)
| 'L'::ns ->
Digit L :: (toRomanDigitListRec ns)
| 'C'::ns ->
Digit C :: (toRomanDigitListRec ns)
| 'D'::ns ->
Digit D :: (toRomanDigitListRec ns)
| 'M'::ns ->
Digit M :: (toRomanDigitListRec ns)
// bad letter matches
| badChar::ns ->
BadChar badChar :: (toRomanDigitListRec ns)
// 0 letter matches
| [] ->
[]

Well, this is much longer than the first version, but otherwise basically the same.
The top level functions are unchanged.

489

Worked example: Roman numerals

let toRomanDigitList (s:string) =

Validation: second version

Finally, let's see how the new domain model affects the validation rules. Now, the rules are
much simpler. In fact, there is only one.
Each digit must be smaller than the preceding digit

490

Worked example: Roman numerals

// check for validity

Alas, after all that, we still didn't fix the bad case that triggered the rewrite!
"VIV" |> toRomanNumeral |> isValid

There is a not-too-complicated fix for this, but I think it's time to leave it alone now!

The entire code for the second version

Here's all the code in one module for the second version:
module RomanNumeralsV2 =
// ==========================================
// Types
// ==========================================
type RomanDigit =

491

Worked example: Roman numerals

| I | II | III | IIII
| IV | V
| IX | X | XX | XXX | XXXX
| XL | L
| XC | C | CC | CCC | CCCC
| CD | D
| CM | M | MM | MMM | MMMM
type RomanNumeral = RomanNumeral of RomanDigit list
// ==========================================
// Output logic
// ==========================================
/// Converts a single RomanDigit to an integer
let digitToInt =
function
| I -> 1 | II -> 2 | III -> 3 | IIII -> 4
| IV -> 4 | V -> 5
| IX -> 9 | X -> 10 | XX -> 20 | XXX -> 30 | XXXX -> 40
| XL -> 40 | L -> 50
| XC -> 90 | C -> 100 | CC -> 200 | CCC -> 300 | CCCC -> 400
| CD -> 400 | D -> 500
| CM -> 900 | M -> 1000 | MM -> 2000 | MMM -> 3000 | MMMM -> 4000
/// converts a RomanNumeral to an integer
let toInt (RomanNumeral digits) = digitsToInt digits
// ==========================================
// Input logic
// ==========================================
type ParsedChar =
| Digit of RomanDigit
| BadChar of char
let rec toRomanDigitListRec charList =
match charList with
// match the longest patterns first
// 4 letter matches
| 'I'::'I'::'I'::'I'::ns ->
Digit IIII :: (toRomanDigitListRec ns)
| 'X'::'X'::'X'::'X'::ns ->
Digit XXXX :: (toRomanDigitListRec ns)
| 'C'::'C'::'C'::'C'::ns ->
Digit CCCC :: (toRomanDigitListRec ns)
| 'M'::'M'::'M'::'M'::ns ->
Digit MMMM :: (toRomanDigitListRec ns)
// 3 letter matches
| 'I'::'I'::'I'::ns ->
Digit III :: (toRomanDigitListRec ns)
| 'X'::'X'::'X'::ns ->

492

Worked example: Roman numerals

Digit XXX :: (toRomanDigitListRec ns)

| 'C'::'C'::'C'::ns ->
Digit CCC :: (toRomanDigitListRec ns)
| 'M'::'M'::'M'::ns ->
Digit MMM :: (toRomanDigitListRec ns)
// 2 letter matches
| 'I'::'I'::ns ->
Digit II :: (toRomanDigitListRec ns)
| 'X'::'X'::ns ->
Digit XX :: (toRomanDigitListRec ns)
| 'C'::'C'::ns ->
Digit CC :: (toRomanDigitListRec ns)
| 'M'::'M'::ns ->
Digit MM :: (toRomanDigitListRec ns)
| 'I'::'V'::ns ->
Digit IV :: (toRomanDigitListRec ns)
| 'I'::'X'::ns ->
Digit IX :: (toRomanDigitListRec ns)
| 'X'::'L'::ns ->
Digit XL :: (toRomanDigitListRec ns)
| 'X'::'C'::ns ->
Digit XC :: (toRomanDigitListRec ns)
| 'C'::'D'::ns ->
Digit CD :: (toRomanDigitListRec ns)
| 'C'::'M'::ns ->
Digit CM :: (toRomanDigitListRec ns)
// 1 letter matches
| 'I'::ns ->
Digit I :: (toRomanDigitListRec ns)
| 'V'::ns ->
Digit V :: (toRomanDigitListRec ns)
| 'X'::ns ->
Digit X :: (toRomanDigitListRec ns)
| 'L'::ns ->
Digit L :: (toRomanDigitListRec ns)
| 'C'::ns ->
Digit C :: (toRomanDigitListRec ns)
| 'D'::ns ->
Digit D :: (toRomanDigitListRec ns)
| 'M'::ns ->
Digit M :: (toRomanDigitListRec ns)
// bad letter matches
| badChar::ns ->
BadChar badChar :: (toRomanDigitListRec ns)
// 0 letter matches
| [] ->
[]

493

Worked example: Roman numerals

let toRomanDigitList (s:string) =

s.ToCharArray()
|> List.ofArray
|> toRomanDigitListRec
/// Convert a string to a RomanNumeral
/// Does not validate the input.E.g. "IVIV" would be valid
let toRomanNumeral s =
toRomanDigitList s
|> List.choose (
function
| Digit digit ->
Some digit
| BadChar ch ->
eprintfn "%c is not a valid character" ch
None
)
|> RomanNumeral
// ==========================================
// Validation logic
// ==========================================
// check for validity
let rec isValidDigitList digitList =
match digitList with
// empty list is valid
| [] -> true
// a following digit that is equal or larger is an error
| d1::d2::_
when d1 <= d2 ->
false
// A single digit is always allowed
| _::ds ->
// check the remainder of the list
isValidDigitList ds
// top level check for validity
let isValid (RomanNumeral digitList) =
isValidDigitList digitList

Comparing the two versions

Which version did you like better? The second one is more longwinded because it has many
more cases, but on the other hand, the actual logic is the same or simpler in all areas, with
no special cases. And as a result, the total number of lines of code is about the same for

494

Worked example: Roman numerals

both versions.
Overall, I prefer the second implementation because of the lack of special cases.
As a fun experiment, try writing the same code in C# or your favorite imperative language!

Making it object-oriented
Finally, let's see how we might make this object oriented. We don't care about the helper
functions, so we probably just need three methods:
A static constructor
A method to convert to a int
A method to convert to a string
And here they are:
type RomanNumeral with
static member FromString s =
toRomanNumeral s
member this.ToInt() =
toInt this
override this.ToString() =
sprintf "%A" this

Note: you can ignore the compiler warning about deprecated overrides.
Let's use this in an object oriented way now:
let r = RomanNumeral.FromString "XXIV"
let s = r.ToString()
let i = r.ToInt()

Summary
In this post we've seen lots and lots of pattern matching!
But again, as with the last post, what's equally important is that we've seen how easy it is to
create a properly designed internal model for very trivial domains. And again, our internal
model used no primitive types -- there is no excuse not to create lots of little types in order to

495

Worked example: Roman numerals

represent the domain better. For example, the ParsedChar type -- would you have bothered
to create that in C#?
And as should be clear, the choice of an internal model can make a lot of difference to the
complexity of the design. But if and when we do refactor, the compiler will almost always
warn us if we have forgotten something.

496

The "Understanding F# types" Series

F# is not just about functions; the powerful type system is another key ingredient. And just
as with functions, understanding the type system is critical to being fluent and comfortable in
the language.
In addition to the common .NET types. F# has some other types that are very common in
functional languages but not available in imperative languages like C# or Java.
This series introduces these types and how to use them.
Understanding F# types: Introduction. A new world of types.
Overview of types in F#. A look at the big picture.
Type abbreviations. Also known as aliases.
Tuples. Multiplying types together.
Records. Extending tuples with labels.
Discriminated Unions. Adding types together.
The Option type. And why it is not null or nullable.
Enum types. Not the same as a union type.
Built-in .NET types. Ints, strings, bools, etc.
Units of measure. Type safety for numerics.
Understanding type inference. Behind the magic curtain.

497

Understanding F# types: Introduction

NOTE: Before reading this series, I suggest that you read the "thinking functionally" and
"expressions and syntax" series as a prerequisite.
F# is not just about functions; the powerful type system is another key ingredient. And just
as with functions, understanding the type system is critical to being fluent and comfortable in
the language.
Now, so far we have seen some basic types that can be used as input and output to
functions:
Primitive types such as int , float , string , and bool
Simple function types such as int->int
The unit type
Generic types.
None of these types should be unfamiliar. Analogues of these are available in C# and other
imperative languages.
But in this series we are going to introduce some new kinds of types that are very common
in functional languages but uncommon in imperative languages.
The extended types we will look at in this series are:
Tuples
Records
Unions
The Option type
Enum types
For all these types, we will discuss both the abstract principles and the details of how to use
them in practice.
Lists and other recursive data types are also very important types, but there is so much to
say about them that they will need their own series!

498

Overview of types in F#

Overview of types in F#
Before we dive into all the specific types, let's look at the big picture.

What are types for?

If you are coming from an object-oriented design background, one of the paradigm shifts
involved in "thinking functionally" is to change how you think about types.
A well designed object-oriented program will have a strong focus on behavior rather than
data, so it will use a lot of polymorphism, either using "duck-typing" or explicit interfaces, and
will try to avoid having explicit knowledge of the actual concrete classes being passed
around.
A well designed functional program, on the other hand, will have a strong focus on data
types rather than behavior. F# puts much more emphasis on designing types correctly than
an imperative language such as C#, and many of the examples in this series and later series
will focus on creating and refining type definitions.
So what is a type? Types are surprisingly hard to define. One definition from a well known
textbook says:
"A type system is a tractable syntactic method of proving the absence of certain
program behaviors by classifying phrases according to the kinds of values they
compute"
(Benjamin Pierce, Types and Programming Languages)
Ok, that definition is a bit technical. So let's turn it around -- what do we use types for in
practice? In the context of F#, you can think of types as being used in two main ways:
Firstly, as an annotation to a value that allows certain checks to be made, especially at
compile time. In other words, types allow you to have "compile time unit tests".
Second, as domains for functions to act upon. That is, a type is a sort of data modeling
tool that allows you to represent a real world domain in your code.
These two definitions interact. The better the type definitions reflect the real-world domain,
the better they will statically encode the business rules. And the better they statically encode
the business rules, the better the "compile time unit tests" work. In the ideal scenario, if your
program compiles, then it really is correct!

499

Overview of types in F#

What kinds of types are there?

F# is a hybrid language, so it has a mixture of types: some from its functional background,
and some from its object-oriented background.
Generally, the types in F# can be grouped into the following categories:
Common .NET types. These are types that conform to the .NET Common Language
Infrastructure (CLI), and which are easily portable to every .NET language.
F# specific types. These are types that are part of the F# language and are designed
for pure functional programming.
If you are familiar with C#, you will know all the CLI types. They include:
Built-in value types (int, bool, etc).
Built-in reference types (string, etc).
User-defined value types (enum and struct).
Classes and interfaces
Delegates
Arrays
The F# specific types include:
Function types (not the same as delegates or C# lambdas)
The unit type
Tuples (now part of .NET 4.0)
Records
Discriminated Unions
Option types
Lists (not the same as the .NET List class)
I strongly recommend that when creating new types you stick with the F# specific types
rather than using classes. They have a number of advantages over the CLI types, such as:
They are immutable
They cannot be null
They have built-in structural equality and comparison
They have built-in pretty printing

Sum and Product types

The key to understanding the power of types in F# is that most new types are constructed by
from other types using two basic operations: sum and product.

500

Overview of types in F#

That is, in F# you can define new types almost as if you were doing algebra:
define typeZ = typeX "plus" typeY
define typeW = typeX "times" typeZ

I will hold off explaining what sum and product mean in practice until we get to the detailed
discussion of tuples (products) and discriminated union (sum) types later in this series.
The key point is that an infinite number of new types can be made by combining existing
types together using these "product" and "sum" methods in various ways. Collectively these
are called "algebraic data types" or ADTs (not to be confused with abstract data types, also
called ADTs). Algebraic data types can be used to model anything, including lists, trees, and
other recursive types.
The sum or "union" types, in particular, are very valuable, and once you get used to them,
you will find them indispensible!

How types are defined

Every type definition is similar, even though the specific details may vary. All type definitions
start with a " type " keyword, followed by an identifier for the type, followed by any generic
type parameters, followed by the definition. For example, here are some type definitions for
a variety of types:
type A = int * int
type B = {FirstName:string; LastName:string}
type C = Circle of int | Rectangle of int * int
type D = Day | Month | Year
type E<'a> = Choice1 of 'a | Choice2 of 'a * 'a
type MyClass(initX:int) =
let x = initX
member this.Method() = printf "x=%i" x

As we said in a previous post, there is a special syntax for defining new types that is
different from the normal expression syntax. So do be aware of this difference.
Types can only be declared in namespaces or modules. But that doesn't mean you always
have to create them at the top level -- you can create types in nested modules if you need to
hide them.

501

Overview of types in F#

module sub =
// type declared in a module
type A = int * int
module private helper =
// type declared in a submodule
type B = B of string list
//internal access is allowed
let b = B ["a";"b"]
//outside access not allowed
let b = sub.helper.B ["a";"b"]

Types cannot be declared inside functions.

let f x =
type A = int * int //unexpected keyword "type"
x * x

Constructing and deconstructing types

After a type is defined, instances of the type are created using a "constructor" expression
that often looks quite similar to the type definition itself.
let a = (1,1)
let b = { FirstName="Bob"; LastName="Smith" }
let c = Circle 99
let c' = Rectangle (2,1)
let d = Month
let e = Choice1 "a"
let myVal = MyClass 99
myVal.Method()

What is interesting is that the same "constructor" syntax is also used to "deconstruct" the
type when doing pattern matching:

502

Overview of types in F#

let a = (1,1) // "construct"

let (a1,a2) = a // "deconstruct"
let b = { FirstName="Bob"; LastName="Smith" } // "construct"
let { FirstName = b1 } = b // "deconstruct"
let c = Circle 99 // "construct"
match c with
| Circle c1 -> printf "circle of radius %i" c1 // "deconstruct"
| Rectangle (c2,c3) -> printf "%i %i" c2 c3 // "deconstruct"
let c' = Rectangle (2,1) // "construct"
match c' with
| Circle c1 -> printf "circle of radius %i" c1 // "deconstruct"
| Rectangle (c2,c3) -> printf "%i %i" c2 c3 // "deconstruct"

As you read through this series, pay attention to how the constructors are used in both ways.

Field guide to the "type" keyword

The same "type" keyword is used to define all the F# types, so they can all look very similar
if you are new to F#. Here is a quick list of these types and how to tell the difference
between them.
Type

Abbrev
(Alias)

Tuple

Example

type ProductCode = string

type transform<'a> = 'a ->
'a

//not explicitly defined

with type keyword
//usage
let t = 1,2
let s = (3,4)

Distinguishing
features

Uses equal sign only.

Always available to be
used and are not
explicitly defined with
the type keyword.
Usage indicated by
comma (with optional
parentheses).

503

Overview of types in F#

Record

type Product =
{code:ProductCode;
price:float }
type Message<'a> = {id:int;
body:'a}

Curly braces.
Uses semicolon to
separate fields.

//usage
let p = {code="X123";
price=9.99}
let m = {id=1;
body="hello"}

Discriminated
Union

Enum

type Gender = | Male = 1 |

Female = 2
//usage
let g = Gender.Male

Vertical bar character.

Uses "of" for types.

Similar to Unions, but

uses equals and an int
value

Has function-style
parameter list after
name for use as

504

Overview of types in F#

Class

type Product (code:string,

price:float) =
let isFree = price=0.0
new (code) =
Product(code,0.0)
member this.Code = code
member this.IsFree =
isFree

Has "member"
keyword.
Has "new" keyword for
secondary
constructors.

//usage
let p =
Product("X123",9.99)
let p2 = Product("X123")

Interface

Struct

type IPrintable =
abstract member Print :
unit -> unit

type Product=
struct
val code:string
val price:float
new(code) = { code =
code; price = 0.0 }
end
//usage
let p = Product()
let p2 = Product("X123")

Same as class but all

members are abstract.
Abstract members
have colon and type
signature rather than a
concrete
implementation.

Has "struct" keyword.

Uses "val" to define
fields.
Can have constructor.

505

Type abbreviations

Type abbreviations
Let's start with the simplest type definition, a type abbreviation or alias.
It has the form:
type [typename] = [existingType]

where "existing type" can be any type: one of the basic types we have already seen, or one
of the extended types we will be seeing soon.
Some examples:
type RealNumber = float
type ComplexNumber = float * float
type ProductCode = string
type CustomerId = int
type AdditionFunction = int->int->int
type ComplexAdditionFunction =
ComplexNumber-> ComplexNumber -> ComplexNumber

And so on -- pretty straightforward.

Type abbreviations are useful to provide documentation and avoid writing a signature
repeatedly. In the above examples, ComplexNumber and AdditionFunction demonstrate this.
Another use is to provide some degree of decoupling between the usage of a type and the
actual implementation of a type. In the above examples, ProductCode and CustomerId
demonstrate this. I could easily change CustomerId to be a string without changing (most
of) my code.
However, one thing is to note is that this really is just an alias or abbreviation; you are not
actually creating a new type. So if I define a function that I explicitly say is an
AdditionFunction :

type AdditionFunction = int->int->int

let f:AdditionFunction = fun a b -> a + b

the compiler will erase the alias and return a plain int->int->int as the function signature.

506

Type abbreviations

In particular, there is no true encapsulation. I could use an explicit int anywhere I used a
CustomerId and the compiler would not complain. And if I had attempted to create safe

versions of entity ids such as this:

type CustomerId = int
type OrderId = int

then I would be disappointed. There would be nothing preventing me from using an

OrderId in place of a CustomerId and vice versa. To get true encapsulated types like this,

we will need to use single case union types, as described in a later post.

507

Tuples

Tuples
We're ready for our first extended type -- the tuple.
Let's start by stepping back again and looking at a type such as "int". As we hinted at before,
rather than thinking of "int" as a abstract thing, you can think of it as concrete collection of all
its possible values, namely the set {...,-3, -2, -1, 0, 2, 3, ...}.
So next, imagine two copies of this "int" collection. We can "multiply" them together by taking
the Cartesian product of them; that is, making a new list of objects by picking every possible
combination of the two "int" lists, as shown below:

As we have already seen, these pairs are called tuples in F#. And now you can see why
they have the type signature that they do. In this example, the "int times int" type is called
" int * int ", and the star symbol means "multiply" of course! The valid instances of this
new type are all the pairs: (-2,2),(-1,0), (2,2) and so on.
Let's see how they might be used in practice:
let t1 = (2,3)
let t2 = (-2,7)

Now if you evaluate the code above you will see that the types of t1 and t2 are int*int as
expected.
val t1 : int * int = (2, 3)
val t2 : int * int = (-2, 7)

508

Tuples

This "product" approach can be used to make tuples out of any mixture of types. Here is one
for "int times bool".

And here is the usage in F#. The tuple type above has the signature " int*bool ".
let t3 = (2,true)
let t4 = (7,false)
// the signatures are:
val t3 : int * bool = (2, true)
val t4 : int * bool = (7, false)

Strings can be used as well, of course. The universe of all possible strings is very large, but
conceptually it is the same thing. The tuple type below has the signature " string*int ".

Test the usage and signatures:

509

Tuples

let t5 = ("hello",42)
let t6 = ("goodbye",99)
// the signatures are:
val t5 : string * int = ("hello", 42)
val t6 : string * int = ("goodbye", 99)

And there is no reason to stop at multiplying just two types together. Why not three? Or four?
For example, here is the type int * bool * string .

Test the usage and signatures:

let t7 = (42,true,"hello")
// the signature is:
val t7 : int * bool * string = (42, true, "hello")

Generic tuples
Generics can be used in tuples too.

The usage is normally associated with functions:

let genericTupleFn aTuple =
let (x,y) = aTuple
printfn "x is %A and y is %A" x y

510

Tuples

And the function signature is:

val genericTupleFn : 'a * 'b -> unit

which means that " genericTupleFn " takes a generic tuple ('a * 'b) and returns a unit

Tuples of complex types

Any kind of type can be used in a tuple: other tuples, classes, function types, etc. Here are
some examples:
// define some types
type Person = {First:string; Last:string}
type Complex = float * float
type ComplexComparisonFunction = Complex -> Complex -> int
// define some tuples using them
type PersonAndBirthday = Person * System.DateTime
type ComplexPair = Complex * Complex
type ComplexListAndSortFunction = Complex list * ComplexComparisonFunction
type PairOfIntFunctions = (int->int) * (int->int)

Key points about tuples

Some key things to know about tuples are:
A particular instance of a tuple type is a single object, similar to a two-element array in
C#, say. When using them with functions they count as a single parameter.
Tuple types cannot be given explicit names. The "name" of the tuple type is determined
by the combination of types that are multiplied together.
The order of the multiplication is important. So int*string is not the same tuple type
as string*int .
The comma is the critical symbol that defines tuples, not the parentheses. You can
define tuples without the parentheses, although it can sometimes be confusing. In F#, if
you see a comma, it is probably part of a tuple.
These points are very important -- if you don't understand them you will get confused quite
quickly!
And it is worth re-iterating the point made in previous posts: don't mistake tuples for multiple
parameters in a function.

511

Tuples

// a function that takes a single tuple parameter

// but looks like it takes two ints
let addConfusingTuple (x,y) = x + y

Making and matching tuples

The tuple types in F# are somewhat more primitive than the other extended types. As you
have seen, you don't need to explicitly define them, and they have no name.
It is easy to make a tuple -- just use a comma!
let x = (1,2)
let y = 1,2 // it's the comma you need, not the parentheses!
let z = 1,true,"hello",3.14 // create arbitrary tuples as needed

And as we have seen, to "deconstruct" a tuple, use the same syntax:

let z = 1,true,"hello",3.14 // "construct"
let z1,z2,z3,z4 = z // "deconstruct"

When pattern matching like this, you must have the same number of elements, otherwise
you will get an error:
let z1,z2 = z // error FS0001: Type mismatch.
// The tuples have differing lengths

If you don't need some of the values, you can use the "don't care" symbol (the underscore)
as a placeholder.
let _,z5,_,z6 = z // ignore 1st and 3rd elements

As you might guess, a two element tuple is commonly called a "pair" and a three element
tuple is called a "triple" and so on. In the special case of pairs, there are functions fst and
snd which extract the first and second element.

let x = 1,2
fst x
snd x

They only work on pairs. Trying to use fst on a triple will give an error.

512

Tuples

let x = 1,2,3
fst x // error FS0001: Type mismatch.
// The tuples have differing lengths of 2 and 3

Using tuples in practice

Tuples have a number of advantages over other more complex types. They can be used on
the fly because they are always available without being defined, and thus are perfect for
small, temporary, lightweight structures.

Using tuples for returning multiple values

It is a common scenario that you want to return two values from a function rather than just
one. For example, in the TryParse style functions, you want to return (a) whether the value
was parsed and (b) if parsed, what the parsed value was.
Here is an implementation of TryParse for integers (assuming it did not already exist, of
course):
let tryParse intStr =
try
let i = System.Int32.Parse intStr
(true,i)
with _ -> (false,0) // any exception
//test it
tryParse "99"
tryParse "abc"

Here's another simple example that returns a pair of numbers:

// return word count and letter count in a tuple
let wordAndLetterCount (s:string) =
let words = s.Split [|' '|]
let letterCount = words |> Array.sumBy (fun word -> word.Length )
(words.Length, letterCount)
//test
wordAndLetterCount "to be or not to be"

Creating tuples from other tuples

513

Tuples

As with most F# values, tuples are immutable and the elements within them cannot be
assigned to. So how do you change a tuple? The short answer is that you can't -- you must
always create a new one.
Say that you need to write a function that, given a tuple, adds one to each element. Here's
an obvious implementation:
let addOneToTuple aTuple =
let (x,y,z) = aTuple
(x+1,y+1,z+1) // create a new one
// try it
addOneToTuple (1,2,3)

This seems a bit long winded -- is there a more compact way? Yes, because you can
deconstruct a tuple directly in the parameters of a function, so that the function becomes a
one liner:
let addOneToTuple (x,y,z) = (x+1,y+1,z+1)
// try it
addOneToTuple (1,2,3)

Equality
Tuples have an automatically defined equality operation: two tuples are equal if they have
the same length and the values in each slot are equal.
(1,2) = (1,2) // true
(1,2,3,"hello") = (1,2,3,"bye") // false
(1,(2,3),4) = (1,(2,3),4) // true

Trying to compare tuples of different lengths is a type error:

(1,2) = (1,2,3) // error FS0001: Type mismatch

And the types in each slot must be the same as well:

(1,2,3) = (1,2,"hello") // element 3 was expected to have type
// int but here has type string
(1,(2,3),4) = (1,2,(3,4)) // elements 2 & 3 have different types

514

Tuples

Tuples also have an automatically defined hash value based on the values in the tuple, so
that tuples can be used as dictionary keys without problems.
(1,2,3).GetHashCode()

Tuple representation
And as noted in a previous post, tuples have a nice default string representation, and can be
serialized easily.
(1,2,3).ToString()

515

Records

Records
As we noted in the previous post, plain tuples are useful in many cases. But they have some
disadvantages too. Because all tuple types are pre-defined, you can't distinguish between a
pair of floats used for geographic coordinates say, vs. a similar tuple used for complex
numbers. And when tuples have more than a few elements, it is easy to get confused about
which element is in which place.
In these situations, what you would like to do is label each slot in the tuple, which will both
document what each element is for and force a distinction between tuples made from the
same types.
Enter the "record" type. A record type is exactly that, a tuple where each element is labeled.
type ComplexNumber = { real: float; imaginary: float }
type GeoCoord = { lat: float; long: float }

A record type has the standard preamble: type [typename] = followed by curly braces.
Inside the curly braces is a list of label: type pairs, separated by semicolons (remember,
all lists in F# use semicolon separators -- commas are for tuples).
Let's compare the "type syntax" for a record type with a tuple type:
type ComplexNumberRecord = { real: float; imaginary: float }
type ComplexNumberTuple = float * float

In the record type, there is no "multiplication", just a list of labeled types.

Relational database theory uses a similar "record type" concept. In the relational model,
a relation is a (possibly empty) finite set of tuples all having the same finite set of
attributes. This set of attributes is familiarly referred to as the set of column names.

Making and matching records

To create a record value, use a similar format to the type definition, but using equals signs
after the labels. This is called a "record expression."

516

Records

type ComplexNumberRecord = { real: float; imaginary: float }

let myComplexNumber = { real = 1.1; imaginary = 2.2 } // use equals!
type GeoCoord = { lat: float; long: float } // use colon in type
let myGeoCoord = { lat = 1.1; long = 2.2 } // use equals in let

And to "deconstruct" a record, use the same syntax:

let myGeoCoord = { lat = 1.1; long = 2.2 } // "construct"
let { lat=myLat; long=myLong } = myGeoCoord // "deconstruct"

As always, if you don't need some of the values, you can use the underscore as a
placeholder; or more cleanly, just leave off the unwanted label altogether.
let { lat=_; long=myLong2 } = myGeoCoord // "deconstruct"
let { long=myLong3 } = myGeoCoord // "deconstruct"

If you just need a single property, you can use dot notation rather than pattern matching.
let x = myGeoCoord.lat
let y = myGeoCoord.long

Note that you can leave a label off when deconstructing, but not when constructing:
let myGeoCoord = { lat = 1.1; } // error FS0764: No assignment
// given for field 'long'

One of the most noticeable features of record types is use of curly braces. Unlike Cstyle languages, curly braces are rarely used in F# -- only for records, sequences,
computation expressions (of which sequences are a special case), and object
expressions (creating implementations of interfaces on the fly). These other uses will be
discussed later.

Label order
Unlike tuples, the order of the labels is not important. So the following two values are the
same:

517

Records

let myGeoCoordA = { lat = 1.1; long = 2.2 }

let myGeoCoordB = { long = 2.2; lat = 1.1 } // same as above

Naming conflicts
In the examples above, we could construct a record by just using the label names " lat "
and " long ". Magically, the compiler knew what record type to create. (Well, in truth, it was
not really that magical, as only one record type had those exact labels.)
But what happens if there are two record types with the same labels? How can the compiler
know which one you mean? The answer is that it can't -- it will use the most recently defined
type, and in some cases, issue a warning. Try evaluating the following:
type Person1 = {first:string; last:string}
type Person2 = {first:string; last:string}
let p = {first="Alice"; last="Jones"}

What type is p ? Answer: Person2 , which was the last type defined with those labels.
And if you try to deconstruct, you will get a warning about ambiguous field labels.
let {first=f; last=l} = p

How can you fix this? Simply by adding the type name as a qualifier to at least one of the
labels.
let p = {Person1.first="Alice"; last="Jones"}
let { Person1.first=f; last=l} = p

If needed, you can even add a fully qualified name (with namespace). Here's an example
using modules.
module Module1 =
type Person = {first:string; last:string}
module Module2 =
type Person = {first:string; last:string}
module Module3 =
let p = {Module1.Person.first="Alice";
Module1.Person.last="Jones"}

518

Records

Of course, if you can ensure there is only one version in the local namespace, you can avoid
having to do this at all.
module Module3b =
open Module1 // bring into the local namespace
let p = {first="Alice"; last="Jones"} // will be Module1.Person

The moral of the story is that when defining record types, you should try to use unique labels
if possible, otherwise you will get ugly code at best, and unexpected behavior at worst.
Note that in F#, unlike some other functional languages, two types with exactly the same
structural definition are not the same type. This is called a "nominal" type system, where
two types are only equal if they have the same name, as opposed to a "structural" type
system, where definitions with identical structures will be the same type regardless of
what they are called.

Using records in practice

How can we use records? Let us count the ways...

Using records for function results

Just like tuples, records are useful for passing back multiple values from a function. Let's
revisit the tuple examples described earlier, rewritten to use records instead:

519

Records

// the tuple version of TryParse

let tryParseTuple intStr =
try
let i = System.Int32.Parse intStr
(true,i)
with _ -> (false,0) // any exception
// for the record version, create a type to hold the return result
type TryParseResult = {success:bool; value:int}
// the record version of TryParse
let tryParseRecord intStr =
try
let i = System.Int32.Parse intStr
{success=true;value=i}
with _ -> {success=false;value=0}
//test it
tryParseTuple "99"
tryParseRecord "99"
tryParseTuple "abc"
tryParseRecord "abc"

You can see that having explicit labels in the return value makes it much easier to
understand (of course, in practice we would probably use an Option type, discussed later).
And here's the word and letter count example using records rather than tuples:
//define return type
type WordAndLetterCountResult = {wordCount:int; letterCount:int}
let wordAndLetterCount (s:string) =
let words = s.Split [|' '|]
let letterCount = words |> Array.sumBy (fun word -> word.Length )
{wordCount=words.Length; letterCount=letterCount}
//test
wordAndLetterCount "to be or not to be"

Creating records from other records

Again, as with most F# values, records are immutable and the elements within them cannot
be assigned to. So how do you change a record? Again the answer is that you can't -- you
must always create a new one.
Say that you need to write a function that, given a GeoCoord record, adds one to each
element. Here it is:

520

Records

let addOneToGeoCoord aGeoCoord =

let {lat=x; long=y} = aGeoCoord
{lat = x + 1.0; long = y + 1.0} // create a new one
// try it
addOneToGeoCoord {lat=1.1; long=2.2}

But again you can simplify by deconstructing directly in the parameters of a function, so that
the function becomes a one liner:
let addOneToGeoCoord {lat=x; long=y} = {lat=x+1.0; long=y+1.0}
// try it
addOneToGeoCoord {lat=1.0; long=2.0}

or depending on your taste, you can also use dot notation to get the properties:
let addOneToGeoCoord aGeoCoord =
{lat=aGeoCoord.lat + 1.0; long= aGeoCoord.long + 1.0}

In many cases, you just need to tweak one or two fields and leave all the others alone. To
make life easier, there is a special syntax for this common case, the " with " keyword. You
start with the original value, followed by "with" and then the fields you want to change. Here
are some examples:
let g1 = {lat=1.1; long=2.2}
let g2 = {g1 with lat=99.9} // create a new one
let p1 = {first="Alice"; last="Jones"}
let p2 = {p1 with last="Smith"}

The technical term for "with" is a copy-and-update record expression.

Record equality
Like tuples, records have an automatically defined equality operation: two records are equal
if they have the same type and the values in each slot are equal.
And records also have an automatically defined hash value based on the values in the
record, so that records can be used as dictionary keys without problems.
{first="Alice"; last="Jones"}.GetHashCode()

521

Records

Record representation
As noted in a previous post, records have a nice default string representation, and can be
serialized easily. But unlike tuples, the ToString() representation is unhelpful.
printfn "%A" {first="Alice"; last="Jones"} // nice
{first="Alice"; last="Jones"}.ToString() // ugly
printfn "%O" {first="Alice"; last="Jones"} // ugly

Sidebar: %A vs. %O in print format strings

We just saw that print format specifiers %A and %O produce very different results for the
same record:
printfn "%A" {first="Alice"; last="Jones"}
printfn "%O" {first="Alice"; last="Jones"}

So why the difference?

%A prints the value using the same pretty printer that is used for interactive output. But %O

uses Object.ToString() , which means that if the ToString method is not overridden, %O
will give the default (and generally unhelpful) output. So in general, you should try to use
%A to %O where possible, because the core F# types do have pretty-printing by default.

But note that the F# "class" types do not have a standard pretty printed format, so %A and
%O are equally uncooperative unless you override ToString .

522

Discriminated Unions

Discriminated Unions
Tuples and records are examples of creating new types by "multiplying" existing types
together. At the beginning of the series, I mentioned that the other way of creating new types
was by "summing" existing types. What does this mean?
Well, let's say that we want to define a function that works with integers OR booleans,
maybe to convert them into strings. But we want to be strict and not accept any other type
(such as floats or strings). Here's a diagram of such as function:

How could we represent the domain of this function?

What we need is a type that represents all possible integers PLUS all possible booleans.

In other words, a "sum" type. In this case the new type is the "sum" of the integer type plus
the boolean type.

523

Discriminated Unions

In F#, a sum type is called a "discriminated union" type. Each component type (called a
union case) must be tagged with a label (called a case identifier or tag) so that they can be
told apart ("discriminated"). The labels can be any identifier you like, but must start with an
uppercase letter.
Here's how we might define the type above:
type IntOrBool =
| I of int
| B of bool

The "I" and the "B" are just arbitrary labels; we could have used any other labels that were
meaningful.
For small types, we can put the definition on one line:
type IntOrBool = I of int | B of bool

The component types can be any other type you like, including tuples, records, other union
types, and so on.
type Person = {first:string; last:string} // define a record type
type IntOrBool = I of int | B of bool
type MixedType =
| Tup of int * int // a tuple
| P of Person // use the record type defined above
| L of int list // a list of ints
| U of IntOrBool // use the union type defined above

You can even have types that are recursive, that is, they refer to themselves. This is typically
how tree structures are defined. Recursive types will be discussed in more detail shortly.

Sum types vs. C++ unions and VB variants

At first glance, a sum type might seem similar to a union type in C++ or a variant type in
Visual Basic, but there is a key difference. The union type in C++ is not type-safe and the
data stored in the type can be accessed using any of the possible tags. An F# discriminated
union type is safe, and the data can only be accessed one way. It really is helpful to think of
it as a sum of two types (as shown in the diagram), rather than as just an overlay of data.

Key points about union types

524

Discriminated Unions

Some key things to know about union types are:

The vertical bar is optional before the first component, so that the following definitions
are all equivalent, as you can see by examining the output of the interactive window:
type IntOrBool = I of int | B of bool // without initial bar
type IntOrBool = | I of int | B of bool // with initial bar
type IntOrBool =
| I of int
| B of bool // with initial bar on separate lines

The tags or labels must start with an uppercase letter. So the following will give an error:
type IntOrBool = int of int| bool of bool
// error FS0053: Discriminated union cases
// must be uppercase identifiers

Other named types (such as Person or IntOrBool ) must be pre-defined outside the
union type. You can't define them "inline" and write something like this:
type MixedType =
| P of {first:string; last:string} // error

or
type MixedType =
| U of (I of int | B of bool) // error

The labels can be any identifier, including the names of the component type
themselves, which can be quite confusing if you are not expecting it. For example, if the
Int32 and Boolean types (from the System namespace) were used instead, and the

labels were named the same, we would have this perfectly valid definition:
open System
type IntOrBool = Int32 of Int32 | Boolean of Boolean

This "duplicate naming" style is actually quite common, because it documents exactly what
the component types are.

Constructing a value of a union type

525

Discriminated Unions

To create a value of a union type, you use a "constructor" that refers to only one of the
possible union cases. The constructor then follows the form of the definition, using the case
label as if it were a function. In the IntOrBool example, you would write:
type IntOrBool = I of int | B of bool
let i = I 99 // use the "I" constructor
// val i : IntOrBool = I 99
let b = B true // use the "B" constructor
// val b : IntOrBool = B true

The resulting value is printed out with the label along with the component type:
val [value name] : [type] = [label] [print of component type]
val i : IntOrBool = I 99
val b : IntOrBool = B true

If the case constructor has more than one "parameter", you construct it in the same way that
you would call a function:
type Person = {first:string; last:string}
type MixedType =
| Tup of int * int
| P of Person
let myTup = Tup (2,99) // use the "Tup" constructor
// val myTup : MixedType = Tup (2,99)
let myP = P {first="Al"; last="Jones"} // use the "P" constructor
// val myP : MixedType = P {first = "Al";last = "Jones";}

The case constructors for union types are normal functions, so you can use them anywhere
a function is expected. For example, in List.map :
type C = Circle of int | Rectangle of int * int
[1..10]
|> List.map Circle
[1..10]
|> List.zip [21..30]
|> List.map Rectangle

526

Discriminated Unions

Naming conflicts
If a particular case has a unique name, then the type to construct will be unambiguous.
But what happens if you have two types which have cases with the same labels?
type IntOrBool1 = I of int | B of bool
type IntOrBool2 = I of int | B of bool

In this case, the last one defined is generally used:

let x = I 99 // val x : IntOrBool2 = I 99

But it is much better to explicitly qualify the type, as shown:

let x1 = IntOrBool1.I 99 // val x1 : IntOrBool1 = I 99
let x2 = IntOrBool2.B true // val x2 : IntOrBool2 = B true

And if the types come from different modules, you can use the module name as well:
module Module1 =
type IntOrBool = I of int | B of bool
module Module2 =
type IntOrBool = I of int | B of bool
module Module3 =
let x = Module1.IntOrBool.I 99 // val x : Module1.IntOrBool = I 99

Matching on union types

For tuples and records, we have seen that "deconstructing" a value uses the same model as
constructing it. This is also true for union types, but we have a complication: which case
should we deconstruct?
This is exactly what the "match" expression is designed for. As you should now realize, the
match expression syntax has parallels to how a union type is defined.

527

Discriminated Unions

// definition of union type

type MixedType =
| Tup of int * int
| P of Person
// "deconstruction" of union type
let matcher x =
match x with
| Tup (x,y) ->
printfn "Tuple matched with %i %i" x y
| P {first=f; last=l} ->
printfn "Person matched with %s %s" f l
let myTup = Tup (2,99) // use the "Tup" constructor
matcher myTup
let myP = P {first="Al"; last="Jones"} // use the "P" constructor
matcher myP

Let's analyze what is going on here:

Each "branch" of the overall match expression is a pattern expression that is designed
to match the corresponding case of the union type.
The pattern starts with the tag for the particular case, and then the rest of the pattern
deconstructs the type for that case in the usual way.
The pattern is followed by an arrow "->" and then the code to execute.

Empty cases
The label for a union case does not have to have to have any type after it. The following are
all valid union types:
type Directory =
| Root // no need to name the root
| Subdirectory of string // other directories need to be named
type Result =
| Success // no string needed for success state
| ErrorMessage of string // error message needed

If all the cases are empty, then we have an "enum style" union:
type Size = Small | Medium | Large
type Answer = Yes | No | Maybe

528

Discriminated Unions

Note that this "enum style" union is not the same as a true C# enum type, discussed later.
To create an empty case, just use the label as a constructor without any parameters:
let myDir1 = Root
let myDir2 = Subdirectory "bin"
let myResult1 = Success
let myResult2 = ErrorMessage "not found"
let mySize1 = Small
let mySize2 = Medium

Single cases
Sometimes it is useful to create union types with only one case. This might be seem
useless, because you don't seem to be adding value. But in fact, this a very useful practice
that can enforce type safety*.
* And in a future series we'll see that, in conjuction with module signatures, single case
unions can also help with data hiding and capability based security.
For example, let's say that we have customer ids and order ids which are both represented
by integers, but that they should never be assigned to each other.
As we saw before, a type alias approach will not work, because an alias is just a synonym
and doesn't create a distinct type. Here's how you might try to do it with aliases:
type CustomerId = int // define a type alias
type OrderId = int // define another type alias
let printOrderId (orderId:OrderId) =
printfn "The orderId is %i" orderId
//try it
let custId = 1 // create a customer id
printOrderId custId // Uh-oh!

But even though I explicitly annotated the orderId parameter to be of type OrderId , I can't
ensure that customer ids are not accidentally passed in.
On the other hand, if we create simple union types, we can easily enforce the type
distinctions.

529

Discriminated Unions

type CustomerId = CustomerId of int // define a union type

type OrderId = OrderId of int // define another union type
let printOrderId (OrderId orderId) = // deconstruct in the param
printfn "The orderId is %i" orderId
//try it
let custId = CustomerId 1 // create a customer id
printOrderId custId // Good! A compiler error now.

This approach is feasible in C# and Java as well, but is rarely used because of the overhead
of creating and managing the special classes for each type. In F# this approach is
lightweight and therefore quite common.
A convenient thing about single case union types is you can pattern match directly against a
value without having to use a full match-with expression.
// deconstruct in the param
let printCustomerId (CustomerId customerIdInt) =
printfn "The CustomerId is %i" customerIdInt
// or deconstruct explicitly through let statement
let printCustomerId2 custId =
let (CustomerId customerIdInt) = custId // deconstruct here
printfn "The CustomerId is %i" customerIdInt
// try it
let custId = CustomerId 1 // create a customer id
printCustomerId custId
printCustomerId2 custId

But a common "gotcha" is that in some cases, the pattern match must have parens around
it, otherwise the compiler will think you are defining a function!
let custId = CustomerId 1
let (CustomerId customerIdInt) = custId // Correct pattern matching
let CustomerId customerIdInt = custId // Wrong! New function?

Similarly, if you ever do need to create an enum-style union type with a single case, you will
have to start the case with a vertical bar in the type definition; otherwise the compiler will
think you are creating an alias.
type TypeAlias = A // type alias!
type SingleCase = | A // single case union type

530

Discriminated Unions

Union equality
Like other core F# types, union types have an automatically defined equality operation: two
unions are equal if they have the same type and the same case and the values for that case
is equal.
type Contact = Email of string | Phone of int
let email1 = Email "bob@example.com"
let email2 = Email "bob@example.com"
let areEqual = (email1=email2)

Union representation
Union types have a nice default string representation, and can be serialized easily. But
unlike tuples, the ToString() representation is unhelpful.
type Contact = Email of string | Phone of int
let email = Email "bob@example.com"
printfn "%A" email // nice
printfn "%O" email // ugly!

531

The Option type

Now let's look at a particular union type, the Option type. It is so common and so useful that
it is actually built into the language.
You have already seen the option type discussed in passing, but let's go back to basics and
understand how it fits into the type system.
A very common situation is when you want to represent missing or invalid values. Using a
diagram, the domain would look like this:

Obviously this calls for some kind of union type!

In F#, it is called the Option type, and is defined as union type with two cases: Some and
None . A similar type is common in functional languages: OCaml and Scala also call it
Option , while Haskell calls it Maybe .

Here is a definition:
type Option<'a> = // use a generic definition
| Some of 'a // valid value
| None // missing

IMPORTANT: if you evaluate this in the interactive window, be sure to reset the session
afterwards, so that the built-in type is restored.

The option type is used in the same way as any union type in construction, by specifying one
of the two cases, the Some case or the None case:

532

The Option type

let validInt = Some 1

let invalidInt = None

and when pattern matching, as with any union type, you must always match all the cases:
match validInt with
| Some x -> printfn "the valid value is %A" x
| None -> printfn "the value is None"

When defining a type that references the Option type, you must specify the generic type to
use. You can do this in an explicit way, with angle brackets, or use the built-in " option "
keyword which comes after the type. The following examples are identical:
type SearchResult1 = Option<string> // Explicit C#-style generics
type SearchResult2 = string option // built-in postfix keyword

Using the Option type

The option type is widely used in the F# libraries for values that might be missing or
otherwise invalid.
For example, the List.tryFind function returns an option, with the None case used
indicate that nothing matches the search predicate.
[1;2;3;4] |> List.tryFind (fun x-> x = 3) // Some 3
[1;2;3;4] |> List.tryFind (fun x-> x = 10) // None

Let's revisit the same example we used for tuples and records, and see how options might
be used instead:

533

The Option type

// the tuple version of TryParse

let tryParseTuple intStr =
try
let i = System.Int32.Parse intStr
(true,i)
with _ -> (false,0) // any exception
// for the record version, create a type to hold the return result
type TryParseResult = {success:bool; value:int}
// the record version of TryParse
let tryParseRecord intStr =
try
let i = System.Int32.Parse intStr
{success=true;value=i}
with _ -> {success=false;value=0}
// the option version of TryParse
let tryParseOption intStr =
try
let i = System.Int32.Parse intStr
Some i
with _ -> None
//test it
tryParseTuple "99"
tryParseRecord "99"
tryParseOption "99"
tryParseTuple "abc"
tryParseRecord "abc"
tryParseOption "abc"

Of these three approaches, the "option" version is generally preferred; no new types need to
be defined and for simple cases, the meaning of None is obvious from the context.
NOTE: The tryParseOption code is just an example. A similar function tryParse is built
into the .NET core libraries and should be used instead.

Option equality
Like other union types, option types have an automatically defined equality operation
let o1 = Some 42
let o2 = Some 42
let areEqual = (o1=o2)

534

The Option type

Option representation
Option types have a nice default string representation, and unlike other union types, the
ToString() representation is also nice.

let o = Some 42
printfn "%A" o // nice
printfn "%O" o // nice

Options are not just for primitive types

The F# option is a true first class type (it's just a normal union type, after all). You can use it
with any type. For example, you can have an option of a complex type like Person, or a tuple
type like int*int , or a function type like int->bool , or even an option of an option type.
type OptionalString = string option
type OptionalPerson = Person option // optional complex type
type OptionalTuple = (int*int) option
type OptionalFunc = (int -> bool) option // optional function
type NestedOptionalString = OptionalString option //nested options!
type StrangeOption = string option option option

How the Option type should not be used

The option type has functions such as IsSome , IsNone and Value , which allow you to
access the "wrapped" value without doing pattern matching. Don't use them! Not only it is
not idiomatic, but it is dangerous and can cause exceptions.
Here is how not to do it:
let x = Some 99
// testing using IsSome
if x.IsSome then printfn "x is %i" x.Value // ugly!!
// no matching at all
printfn "x is %i" x.Value // ugly and dangerous!!

Here is how to do it properly:

535

The Option type

let x = Some 99
match x with
| Some i -> printfn "x is %i" i
| None -> () // what to do here?

The pattern matching approach also forces you to think about and document what happens
in the None case, which you might easily overlook when using IsSome .

The Option module

If you are doing a lot of pattern matching on options, look into the Option module, as it has
some useful helper functions like map , bind , iter and so on.
For example, say that I want to multiply the value of an option by 2 if it is valid. Here's the
pattern matching way:
let x = Some 99
let result = match x with
| Some i -> Some(i * 2)
| None -> None

And here's a more compact version written using Option.map :

let x = Some 99
x |> Option.map (fun v -> v * 2)

Or perhaps I want to multiply the value of an option by 2 if it is valid but return 0 if it is None .
Here's the pattern matching way:
let x = Some 99
let result = match x with
| Some i -> i * 2
| None -> 0

And here's the same thing as a one-liner using Option.fold :

let x = Some 99
x |> Option.fold (fun _ v -> v * 2) 0

In simple cases like the one above, the defaultArg function can be used as well.

536

The Option type

let x = Some 99
defaultArg x 0

Option vs. Null vs. Nullable

The option type often causes confusion to people who are used to dealing with nulls and
nullables in C# and other languages. This section will try to clarify the differences.

Type safety of Option vs. null

In a language like C# or Java, "null" means a reference or pointer to an object that doesn't
exist. The "null" has exactly the same type as the object, so you can't tell from the type
system that you have a null.
For example, in the C# code below we create two string variables, one with a valid string
and one with a null string.
string s1 = "abc";
var len1 = s1.Length;
string s2 = null;
var len2 = s2.Length;

This compiles perfectly, of course. The compiler cannot tell the difference between the two
variables. The null is exactly the same type as the valid string, so all the System.String
methods and properties can be used on it, including the Length property.
Now, we know that this code will fail by just looking at it, but the compiler can't help us.
Instead, as we all know, you have to tediously test for nulls constantly.
Now let's look at the nearest F# equivalent of the C# example above. In F#, to indicate
missing data, you would use an option type and set it to None . (In this artificial example we
have to use an ugly explicitly typed None -- normally this would not be necessary.)
let s1 = "abc"
var len1 = s1.Length
// create a string option with value None
let s2 = Option<string>.None
let len2 = s2.Length

537

The Option type

In the F# version, we get a compile-time error immediately. The None is not a string, it's a
different type altogether, so you can't call Length on it directly. And to be clear, Some
[string] is also not the same type as string , so you can't call Length on it either!

So if Option<string> is not a string, but you want to do something with the string it (might)
contain, you are forced to have to pattern match on it (assuming you don't do bad things as
described earlier).
let s2 = Option<string>.None
//which one is it?
let len2 = match s2 with
| Some s -> s.Length
| None -> 0

You always have to pattern match, because given a value of type Option<string> , you can't
tell whether it is Some or None.
In just the same way Option<int> is not the same type as int , Option<bool> is not the
same type as bool , and so on.
To summarize the critical points:
The type " string option " is not at all the same type as " string ". You cannot cast
from string option to string -- they do not have the same properties. A function that
works with string will not work with string option , and vice versa. So the type
system will prevent any errors.
On the other hand, a "null string" in C# is exactly the same type as "string". You cannot
tell them apart at compile time, only at run time. A "null string" appears to have all the
same properties and functions as a valid string, except that your code will blow up when
you try to use it!

Nulls vs. missing data

A "null" as used in C# is completely different from the concept of "missing" data, which is a
valid part of modeling any system in any language.
In a true functional language there can be a concept of missing data, but there can be no
such thing as "null", because the concepts of "pointers" or "uninitialized variables" do not
exist in the functional way of thinking.
For example, consider a value bound to the result of an expression like this:
let x = "hello world"

538

The Option type

How can that value ever be uninitialized, or become null, or even become any other value at
all?
Unfortunately, additional confusion has been caused because in some cases API designers
have used null to indicate the concept of "missing" data as well! For example, the .NET
library method StreamReader.ReadLine returns null to indicate that there is no more data in a
file.

F# and null
F# is not a pure functional language, and has to interact with the .NET languages that do
have the concept of null. Therefore, F# does include a null keyword in its design, but
makes it hard to use and treats it as an abnormal value.
As a general rule, nulls are never created in "pure" F#, but only by interacting with the .NET
libraries or other external systems.
Here are some examples:
// pure F# type is not allowed to be null (in general)
type Person = {first:string; last:string}
let p : Person = null // error!
// type defined in CLR, so is allowed to be null
let s : string = null // no error!
let line = streamReader.ReadLine() // no error if null

In these cases, it is good practice to immediately check for nulls and convert them into an
option type!
// streamReader example
let line = match streamReader.ReadLine() with
| null -> None
| line -> Some line
// environment example
let GetEnvVar var =
match System.Environment.GetEnvironmentVariable(var) with
| null -> None
| value -> Some value
// try it
GetEnvVar "PATH"
GetEnvVar "TEST"

539

The Option type

And on occasion, you may need to pass a null to an external library. You can do this using
the null keyword as well.

Option vs. Nullable

In addition to null, C# has the concept of a Nullable type, such as Nullable<int> , which
seems similar to the option type. So what's the difference?
The basic idea is the same, but Nullable is much weaker. It only works on value types such
as Int and DateTime , not on reference types such as strings or classes or functions. You
can't nest Nullables, and they don't have much special behavior.
On the other hand, the F# option is a true first class type and can be used consistently
across all types in the same way. (See the examples above in the "Options are not just for
primitive types" section.)

540

Enum types

Enum types
The enum type in F# is the same as the enum type in C#. Its definition is superficially just
like that of a union type, but there are many non-obvious differences to be aware of.

Defining enums
To define an enum you use exactly the same syntax as a union type with empty cases,
except that you must specify a constant value for each case, and the constants must all be
of the same type.
type SizeUnion = Small | Medium | Large // union
type ColorEnum = Red=0 | Yellow=1 | Blue=2 // enum

Strings are not allowed, only ints or compatible types such bytes and chars:
type MyEnum = Yes = "Y" | No ="N" // Error. Strings not allowed.
type MyEnum = Yes = 'Y' | No ='N' // Ok because char was used.

Union types require that their cases start with an uppercase letter. This is not required for
enums.
type SizeUnion = Small | Medium | large // Error - "large" is invalid.
type ColorEnum = Red=0 | Yellow=1 | blue=2 // Ok

Just as with C#, you can use the FlagsAttribute for bit flags:
[<System.FlagsAttribute>]
type PermissionFlags = Read = 1 | Write = 2 | Execute = 4
let permission = PermissionFlags.Read ||| PermissionFlags.Write

Constructing enums
Unlike union types, to construct an enum you must always use a qualified name:

541

Enum types

let red = Red // Error. Enums must be qualified

let red = ColorEnum.Red // Ok
let small = Small // Ok. Unions do not need to be qualified

You can also cast to and from the underlying int type:
let redInt = int ColorEnum.Red
let redAgain:ColorEnum = enum redInt // cast to a specified enum type
let yellowAgain = enum<ColorEnum>(1) // or create directly

You can even create values that are not on the enumerated list at all.
let unknownColor = enum<ColorEnum>(99) // valid

And, unlike unions, you can use the BCL Enum functions to enumerate and parse values,
just as with C#. For example:
let values = System.Enum.GetValues(typeof<ColorEnum>)
let redFromString =
System.Enum.Parse(typeof<ColorEnum>,"Red")
:?> ColorEnum // downcast needed

Matching enums
To match an enum you must again always use a qualified name:
let unqualifiedMatch x =
match x with
| Red -> printfn "red" // warning FS0049
| _ -> printfn "something else"
let qualifiedMatch x =
match x with
| ColorEnum.Red -> printfn "red" //OK. qualified name used.
| _ -> printfn "something else"

Both unions and enums will warn if you have not covered all known cases when pattern
matching:

542

Enum types

let matchUnionIncomplete x =
match x with
| Small -> printfn "small"
| Medium -> printfn "medium"
// Warning: Incomplete pattern matches
let matchEnumIncomplete x =
match x with
| ColorEnum.Red -> printfn "red"
| ColorEnum.Yellow -> printfn "yellow"
// Warning: Incomplete pattern matches

One important difference between unions and enums is that can you make the compiler
happy about exhaustive pattern matching by listing all the union types.
Not so for enums. It is possible to create an enum not on the predeclared list, and try to
match with it, and get a runtime exception, so the compiler will warn you even if you have
explicitly listed all the known enums:
// the compiler is still not happy
let matchEnumIncomplete2 x =
match x with
| ColorEnum.Red -> printfn "red"
| ColorEnum.Yellow -> printfn "yellow"
| ColorEnum.Blue -> printfn "blue"
// the value '3' may indicate a case not covered by the pattern(s).

The only way to fix this is to add a wildcard to the bottom of the cases, to handle enums
outside the predeclared range.
// the compiler is finally happy
let matchEnumComplete x =
match x with
| ColorEnum.Red -> printfn "red"
| ColorEnum.Yellow -> printfn "yellow"
| ColorEnum.Blue -> printfn "blue"
| _ -> printfn "something else"
// test with unknown case
let unknownColor = enum<ColorEnum>(99) // valid
matchEnumComplete unknownColor

Summary

543

Enum types

In general, you should prefer discriminated union types over enums, unless you really need
to have an int value associated with them, or you are writing types that need to be
exposed to other .NET languages.

544

Built-in .NET types

In this post we'll take a quick look at how F# handles the standard types that are built into
.NET.

Literals
F# uses the same syntax for literals that C# does, with a few exceptions.
I'll divide the built-in types into the following groups:
miscellaneous types ( bool , char , etc. )
string types
integer types ( int , uint and byte , etc)
float types ( float , decimal , etc)
pointer types ( IntPtr , etc)
The following tables list the primitive types, with their F# keywords, their suffixes if any, an
example, and the corresponding .NET CLR type.

Miscellaneous types
Object
Keyword

obj

Unit
unit

Bool
bool

Char
(Unicode)

Char
(Ascii)

char

byte

Suffix

Example

let o = obj()

let u = ()

true false

'a'

'a'B

.NET Type

Object

(no equivalent)

Boolean

Char

Byte

Object and unit are not really .NET primitive types, but I have included them for the sake of
completeness.

String types

545

Built-in .NET types

String
(Unicode)
Keyword

Verbatim
string
(Unicode)

Triple quoted string

(Unicode)

String
(Ascii)

string

byte[]

Example

"first\nsecond
line"

@"C:\name"

"""can "contain"" special

chars"""

"aaa"B

.NET
Type

String

Byte[]

Suffix

The usual special characters can be used inside normal strings, such as \n , \t , \\ , etc.
Quotes must be escaped with a backslash: \' and \" .
In verbatim strings, backslashes are ignored (good for Windows filenames and regex
patterns). But quotes need to be doubled.
Triple-quoted strings are new in VS2012. They are useful because special characters do not
need to be escaped at all, and so they can handle embedded quotes nicely (great for XML).

Integer types
8 bit
(Signed)

8 bit
(Unsigned)

16 bit
(Signed)

16 bit
(Unsigned)

32 bit
(Signed)

32 bit
(Unsigned)

Keyword

sbyte

byte

int16

uint16

int

uint32

Suffix

Example

99y

99uy

99s

99us

99u

.NET
Type

SByte

Byte

Int16

UInt16

Int32

UInt32

BigInteger is available in all versions of F#. From .NET 4 it is included as part of the .NET

base library.
Integer types can also be written in hex and octal.
The hex prefix is 0x . So 0xFF is hex for 255.
The octal prefix is 0o . So 0o377 is octal for 255.

Floating point types

546

Built-in .NET types

32 bit
floating point

64 bit (default)
floating point
float, double

High precision
floating point

Keyword

float32, single

decimal

Suffix

Example

123.456f

123.456

123.456m

.NET Type

Single

Double

Decimal

Note that F# natively uses float instead of double , but both can be used.

Pointer types
Pointer/handle
(signed)

Pointer/handle
(unsigned)

Keyword

nativeint

unativeint

Suffix

Example

0xFFFFFFFFn

0xFFFFFFFFun

.NET Type

IntPtr

UIntPtr

Casting between built-in primitive types

Note: this section only covers casting of primitive types. For casting between classes see the
series on object-oriented programming.
There is no direct "cast" syntax in F#, but there are helper functions to cast between types.
These helper functions have the same name as the type (you can see them in the
Microsoft.FSharp.Core namespace).

So for example, in C# you might write:

var x = (int)1.23
var y = (double)1

In F# the equivalent would be:

let x = int 1.23
let y = float 1

In F# there are only casting functions for numeric types. In particular, there is no cast for
bool, and you must use Convert or similar.

547

Built-in .NET types

let x = bool 1 //error

let y = System.Convert.ToBoolean(1) // ok

Boxing and unboxing

Just as in C# and other .NET languages, the primitive int and float types are value objects,
not classes. Although this is normally transparent, there are certain occasions where it can
be an issue.
First, lets look at the transparent case. In the example below, we define a function that takes
a parameter of type Object , and simply returns it. If we pass in an int , it is silently boxed
into an object, as can be seen from the test code, which returns an object not an int .
// create a function with parameter of type Object
let objFunction (o:obj) = o
// test: call with an integer
let result = objFunction 1
// result is
// val result : obj = 1

The fact that result is an object, not an int, can cause type errors if you are not careful.
For example, the result cannot be directly compared with the original value:
let resultIsOne = (result = 1)
// error FS0001: This expression was expected to have type obj
// but here has type int

To work with this situation, and other similar ones, you can convert a primitive type to an
object directly, by using the box keyword:
let o = box 1
// retest the comparison example above, but with boxing
let result = objFunction 1
let resultIsOne = (result = box 1) // OK

To convert an object back to an primitive type, use the unbox keyword, but unlike box , you
must either supply a specific type to unbox to, or be sure that the compiler has enough
information to make an accurate type inference.

548

Built-in .NET types

// box an int
let o = box 1
// type known for target value
let i:int = unbox o // OK
// explicit type given in unbox
let j = unbox<int> o // OK
// type inference, so no type annotation needed
let k = 1 + unbox o // OK

So the comparison example above could also be done with unbox . No explicit type
annotation is needed because it is being compared with an int.
let result = objFunction 1
let resultIsOne = (unbox result = 1) // OK

A common problem occurs if you do not specify enough type information -- you will
encounter the infamous "Value restriction" error, as shown below:
let o = box 1
// no type specified
let i = unbox o // FS0030: Value restriction error

The solution is to reorder the code to help the type inference, or when all else fails, add an
explicit type annotation. See the post on type inference for more tips.

Boxing in combination with type detection

Let's say that you want to have a function that matches based on the type of the parameter,
using the :? operator:
let detectType v =
match v with
| :? int -> printfn "this is an int"
| _ -> printfn "something else"

Unfortunately, this code will fail to compile, with the following error:

549

Built-in .NET types

// error FS0008: This runtime coercion or type test from type 'a to int
// involves an indeterminate type based on information prior to this program point.
// Runtime type tests are not allowed on some types. Further type annotations are need
ed.

550

Units of measure

Units of measure
As we mentioned earlier in the "why use F#?" series, F# has a very cool feature which
allows you to add extra unit-of-measure information to as metadata to numeric types.
The F# compiler will then make sure that only numerics with the same unit-of-measure can
be combined. This can be very useful to stop accidental mismatches and to make your code
safer.

Defining units of measure

A unit of measure definition consists of the attribute [<Measure>] , followed by the type
keyword and then a name. For example:
[<Measure>]
type cm
[<Measure>]
type inch

Often you will see the whole definition written on one line instead:
[<Measure>] type cm
[<Measure>] type inch

Once you have a definition, you can associate a measure type with a numeric type by using
angle brackets with measure name inside:
let x = 1<cm> // int
let y = 1.0<cm> // float
let z = 1.0m<cm> // decimal

You can even combine measures within the angle brackets to create compound measures:

551

Units of measure

[<Measure>] type m
[<Measure>] type sec
[<Measure>] type kg
let distance = 1.0<m>
let time = 2.0<sec>
let speed = 2.0<m/sec>
let acceleration = 2.0<m/sec^2>
let force = 5.0<kg m/sec^2>

Derived units of measure

If you use certain combinations of units a lot, you can define a derived measure and use that
instead.
[<Measure>] type N = kg m/sec^2
let force1 = 5.0<kg m/sec^2>
let force2 = 5.0<N>
force1 = force2 // true

SI units and constants

If you are using the units-of-measure for physics or other scientific applications, you will
definitely want to use the SI units and related constants. You don't need to define all these
yourself! These are predefined for you and available as follows:
In F# 3.0 and higher (which shipped with Visual Studio 2012), these are built into the
core F# libraries in the Microsoft.FSharp.Data.UnitSystems.SI namespace (see the
MSDN page).
In F# 2.0 (which shipped with Visual Studio 2010), you will have to install the F#
powerpack to get them. (The F# powerpack is on Codeplex at
http://fsharppowerpack.codeplex.com).

Type checking and type inference

The units-of-measure are just like proper types; you get static checking and type inference.

552

Units of measure

[<Measure>] type foot

[<Measure>] type inch
let distance = 3.0<foot>
// type inference for result
let distance2 = distance * 2.0
// type inference for input and output
let addThreeFeet ft =
ft + 3.0<foot>

And of course, when using them, the type checking is strict:

addThreeFeet 1.0 //error
addThreeFeet 1.0<inch> //error
addThreeFeet 1.0<foot> //OK

Type annotations
If you want to be explicit in specifying a unit-of-measure type annotation, you can do so in
the usual way. The numeric type must have angle brackets with the unit-of-measure.
let untypedTimesThree (ft:float) =
ft * 3.0
let footTimesThree (ft:float<foot>) =
ft * 3.0

Combining units of measure with multiplication and

division
The compiler understands how units of measure transform when individual values are
multiplied or divided.
For example, in the following, the speed value has been automatically given the measure
<m/sec> .

553

Units of measure

[<Measure>] type m
[<Measure>] type sec
[<Measure>] type kg
let distance = 1.0<m>
let time = 2.0<sec>
let speed = distance/time
let acceleration = speed/time
let mass = 5.0<kg>
let force = mass * speed/time

Look at the types of the acceleration and force values above to see other examples of
how this works.

Dimensionless values
A numeric value without any specific unit of measure is called dimensionless. If you want to
be explicit that a value is dimensionless, you can use the measure called 1 .
// dimensionless
let x = 42
// also dimensionless
let x = 42<1>

Mixing units of measure with dimensionless values

Note that you cannot add a dimensionless value to a value with a unit of measure, but you
can multiply or divide by dimensionless values.
// test addition
3.0<foot> + 2.0<foot> // OK
3.0<foot> + 2.0 // error
// test multiplication
3.0<foot> * 2.0 // OK

But see the section on "generics" below for an alternative approach.

Conversion between units

What if you need to convert between units?

554

Units of measure

It's straightforward. You first need to define a conversion value that uses both units, and then
multiply the source value by the conversion factor.
Here's an example with feet and inches:
[<Measure>] type foot
[<Measure>] type inch
//conversion factor
let inchesPerFoot = 12.0<inch/foot>
// test
let distanceInFeet = 3.0<foot>
let distanceInInches = distanceInFeet * inchesPerFoot

And here's an example with temperature:

[<Measure>] type degC
[<Measure>] type degF
let convertDegCToF c =
c * 1.8<degF/degC> + 32.0<degF>
// test
let f = convertDegCToF 0.0<degC>

The compiler correctly inferred the signature of the conversion function.

val convertDegCToF : float<degC> -> float<degF>

Note that the constant 32.0<degF> was explicitly annotated with the degF so that the result
would be in degF as well. If you leave off this annotation, the result is a plain float, and the
function signature changes to something much stranger! Try it and see:
let badConvertDegCToF c =
c * 1.8<degF/degC> + 32.0

Conversion between dimensionless values and unit-ofmeasure values

To convert from a dimensionless numeric value to a value with a measure type, just multiply
it by one, but with the one annotated with the appropriate unit.

555

Units of measure

[<Measure>] type foot

let ten = 10.0 // normal
//converting from non-measure to measure
let tenFeet = ten * 1.0<foot> // with measure

And to convert the other way, either divide by one, or multiply with the inverse unit.
//converting from measure to non-measure
let tenAgain = tenFeet / 1.0<foot> // without measure
let tenAnotherWay = tenFeet * 1.0<1/foot> // without measure

The above methods are type safe, and will cause errors if you try to convert the wrong type.
If you don't care about type checking, you can do the conversion with the standard casting
functions instead:
let tenFeet = 10.0<foot> // with measure
let tenDimensionless = float tenFeet // without measure

Generic units of measure

Often, we want to write functions that will work with any value, no matter what unit of
measure is associated with it.
For example, here is our old friend square . But when we try to use it with a unit of measure,
we get an error.
let square x = x * x
// test
square 10<foot> // error

What can we do? We don't want to specify a particular unit of measure, but on the other
hand we must specify something, because the simple definition above doesn't work.
The answer is to use generic units of measure, indicated with an underscore where the
measure name normally is.

556

Units of measure

let square (x:int<_>) = x * x

// test
square 10<foot> // OK
square 10<sec> // OK

Now the square function works as desired, and you can see that the function signature has
used the letter 'u to indicate a generic unit of measure. And also note that the compiler
has inferred that the return value is of type "unit squared".
val square : int<'u> -> int<'u ^ 2>

Indeed, you can specify the generic type using letters as well if you like:
// with underscores
let square (x:int<_>) = x * x
// with letters
let square (x:int<'u>) = x * x
// with underscores
let speed (distance:float<_>) (time:float<_>) =
distance / time
// with letters
let speed (distance:float<'u>) (time:float<'v>) =
distance / time

You may need to use letters sometimes to explicitly indicate that the units are the same:
let ratio (distance1:float<'u>) (distance2:float<'u>) =
distance1 / distance2

Using generic measures with lists

You cannot always use a measure directly. For example, you cannot define a list of feet
directly:
//error
[1.0<foot>..10.0<foot>]

Instead, you have to use the "multiply by one" trick mentioned above:

557

Units of measure

//converting using map -- OK

[1.0..10.0] |> List.map (fun i-> i * 1.0<foot>)
//using a generator -- OK
[ for i in [1.0..10.0] -> i * 1.0<foot> ]

Using generic measures for constants

Multiplication by constants is OK (as we saw above), but if you try to do addition, you will get
an error.
let x = 10<foot> + 1 // error

The fix is to add a generic type to the constant, like this:

let x = 10<foot> + 1<_> // ok

A similar situation occurs when passing in constants to a higher order function such as
fold .

let feet = [ for i in [1.0..10.0] -> i * 1.0<foot> ]

// OK
feet |> List.sum
// Error
feet |> List.fold (+) 0.0
// Fixed with generic 0
feet |> List.fold (+) 0.0<_>

Issues with generic measures with functions

There are some cases where type inference fails us. For example, let's try to create a simple
add1 function that uses units.

558

Units of measure

// try to define a generic function

let add1 n = n + 1.0<_>
// warning FS0064: This construct causes code to be less generic than
// indicated by the type annotations. The unit-of-measure variable 'u
// has been constrained to be measure '1'.
// test
add1 10.0<foot>
// error FS0001: This expression was expected to have type float
// but here has type float<foot>

The warning message has the clue. The input parameter n has no measure, so the
measure for 1<_> will always be ignored. The add1 function does not have a unit of
measure so when you try to call it with a value that does have a measure, you get an error.
So maybe the solution is to explicitly annotate the measure type, like this:
// define a function with explicit type annotation
let add1 (n:float<'u>) : float<'u> = n + 1.0<_>

But no, you get the same warning FS0064 again.

Maybe we can replace the underscore with something more explicit such as 1.0<'u> ?
let add1 (n:float<'u>) : float<'u> = n + 1.0<'u>
// error FS0634: Non-zero constants cannot have generic units.

But this time we get a compiler error!

The answer is to use one of the helpful utility functions in the LanguagePrimitives module:
FloatWithMeasure , Int32WithMeasure , etc.

// define the function

let add1 n =
n + (LanguagePrimitives.FloatWithMeasure 1.0)
// test
add1 10.0<foot> // Yes!

And for generic ints, you can use the same approach:

559

Units of measure

open LanguagePrimitives
let add2Int n =
n + (Int32WithMeasure 2)
add2Int 10<foot> // OK

Using generic measures with type definitions

That takes care of functions. What about when we need to use a unit of measure in a type
definition?
Say we want to define a generic coordinate record that works with an unit of measure. Let's
start with a naive approach:
type Coord =
{ X: float<'u>; Y: float<'u>; }
// error FS0039: The type parameter 'u' is not defined

That didn't work, so what about adding the measure as a type parameter:
type Coord<'u> =
{ X: float<'u>; Y: float<'u>; }
// error FS0702: Expected unit-of-measure parameter, not type parameter.
// Explicit unit-of-measure parameters must be marked with the [<Measure>] attribute.

That didn't work either, but the error message tells us what to do. Here is the final, correct
version, using the Measure attribute:
type Coord<[<Measure>] 'u> =
{ X: float<'u>; Y: float<'u>; }
// Test
let coord = {X=10.0<foot>; Y=2.0<foot>}

In some cases, you might need to define more than one measure. In the following example,
the currency exchange rate is defined as the ratio of two currencies, and so needs two
generic measures to be defined.

560

Units of measure

type CurrencyRate<[<Measure>]'u, [<Measure>]'v> =

{ Rate: float<'u/'v>; Date: System.DateTime}
// test
[<Measure>] type EUR
[<Measure>] type USD
[<Measure>] type GBP
let mar1 = System.DateTime(2012,3,1)
let eurToUsdOnMar1 = {Rate= 1.2<USD/EUR>; Date=mar1 }
let eurToGbpOnMar1 = {Rate= 0.8<GBP/EUR>; Date=mar1 }
let tenEur = 10.0<EUR>
let tenEurInUsd = eurToUsdOnMar1.Rate * tenEur

And of course, you can mix regular generic types with unit of measure types.
For example, a product price might consist of a generic product type, plus a price with a
currency:
type ProductPrice<'product, [<Measure>] 'currency> =
{ Product: 'product; Price: float<'currency>; }

Units of measure at runtime

An issue that you may run into is that units of measure are not part of the .NET type system.
F# does stores extra metadata about them in the assembly, but this metadata is only
understood by F#.
This means that there is no (easy) way at runtime to determine what unit of measure a value
has, nor any way to dynamically assign a unit of measure at runtime.
It also means that there is no way to expose units of measure as part of a public API to
another .NET language (except other F# assemblies).

561

Understanding type inference

Before we finish with types, let's revisit type inference: the magic that allows the F# compiler
to deduce what types are used and where. We have seen this happen through all the
examples so far, but how does it work and what can you do if it goes wrong?

How does type inference work?

It does seem to be magic, but the rules are mostly straightforward. The fundamental logic is
based on an algorithm often called "Hindley-Milner" or "HM" (more accurately it should be
called "Damas-Milner's Algorithm W"). If you want to know the details, go ahead and Google
it.
I do recommend that you take some time to understand this algorithm so that you can "think
like the compiler" and troubleshoot effectively when you need to.
Here are some of the rules for determine the types of simple and function values:
Look at the literals
Look at the functions and other values something interacts with
Look at any explicit type constraints
If there are no constraints anywhere, automatically generalize to generic types
Let's look at each of these in turn.

Look at the literals

The literals give the compiler a clue to the context. As we have seen, the type checking is
very strict; ints and floats are not automatically cast to the other. The benefit of this is that
the compiler can deduce types by looking at the literals. If the literal is an int and you are
adding "x" to it, then "x" must be an int as well. But if the literal is a float and you are
adding "x" to it, then "x" must be a float as well.
Here are some examples. Run them and see their signatures in the interactive window:
let inferInt x = x + 1
let inferFloat x = x + 1.0
let inferDecimal x = x + 1m // m suffix means decimal
let inferSByte x = x + 1y // y suffix means signed byte
let inferChar x = x + 'a' // a char
let inferString x = x + "my string"

562

Understanding type inference

Look at the functions and other values it interacts with

If there are no literals anywhere, the compiler tries to work out the types by analyzing the
functions and other values that they interact with. In the cases below, the " indirect "
function calls a function that we do know the types for, which gives us the information to
deduce the types for the " indirect " function itself.
let inferInt x = x + 1
let inferIndirectInt x = inferInt x //deduce that x is an int
let inferFloat x = x + 1.0
let inferIndirectFloat x = inferFloat x //deduce that x is a float

And of course assignment counts as an interaction too. If x is a certain type, and y is bound
(assigned) to x, then y must be the same type as x.
let x = 1
let y = x //deduce that y is also an int

Other interactions might be control structures, or external libraries

// if..else implies a bool
let inferBool x = if x then false else true
// for..do implies a sequence
let inferStringList x = for y in x do printfn "%s" y
// :: implies a list
let inferIntList x = 99::x
// .NET library method is strongly typed
let inferStringAndBool x = System.String.IsNullOrEmpty(x)

Look at any explicit type constraints or annotations

If there are any explicit type constraints or annotations specified, then the compiler will use
them. In the case below, we are explicitly telling the compiler that " inferInt2 " takes an
int parameter. It can then deduce that the return value for " inferInt2 " is also an int ,

which in turn implies that " inferIndirectInt2 " is of type int->int.

let inferInt2 (x:int) = x
let inferIndirectInt2 x = inferInt2 x
let inferFloat2 (x:float) = x
let inferIndirectFloat2 x = inferFloat2 x

Note that the formatting codes in printf statements count as explicit type constraints too!
563

Understanding type inference

let inferIntPrint x = printf "x is %i" x

let inferFloatPrint x = printf "x is %f" x
let inferGenericPrint x = printf "x is %A" x

Automatic generalization
If after all this, there are no constraints found, the compiler just makes the types generic.
let inferGeneric x = x
let inferIndirectGeneric x = inferGeneric x
let inferIndirectGenericAgain x = (inferIndirectGeneric x).ToString()

It works in all directions!

The type inference works top-down, bottom-up, front-to-back, back-to-front, middle-out,
anywhere there is type information, it will be used.
Consider the following example. The inner function has a literal, so we know that it returns
an int . And the outer function has been explicitly told that it returns a string . But what is
the type of the passed in " action " function in the middle?
let outerFn action : string =
let innerFn x = x + 1 // define a sub fn that returns an int
action (innerFn 2) // result of applying action to innerFn

The type inference would work something like this:

1 is an int

Therefore x+1 must be an int , therefore x must be an int

Therefore innerFn must be int->int
Next, (innerFn 2) returns an int , therefore " action " takes an int as input.
The output of action is the return value for outerFn , and therefore the output type of
action is the same as the output type of outerFn .

The output type of outerFn has been explicitly constrained to string , therefore the
output type of action is also string .
Putting this together, we now know that the action function has signature int->string
And finally, therefore, the compiler deduces the type of outerFn as:
val outerFn: (int -> string) -> string

Elementary, my dear Watson!

564

Understanding type inference

The compiler can do deductions worthy of Sherlock Holmes. Here's a tricky example that will
test how well you have understood everything so far.
Let's say we have a doItTwice function that takes any input function (call it " f ") and
generates a new function that simply does the original function twice in a row. Here's the
code for it:
let doItTwice f = (f >> f)

As you can see, it composes f with itself. So in other words, it means: "do f", then "do f" on
the result of that.
Now, what could the compiler possibly deduce about the signature of doItTwice ?
Well, let's look at the signature of " f " first. The output of the first call to " f " is also the
input to the second call to " f ". So therefore the output and input of " f " must be the same
type. So the signature of f must be 'a -> 'a . The type is generic (written as 'a) because
we have no other information about it.
So going back to doItTwice itself, we now know it takes a function parameter of 'a -> 'a .
But what does it return? Well, here's how we deduce it, step by step:
First, note that doItTwice generates a function, so must return a function type.
The input to the generated function is the same type as the input to first call to " f "
The output of the generated function is the same type as the output of the second call to
" f "
So the generated function must also have type 'a -> 'a
Putting it all together, doItTwice has a domain of 'a -> 'a and a range of 'a -> 'a ,
so therefore its signature must be ('a -> 'a) -> ('a -> 'a) .
Is your head spinning yet? You might want to read it again until it sinks in.
Quite a sophisticated deduction for one line of code. Luckily the compiler does all this for us.
But you will need to understand this kind of thing if you have problems and you have to
determine what the compiler is doing.
Let's test it! It's actually much simpler to understand in practice than it is in theory.

565

Understanding type inference

let doItTwice f = (f >> f)

let add3 x = x + 3
let add6 = doItTwice add3
// test
add6 5 // result = 11
let square x = x * x
let fourthPower = doItTwice square
// test
fourthPower 3 // result = 81
let chittyBang x = "Chitty " + x + " Bang"
let chittyChittyBangBang = doItTwice chittyBang
// test
chittyChittyBangBang "&" // result = "Chitty Chitty & Bang Bang"

Hopefully, that makes more sense now.

Things that can go wrong with type inference

The type inference isn't perfect, alas. Sometimes the compiler just doesn't have a clue what
to do. Again, understanding what is happening will really help you stay calm instead of
wanting to kill the compiler. Here are some of the main reasons for type errors:
Declarations out of order
Not enough information
Overloaded methods
Quirks of generic numeric functions

Declarations out of order

A basic rule is that you must declare functions before they are used.
This code fails:
let square2 x = square x // fails: square not defined
let square x = x * x

But this is ok:

let square x = x * x
let square2 x = square x // square already defined earlier

566

Understanding type inference

And unlike C#, in F# the order of file compilation is important, so do make sure the files are
being compiled in the right order. (In Visual Studio, you can change the order from the
context menu).

Recursive or simultaneous declarations

A variant of the "out of order" problem occurs with recursive functions or definitions that have
to refer to each other. No amount of reordering will help in this case -- we need to use
additional keywords to help the compiler.
When a function is being compiled, the function identifier is not available to the body. So if
you define a simple recursive function, you will get a compiler error. The fix is to add the
"rec" keyword as part of the function definition. For example:
// the compiler does not know what "fib" means
let fib n =
if n <= 2 then 1
else fib (n - 1) + fib (n - 2)
// error FS0039: The value or constructor 'fib' is not defined

Here's the fixed version with "rec fib" added to indicate it is recursive:
let rec fib n = // LET REC rather than LET
if n <= 2 then 1
else fib (n - 1) + fib (n - 2)

A similar " let rec ? and " syntax is used for two functions that refer to each other. Here is a
very contrived example that fails if you do not have the " rec " keyword.
let rec showPositiveNumber x = // LET REC rather than LET
match x with
| x when x >= 0 -> printfn "%i is positive" x
| _ -> showNegativeNumber x
and showNegativeNumber x = // AND rather than LET
match x with
| x when x < 0 -> printfn "%i is negative" x
| _ -> showPositiveNumber x

The " and " keyword can also be used to declare simultaneous types in a similar way.

567

Understanding type inference

type A = None | AUsesB of B

// error FS0039: The type 'B' is not defined
type B = None | BUsesA of A

Fixed version:
type A = None | AUsesB of B
and B = None | BUsesA of A // use AND instead of TYPE

Not enough information

Sometimes, the compiler just doesn't have enough information to determine a type. In the
following example, the compiler doesn't know what type the Length method is supposed to
work on. But it can't make it generic either, so it complains.
let stringLength s = s.Length
// error FS0072: Lookup on object of indeterminate type
// based on information prior to this program point.
// A type annotation may be needed ...

These kinds of error can be fixed with explicit annotations.

let stringLength (s:string) = s.Length

Occasionally there does appear to be enough information, but still the compiler doesn't
seem to recognize it. For example, it's obvious to a human that the List.map function
(below) is being applied to a list of strings, so why does x.Length cause an error?
List.map (fun x -> x.Length) ["hello"; "world"] //not ok

The reason is that the F# compiler is currently a one-pass compiler, and so information later
in the program is ignored if it hasn't been parsed yet. (The F# team have said that it is
possible to make the compiler more sophisticated, but it would work less well with
Intellisense and might produce more unfriendly and obscure error messages. So for now, we
will have to live with this limitation.)
So in cases like this, you can always explicitly annotate:
List.map (fun (x:string) -> x.Length) ["hello"; "world"] // ok

568

Understanding type inference

Functional programmers strive to avoid explicit type annotations, so this makes them much
happier!
This technique can be used more generally in other areas as well; a rule of thumb is to try to
put the things that have "known types" earlier than things that have "unknown types".

Overloaded methods
When calling an external class or method in .NET, you will often get errors due to
overloading.
In many cases, such as the concat example below, you will have to explicitly annotate the
parameters of the external function so that the compiler knows which overloaded method to
call.
let concat x = System.String.Concat(x) //fails
let concat (x:string) = System.String.Concat(x) //works
let concat x = System.String.Concat(x:string) //works

Sometimes the overloaded methods have different argument names, in which case you can
also give the compiler a clue by naming the arguments. Here is an example for the
StreamReader constructor.

let makeStreamReader x = new System.IO.StreamReader(x) //fails

let makeStreamReader x = new System.IO.StreamReader(path=x) //works

Quirks of generic numeric functions

Numeric functions can be somewhat confusing. There often appear generic, but once they
are bound to a particular numeric type, they are fixed, and using them with a different
numeric type will cause an error. The following example demonstrates this:

569

Understanding type inference

let myNumericFn x = x * x
myNumericFn 10
myNumericFn 10.0 //fails
// error FS0001: This expression was expected to have
// type int but has type float
let myNumericFn2 x = x * x
myNumericFn2 10.0
myNumericFn2 10 //fails
// error FS0001: This expression was expected to have
// type float but has type int

There is a way round this for numeric types using the "inline" keyword and "static type
parameters". I won't discuss these concepts here, but you can look them up in the F#
reference at MSDN.

"Not enough information" troubleshooting

summary
So to summarize, the things that you can do if the compiler is complaining about missing
types, or not enough information, are:
Define things before they are used (this includes making sure the files are compiled in
the right order)
Put the things that have "known types" earlier than things that have "unknown types". In
particular, you might be able reorder pipes and similar chained functions so that the
typed objects come first.
Annotate as needed. One common trick is to add annotations until everything works,
and then take them away one by one until you have the minimum needed. Do try to
avoid annotating if possible. Not only is it not aesthetically pleasing, but it makes the
code more brittle. It is a lot easier to change types if there are no explicit dependencies
on them.

Debugging type inference issues

Once you have ordered and annotated everything, you will probably still get type errors, or
find that functions are less generic than expected. With what you have learned so far, you
should have the tools to determine why this happened (although it can still be painful).
For example:

570

Understanding type inference

let myBottomLevelFn x = x
let myMidLevelFn x =
let y = myBottomLevelFn x
// some stuff
let z= y
// some stuff
printf "%s" z // this will kill your generic types!
// some more stuff
x
let myTopLevelFn x =
// some stuff
myMidLevelFn x
// some more stuff
x

In this example, we have a chain of functions. The bottom level function is definitely generic,
but what about the top level one? Well often, we might expect it be generic but instead it is
not. In this case we have:
val myTopLevelFn : string -> string

What went wrong? The answer is in the midlevel function. The %s on z forced it be a string,
which forced y and then x to be strings too.
Now this is a pretty obvious example, but with thousands of lines of code, a single line might
be buried away that causes an issue. One thing that can help is to look at all the signatures;
in this case the signatures are:
val myBottomLevelFn : 'a -> 'a // generic as expected
val myMidLevelFn : string -> string // here's the clue! Should be generic
val myTopLevelFn : string -> string

When you find a signature that is unexpected you know that it is the guilty party. You can
then drill down into it and repeat the process until you find the problem.

571

Choosing between collection functions

There's more to learning a new language than the language itself. In order to be productive,
you need to memorize a big chunk of the standard library and be aware of most of the rest of
it. For example, if you know C#, you can pick up Java-the-language quite quickly, but you
won't really get up to speed until you are comfortable with the Java Class Library as well.
Similarly, you can't really be effective in F# until you have some familiarity with all the F#
functions that work with collections.
In C# there are only a few LINQ methods you need to know1 ( Select , Where , and so on).
But in F#, there are currently almost 100 functions in the List module (and similar counts in
the Seq and Array modules). That's a lot!
1 Yes, there are more, but you can get by with just a few. In F# it's more important to know
them all.
If you are coming to F# from C#, then, the large number of list functions can be
overwhelming.
So I have written this post to help guide you to the one you want. And for fun, I've done it in
a "Choose Your Own Adventure" style!

572

Choosing between collection functions

What collection do I want?

First, a table with information about the different kinds of standard collections. There are five
"native" F# ones: list , seq , array , map and set , and ResizeArray and IDictionary
are also often used.

573

Choosing between collection functions

Immutable?

list

Yes

Notes
Pros:
Pattern matching available.
Complex iteration available via recursion.
Forward iteration is fast. Prepending is fast.
Cons:
Indexed access and other access styles are slow.
Alias for IEnumerable .

seq

Yes

Pros:
Lazy evaluation
Memory efficient (only one element at a time
loaded)
Can represent an infinite sequence.
Interop with .NET libraries that use IEnumerable.
Cons:
No pattern matching.
Forward only iteration.
Indexed access and other access styles are slow.
Same as BCL Array .

array

Pros:
Fast random access
Memory efficient and cache locality, especially
with structs.
Interop with .NET libraries that use Array.
Support for 2D, 3D and 4D arrays
Cons:
Limited pattern matching.
Not persistent.

map

Yes

Immutable dictionary. Requires keys to implement

IComparable .

set

Yes

Immutable set. Requires elements to implement

IComparable .

ResizeArray

Alias for BCL List . Pros and cons similar to array,

but resizable.

IDictionary

Yes

For an alternate dictionary that does not requires

elements to implement IComparable , you can use the
BCL IDictionary. The constructor is dict in F#.
Note that mutation methods such as Add are present,
but will cause a runtime error if called.

574

Choosing between collection functions

These are the main collection types that you will encounter in F#, and will be good enough
for all common cases.
If you need other kinds of collections though, there are lots of choices:
You can use the collection classes in .NET, either the traditional, mutable ones or the
newer ones such as those in the System.Collections.Immutable namespace.
Alternatively, you can use one of the F# collection libraries:
FSharpx.Collections, part of the FSharpx series of projects.
ExtCore. Some of these are drop-in (almost) replacements for the Map and Set
types in FSharp.Core which provide improved performance in specific scenarios
(e.g., HashMap). Others provide unique functionality to help tackle specific coding
tasks (e.g., LazyList and LruCache).
Funq: high performance, immutable data structures for .NET.
Persistent: some efficient persistent (immutable) data structures.

About the documentation

All functions are available for list , seq and array in F# v4 unless noted. The Map and
Set modules have some of them as well, but I won't be discussing map and set here.

For the function signatures I will use list as the standard collection type. The signatures
for the seq and array versions will be similar.
Many of these functions are not yet documented on MSDN so I'm going to link directly to the
source code on GitHub, which has the up-to-date comments. Click on the function name for
the link.

Note on availability
The availability of these functions may depend on which version of F# you use.
In F# version 3 (Visual Studio 2013), there was some degree of inconsistency between
Lists, Arrays and Sequences.
In F# version 4 (Visual Studio 2015), this inconsistency has been eliminated, and almost
all functions are available for all three collection types.
If you want to know what changed between F# v3 and F# v4, please see this chart (from
here). The chart shows the new APIs in F# v4 (green), previously-existing APIs (blue), and
intentional remaining gaps (white).

575

Choosing between collection functions

Some of the functions documented below are not in this chart -- these are newer still! If you
are using an older version of F#, you can simply reimplement them yourself using the code
on GitHub.
With that disclaimer out of the way, you can start your adventure!

Table of contents
1. What kind of collection do you have?
2. Creating a new collection
3. Creating a new empty or one-element collection
4. Creating a new collection of known size
5. Creating a new collection of known size with each element having the same value
6. Creating a new collection of known size with each element having a different value
7. Creating a new infinite collection
8. Creating a new collection of indefinite size
9. Working with one list
10. Getting an element at a known position
11. Getting an element by searching
12. Getting a subset of elements from a collection
13. Partitioning, chunking and grouping
14. Aggregating or summarizing a collection
15. Changing the order of the elements
16. Testing the elements of a collection
17. Transforming each element to something different
18. Iterating over each element
19. Threading state through an iteration
20. Working with the index of each element
21. Transforming the whole collection to a different collection type
22. Changing the behavior of the collection as a whole
23. Working with two collections
24. Working with three collections
25. Working with more than three collections
26. Combining and uncombining collections
27. Other array-only functions
28. Using sequences with disposables

576

Choosing between collection functions

1. What kind of collection do you have?

What kind of collection do you have?
If you don't have a collection, and want to create one, go to section 2.
If you already have a collection that you want to work with, go to section 9.
If you have two collections that you want to work with, go to section 23.
If you have three collections that you want to work with, go to section 24.
If you have more than three collections that you want to work with, go to section 25.
If you want to combine or uncombine collections, go to section 26.

2. Creating a new collection

So you want to create a new collection. How do you want to create it?
If the new collection will be empty or will have one element, go to section 3.
If the new collection is a known size, go to section 4.
If the new collection is potentially infinite, go to section 7.
If you don't know how big the collection will be, go to section 8.

3. Creating a new empty or one-element

collection
If you want to create a new empty or one-element collection, use these functions:
empty : 'T list . Returns an empty list of the given type.
singleton : value:'T -> 'T list . Returns a list that contains one item only.

If you know the size of the collection in advance, it is generally more efficient to use a
different function. See section 4 below.

Usage examples
let list0 = List.empty
// list0 = []
let list1 = List.singleton "hello"
// list1 = ["hello"]

577

Choosing between collection functions

4. Creating a new collection of known size

If all elements of the collection will have the same value, go to section 5.
If elements of the collection could be different, go to section 6.

5. Creating a new collection of known size with

each element having the same value
If you want to create a new collection of known size with each element having the same
value, you want to use replicate :
replicate : count:int -> initial:'T -> 'T list . Creates a collection by replicating the

given initial value.

(Array only) create : count:int -> value:'T -> 'T[] . Creates an array whose elements
are all initially the supplied value.
(Array only) zeroCreate : count:int -> 'T[] . Creates an array where the entries are
initially the default value.
Array.create is basically the same as replicate (although with a subtly different

implementation!) but replicate was only implemented for Array in F# v4.

Usage examples
let repl = List.replicate 3 "hello"
// val repl : string list = ["hello"; "hello"; "hello"]
let arrCreate = Array.create 3 "hello"
// val arrCreate : string [] = [|"hello"; "hello"; "hello"|]
let intArr0 : int[] = Array.zeroCreate 3
// val intArr0 : int [] = [|0; 0; 0|]
let stringArr0 : string[] = Array.zeroCreate 3
// val stringArr0 : string [] = [|null; null; null|]

Note that for zeroCreate , the target type must be known to the compiler.

578

Choosing between collection functions

6. Creating a new collection of known size with

each element having a different value
If you want to create a new collection of known size with each element having a potentially
different value, you can choose one of three ways:
init : length:int -> initializer:(int -> 'T) -> 'T list . Creates a collection by

calling the given generator on each index.

For lists and arrays, you can also use the literal syntax such as [1; 2; 3] (lists) and
[|1; 2; 3|] (arrays).

For lists and arrays and seqs, you can use the comprehension syntax for .. in .. do
.. yield .

Usage examples
// using list initializer
let listInit1 = List.init 5 (fun i-> i*i)
// val listInit1 : int list = [0; 1; 4; 9; 16]
// using list comprehension
let listInit2 = [for i in [1..5] do yield i*i]
// val listInit2 : int list = [1; 4; 9; 16; 25]
// literal
let listInit3 = [1; 4; 9; 16; 25]
// val listInit3 : int list = [1; 4; 9; 16; 25]
let arrayInit3 = [|1; 4; 9; 16; 25|]
// val arrayInit3 : int [] = [|1; 4; 9; 16; 25|]

Literal syntax allows for an increment as well:

// literal with +2 increment
let listOdd= [1..2..10]
// val listOdd : int list = [1; 3; 5; 7; 9]

The comprehension syntax is even more flexible because you can yield more than once:

579

Choosing between collection functions

// using list comprehension

let listFunny = [
for i in [2..3] do
yield i
yield i*i
yield i*i*i
]
// val listFunny : int list = [2; 4; 8; 3; 9; 27]

and it can also be used as a quick and dirty inline filter:

let primesUpTo n =
let rec sieve l =
match l with
| [] -> []
| p::xs ->
p :: sieve [for x in xs do if (x % p) > 0 then yield x]
[2..n] |> sieve
primesUpTo 20
// [2; 3; 5; 7; 11; 13; 17; 19]

Two other tricks:

You can use yield! to return a list rather than a single value
You can also use recursion
Here is an example of both tricks being used to count up to 10 by twos:
let rec listCounter n = [
if n <= 10 then
yield n
yield! listCounter (n+2)
]
listCounter 3
// val it : int list = [3; 5; 7; 9]
listCounter 4
// val it : int list = [4; 6; 8; 10]

7. Creating a new infinite collection

If you want an infinite list, you have to use a seq rather than a list or array.
initInfinite : initializer:(int -> 'T) -> seq<'T> . Generates a new sequence which,

580

Choosing between collection functions

when iterated, will return successive elements by calling the given function.
You can also use a seq comprehension with a recursive loop to generate an infinite
sequence.

Usage examples
// generator version
let seqOfSquares = Seq.initInfinite (fun i -> i*i)
let firstTenSquares = seqOfSquares |> Seq.take 10
firstTenSquares |> List.ofSeq // [0; 1; 4; 9; 16; 25; 36; 49; 64; 81]
// recursive version
let seqOfSquares_v2 =
let rec loop n = seq {
yield n * n
yield! loop (n+1)
}
loop 1
let firstTenSquares_v2 = seqOfSquares_v2 |> Seq.take 10

8. Creating a new collection of indefinite size

Sometimes you don't know how big the collection will be in advance. In this case you need a
function that will keep adding elements until it gets a signal to stop. unfold is your friend
here, and the "signal to stop" is whether you return a None (stop) or a Some (keep going).
unfold : generator:('State -> ('T * 'State) option) -> state:'State -> 'T list .

Returns a collection that contains the elements generated by the given computation.

Usage examples
This example reads from the console in a loop until an empty line is entered:

581

Choosing between collection functions

let getInputFromConsole lineNo =

let text = System.Console.ReadLine()
if System.String.IsNullOrEmpty(text) then
None
else
// return value and new threaded state
// "text" will be in the generated sequence
Some (text,lineNo+1)
let listUnfold = List.unfold getInputFromConsole 1

unfold requires that a state be threaded through the generator. You can ignore it (as in the
ReadLine example above), or you can use it to keep track of what you have done so far. For

example, you can create a Fibonacci series generator using unfold :

let fibonacciUnfolder max (f1,f2) =
if f1 > max then
None
else
// return value and new threaded state
let fNext = f1 + f2
let newState = (f2,fNext)
// f1 will be in the generated sequence
Some (f1,newState)
let fibonacci max = List.unfold (fibonacciUnfolder max) (1,1)
fibonacci 100
// int list = [1; 1; 2; 3; 5; 8; 13; 21; 34; 55; 89]

9. Working with one list

If you are working with one list and...
If you want to get an element at a known position, go to section 10
If you want to get one element by searching, go to section 11
If you want to get a subset of the collection, go to section 12
If you want to partition, chunk, or group a collection into smaller collections, go to
section 13
If you want to aggregate or summarize the collection into a single value, go to section
14
If you want to change the order of the elements, go to section 15
If you want to test the elements in the collection, go to section 16
If you want to transform each element to something different, go to section 17

582

Choosing between collection functions

If you want to iterate over each element, go to section 18

If you want to thread state through an iteration, go to section 19
If you need to know the index of each element while you are iterating or mapping, go to
section 20
If you want to transform the whole collection to a different collection type, go to section
21
If you want to change the behaviour of the collection as a whole, go to section 22
If you want to mutate the collection in place, go to section 27
If you want to use a lazy collection with an IDisposable, go to section 28

10. Getting an element at a known position

The following functions get a element in the collection by position:
head : list:'T list -> 'T . Returns the first element of the collection.
last : list:'T list -> 'T . Returns the last element of the collection.
item : index:int -> list:'T list -> 'T . Indexes into the collection. The first element

has index 0.
NOTE: Avoid using nth and item for lists and sequences. They are not designed for
random access, and so they will be slow in general.
nth : list:'T list -> index:int -> 'T . The older version of item . NOTE: Deprecated

in v4 -- use item instead.

(Array only) get : array:'T[] -> index:int -> 'T . Yet another version of item .
exactlyOne : list:'T list -> 'T . Returns the only element of the collection.

But what if the collection is empty? Then head and last will fail with an exception
(ArgumentException).
And if the index is not found in the collection? Then another exception again
(ArgumentException for lists, IndexOutOfRangeException for arrays).
I would therefore recommend that you avoid these functions in general and use the tryXXX
equivalents below:
tryHead : list:'T list -> 'T option . Returns the first element of the collection, or

None if the collection is empty.

tryLast : list:'T list -> 'T option . Returns the last element of the collection, or

None if the collection is empty.

tryItem : index:int -> list:'T list -> 'T option . Indexes into the collection, or None

if the index is not valid.

583

Choosing between collection functions

Usage examples
let head = [1;2;3] |> List.head
// val head : int = 1
let badHead : int = [] |> List.head
// System.ArgumentException: The input list was empty.
let goodHeadOpt =
[1;2;3] |> List.tryHead
// val goodHeadOpt : int option = Some 1
let badHeadOpt : int option =
[] |> List.tryHead
// val badHeadOpt : int option = None
let goodItemOpt =
[1;2;3] |> List.tryItem 2
// val goodItemOpt : int option = Some 3
let badItemOpt =
[1;2;3] |> List.tryItem 99
// val badItemOpt : int option = None

As noted, the item function should be avoided for lists. For example, if you want to process
each item in a list, and you come from an imperative background, you might write a loop with
something like this:
// Don't do this!
let helloBad =
let list = ["a";"b";"c"]
let listSize = List.length list
[ for i in [0..listSize-1] do
let element = list |> List.item i
yield "hello " + element
]
// val helloBad : string list = ["hello a"; "hello b"; "hello c"]

Don't do that! Use something like map instead. It's both more concise and more efficient:
let helloGood =
let list = ["a";"b";"c"]
list |> List.map (fun element -> "hello " + element)
// val helloGood : string list = ["hello a"; "hello b"; "hello c"]

584

Choosing between collection functions

11. Getting an element by searching

You can search for an element or its index using find and findIndex :
find : predicate:('T -> bool) -> list:'T list -> 'T . Returns the first element for

which the given function returns true.

findIndex : predicate:('T -> bool) -> list:'T list -> int . Returns the index of the

first element for which the given function returns true.

And you can also search backwards:
findBack : predicate:('T -> bool) -> list:'T list -> 'T . Returns the last element for

which the given function returns true.

findIndexBack : predicate:('T -> bool) -> list:'T list -> int . Returns the index of

the last element for which the given function returns true.
But what if the item cannot be found? Then these will fail with an exception
( KeyNotFoundException ).
I would therefore recommend that, as with head and item , you avoid these functions in
general and use the tryXXX equivalents below:
tryFind : predicate:('T -> bool) -> list:'T list -> 'T option . Returns the first

element for which the given function returns true, or None if no such element exists.
tryFindBack : predicate:('T -> bool) -> list:'T list -> 'T option . Returns the last

element for which the given function returns true, or None if no such element exists.
tryFindIndex : predicate:('T -> bool) -> list:'T list -> int option . Returns the

index of the first element for which the given function returns true, or None if no such
element exists.
tryFindIndexBack : predicate:('T -> bool) -> list:'T list -> int option . Returns the

index of the last element for which the given function returns true, or None if no such
element exists.
If you are doing a map before a find you can often combine the two steps into a single
one using pick (or better, tryPick ). See below for a usage example.
pick : chooser:('T -> 'U option) -> list:'T list -> 'U . Applies the given function to

successive elements, returning the first result where the chooser function returns Some.
tryPick : chooser:('T -> 'U option) -> list:'T list -> 'U option . Applies the given

function to successive elements, returning the first result where the chooser function
returns Some, or None if no such element exists.

Usage examples

585

Choosing between collection functions

let listOfTuples = [ (1,"a"); (2,"b"); (3,"b"); (4,"a"); ]

listOfTuples |> List.find ( fun (x,y) -> y = "b")
// (2, "b")
listOfTuples |> List.findBack ( fun (x,y) -> y = "b")
// (3, "b")
listOfTuples |> List.findIndex ( fun (x,y) -> y = "b")
// 1
listOfTuples |> List.findIndexBack ( fun (x,y) -> y = "b")
// 2
listOfTuples |> List.find ( fun (x,y) -> y = "c")
// KeyNotFoundException

With pick , rather than returning a bool, you return an option:

listOfTuples |> List.pick ( fun (x,y) -> if y = "b" then Some (x,y) else None)
// (2, "b")

Pick vs. Find

That 'pick' function might seem unnecessary, but it is useful when dealing with functions that
return options.
For example, say that there is a function tryInt that parses a string and returns Some int
if the string is a valid int, otherwise None .
// string -> int option
let tryInt str =
match System.Int32.TryParse(str) with
| true, i -> Some i
| false, _ -> None

And now say that we want to find the first valid int in a list. The crude way would be:
map the list using tryInt
find the first one that is a Some using find
get the value from inside the option using Option.get
The code might look something like this:

586

Choosing between collection functions

let firstValidNumber =
["a";"2";"three"]
// map the input
|> List.map tryInt
// find the first Some
|> List.find (fun opt -> opt.IsSome)
// get the data from the option
|> Option.get
// val firstValidNumber : int = 2

But pick will do all these steps at once! So the code becomes much simpler:
let firstValidNumber =
["a";"2";"three"]
|> List.pick tryInt

If you want to return many elements in the same way as pick , consider using choose (see
section 12).

12. Getting a subset of elements from a

collection
The previous section was about getting one element. How can you get more than one
element? Well you're in luck! There's lots of functions to choose from.
To extract elements from the front, use one of these:
take: count:int -> list:'T list -> 'T list . Returns the first N elements of the

collection.
takeWhile: predicate:('T -> bool) -> list:'T list -> 'T list . Returns a collection

that contains all elements of the original collection while the given predicate returns true,
and then returns no further elements.
truncate: count:int -> list:'T list -> 'T list . Returns at most N elements in a new

collection.
To extract elements from the rear, use one of these:
skip: count:int -> list: 'T list -> 'T list . Returns the collection after removing the

first N elements.
skipWhile: predicate:('T -> bool) -> list:'T list -> 'T list . Bypasses elements in

a collection while the given predicate returns true, and then returns the remaining

587

Choosing between collection functions

elements of the collection.

tail: list:'T list -> 'T list . Returns the collection after removing the first element.

To extract other subsets of elements, use one of these:

filter: predicate:('T -> bool) -> list:'T list -> 'T list . Returns a new collection

containing only the elements of the collection for which the given function returns true.
except: itemsToExclude:seq<'T> -> list:'T list -> 'T list when 'T : equality .

Returns a new collection with the distinct elements of the input collection which do not
appear in the itemsToExclude sequence, using generic hash and equality comparisons
to compare values.
choose: chooser:('T -> 'U option) -> list:'T list -> 'U list . Applies the given

function to each element of the collection. Returns a collection comprised of the

elements where the function returns Some.
where: predicate:('T -> bool) -> list:'T list -> 'T list . Returns a new collection

containing only the elements of the collection for which the given predicate returns true.
NOTE: "where" is a synonym for "filter".
(Array only) sub : 'T [] -> int -> int -> 'T [] . Creates an array that contains the
supplied subrange, which is specified by starting index and length.
You can also use slice syntax: myArray.[2..5] . See below for examples.
To reduce the list to distinct elements, use one of these:
distinct: list:'T list -> 'T list when 'T : equality . Returns a collection that

contains no duplicate entries according to generic hash and equality comparisons on

the entries.
distinctBy: projection:('T -> 'Key) -> list:'T list -> 'T list when 'Key : equality .

Returns a collection that contains no duplicate entries according to the generic hash
and equality comparisons on the keys returned by the given key-generating function.

Usage examples
Taking elements from the front:

588

Choosing between collection functions

[1..10] |> List.take 3

// [1; 2; 3]
[1..10] |> List.takeWhile (fun i -> i < 3)
// [1; 2]
[1..10] |> List.truncate 4
// [1; 2; 3; 4]
[1..2] |> List.take 3
// System.InvalidOperationException: The input sequence has an insufficient number of
elements.
[1..2] |> List.takeWhile (fun i -> i < 3)
// [1; 2]
[1..2] |> List.truncate 4
// [1; 2] // no error!

Taking elements from the rear:

[1..10] |> List.skip 3
// [4; 5; 6; 7; 8; 9; 10]
[1..10] |> List.skipWhile (fun i -> i < 3)
// [3; 4; 5; 6; 7; 8; 9; 10]
[1..10] |> List.tail
// [2; 3; 4; 5; 6; 7; 8; 9; 10]
[1..2] |> List.skip 3
// System.ArgumentException: The index is outside the legal range.
[1..2] |> List.skipWhile (fun i -> i < 3)
// []
[1] |> List.tail |> List.tail
// System.ArgumentException: The input list was empty.

To extract other subsets of elements:

[1..10] |> List.filter (fun i -> i%2 = 0) // even
// [2; 4; 6; 8; 10]
[1..10] |> List.where (fun i -> i%2 = 0) // even
// [2; 4; 6; 8; 10]
[1..10] |> List.except [3;4;5]
// [1; 2; 6; 7; 8; 9; 10]

589

Choosing between collection functions

To extract a slice:
Array.sub [|1..10|] 3 5
// [|4; 5; 6; 7; 8|]
[1..10].[3..5]
// [4; 5; 6]
[1..10].[3..]
// [4; 5; 6; 7; 8; 9; 10]
[1..10].[..5]
// [1; 2; 3; 4; 5; 6]

Note that slicing on lists can be slow, because they are not random access. Slicing on arrays
is fast however.
To extract the distinct elements:
[1;1;1;2;3;3] |> List.distinct
// [1; 2; 3]
[ (1,"a"); (1,"b"); (1,"c"); (2,"d")] |> List.distinctBy fst
// [(1, "a"); (2, "d")]

Choose vs. Filter

As with pick , the choose function might seem awkward, but it is useful when dealing with
functions that return options.
In fact, choose is to filter as pick is to find , Rather than using a boolean filter, the
signal is Some vs. None .
As before, say that there is a function tryInt that parses a string and returns Some int if
the string is a valid int, otherwise None .
// string -> int option
let tryInt str =
match System.Int32.TryParse(str) with
| true, i -> Some i
| false, _ -> None

And now say that we want to find all the valid ints in a list. The crude way would be:
map the list using tryInt
filter to only include the ones that are Some

590

Choosing between collection functions

get the value from inside each option using Option.get

The code might look something like this:
let allValidNumbers =
["a";"2";"three"; "4"]
// map the input
|> List.map tryInt
// include only the "Some"
|> List.filter (fun opt -> opt.IsSome)
// get the data from each option
|> List.map Option.get
// val allValidNumbers : int list = [2; 4]

But choose will do all these steps at once! So the code becomes much simpler:
let allValidNumbers =
["a";"2";"three"; "4"]
|> List.choose tryInt

If you already have a list of options, you can filter and return the "Some" in one step by
passing id into choose :
let reduceOptions =
[None; Some 1; None; Some 2]
|> List.choose id
// val reduceOptions : int list = [1; 2]

If you want to return the first element in the same way as choose , consider using pick
(see section 11).
If you want to do a similar action as choose but for other wrapper types (such as a
Success/Failure result), there is a discussion here.

13. Partitioning, chunking and grouping

There are lots of different ways to split a collection! Have a look at the usage examples to
see the differences:
chunkBySize: chunkSize:int -> list:'T list -> 'T list list . Divides the input

collection into chunks of size at most chunkSize .

groupBy : projection:('T -> 'Key) -> list:'T list -> ('Key * 'T list) list when 'Key

591

Choosing between collection functions

: equality . Applies a key-generating function to each element of a collection and yields

a list of unique keys. Each unique key contains a list of all elements that match to this
key.
pairwise: list:'T list -> ('T * 'T) list . Returns a collection of each element in the

input collection and its predecessor, with the exception of the first element which is only
returned as the predecessor of the second element.
(Except Seq) partition: predicate:('T -> bool) -> list:'T list -> ('T list * 'T
list) . Splits the collection into two collections, containing the elements for which the

given predicate returns true and false respectively.

(Except Seq) splitAt: index:int -> list:'T list -> ('T list * 'T list) . Splits a
collection into two collections at the given index.
splitInto: count:int -> list:'T list -> 'T list list . Splits the input collection into at

most count chunks.

windowed : windowSize:int -> list:'T list -> 'T list list . Returns a list of sliding

windows containing elements drawn from the input collection. Each window is returned
as a fresh collection. Unlike pairwise the windows are collections, not tuples.

Usage examples
[1..10] |> List.chunkBySize 3
// [[1; 2; 3]; [4; 5; 6]; [7; 8; 9]; [10]]
// note that the last chunk has one element
[1..10] |> List.splitInto 3
// [[1; 2; 3; 4]; [5; 6; 7]; [8; 9; 10]]
// note that the first chunk has four elements
['a'..'i'] |> List.splitAt 3
// (['a'; 'b'; 'c'], ['d'; 'e'; 'f'; 'g'; 'h'; 'i'])
['a'..'e'] |> List.pairwise
// [('a', 'b'); ('b', 'c'); ('c', 'd'); ('d', 'e')]
['a'..'e'] |> List.windowed 3
// [['a'; 'b'; 'c']; ['b'; 'c'; 'd']; ['c'; 'd'; 'e']]
let isEven i = (i%2 = 0)
[1..10] |> List.partition isEven
// ([2; 4; 6; 8; 10], [1; 3; 5; 7; 9])
let firstLetter (str:string) = str.[0]
["apple"; "alice"; "bob"; "carrot"] |> List.groupBy firstLetter
// [('a', ["apple"; "alice"]); ('b', ["bob"]); ('c', ["carrot"])]

All the functions other than splitAt and pairwise handle edge cases gracefully:

592

Choosing between collection functions

[1] |> List.chunkBySize 3

// [[1]]
[1] |> List.splitInto 3
// [[1]]
['a'; 'b'] |> List.splitAt 3
// InvalidOperationException: The input sequence has an insufficient number of element
s.
['a'] |> List.pairwise
// InvalidOperationException: The input sequence has an insufficient number of element
s.
['a'] |> List.windowed 3
// []
[1] |> List.partition isEven
// ([], [1])
[] |> List.groupBy firstLetter
// []

14. Aggregating or summarizing a collection

The most generic way to aggregate the elements in a collection is to use reduce :
reduce : reduction:('T -> 'T -> 'T) -> list:'T list -> 'T . Apply a function to each

element of the collection, threading an accumulator argument through the computation.

reduceBack : reduction:('T -> 'T -> 'T) -> list:'T list -> 'T . Applies a function to

each element of the collection, starting from the end, threading an accumulator
argument through the computation.
and there are specific versions of reduce for frequently used aggregations:
max : list:'T list -> 'T when 'T : comparison . Return the greatest of all elements of

the collection, compared via Operators.max.

maxBy : projection:('T -> 'U) -> list:'T list -> 'T when 'U : comparison . Returns

the greatest of all elements of the collection, compared via Operators.max on the
function result.
min : list:'T list -> 'T when 'T : comparison . Returns the lowest of all elements of

the collection, compared via Operators.min.

minBy : projection:('T -> 'U) -> list:'T list -> 'T when 'U : comparison . Returns

the lowest of all elements of the collection, compared via Operators.min on the function

593

Choosing between collection functions

result.
sum : list:'T list -> 'T when 'T has static members (+) and Zero . Returns the sum of

the elements in the collection.

sumBy : projection:('T -> 'U) -> list:'T list -> 'U when 'U has static members (+)
and Zero . Returns the sum of the results generated by applying the function to each

element of the collection.

average : list:'T list -> 'T when 'T has static members (+) and Zero and DivideByInt .

Returns the average of the elements in the collection. Note that a list of ints cannot be
averaged -- they must be cast to floats or decimals.
averageBy : projection:('T -> 'U) -> list:'T list -> 'U when 'U has static members
(+) and Zero and DivideByInt . Returns the average of the results generated by applying

the function to each element of the collection.

Finally there are some counting functions:
length: list:'T list -> int . Returns the length of the collection.
countBy : projection:('T -> 'Key) -> list:'T list -> ('Key * int) list when 'Key :
equality . Applies a key-generating function to each element and returns a collection

yielding unique keys and their number of occurrences in the original collection.

Usage examples
reduce is a variant of fold without an initial state -- see section 19 for more on fold .

One way to think of it is just inserting a operator between each element.

["a";"b";"c"] |> List.reduce (+)
// "abc"

is the same as
"a" + "b" + "c"

Here's another example:

[2;3;4] |> List.reduce (*)
// is same as
2 * 3 * 4
// Result is 24

Some ways of combining elements depend on the order of combining, and so there are two
variants of "reduce":

594

Choosing between collection functions

reduce moves forward through the list.

reduceBack , not surprisingly, moves backwards through the list.

Here's a demonstration of the difference. First reduce :

[1;2;3;4] |> List.reduce (fun state x -> (state)*10 + x)
// built up from // state at each step
1 // 1
(1)*10 + 2 // 12
((1)*10 + 2)*10 + 3 // 123
(((1)*10 + 2)*10 + 3)*10 + 4 // 1234
// Final result is 1234

Using the same combining function with reduceBack produces a different result! It looks like
this:
[1;2;3;4] |> List.reduceBack (fun x state -> x + 10*(state))
// built up from // state at each step
4 // 4
3 + 10*(4) // 43
2 + 10*(3 + 10*(4)) // 432
1 + 10*(2 + 10*(3 + 10*(4))) // 4321
// Final result is 4321

Again, see section 19 for a more detailed discussion of the related functions fold and
foldBack .

The other aggregation functions are much more straightforward.

595

Choosing between collection functions

type Suit = Club | Diamond | Spade | Heart

type Rank = Two | Three | King | Ace
let cards = [ (Club,King); (Diamond,Ace); (Spade,Two); (Heart,Three); ]
cards |> List.max // (Heart, Three)
cards |> List.maxBy snd // (Diamond, Ace)
cards |> List.min // (Club, King)
cards |> List.minBy snd // (Spade, Two)
[1..10] |> List.sum
// 55
[ (1,"a"); (2,"b") ] |> List.sumBy fst
// 3
[1..10] |> List.average
// The type 'int' does not support the operator 'DivideByInt'
[1..10] |> List.averageBy float
// 5.5
[ (1,"a"); (2,"b") ] |> List.averageBy (fst >> float)
// 1.5
[1..10] |> List.length
// 10
[ ("a","A"); ("b","B"); ("a","C") ] |> List.countBy fst
// [("a", 2); ("b", 1)]
[ ("a","A"); ("b","B"); ("a","C") ] |> List.countBy snd
// [("A", 1); ("B", 1); ("C", 1)]

Most of the aggregation functions do not like empty lists! You might consider using one of
the fold functions to be safe -- see section 19.

596

Choosing between collection functions

let emptyListOfInts : int list = []

emptyListOfInts |> List.reduce (+)
// ArgumentException: The input list was empty.
emptyListOfInts |> List.max
// ArgumentException: The input sequence was empty.
emptyListOfInts |> List.min
// ArgumentException: The input sequence was empty.
emptyListOfInts |> List.sum
// 0
emptyListOfInts |> List.averageBy float
// ArgumentException: The input sequence was empty.
let emptyListOfTuples : (int*int) list = []
emptyListOfTuples |> List.countBy fst
// (int * int) list = []

15. Changing the order of the elements

You can change the order of the elements using reversing, sorting and permuting. All of the
following return new collections:
rev: list:'T list -> 'T list . Returns a new collection with the elements in reverse

order.
sort: list:'T list -> 'T list when 'T : comparison . Sorts the given collection using

Operators.compare.
sortDescending: list:'T list -> 'T list when 'T : comparison . Sorts the given

collection in descending order using Operators.compare.

sortBy: projection:('T -> 'Key) -> list:'T list -> 'T list when 'Key : comparison .

Sorts the given collection using keys given by the given projection. Keys are compared
using Operators.compare.
sortByDescending: projection:('T -> 'Key) -> list:'T list -> 'T list when 'Key :
comparison . Sorts the given collection in descending order using keys given by the

given projection. Keys are compared using Operators.compare.

sortWith: comparer:('T -> 'T -> int) -> list:'T list -> 'T list . Sorts the given

collection using the given comparison function.

permute : indexMap:(int -> int) -> list:'T list -> 'T list . Returns a collection with

all elements permuted according to the specified permutation.

597

Choosing between collection functions

And there are also some array-only functions that sort in place:
(Array only) sortInPlace: array:'T[] -> unit when 'T : comparison . Sorts the elements
of an array by mutating the array in-place. Elements are compared using
Operators.compare.
(Array only) sortInPlaceBy: projection:('T -> 'Key) -> array:'T[] -> unit when 'Key :
comparison . Sorts the elements of an array by mutating the array in-place, using the

given projection for the keys. Keys are compared using Operators.compare.
(Array only) sortInPlaceWith: comparer:('T -> 'T -> int) -> array:'T[] -> unit . Sorts
the elements of an array by mutating the array in-place, using the given comparison
function as the order.

Usage examples
[1..5] |> List.rev
// [5; 4; 3; 2; 1]
[2;4;1;3;5] |> List.sort
// [1; 2; 3; 4; 5]
[2;4;1;3;5] |> List.sortDescending
// [5; 4; 3; 2; 1]
[ ("b","2"); ("a","3"); ("c","1") ] |> List.sortBy fst
// [("a", "3"); ("b", "2"); ("c", "1")]
[ ("b","2"); ("a","3"); ("c","1") ] |> List.sortBy snd
// [("c", "1"); ("b", "2"); ("a", "3")]
// example of a comparer
let tupleComparer tuple1 tuple2 =
if tuple1 < tuple2 then
-1
elif tuple1 > tuple2 then
1
else
0
[ ("b","2"); ("a","3"); ("c","1") ] |> List.sortWith tupleComparer
// [("a", "3"); ("b", "2"); ("c", "1")]
[1..10] |> List.permute (fun i -> (i + 3) % 10)
// [8; 9; 10; 1; 2; 3; 4; 5; 6; 7]
[1..10] |> List.permute (fun i -> 9 - i)
// [10; 9; 8; 7; 6; 5; 4; 3; 2; 1]

598

Choosing between collection functions

16. Testing the elements of a collection

These set of functions all return true or false.
contains: value:'T -> source:'T list -> bool when 'T : equality . Tests if the

collection contains the specified element.

exists: predicate:('T -> bool) -> list:'T list -> bool . Tests if any element of the

collection satisfies the given predicate.

forall: predicate:('T -> bool) -> list:'T list -> bool . Tests if all elements of the

collection satisfy the given predicate.

isEmpty: list:'T list -> bool . Returns true if the collection contains no elements,

false otherwise.

17. Transforming each element to something

different
I sometimes like to think of functional programming as "transformation-oriented
programming", and map (aka Select in LINQ) is one of the most fundamental ingredients
for this approach. In fact, I have devoted a whole series to it here.

599

Choosing between collection functions

map: mapping:('T -> 'U) -> list:'T list -> 'U list . Builds a new collection whose

elements are the results of applying the given function to each of the elements of the
collection.
Sometimes each element maps to a list, and you want to flatten out all the lists. For this
case, use collect (aka SelectMany in LINQ).
collect: mapping:('T -> 'U list) -> list:'T list -> 'U list . For each element of the

list, applies the given function. Concatenates all the results and return the combined list.
Other transformation functions include:
choose in section 12 is a map and option filter combined.

(Seq only) cast: source:IEnumerable -> seq<'T> . Wraps a loosely-typed

System.Collections sequence as a typed sequence.

Usage examples
Here are some examples of using map in the conventional way, as a function that accepts a
list and a mapping function and returns a new transformed list:
let add1 x = x + 1
// map as a list transformer
[1..5] |> List.map add1
// [2; 3; 4; 5; 6]
// the list being mapped over can contain anything!
let times2 x = x * 2
[ add1; times2] |> List.map (fun f -> f 5)
// [6; 10]

You can also think of map as a function transformer. It turns an element-to-element function
into a list-to-list function.
let add1ToEachElement = List.map add1
// "add1ToEachElement" transforms lists to lists rather than ints to ints
// val add1ToEachElement : (int list -> int list)
// now use it
[1..5] |> add1ToEachElement
// [2; 3; 4; 5; 6]

collect works to flatten lists. If you already have a list of lists, you can use collect with
id to flatten them.

600

Choosing between collection functions

[2..5] |> List.collect (fun x -> [x; xx; xx*x] )

// [2; 4; 8; 3; 9; 27; 4; 16; 64; 5; 25; 125]
// using "id" with collect
let list1 = [1..3]
let list2 = [4..6]
[list1; list2] |> List.collect id
// [1; 2; 3; 4; 5; 6]

Seq.cast
Finally, Seq.cast is useful when working with older parts of the BCL that have specialized
collection classes rather than generics.
For example, the Regex library has this problem, and so the code below won't compile
because MatchCollection is not an IEnumerable<T>
open System.Text.RegularExpressions
let matches =
let pattern = "\d\d\d"
let matchCollection = Regex.Matches("123 456 789",pattern)
matchCollection
|> Seq.map (fun m -> m.Value) // ERROR
// ERROR: The type 'MatchCollection' is not compatible with the type 'seq<'a>'
|> Seq.toList

The fix is to cast MatchCollection to a Seq<Match> and then the code will work nicely:
let matches =
let pattern = "\d\d\d"
let matchCollection = Regex.Matches("123 456 789",pattern)
matchCollection
|> Seq.cast<Match>
|> Seq.map (fun m -> m.Value)
|> Seq.toList
// output = ["123"; "456"; "789"]

18. Iterating over each element

Normally, when processing a collection, we transform each element to a new value using
map . But occasionally we need to process all the elements with a function which doesn't

produce a useful value (a "unit function").

601

Choosing between collection functions

iter: action:('T -> unit) -> list:'T list -> unit . Applies the given function to each

element of the collection.

Alternatively, you can use a for-loop. The expression inside a for-loop must return
unit .

Usage examples
The most common examples of unit functions are all about side-effects: printing to the
console, updating a database, putting a message on a queue, etc. For the examples below, I
will just use printfn as my unit function.
[1..3] |> List.iter (fun i -> printfn "i is %i" i)
(*
i is 1
i is 2
i is 3
*)
// or using partial application
[1..3] |> List.iter (printfn "i is %i")
// or using a for loop
for i = 1 to 3 do
printfn "i is %i" i
// or using a for-in loop
for i in [1..3] do
printfn "i is %i" i

As noted above, the expression inside an iter or for-loop must return unit. In the following
examples, we try to add 1 to the element, and get a compiler error:
[1..3] |> List.iter (fun i -> i + 1)
// ~~~
// ERROR error FS0001: The type 'unit' does not match the type 'int'
// a for-loop expression *must* return unit
for i in [1..3] do
i + 1 // ERROR
// This expression should have type 'unit',
// but has type 'int'. Use 'ignore' ...

If you are sure that this is not a logic bug in your code, and you want to get rid of this error,
you can pipe the results into ignore :

602

Choosing between collection functions

[1..3] |> List.iter (fun i -> i + 1 |> ignore)

for i in [1..3] do
i + 1 |> ignore

19. Threading state through an iteration

The fold function is the most basic and powerful function in the collection arsenal. All other
functions (other than generators like unfold ) can be written in terms of it. See the examples
below.
fold<'T,'State> : folder:('State -> 'T -> 'State) -> state:'State -> list:'T list ->
'State . Applies a function to each element of the collection, threading an accumulator

argument through the computation.

foldBack<'T,'State> : folder:('T -> 'State -> 'State) -> list:'T list -> state:'State
-> 'State . Applies a function to each element of the collection, starting from the end,

threading an accumulator argument through the computation. WARNING: Watch out for
using Seq.foldBack on infinite lists! The runtime will laugh at you ha ha ha and then go
very quiet.
The fold function is often called "fold left" and foldBack is often called "fold right".
The scan function is like fold but returns the intermediate results and thus can be used to
trace or monitor the iteration.
scan<'T,'State> : folder:('State -> 'T -> 'State) -> state:'State -> list:'T list ->
'State list . Like fold , but returns both the intermediary and final results.
scanBack<'T,'State> : folder:('T -> 'State -> 'State) -> list:'T list -> state:'State
-> 'State list . Like foldBack , but returns both the intermediary and final results.

Just like the fold twins, scan is often called "scan left" and scanBack is often called "scan
right".
Finally, mapFold combines map and fold into one awesome superpower. More
complicated than using map and fold separately but also more efficient.
mapFold<'T,'State,'Result> : mapping:('State -> 'T -> 'Result * 'State) ->
state:'State -> list:'T list -> 'Result list * 'State . Combines map and fold. Builds

a new collection whose elements are the results of applying the given function to each
of the elements of the input collection. The function is also used to accumulate a final
value.
mapFoldBack<'T,'State,'Result> : mapping:('T -> 'State -> 'Result * 'State) ->

603

Choosing between collection functions

list:'T list -> state:'State -> 'Result list * 'State . Combines map and foldBack.

Builds a new collection whose elements are the results of applying the given function to
each of the elements of the input collection. The function is also used to accumulate a
final value.

fold examples
One way of thinking about fold is that it is like reduce but with an extra parameter for the
initial state:
["a";"b";"c"] |> List.fold (+) "hello: "
// "hello: abc"
// "hello: " + "a" + "b" + "c"
[1;2;3] |> List.fold (+) 10
// 16
// 10 + 1 + 2 + 3

As with reduce , fold and foldBack can give very different answers.
[1;2;3;4] |> List.fold (fun state x -> (state)*10 + x) 0
// state at each step
1 // 1
(1)*10 + 2 // 12
((1)*10 + 2)*10 + 3 // 123
(((1)*10 + 2)*10 + 3)*10 + 4 // 1234
// Final result is 1234

And here's the foldBack version:

List.foldBack (fun x state -> x + 10*(state)) [1;2;3;4] 0
// state at each step
4 // 4
3 + 10*(4) // 43
2 + 10*(3 + 10*(4)) // 432
1 + 10*(2 + 10*(3 + 10*(4))) // 4321
// Final result is 4321

Note that foldBack has a different parameter order to fold : the list is second last, and the
initial state is last, which means that piping is not as convenient.

Recursing vs iterating

604

Choosing between collection functions

It's easy to get confused between fold vs. foldBack . I find it helpful to think of fold as
being about iteration while foldBack is about recursion.
Let's say we want to calculate the sum of a list. The iterative way would be to use a for-loop.
You start with a (mutable) accumulator and thread it through each iteration, updating it as
you go.
let iterativeSum list =
let mutable total = 0
for e in list do
total <- total + e
total // return sum

On the other hand, the recursive approach says that if the list has a head and tail, calculate
the sum of the tail (a smaller list) first, and then add the head to it.
Each time the tail gets smaller and smaller until it is empty, at which point you're done.
let rec recursiveSum list =
match list with
| [] ->
0
| head::tail ->
head + (recursiveSum tail)

Which approach is better?

For aggregation, the iterative way is ( fold ) often easiest to understand. But for things like
constructing new lists, the recursive way is ( foldBack ) is easier to understand.
For example, if we were going to going to create a function from scratch that turned each
element into the corresponding string, we might write something like this:
let rec mapToString list =
match list with
| [] ->
[]
| head::tail ->
head.ToString() :: (mapToString tail)
[1..3] |> mapToString
// ["1"; "2"; "3"]

Using foldBack we can transfer that same logic "as is":

action for empty list = []

605

Choosing between collection functions

action for non-empty list = head.ToString() :: state

Here is the resulting function:
let foldToString list =
let folder head state =
head.ToString() :: state
List.foldBack folder list []
[1..3] |> foldToString
// ["1"; "2"; "3"]

On the other hand, a big advantage of fold is that it is easier to use "inline" because it
plays better with piping.
Luckily, you can use fold (for list construction at least) just like foldBack as long as you
reverse the list at the end.
// inline version of "foldToString"
[1..3]
|> List.fold (fun state head -> head.ToString() :: state) []
|> List.rev
// ["1"; "2"; "3"]

Using fold to implement other functions

As I mentioned above, fold is the core function for operating on lists and can emulate
most other functions, although perhaps not as efficiently as a custom implementation.
For example, here is map implemented using fold :
/// map a function "f" over all elements
let myMap f list =
// helper function
let folder state head =
f head :: state
// main flow
list
|> List.fold folder []
|> List.rev
[1..3] |> myMap (fun x -> x + 2)
// [3; 4; 5]

And here is filter implemented using fold :

606

Choosing between collection functions

/// return a new list of elements for which "pred" is true

let myFilter pred list =
// helper function
let folder state head =
if pred head then
head :: state
else
state
// main flow
list
|> List.fold folder []
|> List.rev
let isOdd n = (n%2=1)
[1..5] |> myFilter isOdd
// [1; 3; 5]

And of course, you can emulate the other functions in a similar way.

scan examples
Earlier, I showed an example of the intermediate steps of fold :
[1;2;3;4] |> List.fold (fun state x -> (state)*10 + x) 0
// state at each step
1 // 1
(1)*10 + 2 // 12
((1)*10 + 2)*10 + 3 // 123
(((1)*10 + 2)*10 + 3)*10 + 4 // 1234
// Final result is 1234

For that example, I had to manually calculate the intermediate states,

Well, if I had used scan , I would have got those intermediate states for free!
[1;2;3;4] |> List.scan (fun state x -> (state)*10 + x) 0
// accumulates from left ===> [0; 1; 12; 123; 1234]

scanBack works the same way, but backwards of course:

List.scanBack (fun x state -> (state)*10 + x) [1;2;3;4] 0

// [4321; 432; 43; 4; 0] <=== accumulates from right

607

Choosing between collection functions

Just as with foldBack the parameter order for "scan right" is inverted compared with "scan
left".

Truncating a string with scan

Here's an example where scan is useful. Say that you have a news site, and you need to
make sure headlines fit into 50 chars.
You could just truncate the string at 50, but that would look ugly. Instead you want to have
the truncation end at a word boundary.
Here's one way of doing it using scan :
Split the headline into words.
Use scan to concat the words back together, generating a list of fragments, each with
an extra word added.
Get the longest fragment under 50 chars.
// start by splitting the text into words
let text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod te
mpor."
let words = text.Split(' ')
// [|"Lorem"; "ipsum"; "dolor"; "sit"; ... ]
// accumulate a series of fragments
let fragments = words |> Seq.scan (fun frag word -> frag + " " + word) ""
(*
" Lorem"
" Lorem ipsum"
" Lorem ipsum dolor"
" Lorem ipsum dolor sit"
" Lorem ipsum dolor sit amet,"
etc
*)
// get the longest fragment under 50
let longestFragUnder50 =
fragments
|> Seq.takeWhile (fun s -> s.Length <= 50)
|> Seq.last
// trim off the first blank
let longestFragUnder50Trimmed =
longestFragUnder50 |> (fun s -> s.[1..])
// The result is:
// "Lorem ipsum dolor sit amet, consectetur"

608

Choosing between collection functions

Note that I'm using Seq.scan rather than Array.scan . This does a lazy scan and avoids
having to create fragments that are not needed.
Finally, here is the complete logic as a utility function:
// the whole thing as a function
let truncText max (text:string) =
if text.Length <= max then
text
else
text.Split(' ')
|> Seq.scan (fun frag word -> frag + " " + word) ""
|> Seq.takeWhile (fun s -> s.Length <= max-3)
|> Seq.last
|> (fun s -> s.[1..] + "...")
"a small headline" |> truncText 50
// "a small headline"
text |> truncText 50
// "Lorem ipsum dolor sit amet, consectetur..."

Yes, I know that there is a more efficient implementation than this, but I hope that this little
example shows off the power of scan .

mapFold examples
The mapFold function can do a map and a fold in one step, which can be convenient on
occasion.
Here's an example of combining an addition and a sum in one step using mapFold :
let add1 x = x + 1
// add1 using map
[1..5] |> List.map (add1)
// Result => [2; 3; 4; 5; 6]
// sum using fold
[1..5] |> List.fold (fun state x -> state + x) 0
// Result => 15
// map and sum using mapFold
[1..5] |> List.mapFold (fun state x -> add1 x, (state + x)) 0
// Result => ([2; 3; 4; 5; 6], 15)

609

Choosing between collection functions

20. Working with the index of each element

Often, you need the index of the element as you do an iteration. You could use a mutable
counter, but why not sit back and let the library do the work for you?
mapi: mapping:(int -> 'T -> 'U) -> list:'T list -> 'U list . Like map , but with the

integer index passed to the function as well. See section 17 for more on map .
iteri: action:(int -> 'T -> unit) -> list:'T list -> unit . Like iter , but with the

integer index passed to the function as well. See section 18 for more on iter .
indexed: list:'T list -> (int * 'T) list . Returns a new list whose elements are the

corresponding elements of the input list paired with the index (from 0) of each element.

Usage examples
['a'..'c'] |> List.mapi (fun index ch -> sprintf "the %ith element is '%c'" index ch)
// ["the 0th element is 'a'"; "the 1th element is 'b'"; "the 2th element is 'c'"]
// with partial application
['a'..'c'] |> List.mapi (sprintf "the %ith element is '%c'")
// ["the 0th element is 'a'"; "the 1th element is 'b'"; "the 2th element is 'c'"]
['a'..'c'] |> List.iteri (printfn "the %ith element is '%c'")
(*
the 0th element is 'a'
the 1th element is 'b'
the 2th element is 'c'
*)

indexed generates a tuple with the index -- a shortcut for a specific use of mapi :

['a'..'c'] |> List.mapi (fun index ch -> (index, ch) )

// [(0, 'a'); (1, 'b'); (2, 'c')]
// "indexed" is a shorter version of above
['a'..'c'] |> List.indexed
// [(0, 'a'); (1, 'b'); (2, 'c')]

21. Transforming the whole collection to a

different collection type
You often need to convert from one kind of collection to another. These functions do this.

610

Choosing between collection functions

The ofXXX functions are used to convert from XXX to the module type. For example,
List.ofArray will turn an array into a list.

(Except Array) ofArray : array:'T[] -> 'T list . Builds a new collection from the given
array.
(Except Seq) ofSeq: source:seq<'T> -> 'T list . Builds a new collection from the given
enumerable object.
(Except List) ofList: source:'T list -> seq<'T> . Builds a new collection from the given
list.
The toXXX are used to convert from the module type to the type XXX . For example,
List.toArray will turn an list into an array.

(Except Array) toArray: list:'T list -> 'T[] . Builds an array from the given
collection.
(Except Seq) toSeq: list:'T list -> seq<'T> . Views the given collection as a
sequence.
(Except List) toList: source:seq<'T> -> 'T list . Builds a list from the given collection.

Usage examples
[1..5] |> List.toArray // [|1; 2; 3; 4; 5|]
[1..5] |> Array.ofList // [|1; 2; 3; 4; 5|]
// etc

Using sequences with disposables

One important use of these conversion functions is to convert a lazy enumeration ( seq ) to
a fully evaluated collection such as list . This is particularly important when there is a
disposable resource involved, such as file handle or database connection. If the sequence is
not converted into a list you may encounter errors accessing the elements. See section 28
for more.

22. Changing the behavior of the collection as

a whole
There are some special functions (for Seq only) that change the behavior of the collection as
a whole.
(Seq only) cache: source:seq<'T> -> seq<'T> . Returns a sequence that corresponds to

611

Choosing between collection functions

a cached version of the input sequence. This result sequence will have the same
elements as the input sequence. The result can be enumerated multiple times. The
input sequence will be enumerated at most once and only as far as is necessary.
(Seq only) readonly : source:seq<'T> -> seq<'T> . Builds a new sequence object that
delegates to the given sequence object. This ensures the original sequence cannot be
rediscovered and mutated by a type cast.
(Seq only) delay : generator:(unit -> seq<'T>) -> seq<'T> . Returns a sequence that is
built from the given delayed specification of a sequence.

cache example
Here's an example of cache in use:
let uncachedSeq = seq {
for i = 1 to 3 do
printfn "Calculating %i" i
yield i
}
// iterate twice
uncachedSeq |> Seq.iter ignore
uncachedSeq |> Seq.iter ignore

The result of iterating over the sequence twice is as you would expect:
Calculating 1
Calculating 2
Calculating 3
Calculating 1
Calculating 2
Calculating 3

But if we cache the sequence...

let cachedSeq = uncachedSeq |> Seq.cache
// iterate twice
cachedSeq |> Seq.iter ignore
cachedSeq |> Seq.iter ignore

... then each item is only printed once:

612

Choosing between collection functions

Calculating 1
Calculating 2
Calculating 3

readonly example
Here's an example of readonly being used to hide the underlying type of the sequence:
// print the underlying type of the sequence
let printUnderlyingType (s:seq<_>) =
let typeName = s.GetType().Name
printfn "%s" typeName
[|1;2;3|] |> printUnderlyingType
// Int32[]
[|1;2;3|] |> Seq.readonly |> printUnderlyingType
// mkSeq@589 // a temporary type

delay example
Here's an example of delay .
let makeNumbers max =
[ for i = 1 to max do
printfn "Evaluating %d." i
yield i ]
let eagerList =
printfn "Started creating eagerList"
let list = makeNumbers 5
printfn "Finished creating eagerList"
list
let delayedSeq =
printfn "Started creating delayedSeq"
let list = Seq.delay (fun () -> makeNumbers 5 |> Seq.ofList)
printfn "Finished creating delayedSeq"
list

If we run the code above, we find that just by creating eagerList , we print all the
"Evaluating" messages. But creating delayedSeq does not trigger the list iteration.

613

Choosing between collection functions

Started creating eagerList

Evaluating 1.
Evaluating 2.
Evaluating 3.
Evaluating 4.
Evaluating 5.
Finished creating eagerList
Started creating delayedSeq
Finished creating delayedSeq

Only when the sequence is iterated over does the list creation happen:
eagerList |> Seq.take 3 // list already created
delayedSeq |> Seq.take 3 // list creation triggered

An alternative to using delay is just to embed the list in a seq like this:
let embeddedList = seq {
printfn "Started creating embeddedList"
yield! makeNumbers 5
printfn "Finished creating embeddedList"
}

As with delayedSeq , the makeNumbers function will not be called until the sequence is
iterated over.

23. Working with two lists

If you have two lists, there are analogues of most of the common functions like map and
fold.
map2: mapping:('T1 -> 'T2 -> 'U) -> list1:'T1 list -> list2:'T2 list -> 'U list .

Builds a new collection whose elements are the results of applying the given function to
the corresponding elements of the two collections pairwise.
mapi2: mapping:(int -> 'T1 -> 'T2 -> 'U) -> list1:'T1 list -> list2:'T2 list -> 'U
list . Like mapi , but mapping corresponding elements from two lists of equal length.
iter2: action:('T1 -> 'T2 -> unit) -> list1:'T1 list -> list2:'T2 list -> unit .

Applies the given function to two collections simultaneously. The collections must have
identical size.
iteri2: action:(int -> 'T1 -> 'T2 -> unit) -> list1:'T1 list -> list2:'T2 list ->

614

Choosing between collection functions

unit . Like iteri , but mapping corresponding elements from two lists of equal length.
forall2: predicate:('T1 -> 'T2 -> bool) -> list1:'T1 list -> list2:'T2 list -> bool .

The predicate is applied to matching elements in the two collections up to the lesser of
the two lengths of the collections. If any application returns false then the overall result
is false, else true.
exists2: predicate:('T1 -> 'T2 -> bool) -> list1:'T1 list -> list2:'T2 list -> bool .

The predicate is applied to matching elements in the two collections up to the lesser of
the two lengths of the collections. If any application returns true then the overall result is
true, else false.
fold2<'T1,'T2,'State> : folder:('State -> 'T1 -> 'T2 -> 'State) -> state:'State ->
list1:'T1 list -> list2:'T2 list -> 'State . Applies a function to corresponding

elements of two collections, threading an accumulator argument through the

computation.
foldBack2<'T1,'T2,'State> : folder:('T1 -> 'T2 -> 'State -> 'State) -> list1:'T1 list
-> list2:'T2 list -> state:'State -> 'State . Applies a function to corresponding

elements of two collections, threading an accumulator argument through the

computation.
compareWith: comparer:('T -> 'T -> int) -> list1:'T list -> list2:'T list -> int .

Compares two collections using the given comparison function, element by element.
Returns the first non-zero result from the comparison function. If the end of a collection
is reached it returns a -1 if the first collection is shorter and a 1 if the second collection is
shorter.
See also append , concat , and zip in section 26: combining and uncombining
collections.

Usage examples
These functions are straightforward to use:

615

Choosing between collection functions

let intList1 = [2;3;4]

let intList2 = [5;6;7]
List.map2 (fun i1 i2 -> i1 + i2) intList1 intList2
// [7; 9; 11]
// TIP use the ||> operator to pipe a tuple as two arguments
(intList1,intList2) ||> List.map2 (fun i1 i2 -> i1 + i2)
// [7; 9; 11]
(intList1,intList2) ||> List.mapi2 (fun index i1 i2 -> index,i1 + i2)
// [(0, 7); (1, 9); (2, 11)]
(intList1,intList2) ||> List.iter2 (printf "i1=%i i2=%i; ")
// i1=2 i2=5; i1=3 i2=6; i1=4 i2=7;
(intList1,intList2) ||> List.iteri2 (printf "index=%i i1=%i i2=%i; ")
// index=0 i1=2 i2=5; index=1 i1=3 i2=6; index=2 i1=4 i2=7;
(intList1,intList2) ||> List.forall2 (fun i1 i2 -> i1 < i2)
// true
(intList1,intList2) ||> List.exists2 (fun i1 i2 -> i1+10 > i2)
// true
(intList1,intList2) ||> List.fold2 (fun state i1 i2 -> (10*state) + i1 + i2) 0
// 801 = 234 + 567
List.foldBack2 (fun i1 i2 state -> i1 + i2 + (10*state)) intList1 intList2 0
// 1197 = 432 + 765
(intList1,intList2) ||> List.compareWith (fun i1 i2 -> i1.CompareTo(i2))
// -1
(intList1,intList2) ||> List.append
// [2; 3; 4; 5; 6; 7]
[intList1;intList2] |> List.concat
// [2; 3; 4; 5; 6; 7]
(intList1,intList2) ||> List.zip
// [(2, 5); (3, 6); (4, 7)]

Need a function that's not here?

By using fold2 and foldBack2 you can easily create your own functions. For example,
some filter2 functions can be defined like this:

616

Choosing between collection functions

/// Apply a function to each element in a pair

/// If either result passes, include that pair in the result
let filterOr2 filterPredicate list1 list2 =
let pass e = filterPredicate e
let folder e1 e2 state =
if (pass e1) || (pass e2) then
(e1,e2)::state
else
state
List.foldBack2 folder list1 list2 ([])
/// Apply a function to each element in a pair
/// Only if both results pass, include that pair in the result
let filterAnd2 filterPredicate list1 list2 =
let pass e = filterPredicate e
let folder e1 e2 state =
if (pass e1) && (pass e2) then
(e1,e2)::state
else
state
List.foldBack2 folder list1 list2 []
// test it
let startsWithA (s:string) = (s.[0] = 'A')
let strList1 = ["A1"; "A3"]
let strList2 = ["A2"; "B1"]
(strList1, strList2) ||> filterOr2 startsWithA
// [("A1", "A2"); ("A3", "B1")]
(strList1, strList2) ||> filterAnd2 startsWithA
// [("A1", "A2")]

24. Working with three lists

If you have three lists, you only have one built-in function available. But see section 25 for an
example of how you can build your own three-list functions.
map3: mapping:('T1 -> 'T2 -> 'T3 -> 'U) -> list1:'T1 list -> list2:'T2 list ->
list3:'T3 list -> 'U list . Builds a new collection whose elements are the results of

applying the given function to the corresponding elements of the three collections
simultaneously.
See also append , concat , and zip3 in section 26: combining and uncombining
collections.

617

Choosing between collection functions

25. Working with more than three lists

If you are working with more than three lists, there are no built in functions for you.
If this happens infrequently, then you could just collapse the lists into a single tuple using
zip2 and/or zip3 in succession, and then process that tuple using map .

Alternatively you can "lift" your function to the world of "zip lists" using applicatives.
let (<*>) fList xList =
List.map2 (fun f x -> f x) fList xList
let (<!>) = List.map
let addFourParams x y z w =
x + y + z + w
// lift "addFourParams" to List world and pass lists as parameters rather than ints
addFourParams <!> [1;2;3] <*> [1;2;3] <*> [1;2;3] <*> [1;2;3]
// Result = [4; 8; 12]

If that seems like magic, see this series for a explanation of what this code is doing.

26. Combining and uncombining collections

Finally, there are a number of functions that combine and uncombine collections.
append: list1:'T list -> list2:'T list -> 'T list . Returns a new collection that

contains the elements of the first collection followed by elements of the second.
@ is an infix version of append for lists.
concat: lists:seq<'T list> -> 'T list . Builds a new collection whose elements are

the results of applying the given function to the corresponding elements of the
collections simultaneously.
zip: list1:'T1 list -> list2:'T2 list -> ('T1 * 'T2) list . Combines two collections

into a list of pairs. The two collections must have equal lengths.
zip3: list1:'T1 list -> list2:'T2 list -> list3:'T3 list -> ('T1 * 'T2 * 'T3) list .

Combines three collections into a list of triples. The collections must have equal lengths.
(Except Seq) unzip: list:('T1 * 'T2) list -> ('T1 list * 'T2 list) . Splits a
collection of pairs into two collections.
(Except Seq) unzip3: list:('T1 * 'T2 * 'T3) list -> ('T1 list * 'T2 list * 'T3

618

Choosing between collection functions

list) . Splits a collection of triples into three collections.

Usage examples
These functions are straightforward to use:
List.append [1;2;3] [4;5;6]
// [1; 2; 3; 4; 5; 6]
[1;2;3] @ [4;5;6]
// [1; 2; 3; 4; 5; 6]
List.concat [ [1]; [2;3]; [4;5;6] ]
// [1; 2; 3; 4; 5; 6]
List.zip [1;2] [10;20]
// [(1, 10); (2, 20)]
List.zip3 [1;2] [10;20] [100;200]
// [(1, 10, 100); (2, 20, 200)]
List.unzip [(1, 10); (2, 20)]
// ([1; 2], [10; 20])
List.unzip3 [(1, 10, 100); (2, 20, 200)]
// ([1; 2], [10; 20], [100; 200])

Note that the zip functions require the lengths to be the same.
List.zip [1;2] [10]
// ArgumentException: The lists had different lengths.

27. Other array-only functions

Arrays are mutable, and therefore have some functions that are not applicable to lists and
sequences.
See the "sort in place" functions in section 15
Array.blit: source:'T[] -> sourceIndex:int -> target:'T[] -> targetIndex:int ->
count:int -> unit . Reads a range of elements from the first array and write them into

the second.
Array.copy: array:'T[] -> 'T[] . Builds a new array that contains the elements of the

given array.

619

Choosing between collection functions

Array.fill: target:'T[] -> targetIndex:int -> count:int -> value:'T -> unit . Fills a

range of elements of the array with the given value.

Array.set: array:'T[] -> index:int -> value:'T -> unit . Sets an element of an array.

In addition to these, all the other BCL array functions are available as well.
I won't give examples. See the MSDN documentation.

28. Using sequences with disposables

One important use of conversion functions like List.ofSeq is to convert a lazy enumeration
( seq ) to a fully evaluated collection such as list . This is particularly important when there
is a disposable resource involved such as file handle or database connection. If the
sequence is not converted into a list while the resource is available you may encounter
errors accessing the elements later, after the resource has been disposed.
This will be an extended example, so let's start with some helper functions that emulate a
database and a UI:
// a disposable database connection
let DbConnection() =
printfn "Opening connection"
{ new System.IDisposable with
member this.Dispose() =
printfn "Disposing connection" }
// read some records from the database
let readNCustomersFromDb dbConnection n =
let makeCustomer i =
sprintf "Customer %i" i
seq {
for i = 1 to n do
let customer = makeCustomer i
printfn "Loading %s from db" customer
yield customer
}
// show some records on the screen
let showCustomersinUI customers =
customers |> Seq.iter (printfn "Showing %s in UI")

A naive implementation will cause the sequence to be evaluated after the connection is
closed:

620

Choosing between collection functions

let readCustomersFromDb() =
use dbConnection = DbConnection()
let results = readNCustomersFromDb dbConnection 2
results
let customers = readCustomersFromDb()
customers |> showCustomersinUI

The output is below. You can see that the connection is closed and only then is the
sequence evaluated.
Opening connection
Disposing connection
Loading Customer 1 from db // error! connection closed!
Showing Customer 1 in UI
Loading Customer 2 from db
Showing Customer 2 in UI

A better implementation will convert the sequence to a list while the connection is open,
causing the sequence to be evaluated immediately:
let readCustomersFromDb() =
use dbConnection = DbConnection()
let results = readNCustomersFromDb dbConnection 2
results |> List.ofSeq
// Convert to list while connection is open
let customers = readCustomersFromDb()
customers |> showCustomersinUI

The result is much better. All the records are loaded before the connection is disposed:
Opening connection
Loading Customer 1 from db
Loading Customer 2 from db
Disposing connection
Showing Customer 1 in UI
Showing Customer 2 in UI

A third alternative is to embed the disposable in the sequence itself:

621

Choosing between collection functions

let readCustomersFromDb() =
seq {
// put disposable inside the sequence
use dbConnection = DbConnection()
yield! readNCustomersFromDb dbConnection 2
}
let customers = readCustomersFromDb()
customers |> showCustomersinUI

The output shows that now the UI display is also done while the connection is open:
Opening connection
Loading Customer 1 from db
Showing Customer 1 in UI
Loading Customer 2 from db
Showing Customer 2 in UI
Disposing connection

This may be a bad thing (longer time for the connection to stay open) or a good thing
(minimal memory use), depending on the context.

29. The end of the adventure

You made it to the end -- well done! Not really much of an adventure, though, was it? No
dragons or anything. Nevertheless, I hope it was helpful.

622

The "Object-oriented programming in F#" Series

As has been stressed many times before, F# is fundamentally a functional language at

heart, yet the OO features have been nicely integrated and do not have a "tacked-on"
feeling. As a result, it is quite viable to use F# just as an OO language, as an alternative to
C#, say.
In this series, we'll look at how F# supports object-oriented classes and methods.
Object-oriented programming in F#: Introduction. .
Classes. .
Inheritance and abstract classes. .
Interfaces. .
Object expressions. .

623

Object-oriented programming in F#: Introduction

Object-oriented programming in F#:

Introduction
In this series, we'll look at how F# supports object-oriented classes and methods.

Should you use object-oriented features at all?

As has been stressed many times before, F# is fundamentally a functional language at
heart, yet the OO features have been nicely integrated and do not have a "tacked-on"
feeling. As a result, it is quite viable to use F# just as an OO language, as an alternative to
C#, say.
Whether to use the OO style or the functional style is, of course, up to you. Here are some
arguments for and against.
Reasons in favor of using OO features:
If you just want to do a direct port from C# without refactoring. (For more on this, there
is a entire series on how to port from C# to F#.)
If you want to use F# primarily as an OO language, as an alternative to C#.
If you need to integrate with other .NET languages
Reasons against using OO features:
If you are a beginner coming from an imperative language, classes can be a crutch that
hinder your understanding of functional programming.
Classes do not have the convenient "out of the box" features that the "pure" F# data
types have, such as built-in equality and comparison, pretty printing, etc.
Classes and methods do not play well with the type inference system and higher order
functions (see discussion here), so using them heavily means that you are making it
harder to benefit from the most powerful parts of F#.
In most cases, the best approach is a hybrid one, primarily using pure F# types and
functions to benefit from type inference, but occasionally using interfaces and classes when
polymorphism is needed.

Understanding the object-oriented features of

F#
624

Object-oriented programming in F#: Introduction

If you do decide to use the object-oriented features of F#, the following series of posts
should cover everything you need to know to be productive with classes and methods in F#.
First up, how to create classes!

625

Classes

Classes
This post and the next will cover the basics of creating and using classes and methods in
F#.

Defining a class
Just like all other data types in F#, class definitions start with the type keyword.
The thing that distinguishes them from other types is that classes always have some
parameters passed in when they are created -- the constructor -- and so there are always
parentheses after the class name.
Also, unlike other types, classes must have functions attached to them as members. This
post will explain how you do this for classes, but for a general discussion of attaching
functions to other types see the post on type extensions.
So, for example, if we want to have a class called CustomerName that requires three
parameters to construct it, it would be written like this:
type CustomerName(firstName, middleInitial, lastName) =
member this.FirstName = firstName
member this.MiddleInitial = middleInitial
member this.LastName = lastName

Let's compare this with the C# equivalent:

public class CustomerName
{
public CustomerName(string firstName,
string middleInitial, string lastName)
{
this.FirstName = firstName;
this.MiddleInitial = middleInitial;
this.LastName = lastName;
}
public string FirstName { get; private set; }
public string MiddleInitial { get; private set; }
public string LastName { get; private set; }
}

626

Classes

You can see that in the F# version, the primary constructor is embedded into the class
declaration itself --- it is not a separate method. That is, the class declaration has the same
parameters as the constructor, and the parameters automatically become immutable private
fields that store the original values that were passed in.
So in the above example, because we declared the CustomerName class as:
type CustomerName(firstName, middleInitial, lastName)

therefore firstName , middleInitial , and lastName automatically became immutable

private fields.

Specifying types in the constructor

You might not have noticed, but the CustomerName class defined above does not constrain
the parameters to be strings, unlike the C# version. In general, type inference from usage
will probably force the values to be strings, but if you do need to specify the types explicitly,
you can do so in the usual way with a colon followed by the type name.
Here's a version of the class with explicit types in the constructor:
type CustomerName2(firstName:string,
middleInitial:string, lastName:string) =
member this.FirstName = firstName
member this.MiddleInitial = middleInitial
member this.LastName = lastName

One little quirk about F# is that if you ever need to pass a tuple as a parameter to a
constructor, you will have to annotate it explicitly, because the call to the constructor will look
identical:
type NonTupledConstructor(x:int,y: int) =
do printfn "x=%i y=%i" x y
type TupledConstructor(tuple:int * int) =
let x,y = tuple
do printfn "x=%i y=%i" x y
// calls look identical
let myNTC = new NonTupledConstructor(1,2)
let myTC = new TupledConstructor(1,2)

Class members

627

Classes

The example class above has three read-only instance properties. In F#, both properties and
methods use the member keyword.
Also, in the example above, you see the word " this " in front of each member name. This is
a "self-identifier" that can be used to refer to the current instance of the class. Every nonstatic member must have a self-identifier, even it is not used (as in the properties above).
There is no requirement to use a particular word, just as long as it is consistent. You could
use "this" or "self" or "me" or any other word that commonly indicates a self reference.

Understanding class signatures

When a class is compiled (or when you over hover the definition in the editor), you see the
"class signature" for the class. For example, for the class definition:
type MyClass(intParam:int, strParam:string) =
member this.Two = 2
member this.Square x = x * x

the corresponding signature is:

type MyClass =
class
new : intParam:int * strParam:string -> MyClass
member Square : x:int -> int
member Two : int
end

The class signature contains the signatures for all the constructors, methods and properties
in the class. It is worth understanding what these signatures mean, because, just as with
functions, you can understand what the class does by looking at them. It is also important
because you will need to write these signatures when creating abstract methods and
interfaces.

Method signatures
Method signatures such as are very similar to the signatures for standalone functions,
except that the parameter names are part of the signature itself.
So in this case, the method signature is:
member Square : x:int -> int

628

Classes

And for comparison, the corresponding signature for a standalone function would be:
val Square : int -> int

Constructor signatures
Constructor signatures are always called new , but other than that, they look like a method
signature.
Constructor signatures always take tuple values as their only parameter. In this case the
tuple type is int * string , as you would expect. The return type is the class itself, again as
you would expect.
Again, we can compare the constructor signature with a similar standalone function:
// class constructor signature
new : intParam:int * strParam:string -> MyClass
// standalone function signature
val new : int * string -> MyClass

Property signatures
Finally, property signatures such as member Two : int are very similar to the signatures for
standalone simple values, except that no explicit value is given.
// member property
member Two : int
// standalone value
val Two : int = 2

Private fields and functions using "let"

bindings
After the class declaration, you can optionally have a set of "let" bindings, typically used for
defining private fields and functions.
Here's some sample code to demonstrate this:

629

Classes

type PrivateValueExample(seed) =
// private immutable value
let privateValue = seed + 1
// private mutable value
let mutable mutableValue = 42
// private function definition
let privateAddToSeed input =
seed + input
// public wrapper for private function
member this.AddToSeed x =
privateAddToSeed x
// public wrapper for mutable value
member this.SetMutableValue x =
mutableValue <- x
// test
let instance = new PrivateValueExample(42)
printf "%i" (instance.AddToSeed 2)
instance.SetMutableValue 43

In the example above, there are three let bindings:

privateValue is set to the initial seed plus 1
mutableValue is set to 42

The privateAddToSeed function uses the initial seed plus a parameter

Because they are let bindings, they are automatically private, so to access them externally,
there must be a public member to act as a wrapper.
Note that the seed value passed into the constructor is also available as a private field, just
like the let-bound values.

Mutable constructor parameters

Sometimes, you want a parameter passed to the constructor to be mutable. You cannot
specify this in the parameter itself, so the standard technique is to create a mutable letbound value and assign it from the parameter, as shown below:

630

Classes

type MutableConstructorParameter(seed) =
let mutable mutableSeed = seed
// public wrapper for mutable value
member this.SetSeed x =
mutableSeed <- x

In cases, like this, it is quite common to give the mutable value the same name as the
parameter itself, like this:
type MutableConstructorParameter2(seed) =
let mutable seed = seed // shadow the parameter
// public wrapper for mutable value
member this.SetSeed x =
seed <- x

Additional constructor behavior with "do"

blocks
In the CustomerName example earlier, the constructor just allowed some values to be passed
in but didn't do anything else. However, in some cases, you might need to execute some
code as part of the constructor. This is done using do blocks.
Here's an example:
type DoExample(seed) =
let privateValue = seed + 1
//extra code to be done at construction time
do printfn "the privateValue is now %i" privateValue
// test
new DoExample(42)

The "do" code can also call any let-bound functions defined before it, as shown in this
example:

631

Classes

type DoPrivateFunctionExample(seed) =
let privateValue = seed + 1
// some code to be done at construction time
do printfn "hello world"
// must come BEFORE the do block that calls it
let printPrivateValue() =
do printfn "the privateValue is now %i" privateValue
// more code to be done at construction time
do printPrivateValue()
// test
new DoPrivateFunctionExample(42)

Accessing the instance via "this" in a do block

One of the differences between the "do" and "let" bindings is that the "do" bindings can
access the instance while "let" bindings cannot. This is because "let" bindings are actually
evaluated before the constructor itself (similar to field initializers in C#), so the instance in a
sense does not exist yet.
If you need to call members of the instance from a "do" block, you need some way to refer to
the instance itself. This is again done using a "self-identifier", but this time it is attached to
the class declaration itself.
type DoPublicFunctionExample(seed) as this =
// Note the "this" keyword in the declaration
let privateValue = seed + 1
// extra code to be done at construction time
do this.PrintPrivateValue()
// member
member this.PrintPrivateValue() =
do printfn "the privateValue is now %i" privateValue
// test
new DoPublicFunctionExample(42)

In general though, it is not best practice to call members from constructors unless you have
to (e.g. calling a virtual method). Better to call private let-bound functions, and if necessary,
have the public members call those same private functions.

632

Classes

Methods
A method definition is very like a function definition, except that it has the member keyword
and the self-identifier instead of just the let keyword.
Here are some examples:
type MethodExample() =
// standalone method
member this.AddOne x =
x + 1
// calls another method
member this.AddTwo x =
this.AddOne x |> this.AddOne
// parameterless method
member this.Pi() =
3.14159
// test
let me = new MethodExample()
printfn "%i" <| me.AddOne 42
printfn "%i" <| me.AddTwo 42
printfn "%f" <| me.Pi()

You can see that, just like normal functions, methods can have parameters, call other
methods, and be parameterless (or to be precise, take a unit parameter)

Tuple form vs. curried form

Unlike normal functions, methods with more than one parameter can be defined in two
different ways:
The curried form, where parameters are separated with spaces, and partial application
is supported. (Why "curried"? See the explanation of currying.)
The tuple form, where all the parameters as passed in at the same time, commaseparated, in a single tuple.
The curried approach is more functional, and the tuple approach is more object-oriented.
Here is an example class with a method for each approach:

633

Classes

type TupleAndCurriedMethodExample() =
// curried form
member this.CurriedAdd x y =
x + y
// tuple form
member this.TupleAdd(x,y) =
x + y
// test
let tc = new TupleAndCurriedMethodExample()
printfn "%i" <| tc.CurriedAdd 1 2
printfn "%i" <| tc.TupleAdd(1,2)
// use partial application
let addOne = tc.CurriedAdd 1
printfn "%i" <| addOne 99

So which approach should you use?

The advantages of tuple form are:
Compatible with other .NET code
Supports named parameters and optional parameters
Supports method overloads (multiple methods with the same name that differ only in
their function signature)
On the other hand, the disadvantages of tuple form are:
Doesn't support partial application
Doesn't work well with higher order functions
Doesn't work well with type inference
For a more detailed discussion on tuple form vs. curried form see the post on type
extensions.

Let- bound functions in conjunction with class methods

A common pattern is to create let-bound functions that do all the heavy lifting, and then have
the public methods call these internal functions directly. This has the benefit that the type
inference works much better with functional-style code than with methods.
Here's an example:

634

Classes

type LetBoundFunctions() =
let listReduce reducer list =
list |> List.reduce reducer
let reduceWithSum sum elem =
sum + elem
let sum list =
list |> listReduce reduceWithSum
// finally a public wrapper
member this.Sum = sum
// test
let lbf = new LetBoundFunctions()
printfn "Sum is %i" <| lbf.Sum [1..10]

For more details on how to do this, see this discussion.

Recursive methods
Unlike normal let-bound functions, methods that are recursive do not need the special rec
keyword. Here's the boringly familiar Fibonacci function as a method:
type MethodExample() =
// recursive method without "rec" keyword
member this.Fib x =
match x with
| 0 | 1 -> 1
| _ -> this.Fib (x-1) + this.Fib (x-2)
// test
let me = new MethodExample()
printfn "%i" <| me.Fib 10

Type annotation for methods

As usual, the types for a method's parameters and return value can normally be inferred by
the compiler, but if you need to specify them, you do so in the same way that you would for a
standard function:

635

Classes

type MethodExample() =
// explicit type annotation
member this.AddThree (x:int) :int =
x + 3

Properties
Properties can be divided into three groups:
Immutable properties, where there is a "get" but no "set".
Mutable properties, where there is a "get" and also a (possibly private) "set".
Write-only properties, where there is a "set" but no "get". These are so unusual that I
won't discuss them here, but the MSDN documentation describes the syntax if you ever
need it.
The syntax for immutable and mutable properties is slightly different.
For immutable properties, the syntax is simple. There is a "get" member that is similar to a
standard "let" value binding. The expression on the right-hand side of the binding can be any
standard expression, typically a combination of the constructor parameters, private let-bound
fields, and private functions.
Here's an example:
type PropertyExample(seed) =
// immutable property
// using a constructor parameter
member this.Seed = seed

For mutable properties however, the syntax is more complicated. You need to provide two
functions, one to get and one to set. This is done by using the syntax:
with get() = ...
and set(value) = ...

Here's an example:

636

Classes

type PropertyExample(seed) =
// private mutable value
let mutable myProp = seed
// mutable property
// changing a private mutable value
member this.MyProp
with get() = myProp
and set(value) = myProp <- value

To make the set function private, use the keywords private set instead.

Automatic properties
Starting in VS2012, F# supports automatic properties, which remove the requirement to
create a separate backing store for them.
To create an immutable auto property, use the syntax:
member val MyProp = initialValue

To create a mutable auto property, use the syntax:

member val MyProp = initialValue with get,set

Note that in this syntax there is a new keyword val and the self-identifier has gone.

Complete property example

Here's a complete example that demonstrates all the property types:

637

Classes

type PropertyExample(seed) =
// private mutable value
let mutable myProp = seed
// private function
let square x = x * x
// immutable property
// using a constructor parameter
member this.Seed = seed
// immutable property
// using a private function
member this.SeedSquared = square seed
// mutable property
// changing a private mutable value
member this.MyProp
with get() = myProp
and set(value) = myProp <- value
// mutable property with private set
member this.MyProp2
with get() = myProp
and private set(value) = myProp <- value
// automatic immutable property (in VS2012)
member val ReadOnlyAuto = 1
// automatic mutable property (in VS2012)
member val ReadWriteAuto = 1 with get,set
// test
let pe = new PropertyExample(42)
printfn "%i" <| pe.Seed
printfn "%i" <| pe.SeedSquared
printfn "%i" <| pe.MyProp
printfn "%i" <| pe.MyProp2
// try calling set
pe.MyProp <- 43 // Ok
printfn "%i" <| pe.MyProp
// try calling private set
pe.MyProp2 <- 43 // Error

Properties vs. parameterless methods

638

Classes

At this point you might be confused by the difference between properties and parameterless
methods. They look identical at first glance, but there is a subtle difference -"parameterless" methods are not really parameterless; they always have a unit parameter.
Here's an example of the difference in both definition and usage:
type ParameterlessMethodExample() =
member this.MyProp = 1 // No parens!
member this.MyFunc() = 1 // Note the ()
// in use
let x = new ParameterlessMethodExample()
printfn "%i" <| x.MyProp // No parens!
printfn "%i" <| x.MyFunc() // Note the ()

You can also tell the difference by looking at the signature of the class definition
The class definition looks like this:
type ParameterlessMethodExample =
class
new : unit -> ParameterlessMethodExample
member MyFunc : unit -> int
member MyProp : int
end

The method has signature MyFunc : unit -> int and the property has signature MyProp :
int .

This is very similar to what the signatures would be if the function and property were
declared standalone, outside of any class:
let MyFunc2() = 1
let MyProp2 = 1

The signatures for these would look like:

val MyFunc2 : unit -> int
val MyProp2 : int = 1

which is almost exactly the same.

If you are unclear on the difference and why the unit parameter is needed for the function,
please read the discussion of parameterless methods.

639

Classes

Secondary constructors
In addition to the primary constructor embedded in its declaration, a class can have
additional constructors. These are indicated by the new keyword and must call the primary
constructor as their last expression.
type MultipleConstructors(param1, param2) =
do printfn "Param1=%i Param12=%i" param1 param2
// secondary constructor
new(param1) =
MultipleConstructors(param1,-1)
// secondary constructor
new() =
printfn "Constructing..."
MultipleConstructors(13,17)
// test
let mc1 = new MultipleConstructors(1,2)
let mc2 = new MultipleConstructors(42)
let mc3 = new MultipleConstructors()

Static members
Just as in C#, classes can have static members, and this is indicated with the static
keyword. The static modifier comes before the member keyword.
Members which are static cannot have a self-identifier such as "this" because there is no
instance for them to refer to.
type StaticExample() =
member this.InstanceValue = 1
static member StaticValue = 2 // no "this"
// test
let instance = new StaticExample()
printf "%i" instance.InstanceValue
printf "%i" StaticExample.StaticValue

Static constructors

640

Classes

There is no direct equivalent of a static constructor in F#, but you can create static let-bound
values and static do-blocks that are executed when the class is first used.
type StaticConstructor() =
// static field
static let rand = new System.Random()
// static do
static do printfn "Class initialization!"
// instance member accessing static field
member this.GetRand() = rand.Next()

Accessibility of members
You can control the accessibility of a member with the standard .NET keywords public ,
private and internal . The accessibility modifiers come after the member keyword and

before the member name.

Unlike C#, all class members are public by default, not private. This includes both properties
and methods. However, non-members (e.g. let declarations) are private and cannot be made
public.
Here's an example:
type AccessibilityExample() =
member this.PublicValue = 1
member private this.PrivateValue = 2
member internal this.InternalValue = 3
// test
let a = new AccessibilityExample();
printf "%i" a.PublicValue
printf "%i" a.PrivateValue // not accessible

For properties, if the set and get have different accessibilities, you can tag each part with a
separate accessibility modifier.

641

Classes

type AccessibilityExample2() =
let mutable privateValue = 42
member this.PrivateSetProperty
with get() =
privateValue
and private set(value) =
privateValue <- value
// test
let a2 = new AccessibilityExample2();
printf "%i" a2.PrivateSetProperty // ok to read
a2.PrivateSetProperty <- 43 // not ok to write

In practice, the "public get, private set" combination that is so common in C# is not generally
needed in F#, because immutable properties can be defined more elegantly, as described
earlier.

Tip: defining classes for use by other .NET

code
If you are defining classes that need to interop with other .NET code, do not define them
inside a module! Define them in a namespace instead, outside of any module.
The reason for this is that F# modules are exposed as static classes, and any F# classes
defined inside a module are then defined as nested classes within the static class, which can
mess up your interop. For example, some unit test runners don't like static classes.
F# classes which are defined outside a module are generated as normal top-level .NET
classes, which is probably what you want. But remember that (as discussed in a previous
post) if you don't declare a namespace specifically, your class will be placed in an
automatically generated module, and will be nested without your knowledge.
Here's an example of two F# classes, one defined outside a module and one defined inside:
// Note: this code will not work in an .FSX script,
// only in an .FS source file.
namespace MyNamespace
type TopLevelClass() =
let nothing = 0
module MyModule =
type NestedClass() =
let nothing = 0

642

Classes

And here's how the same code might look in C#:

namespace MyNamespace
{
public class TopLevelClass
{
// code
}
public static class MyModule
{
public class NestedClass
{
// code
}
}
}

Constructing and using a class

Now that we have defined the class, how do we go about using it?
One way to create an instance of a class is straightfoward and just like C# -- use the new
keyword and pass in the arguments to the constructor.
type MyClass(intParam:int, strParam:string) =
member this.Two = 2
member this.Square x = x * x
let myInstance = new MyClass(1,"hello")

However, in F#, the constructor is considered to be just another function, so you can
normally eliminate the new and call the constructor function on its own, like this:
let myInstance2 = MyClass(1,"hello")
let point = System.Drawing.Point(1,2) // works with .NET classes too!

In the case when you are creating a class that implements IDisposible , you will get a
compiler warning if you do not use new .
let sr1 = System.IO.StringReader("") // Warning
let sr2 = new System.IO.StringReader("") // OK

643

Classes

This can be a useful reminder to use the use keyword instead of the let keyword for
disposables. See the post on use for more.

Calling methods and properties

And once you have an instance, you can "dot into" the instance and use any methods and
properties in the standard way.
myInstance.Two
myInstance.Square 2

We have seen many examples of member usage in the above discussion, and there's not
too much to say about it.
Remember that, as discussed above, tuple-style methods and curried-style methods can be
called in distinct ways:
type TupleAndCurriedMethodExample() =
member this.TupleAdd(x,y) = x + y
member this.CurriedAdd x y = x + y
let tc = TupleAndCurriedMethodExample()
tc.TupleAdd(1,2) // called with parens
tc.CurriedAdd 1 2 // called without parens
2 |> tc.CurriedAdd 1 // partial application

644

Inheritance and abstract classes

This is a follow-on from the previous post on classes. This post will focus on inheritance in
F#, and how to define and use abstract classes and interfaces.

Inheritance
To declare that a class inherits from another class, use the syntax:
type DerivedClass(param1, param2) =
inherit BaseClass(param1)

The inherit keyword signals that DerivedClass inherits from BaseClass . In addition,
some BaseClass constructor must be called at the same time.
It might be useful to compare F# with C# at this point. Here is some C# code for a very
simple pair of classes.
public class MyBaseClass
{
public MyBaseClass(int param1)
{
this.Param1 = param1;
}
public int Param1 { get; private set; }
}
public class MyDerivedClass: MyBaseClass
{
public MyDerivedClass(int param1,int param2): base(param1)
{
this.Param2 = param2;
}
public int Param2 { get; private set; }
}

Note that the inheritance declaration class MyDerivedClass: MyBaseClass is distinct from the
constructor which calls base(param1) .
Now here is the F# version:

645

Inheritance and abstract classes

type BaseClass(param1) =
member this.Param1 = param1
type DerivedClass(param1, param2) =
inherit BaseClass(param1)
member this.Param2 = param2
// test
let derived = new DerivedClass(1,2)
printfn "param1=%O" derived.Param1
printfn "param2=%O" derived.Param2

Unlike C#, the inheritance part of the declaration, inherit BaseClass(param1) , contains both
the class to inherit from and its constructor.

Abstract and virtual methods

Obviously, part of the point of inheritance is to be able to have abstract methods, virtual
methods, and so on.

Defining abstract methods in the base class

In C#, an abstract method is indicated by the abstract keyword plus the method signature.
In F#, it is the same concept, except that the way that function signatures are written in F# is
quite different from C#.
// concrete function definition
let Add x y = x + y
// function signature
// val Add : int -> int -> int

So to define an abstract method, we use the signature syntax, along with the abstract
member keywords:

type BaseClass() =
abstract member Add: int -> int -> int

Notice that the equals sign has been replaced with a colon. This is what you would expect,
as the equals sign is used for binding values, while the colon is used for type annotation.
Now, if you try to compile the code above, you will get an error! The compiler will complain
that there is no implementation for the method. To fix this, you need to:

646

Inheritance and abstract classes

provide a default implementation of the method, or

tell the compiler that the class as whole is also abstract.
We'll look at both of these alternatives shortly.

Defining abstract properties

An abstract immutable property is defined in a similar way. The signature is just like that of a
simple value.
type BaseClass() =
abstract member Pi : float

If the abstract property is read/write, you add the get/set keywords.

type BaseClass() =
abstract Area : float with get,set

Default implementations (but no virtual methods)

To provide a default implementation of an abstract method in the base class, use the
default keyword instead of the member keyword:

// with default implementations

type BaseClass() =
// abstract method
abstract member Add: int -> int -> int
// abstract property
abstract member Pi : float
// defaults
default this.Add x y = x + y
default this.Pi = 3.14

You can see that the default method is defined in the usual way, except for the use of
default instead of member .

One major difference between F# and C# is that in C# you can combine the abstract
definition and the default implementation into a single method, using the virtual keyword.
In F#, you cannot. You must declare the abstract method and the default implementation
separately. The abstract member has the signature, and the default has the
implementation.

647

Inheritance and abstract classes

Abstract classes
If at least one abstract method does not have a default implementation, then the entire class
is abstract, and you must indicate this by annotating it with the AbstractClass attribute.
[<AbstractClass>]
type AbstractBaseClass() =
// abstract method
abstract member Add: int -> int -> int
// abstract immutable property
abstract member Pi : float
// abstract read/write property
abstract member Area : float with get,set

If this is done, then the compiler will no longer complain about a missing implementation.

Overriding methods in subclasses

To override an abstract method or property in a subclass, use the override keyword
instead of the member keyword. Other than that change, the overridden method is defined in
the usual way.
[<AbstractClass>]
type Animal() =
abstract member MakeNoise: unit -> unit
type Dog() =
inherit Animal()
override this.MakeNoise () = printfn "woof"
// test
// let animal = new Animal() // error creating ABC
let dog = new Dog()
dog.MakeNoise()

And to call a base method, use the base keyword, just as in C#.

648

Inheritance and abstract classes

type Vehicle() =
abstract member TopSpeed: unit -> int
default this.TopSpeed() = 60
type Rocket() =
inherit Vehicle()
override this.TopSpeed() = base.TopSpeed() * 10
// test
let vehicle = new Vehicle()
printfn "vehicle.TopSpeed = %i" <| vehicle.TopSpeed()
let rocket = new Rocket()
printfn "rocket.TopSpeed = %i" <| rocket.TopSpeed()

Summary of abstract methods

Abstract methods are basically straightforward and similar to C#. There are only two areas
that might be tricky if you are used to C#:
You must understand how function signatures work and what their syntax is! For a
detailed discussion see the post on function signatures.
There is no all-in-one virtual method. You must define the abstract method and the
default implementation separately.

649

Interfaces

Interfaces
Interfaces are available and fully supported in F#, but there are number of important ways in
which their usage differs from what you might be used to in C#.

Defining interfaces
Defining an interface is similar to defining an abstract class. So similar, in fact, that you might
easily get them confused.
Here's an interface definition:
type MyInterface =
// abstract method
abstract member Add: int -> int -> int
// abstract immutable property
abstract member Pi : float
// abstract read/write property
abstract member Area : float with get,set

And here's the definition for the equivalent abstract base class:
[<AbstractClass>]
type AbstractBaseClass() =
// abstract method
abstract member Add: int -> int -> int
// abstract immutable property
abstract member Pi : float
// abstract read/write property
abstract member Area : float with get,set

So what's the difference? As usual, all abstract members are defined by signatures only. The
only difference seems to be the lack of the [<AbstractClass>] attribute.
But in the earlier discussion on abstract methods, we stressed that the [<AbstractClass>]
attribute was required; the compiler would complain that the methods have no
implementation otherwise. So how does the interface definition get away with it?

650

Interfaces

The answer is trivial, but subtle. The interface has no constructor. That is, it does not have
any parentheses after the interface name:
type MyInterface = // <- no parens!

That's it. Removing the parens will convert a class definition into an interface!

Explicit and implicit interface implementations

When it comes time to implement an interface in a class, F# is quite different from C#. In C#,
you can add a list of interfaces to the class definition and implement the interfaces implicitly.
Not so in F#. In F#, all interfaces must be explicitly implemented.
In an explicit interface implementation, the interface members can only be accessed through
an interface instance (e.g. by casting the class to the interface type). The interface members
are not visible as part of the class itself.
C# has support for both explicit and implicit interface implementations, but almost always,
the implicit approach is used, and many programmers are not even aware of explicit
interfaces in C#.

Implementing interfaces in F#
So, how do you implement an interface in F#? You cannot just "inherit" from it, as you would
an abstract base class. You have to provide an explicit implementation for each interface
member using the syntax interface XXX with , as shown below:
type IAddingService =
abstract member Add: int -> int -> int
type MyAddingService() =
interface IAddingService with
member this.Add x y =
x + y
interface System.IDisposable with
member this.Dispose() =
printfn "disposed"

The above code shows how the class MyAddingService explicitly implements the
IAddingService and the IDisposable interfaces. After the required interface XXX with

section, the members are implemented in the normal way.

651

Interfaces

(As an aside, note again that MyAddingService() has a constructor, while IAddingService
does not.)

Using interfaces
So now let's try to use the adding service interface:
let mas = new MyAddingService()
mas.Add 1 2 // error

Immediately, we run into an error. It appears that the instance does not implement the Add
method at all. Of course, what this really means is that we must cast it to the interface first
using the :> operator:
// cast to the interface
let mas = new MyAddingService()
let adder = mas :> IAddingService
adder.Add 1 2 // ok

This might seem incredibly awkward, but in practice it is not a problem as in most cases the
casting is done implicitly for you.
For example, you will typically be passing an instance to a function that specifies an
interface parameter. In this case, the casting is done automatically:
// function that requires an interface
let testAddingService (adder:IAddingService) =
printfn "1+2=%i" <| adder.Add 1 2 // ok
let mas = new MyAddingService()
testAddingService mas // cast automatically

And in the special case of IDisposable , the use keyword will also automatically cast the
instance as needed:
let testDispose =
use mas = new MyAddingService()
printfn "testing"
// Dispose() is called here

652

Object expressions

Object expressions
So as we saw in the previous post, implementing interfaces in F# is a bit more awkward than
in C#. But F# has a trick up its sleeve, called "object expressions".
With object expressions, you can implement an interface on-the-fly, without having to create
a class.

Implementing interfaces with object expressions

Object expressions are most commonly used to implement interfaces. To do this, you use
the syntax new MyInterface with ... , and the wrap the whole thing in curly braces (one of
the few uses for them in F#!)
Here is some example code that creates a number of objects, each of which implements
IDisposable .

// create a new object that implements IDisposable

If you execute this code, you will see the output below. You can see that Dispose() is
indeed being called when the objects go out of scope.

653

Object expressions

using first resource

We can take the same approach with the IAddingService and create one on the fly as well.
let makeAdder id =
{ new IAddingService with
member this.Add x y =
printfn "Adder%i is adding" id
let result = x + y
printfn "%i + %i = %i" x y result
result
}
let testAdders =
for i in [1..3] do
let adder = makeAdder i
let result = adder.Add i i
() //ignore result

Object expressions are extremely convenient, and can greatly reduce the number of classes
you need to create if you are interacting with an interface heavy library.

654

The "Computation Expressions" Series

In this series, you'll learn what computation expressions are, some common patterns, and
how to make your own. In the process, we'll also look at continuations, the bind function,
wrapper types, and more.
Computation expressions: Introduction. Unwrapping the enigma....
Understanding continuations. How 'let' works behind the scenes.
Introducing 'bind'. Steps towards creating our own 'let!' .
Computation expressions and wrapper types. Using types to assist the workflow.
More on wrapper types. We discover that even lists can be wrapper types.
Implementing a builder: Zero and Yield. Getting started with the basic builder methods.
Implementing a builder: Combine. How to return multiple values at once.
Implementing a builder: Delay and Run. Controlling when functions execute.
Implementing a builder: Overloading. Stupid method tricks.
Implementing a builder: Adding laziness. Delaying a workflow externally.
Implementing a builder: The rest of the standard methods. Implementing While, Using,
and exception handling.

655

Computation expressions: Introduction

By popular request, it is time to talk about the mysteries of computation expressions, what
they are, and how they can be useful in practice (and I will try to avoid using the forbidden
m-word).
In this series, you'll learn what computation expressions are, how to make your own, and
what some common patterns involving them. In the process, we'll also look at continuations,
the bind function, wrapper types, and more.

Background
Computation expressions seem to have a reputation for being abstruse and difficult to
understand.
On one hand, they're easy enough to use. Anyone who has written much F# code has
certainly used standard ones like seq{...} or async{...} .
But how do you make a new one of these things? How do they work behind the scenes?
Unfortunately, many explanations seem to make things even more confusing. There seems
to be some sort of mental bridge that you have to cross. Once you are on the other side, it is
all obvious, but to someone on this side, it is baffling.
If we turn for guidance to the official MSDN documention, it is explicit, but quite unhelpful to
a beginner.
For example, it says that when you see the following code within a computation expression:
{| let! pattern = expr in cexpr |}

it is simply syntactic sugar for this method call:

builder.Bind(expr, (fun pattern -> {| cexpr |}))

But... what does this mean exactly?

I hope that by the end of this series, the documentation above will become obvious. Don't
believe me? Read on!

656

Computation expressions: Introduction

Computation expressions in practice

Before going into the mechanics of computation expressions, let's look at a few trivial
examples that show the same code before and after using computation expressions.
Let's start with a simple one. Let's say we have some code, and we want to log each step.
So we define a little logging function, and call it after every value is created, like so:
let log p = printfn "expression is %A" p
let loggedWorkflow =
let x = 42
log x
let y = 43
log y
let z = x + y
log z
//return
z

If you run this, you will see the output:

expression is 42
expression is 43
expression is 85

Simple enough.
But it is annoying to have to explicitly write all the log statements each time. Is there a way to
hide them?
Funny you should ask... A computation expression can do that. Here's one that does exactly
the same thing.
First we define a new type called LoggingBuilder :
type LoggingBuilder() =
let log p = printfn "expression is %A" p
member this.Bind(x, f) =
log x
f x
member this.Return(x) =
x

657

Computation expressions: Introduction

Don't worry about what the mysterious Bind and Return are for yet -- they will be
explained soon.
Next we create an instance of the type, logger in this case.
let logger = new LoggingBuilder()

So with this logger value, we can rewrite the original logging example like this:
let loggedWorkflow =
logger
{
let! x = 42
let! y = 43
let! z = x + y
return z
}

If you run this, you get exactly the same output, but you can see that the use of the
logger{...} workflow has allowed us to hide the repetitive code.

Safe division
Now let's look at an old chestnut.
Say that we want to divide a series of numbers, one after another, but one of them might be
zero. How can we handle it? Throwing an exception is ugly. Sounds like a good match for
the option type though.
First we need to create a helper function that does the division and gives us back an int
option . If everything is OK, we get a Some and if the division fails, we get a None .

Then we can chain the divisions together, and after each division we need to test whether it
failed or not, and keep going only if it was successful.
Here's the helper function first, and then the main workflow:
let divideBy bottom top =
if bottom = 0
then None
else Some(top/bottom)

Note that I have put the divisor first in the parameter list. This is so we can write an
expression like 12 |> divideBy 3 , which makes chaining easier.

658

Computation expressions: Introduction

And here it is in use:

let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

The bad workflow fails on the third step and returns None for the whole thing.
It is very important to note that the entire workflow has to return an int option as well. It
can't just return an int because what would it evaluate to in the bad case? And can you
see how the type that we used "inside" the workflow, the option type, has to be the same
type that comes out finally at the end. Remember this point -- it will crop up again later.
Anyway, this continual testing and branching is really ugly! Does turning it into a computation
expression help?
Once more we define a new type ( MaybeBuilder ) and make an instance of the type
( maybe ).
type MaybeBuilder() =
member this.Bind(x, f) =
match x with
| None -> None
| Some a -> f a
member this.Return(x) =
Some x
let maybe = new MaybeBuilder()

659

Computation expressions: Introduction

I have called this one MaybeBuilder rather than divideByBuilder because the issue of
dealing with option types this way, using a computation expression, is quite common, and
maybe is the standard name for this thing.

So now that we have defined the maybe workflow, let's rewrite the original code to use it.
let divideByWorkflow init x y z =
maybe
{
let! a = init |> divideBy x
let! b = a |> divideBy y
let! c = b |> divideBy z
return c
}

Much, much nicer. The maybe expression has completely hidden the branching logic!
And if we test it we get the same result as before:
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

Chains of "or else" tests

In the previous example of "divide by", we only wanted to continue if each step was
successful.
But sometimes it is the other way around. Sometimes the flow of control depends on a
series of "or else" tests. Try one thing, and if that succeeds, you're done. Otherwise try
another thing, and if that fails, try a third thing, and so on.
Let's look at a simple example. Say that we have three dictionaries and we want to find the
value corresponding to a key. Each lookup might succeed or fail, so we need to chain the
lookups in a series.

660

Computation expressions: Introduction

let map1 = [ ("1","One"); ("2","Two") ] |> Map.ofList

let map2 = [ ("A","Alice"); ("B","Bob") ] |> Map.ofList
let map3 = [ ("CA","California"); ("NY","New York") ] |> Map.ofList
let multiLookup key =
match map1.TryFind key with
| Some result1 -> Some result1 // success
| None -> // failure
match map2.TryFind key with
| Some result2 -> Some result2 // success
| None -> // failure
match map3.TryFind key with
| Some result3 -> Some result3 // success
| None -> None // failure

Because everything is an expression in F# we can't do an early return, we have to cascade

all the tests in a single expression.
Here's how this might be used:
multiLookup "A" |> printfn "Result for A is %A"
multiLookup "CA" |> printfn "Result for CA is %A"
multiLookup "X" |> printfn "Result for X is %A"

It works fine, but can it be simplified?

Yes indeed. Here is an "or else" builder that allows us to simplify these kinds of lookups:
type OrElseBuilder() =
member this.ReturnFrom(x) = x
member this.Combine (a,b) =
match a with
| Some _ -> a // a succeeds -- use it
| None -> b // a fails -- use b instead
member this.Delay(f) = f()
let orElse = new OrElseBuilder()

Here's how the lookup code could be altered to use it:

661

Computation expressions: Introduction

let map1 = [ ("1","One"); ("2","Two") ] |> Map.ofList

let map2 = [ ("A","Alice"); ("B","Bob") ] |> Map.ofList
let map3 = [ ("CA","California"); ("NY","New York") ] |> Map.ofList
let multiLookup key = orElse {
return! map1.TryFind key
return! map2.TryFind key
return! map3.TryFind key
}

Again we can confirm that the code works as expected.

multiLookup "A" |> printfn "Result for A is %A"
multiLookup "CA" |> printfn "Result for CA is %A"
multiLookup "X" |> printfn "Result for X is %A"

Asynchronous calls with callbacks

Finally, let's look at callbacks. The standard approach for doing asynchronous operations in
.NET is to use a AsyncCallback delegate which gets called when the async operation is
complete.
Here is an example of how a web page might be downloaded using this technique:
open System.Net
let req1 = HttpWebRequest.Create("http://tryfsharp.org")
let req2 = HttpWebRequest.Create("http://google.com")
let req3 = HttpWebRequest.Create("http://bing.com")
req1.BeginGetResponse((fun r1 ->
use resp1 = req1.EndGetResponse(r1)
printfn "Downloaded %O" resp1.ResponseUri
req2.BeginGetResponse((fun r2 ->
use resp2 = req2.EndGetResponse(r2)
printfn "Downloaded %O" resp2.ResponseUri
req3.BeginGetResponse((fun r3 ->
use resp3 = req3.EndGetResponse(r3)
printfn "Downloaded %O" resp3.ResponseUri
),null) |> ignore
),null) |> ignore
),null) |> ignore

662

Computation expressions: Introduction

Lots of calls to BeginGetResponse and EndGetResponse , and the use of nested lambdas,
makes this quite complicated to understand. The important code (in this case, just print
statements) is obscured by the callback logic.
In fact, managing this cascading approach is always a problem in code that requires a chain
of callbacks; it has even been called the "Pyramid of Doom" (although none of the solutions
are very elegant, IMO).
Of course, we would never write that kind of code in F#, because F# has the async
computation expression built in, which both simplifies the logic and flattens the code.
open System.Net
let req1 = HttpWebRequest.Create("http://tryfsharp.org")
let req2 = HttpWebRequest.Create("http://google.com")
let req3 = HttpWebRequest.Create("http://bing.com")
async {
use! resp1 = req1.AsyncGetResponse()
printfn "Downloaded %O" resp1.ResponseUri
use! resp2 = req2.AsyncGetResponse()
printfn "Downloaded %O" resp2.ResponseUri
use! resp3 = req3.AsyncGetResponse()
printfn "Downloaded %O" resp3.ResponseUri
} |> Async.RunSynchronously

We'll see exactly how the async workflow is implemented later in this series.

Summary
So we've seen some very simple examples of computation expressions, both "before" and
"after", and they are quite representative of the kinds of problems that computation
expressions are useful for.
In the logging example, we wanted to perform some side-effect between each step.
In the safe division example, we wanted to handle errors elegantly so that we could
focus on the happy path.
In the multiple dictionary lookup example, we wanted to return early with the first
success.
And finally, in the async example, we wanted to hide the use of callbacks and avoid the
"pyramid of doom".

663

Computation expressions: Introduction

What all the cases have in common is that the computation expression is "doing something
behind the scenes" between each expression.
If you want a bad analogy, you can think of a computation expression as somewhat like a
post-commit hook for SVN or git, or a database trigger that gets called on every update. And
really, that's all that a computation expression is: something that allows you to sneak your
own code in to be called in the background, which in turn allows you to focus on the
important code in the foreground.
Why are they called "computation expressions"? Well, it's obviously some kind of
expression, so that bit is obvious. I believe that the F# team did originally want to call it
"expression-that-does-something-in-the-background-between-each-let" but for some reason,
people thought that was a bit unwieldy, so they settled on the shorter name "computation
expression" instead.
And as to the difference between a "computation expression" and a "workflow", I use
"computation expression" to mean the {...} and let! syntax, and reserve "workflow" for
particular implementations where appropriate. Not all computation expression
implementations are workflows. For example, it is appropriate to talk about the "async
workflow" or the "maybe workflow", but the "seq workflow" doesn't sound right.
In other words, in the following code, I would say that maybe is the workflow we are using,
and the particular chunk of code { let! a = .... return c } is the computation expression.
maybe
{
let! a = x |> divideBy y
let! b = a |> divideBy w
let! c = b |> divideBy z
return c
}

You probably want to start creating your own computation expressions now, but first we
need to take a short detour into continuations. That's up next.
Update on 2015-01-11: I have removed the counting example that used a "state"
computation expression. It was too confusing and distracted from the main concepts.

664

Understanding continuations

Understanding continuations
In the previous post we saw how some complex code could be condensed using
computation expressions.
Here's the code before using a computation expression:

let log p = printfn "expression is %A" p

let loggedWorkflow =
let x = 42
log x
let y = 43
log y
let z = x + y
log z
//return
z

And here's the same code after using a computation expression:

let loggedWorkflow =
logger
{
let! x = 42
let! y = 43
let! z = x + y
return z
}

The use of let! rather than a normal let is important. Can we emulate this ourselves so
we can understand what is going on? Yes, but we need to understand continuations first.

Continuations
In imperative programming, we have the concept of "returning" from a function. When you
call a function, you "go in", and then you "come out", just like pushing and popping a stack.
Here is some typical C# code which works like this. Notice the use of the return keyword.

665

Understanding continuations

public int Divide(int top, int bottom)

{
if (bottom==0)
{
throw new InvalidOperationException("div by 0");
}
else
{
return top/bottom;
}
}
public bool IsEven(int aNumber)
{
var isEven = (aNumber % 2 == 0);
return isEven;
}

You've seen this a million times, but there is a subtle point about this approach that you
might not have considered: the called function always decides what to do.
For example, the implemention of Divide has decided that it is going to throw an exception.
But what if I don't want an exception? Maybe I want a nullable<int> , or maybe I am going
to display it on a screen as "#DIV/0". Why throw an exception that I am immediately going to
have to catch? In other words, why not let the caller decide what should happen, rather the
callee.
Similarly in the IsEven example, what am I going to do with the boolean return value?
Branch on it? Or maybe print it in a report? I don't know, but again, rather than returning a
boolean that the caller has to deal with, why not let the caller tell the callee what to do next?
So this is what continuations are. A continuation is simply a function that you pass into
another function to tell it what to do next.
Here's the same C# code rewritten to allow the caller to pass in functions which the callee
uses to handle each case. If it helps, you can think of this as somewhat analogous to a
visitor pattern. Or maybe not.

666

Understanding continuations

public T Divide<T>(int top, int bottom, Func<T> ifZero, Func<int,T> ifSuccess)

{
if (bottom==0)
{
return ifZero();
}
else
{
return ifSuccess( top/bottom );
}
}
public T IsEven<T>(int aNumber, Func<int,T> ifOdd, Func<int,T> ifEven)
{
if (aNumber % 2 == 0)
{
return ifEven(aNumber);
}
else
{ return ifOdd(aNumber);
}
}

Note that the C# functions have been changed to return a generic T now, and both
continuations are a Func that returns a T .
Well, passing in lots of Func parameters always looks pretty ugly in C#, so it is not done
very often. But passing functions is easy in F#, so let's see how this code ports over.
Here's the "before" code:
let divide top bottom =
if (bottom=0)
then invalidOp "div by 0"
else (top/bottom)
let isEven aNumber =
aNumber % 2 = 0

and here's the "after" code:

667

Understanding continuations

let divide ifZero ifSuccess top bottom =

if (bottom=0)
then ifZero()
else ifSuccess (top/bottom)
let isEven ifOdd ifEven aNumber =
if (aNumber % 2 = 0)
then aNumber |> ifEven
else aNumber |> ifOdd

A few things to note. First, you can see that I have put the extra functions ( ifZero , etc) first
in the parameter list, rather than last, as in the C# example. Why? Because I am probably
going to want to use partial application.
And also, in the isEven example, I wrote aNumber |> ifEven and aNumber |> ifOdd . This
makes it clear that we are piping the current value into the continuation and the continuation
is always the very last step to be evaluated. We will be using this exact same pattern later in
this post, so make sure you understand what is going on here.

Continuation examples
With the power of continuations at our disposal, we can use the same divide function in
three completely different ways, depending on what the caller wants.
Here are three scenarios we can create quickly:
pipe the result into a message and print it,
convert the result to an option using None for the bad case and Some for the good
case,
or throw an exception in the bad case and just return the result in the good case.

668

Understanding continuations

// Scenario 1: pipe the result into a message

// ---------------------------------------// setup the functions to print a message
let ifZero1 () = printfn "bad"
let ifSuccess1 x = printfn "good %i" x
// use partial application
let divide1 = divide ifZero1 ifSuccess1
//test
let good1 = divide1 6 3
let bad1 = divide1 6 0
// Scenario 2: convert the result to an option
// ---------------------------------------// setup the functions to return an Option
let ifZero2() = None
let ifSuccess2 x = Some x
let divide2 = divide ifZero2 ifSuccess2
//test
let good2 = divide2 6 3
let bad2 = divide2 6 0
// Scenario 3: throw an exception in the bad case
// ---------------------------------------// setup the functions to throw exception
let ifZero3() = failwith "div by 0"
let ifSuccess3 x = x
let divide3 = divide ifZero3 ifSuccess3
//test
let good3 = divide3 6 3
let bad3 = divide3 6 0

Notice that with this approach, the caller never has to catch an exception from divide
anywhere. The caller decides whether an exception will be thrown, not the callee. So not
only has the divide function become much more reusable in different contexts, but the
cyclomatic complexity has just dropped a level as well.
The same three scenarios can be applied to the isEven implementation:

669

Understanding continuations

// Scenario 1: pipe the result into a message

// ---------------------------------------// setup the functions to print a message
let ifOdd1 x = printfn "isOdd %i" x
let ifEven1 x = printfn "isEven %i" x
// use partial application
let isEven1 = isEven ifOdd1 ifEven1
//test
let good1 = isEven1 6
let bad1 = isEven1 5
// Scenario 2: convert the result to an option
// ---------------------------------------// setup the functions to return an Option
let ifOdd2 _ = None
let ifEven2 x = Some x
let isEven2 = isEven ifOdd2 ifEven2
//test
let good2 = isEven2 6
let bad2 = isEven2 5
// Scenario 3: throw an exception in the bad case
// ---------------------------------------// setup the functions to throw exception
let ifOdd3 _ = failwith "assert failed"
let ifEven3 x = x
let isEven3 = isEven ifOdd3 ifEven3
//test
let good3 = isEven3 6
let bad3 = isEven3 5

In this case, the benefits are subtler, but the same: the caller never had to handle booleans
with an if/then/else anywhere. There is less complexity and less chance of error.
It might seem like a trivial difference, but by passing functions around like this, we can use
all our favorite functional techniques such as composition, partial application, and so on.
We have also met continuations before, in the series on designing with types. We saw that
their use enabled the caller to decide what would happen in case of possible validation
errors in a constructor, rather than just throwing an exception.

670

Understanding continuations

type EmailAddress = EmailAddress of string

let CreateEmailAddressWithContinuations success failure (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then success (EmailAddress s)
else failure "Email address must contain an @ sign"

The success function takes the email as a parameter and the error function takes a string.
Both functions must return the same type, but the type is up to you.
And here is a simple example of the continuations in use. Both functions do a printf, and
return nothing (i.e. unit).
// setup the functions
let success (EmailAddress s) = printfn "success creating email %s" s
let failure msg = printfn "error creating email: %s" msg
let createEmail = CreateEmailAddressWithContinuations success failure
// test
let goodEmail = createEmail "x@example.com"
let badEmail = createEmail "example.com"

Continuation passing style

Using continuations like this leads to a style of programming called "continuation passing
style" (or CPS), whereby every function is called with an extra "what to do next" function
parameter.
To see the difference, let's look at the standard, direct style of programming.
When you use the direct style, you go "in" and "out" of functions, like this
call a function ->
<- return from the function
call another function ->
<- return from the function
call yet another function ->
<- return from the function

In continuation passing style, on the other hand, you end up with a chain of functions, like
this:

671

Understanding continuations

evaluate something and pass it into ->

a function that evaluates something and passes it into ->
another function that evaluates something and passes it into ->
yet another function that evaluates something and passes it into ->
...etc...

There is obviously a big difference between the two styles.

In the direct style, there is a hierarchy of functions. The top level function is a sort of "master
controller" who calls one subroutine, and then another, deciding when to branch, when to
loop, and generally coordinating the control flow explicitly.
In the contination passing style, though, there is no "master controller". Instead there is a
sort of "pipeline", not of data but of control flow, where the "function in charge" changes as
the execution logic flows through the pipe.
If you have ever attached a event handler to a button click in a GUI, or used a callback with
BeginInvoke, then you have used this style without being aware of it. And in fact, this style
will be key to understanding the async workflow, which I'll discuss later in this series.

Continuations and 'let'

So how does all this fit in with let ?
Let's go back and revisit what 'let` actually does.
Remember that a (non-top-level) "let" can never be used in isolation -- it must always be part
of a larger code block.
That is:
let x = someExpression

really means:
let x = someExpression in [an expression involving x]

And then every time you see the x in the second expression (the body expression),
substitute it with the first expression ( someExpression ).
So for example, the expression:

672

Understanding continuations

let x = 42
let y = 43
let z = x + y

really means (using the verbose in keyword):

let x = 42 in
let y = 43 in
let z = x + y in
z // the result

Now funnily enough, a lambda looks very similar to a let :

fun x -> [an expression involving x]

and if we pipe in the value of x as well, we get the following:

someExpression |> (fun x -> [an expression involving x] )

Doesn't this look awfully like a let to you? Here is a let and a lambda side by side:
// let
let x = someExpression in [an expression involving x]
// pipe a value into a lambda
someExpression |> (fun x -> [an expression involving x] )

They both have an x , and a someExpression , and everywhere you see x in the body of
the lambda you replace it with someExpression . Yes, the x and the someExpression are
reversed in the lambda case, but otherwise it is basically the same thing as a let .
So, using this technique, we can rewrite the original example in this style:
42 |> (fun x ->
43 |> (fun y ->
x + y |> (fun z ->
z)))

When it is written this way, you can see that we have transformed the let style into a
continuation passing style!
In the first line we have a value 42 -- what do we want to do with it? Let's pass it into a
continuation, just as we did with the isEven function earlier. And in the context of the

673

Understanding continuations

continuation, we will relabel 42 as x .

In the second line we have a value 43 -- what do we want to do with it? Let's pass it
too into a continuation, calling it y in that context.
In the third line we add the x and y together to create a new value. And what do we
want to do with it? Another continuation, another label ( z ).
Finally in the last line we are done and the whole expression evaluates to z .

Wrapping the continuation in a function

Let's get rid of the explicit pipe and write a little function to wrap this logic. We can't call it
"let" because that is a reserved word, and more importantly, the parameters are backwards
from 'let'. The "x" is on the right hand side, and the "someExpression" is on the left hand
side. So we'll call it pipeInto for now.
The definition of pipeInto is really obvious:
let pipeInto (someExpression,lambda) =
someExpression |> lambda

Note that we are passing both parameters in at once using a tuple rather than as two distinct
parameters separated by whitespace. They will always come as a pair.
So, with this pipeInto function we can then rewrite the example once more as:
pipeInto (42, fun x ->
pipeInto (43, fun y ->
pipeInto (x + y, fun z ->
z)))

or we can eliminate the indents and write it like this:

pipeInto (42, fun x ->
pipeInto (43, fun y ->
pipeInto (x + y, fun z ->
z)))

You might be thinking: so what? Why bother to wrap the pipe into a function?
The answer is that we can add extra code in the pipeInto function to do stuff "behine the
scenes", just as in a computation expression.

The "logging" example revisited

674

Understanding continuations

Let's redefine pipeInto to add a little bit of logging, like this:

let pipeInto (someExpression,lambda) =
printfn "expression is %A" someExpression
someExpression |> lambda

Now... run that code again.

pipeInto (42, fun x ->
pipeInto (43, fun y ->
pipeInto (x + y, fun z ->
z
)))

What is the output?

expression is 42
expression is 43
expression is 85

This is exactly the same output as we had in the earlier implementations. We have created
our own little computation expression workflow!
If we compare this side by side with the computation expression version, we can see that
our homebrew version is very similar to the let! , except that we have the parameters
reversed, and we have the explicit arrow for the continuation.

The "safe divide" example revisited

Let's do the same thing with the "safe divide" example. Here was the original code:

675

Understanding continuations

let divideBy bottom top =

You should see now that this "stepped" style is an obvious clue that we really should be
using continuations.
Let's see if we can add extra code to pipeInto to do the matching for us. The logic we want
is:
If the someExpression parameter is None , then don't call the continuation lambda.
If the someExpression parameter is Some , then do call the continuation lambda,
passing in the contents of the Some .
Here it is:
let pipeInto (someExpression,lambda) =
match someExpression with
| None ->
None
| Some x ->
x |> lambda

With this new version of pipeInto we can rewrite the original code like this:

676

Understanding continuations

let divideByWorkflow x y w z =
let a = x |> divideBy y
pipeInto (a, fun a' ->
let b = a' |> divideBy w
pipeInto (b, fun b' ->
let c = b' |> divideBy z
pipeInto (c, fun c' ->
Some c' //return
)))

We can clean this up quite a bit.

First we can eliminate the a , b and c , and replace them with the divideBy expression
directly. So that this:
let a = x |> divideBy y
pipeInto (a, fun a' ->

becomes just this:

pipeInto (x |> divideBy y, fun a' ->

Now we can relabel a' as just a , and so on, and we can also remove the stepped
indentation, so that we get this:
let divideByResult x y w z =
pipeInto (x |> divideBy y, fun a ->
pipeInto (a |> divideBy w, fun b ->
pipeInto (b |> divideBy z, fun c ->
Some c //return
)))

Finally, we'll create a little helper function called return' to wrap the result in an option.
Putting it all together, the code looks like this:

677

Understanding continuations

let divideBy bottom top =

if bottom = 0
then None
else Some(top/bottom)
let pipeInto (someExpression,lambda) =
match someExpression with
| None ->
None
| Some x ->
x |> lambda
let return' c = Some c
let divideByWorkflow x y w z =
pipeInto (x |> divideBy y, fun a ->
pipeInto (a |> divideBy w, fun b ->
pipeInto (b |> divideBy z, fun c ->
return' c
)))
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

Again, if we compare this side by side with the computation expression version, we can see
that our homebrew version is identical in meaning. Only the syntax is different.

Summary
In this post, we talked about continuations and continuation passing style, and how we can
think of let as a nice syntax for doing continuations behind scenes.
So now we have everything we need to start creating our own version of let . In the next
post, we'll put this knowledge into practice.

678

Understanding continuations

679

Introducing 'bind'

Introducing 'bind'
In the last post we talked about how we can think of let as a nice syntax for doing
continuations behind scenes. And we introduced a pipeInto function that allowed us to add
hooks into the continuation pipeline.
Now we are ready to look at our first builder method, Bind , which formalizes this approach
and is the core of any computation expression.

Introducing "Bind "

The MSDN page on computation expressions describes the let! expression as syntactic
sugar for a Bind method. Let's look at this again:
Here's the let! expression documentation, along with a real example:
// documentation
{| let! pattern = expr in cexpr |}
// real example
let! x = 43 in some expression

And here's the Bind method documentation, along with a real example:
// documentation
builder.Bind(expr, (fun pattern -> {| cexpr |}))
// real example
builder.Bind(43, (fun x -> some expression))

Notice a few interesting things about this:

Bind takes two parameters, an expression ( 43 ) and a lambda.

The parameter of the lambda ( x ) is bound to the expression passed in as the first
parameter. (In this case at least. More on this later.)
The parameters of Bind are reversed from the order they are in let! .
So in other words, if we chain a number of let! expressions together like this:
let! x = 1
let! y = 2
let! z = x + y

680

Introducing 'bind'

the compiler converts it to calls to Bind , like this:

Bind(1, fun x ->
Bind(2, fun y ->
Bind(x + y, fun z ->
etc

I think you can see where we are going with this by now.
Indeed, our pipeInto function is exactly the same as the Bind method.
This is a key insight: computation expressions are just a way to create nice syntax for
something that we could do ourselves.

A standalone bind function

Having a "bind" function like this is actually a standard functional pattern, and it is not
dependent on computation expressions at all.
First, why is it called "bind"? Well, as we've seen, a "bind" function or method can be thought
of as feeding an input value to a function. This is known as "binding" a value to the
parameter of the function (recall that all functions have only one parameter).
So when you think of bind this this way, you can see that it is similar to piping or
composition.
In fact, you can turn it into an infix operation like this:
let (>>=) m f = pipeInto(m,f)

By the way, this symbol ">>=" is the standard way of writing bind as an infix operator. If you
ever see it used in other F# code, that is probably what it represents.
Going back to the safe divide example, we can now write the workflow on one line, like this:
let divideByWorkflow x y w z =
x |> divideBy y >>= divideBy w >>= divideBy z

You might be wondering exactly how this is different from normal piping or composition? It's
not immediately obvious.
The answer is twofold:
First, the bind function has extra customized behavior for each situation. It is not a
generic function, like pipe or composition.

681

Introducing 'bind'

Second, the input type of the value parameter ( m above) is not necessarily the same
as the output type of the function parameter ( f above), and so one of the things that
bind does is handle this mismatch elegantly so that functions can be chained.
As we will see in the next post, bind generally works with some "wrapper" type. The value
parameter might be of WrapperType<TypeA> , and then the signature of the function parameter
of bind function is always TypeA -> WrapperType<TypeB> .
In the particular case of the bind for safe divide, the wrapper type is Option . The type of
the value parameter ( m above) is Option<int> and the signature of the function parameter
( f above) is int -> Option<int> .
To see bind used in a different context, here is an example of the logging workflow
expressed using a infix bind function:
let (>>=) m f =
printfn "expression is %A" m
f m
let loggingWorkflow =
1 >>= (+) 2 >>= (*) 42 >>= id

In this case, there is no wrapper type. Everything is an int . But even so, bind has the
special behavior that performs the logging behind the scenes.

Option.bind and the "maybe" workflow

revisited
In the F# libraries, you will see Bind functions or methods in many places. Now you know
what they are for!
A particularly useful one is Option.bind , which does exactly what we wrote by hand above,
namely
If the input parameter is None , then don't call the continuation function.
If the input parameter is Some , then do call the continuation function, passing in the
contents of the Some .
Here was our hand-crafted function:

682

Introducing 'bind'

let pipeInto (m,f) =

match m with
| None ->
None
| Some x ->
x |> f

And here is the implementation of Option.bind :

module Option =
let bind f m =
match m with
| None ->
None
| Some x ->
x |> f

There is a moral in this -- don't be too hasty to write your own functions. There may well be
library functions that you can reuse.
Here is the "maybe" workflow, rewritten to use Option.bind :
type MaybeBuilder() =
member this.Bind(m, f) = Option.bind f m
member this.Return(x) = Some x

Reviewing the different approaches so far

We've used four different approaches for the "safe divide" example so far. Let's put them
together side by side and compare them once more.
Note: I have renamed the original pipeInto function to bind , and used Option.bind
instead of our original custom implementation.
First the original version, using an explicit workflow:

683

Introducing 'bind'

module DivideByExplicit =
let divideBy bottom top =
if bottom = 0
then None
else Some(top/bottom)
let divideByWorkflow x y w z =
let a = x |> divideBy y
match a with
| None -> None // give up
| Some a' -> // keep going
let b = a' |> divideBy w
match b with
| None -> None // give up
| Some b' -> // keep going
let c = b' |> divideBy z
match c with
| None -> None // give up
| Some c' -> // keep going
//return
Some c'
// test
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

Next, using our own version of "bind" (a.k.a. "pipeInto")

module DivideByWithBindFunction =
let divideBy bottom top =
if bottom = 0
then None
else Some(top/bottom)
let bind (m,f) =
Option.bind f m
let return' x = Some x
let divideByWorkflow x y w z =
bind (x |> divideBy y, fun a ->
bind (a |> divideBy w, fun b ->
bind (b |> divideBy z, fun c ->
return' c
)))
// test
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

684

Introducing 'bind'

Next, using a computation expression:

module DivideByWithCompExpr =
let divideBy bottom top =
if bottom = 0
then None
else Some(top/bottom)
type MaybeBuilder() =
member this.Bind(m, f) = Option.bind f m
member this.Return(x) = Some x
let maybe = new MaybeBuilder()
let divideByWorkflow x y w z =
maybe
{
let! a = x |> divideBy y
let! b = a |> divideBy w
let! c = b |> divideBy z
return c
}
// test
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

And finally, using bind as an infix operation:

module DivideByWithBindOperator =
let divideBy bottom top =
if bottom = 0
then None
else Some(top/bottom)
let (>>=) m f = Option.bind f m
let divideByWorkflow x y w z =
x |> divideBy y
>>= divideBy w
>>= divideBy z
// test
let good = divideByWorkflow 12 3 2 1
let bad = divideByWorkflow 12 3 0 1

685

Introducing 'bind'

Bind functions turn out to be very powerful. In the next post we'll see that combining bind
with wrapper types creates an elegant way of passing extra information around in the
background.

Exercise: How well do you understand?

Before you move on to the next post, why don't you test yourself to see if you have
understood everything so far?
Here is a little exercise for you.
Part 1 - create a workflow
First, create a function that parses a string into a int:
let strToInt str = ???

and then create your own computation expression builder class so that you can use it in a
workflow, as shown below.
let stringAddWorkflow x y z =
yourWorkflow
{
let! a = strToInt x
let! b = strToInt y
let! c = strToInt z
return a + b + c
}
// test
let good = stringAddWorkflow "12" "3" "2"
let bad = stringAddWorkflow "12" "xyz" "2"

Part 2 -- create a bind function

Once you have the first part working, extend the idea by adding two more functions:
let strAdd str i = ???
let (>>=) m f = ???

And then with these functions, you should be able to write code like this:
let good = strToInt "1" >>= strAdd "2" >>= strAdd "3"
let bad = strToInt "1" >>= strAdd "xyz" >>= strAdd "3"

686

Introducing 'bind'

Summary
Here's a summary of the points covered in this post:
Computation expressions provide a nice syntax for continuation passing, hiding the
chaining logic for us.
bind is the key function that links the output of one step to the input of the next step.

The symbol >>= is the standard way of writing bind as an infix operator.

687

Computation expressions and wrapper types

Computation expressions and wrapper

types
In the previous post, we were introduced to the "maybe" workflow, which allowed us to hide
the messiness of chaining together option types.
A typical use of the "maybe" workflow looked something like this:
let result =
maybe
{
let! anInt = expression of Option<int>
let! anInt2 = expression of Option<int>
return anInt + anInt2
}

As we saw before, there is some apparently strange behavior going on here:

In the let! lines, the expression on the right of the equals is an int option , but the
value on the left is just an int . The let! has "unwrapped" the option before binding it
to the value.
And in the return line, the opposite occurs. The expression being returned is an int ,
but the value of the whole workflow ( result ) is an int option . That is, the return
has "wrapped" the raw value back into an option.
We will follow up these observations in this post, and we will see that this leads to one of the
major uses of computation expressions: namely, to implicitly unwrap and rewrap values that
are stored in some sort of wrapper type.

Another example
Let's look at another example. Say that we are accessing a database, and we want to
capture the result in a Success/Error union type, like this:
type DbResult<'a> =
| Success of 'a
| Error of string

688

Computation expressions and wrapper types

We then use this type in our database access methods. Here are some very simple stubs to
give you an idea of how the DbResult type might be used:
let getCustomerId name =
if (name = "")
then Error "getCustomerId failed"
else Success "Cust42"
let getLastOrderForCustomer custId =
if (custId = "")
then Error "getLastOrderForCustomer failed"
else Success "Order123"
let getLastProductForOrder orderId =
if (orderId = "")
then Error "getLastProductForOrder failed"
else Success "Product456"

Now let's say we want to chain these calls together. First get the customer id from the name,
and then get the order for the customer id, and then get the product from the order.
Here's the most explicit way of doing it. As you can see, we have to have pattern matching
at each step.
let product =
let r1 = getCustomerId "Alice"
match r1 with
| Error _ -> r1
| Success custId ->
let r2 = getLastOrderForCustomer custId
match r2 with
| Error _ -> r2
| Success orderId ->
let r3 = getLastProductForOrder orderId
match r3 with
| Error _ -> r3
| Success productId ->
printfn "Product is %s" productId
r3

Really ugly code. And the top-level flow has been submerged in the error handling logic.
Computation expressions to the rescue! We can write one that handles the branching of
Success/Error behind the scenes:

689

Computation expressions and wrapper types

type DbResultBuilder() =
member this.Bind(m, f) =
match m with
| Error _ -> m
| Success a ->
printfn "\tSuccessful: %s" a
f a
member this.Return(x) =
Success x
let dbresult = new DbResultBuilder()

And with this workflow, we can focus on the big picture and write much cleaner code:
let product' =
dbresult {
let! custId = getCustomerId "Alice"
let! orderId = getLastOrderForCustomer custId
let! productId = getLastProductForOrder orderId
printfn "Product is %s" productId
return productId
}
printfn "%A" product'

And if there are errors, the workflow traps them nicely and tells us where the error was, as in
this example below:
let product'' =
dbresult {
let! custId = getCustomerId "Alice"
let! orderId = getLastOrderForCustomer "" // error!
let! productId = getLastProductForOrder orderId
printfn "Product is %s" productId
return productId
}
printfn "%A" product''

The role of wrapper types in workflows

So now we have seen two workflows (the maybe workflow and the dbresult workflow),
each with their own corresponding wrapper type ( Option<T> and DbResult<T>
respectively).

690

Computation expressions and wrapper types

These are not just special cases. In fact, every computation expression must have an
associated wrapper type. And the wrapper type is often designed specifically to go hand-inhand with the workflow that we want to manage.
The example above demonstrates this clearly. The DbResult type we created is more than
just a simple type for return values; it actually has a critical role in the workflow by "storing"
the current state of the workflow, and whether it is succeeding or failing at each step. By
using the various cases of the type itself, the dbresult workflow can manage the transitions
for us, hiding them from view and enabling us to focus on the big picture.
We'll learn how to design a good wrapper type later in the series, but first let's look at how
they are manipulated.

Bind and Return and wrapper types

Let's look again at the definition of the Bind and Return methods of a computation
expression.
We'll start off with the easy one, Return . The signature of Return as documented on
MSDN is just this:
member Return : 'T -> M<'T>

In other words, for some type T , the Return method just wraps it in the wrapper type.
Note: In signatures, the wrapper type is normally called M , so M<int> is the wrapper type
applied to int and M<string> is the wrapper type applied to string , and so on.
And we've seen two examples of this usage. The maybe workflow returns a Some , which is
an option type, and the dbresult workflow returns Success , which is part of the DbResult
type.
// return for the maybe workflow
member this.Return(x) =
Some x
// return for the dbresult workflow
member this.Return(x) =
Success x

Now let's look at Bind . The signature of Bind is this:

member Bind : M<'T> * ('T -> M<'U>) -> M<'U>

691

Computation expressions and wrapper types

It looks complicated, so let's break it down. It takes a tuple M<'T> * ('T -> M<'U>) and
returns a M<'U> , where M<'U> means the wrapper type applied to type U .
The tuple in turn has two parts:
M<'T> is a wrapper around type T , and
'T -> M<'U> is a function that takes a unwrapped T and creates a wrapped U .

In other words, what Bind does is:

Take a wrapped value.
Unwrap it and do any special "behind the scenes" logic.
Then, optionally apply the function to the unwrapped value to create a new wrapped
value.
Even if the function is not applied, Bind must still return a wrapped U .
With this understanding, here are the Bind methods that we have seen already:
// return for the maybe workflow
member this.Bind(m,f) =
match m with
| None -> None
| Some x -> f x
// return for the dbresult workflow
member this.Bind(m, f) =
match m with
| Error _ -> m
| Success x ->
printfn "\tSuccessful: %s" x
f x

Look over this code and make sure that you understand why these methods do indeed
follow the pattern described above.
Finally, a picture is always useful. Here is a diagram of the various types and functions:

For Bind , we start with a wrapped value ( m here), unwrap it to a raw value of type

692

Computation expressions and wrapper types

T , and then (maybe) apply the function f to it to get a wrapped value of type U .

For Return , we start with a value ( x here), and simply wrap it.

The type wrapper is generic

Note that all the functions use generic types ( T and U ) other than the wrapper type itself,
which must be the same throughout. For example, there is nothing stopping the maybe
binding function from taking an int and returning a Option<string> , or taking a string
and then returning an Option<bool> . The only requirement is that it always return an
Option<something> .

To see this, we can revisit the example above, but rather than using strings everywhere, we
will create special types for the customer id, order id, and product id. This means that each
step in the chain will be using a different type.
We'll start with the types again, this time defining CustomerId , etc.
type DbResult<'a> =
| Success of 'a
| Error of string
type CustomerId = CustomerId of string
type OrderId = OrderId of int
type ProductId = ProductId of string

The code is almost identical, except for the use of the new types in the Success line.
let getCustomerId name =
if (name = "")
then Error "getCustomerId failed"
else Success (CustomerId "Cust42")
let getLastOrderForCustomer (CustomerId custId) =
if (custId = "")
then Error "getLastOrderForCustomer failed"
else Success (OrderId 123)
let getLastProductForOrder (OrderId orderId) =
if (orderId = 0)
then Error "getLastProductForOrder failed"
else Success (ProductId "Product456")

Here's the long-winded version again.

693

Computation expressions and wrapper types

let product =
let r1 = getCustomerId "Alice"
match r1 with
| Error e -> Error e
| Success custId ->
let r2 = getLastOrderForCustomer custId
match r2 with
| Error e -> Error e
| Success orderId ->
let r3 = getLastProductForOrder orderId
match r3 with
| Error e -> Error e
| Success productId ->
printfn "Product is %A" productId
r3

There are a couple of changes worth discussing:

First, the printfn at the bottom uses the "%A" format specifier rather than "%s". This
is required because the ProductId type is a union type now.
More subtly, there seems to be unnecessary code in the error lines. Why write | Error
e -> Error e ? The reason is that the incoming error that is being matched against is of

type DbResult<CustomerId> or DbResult<OrderId> , but the return value must be of type

DbResult<ProductId> . So, even though the two Error s look the same, they are

actually of different types.

Next up, the builder, which hasn't changed at all except for the | Error e -> Error e line.
type DbResultBuilder() =
member this.Bind(m, f) =
match m with
| Error e -> Error e
| Success a ->
printfn "\tSuccessful: %A" a
f a
member this.Return(x) =
Success x
let dbresult = new DbResultBuilder()

Finally, we can use the workflow as before.

694

Computation expressions and wrapper types

let product' =
dbresult {
let! custId = getCustomerId "Alice"
let! orderId = getLastOrderForCustomer custId
let! productId = getLastProductForOrder orderId
printfn "Product is %A" productId
return productId
}
printfn "%A" product'

At each line, the returned value is of a different type

( DbResult<CustomerId> , DbResult<OrderId> , etc), but because they have the same wrapper
type in common, the bind works as expected.
And finally, here's the workflow with an error case.
let product'' =
dbresult {
let! custId = getCustomerId "Alice"
let! orderId = getLastOrderForCustomer (CustomerId "") //error
let! productId = getLastProductForOrder orderId
printfn "Product is %A" productId
return productId
}
printfn "%A" product''

Composition of computation expressions

We've seen that every computation expression must have an associated wrapper type. This
wrapper type is used in both Bind and Return , which leads to a key benefit:
the output of a Return can be fed to the input of a Bind
In other words, because a workflow returns a wrapper type, and because let! consumes a
wrapper type, you can put a "child" workflow on the right hand side of a let! expression.
For example, say that you have a workflow called myworkflow . Then you can write the
following:

695

Computation expressions and wrapper types

let subworkflow1 = myworkflow { return 42 }

let subworkflow2 = myworkflow { return 43 }
let aWrappedValue =
myworkflow {
let! unwrappedValue1 = subworkflow1
let! unwrappedValue2 = subworkflow2
return unwrappedValue1 + unwrappedValue2
}

Or you can even "inline" them, like this:

let aWrappedValue =
myworkflow {
let! unwrappedValue1 = myworkflow {
let! x = myworkflow { return 1 }
return x
}
let! unwrappedValue2 = myworkflow {
let! y = myworkflow { return 2 }
return y
}
return unwrappedValue1 + unwrappedValue2
}

If you have used the async workflow, you probably have done this already, because an
async workflow typically contains other asyncs embedded in it:
let a =
async {
let! x = doAsyncThing // nested workflow
let! y = doNextAsyncThing x // nested workflow
return x + y
}

Introducing "ReturnFrom"
We have been using return as a way of easily wrapping up an unwrapped return value.
But sometimes we have a function that already returns a wrapped value, and we want to
return it directly. return is no good for this, because it requires an unwrapped type as input.
The solution is a variant on return called return! , which takes a wrapped type as input
and returns it.

696

Computation expressions and wrapper types

The corresponding method in the "builder" class is called ReturnFrom . Typically the
implementation just returns the wrapped type "as is" (although of course, you can always
add extra logic behind the scenes).
Here is a variant on the "maybe" workflow to show how it can be used:
type MaybeBuilder() =
member this.Bind(m, f) = Option.bind f m
member this.Return(x) =
printfn "Wrapping a raw value into an option"
Some x
member this.ReturnFrom(m) =
printfn "Returning an option directly"
m
let maybe = new MaybeBuilder()

And here it is in use, compared with a normal return .

// return an int
maybe { return 1 }
// return an Option
maybe { return! (Some 2) }

For a more realistic example, here is return! used in conjunction with divideBy :
// using return
maybe
{
let! x = 12 |> divideBy 3
let! y = x |> divideBy 2
return y // return an int
}
// using return!
maybe
{
let! x = 12 |> divideBy 3
return! x |> divideBy 2 // return an Option
}

Summary
This post introduced wrapper types and how they related to Bind , Return and
ReturnFrom , the core methods of any builder class.

697

Computation expressions and wrapper types

In the next post, we'll continue to look at wrapper types, including using lists as wrapper
types.

698

More on wrapper types

In the previous post, we looked at the concept of "wrapper types" and their relation to
computation expressions. In this post, we'll investigate what types are suitable for being
wrapper types.

What kinds of types can be wrapper types?

If every computation expression must have an associated wrapper type, then what kinds of
type can be used as wrapper types? Are there any special constraints or limitations that
apply?
There is one general rule, which is:
Any type with a generic parameter can be used as a wrapper type
So for example, you can use Option<T> , DbResult<T> , etc., as wrapper types, as we have
seen. And you can use wrapper types that restrict the type parameter, such as
Vector<int> .

But what about other generic types like List<T> or IEnumerable<T> ? Surely they can't be
used? Actually, yes, they can be used! We'll see how shortly.

Can non-generic wrapper types work?

Is it possible to use a wrapper type that does not have a generic parameter?
For example, we saw in an earlier example an attempt to do addition on strings, like this:
"1" + "2" . Can't we be clever and treat string as a wrapper type for int in this case?

That would be cool, yes?

Let's try. We can use the signatures of Bind and Return to guide our implementation.
Bind takes a tuple. The first part of the tuple is the wrapped type ( string in this

case), and the second part of the tuple is a function that takes an unwrapped type and
converts it to a wrapped type. In this case, that would be int -> string .
Return takes an unwrapped type ( int in this case) and converts it to a wrapped type.

So in this case, the signature of Return would be int -> string .

How does this guide the implementation?

699

More on wrapper types

The implementation of the "rewrapping" function, int -> string , is easy. It is just
"toString" on an int.
The bind function has to unwrap a string to an int, and then pass it to the function. We
can use int.Parse for that.
But what happens if the bind function can't unwrap a string, because it is not a valid
number? In this case, the bind function must still return a wrapped type (a string), so we
can just return a string such as "error".
Here's the implementation of the builder class:
type StringIntBuilder() =
member this.Bind(m, f) =
let b,i = System.Int32.TryParse(m)
match b,i with
| false,_ -> "error"
| true,i -> f i
member this.Return(x) =
sprintf "%i" x
let stringint = new StringIntBuilder()

Now we can try using it:

let good =
stringint {
let! i = "42"
let! j = "43"
return i+j
}
printfn "good=%s" good

And what happens if one of the strings is invalid?

let bad =
stringint {
let! i = "42"
let! j = "xxx"
return i+j
}
printfn "bad=%s" bad

That looks really good -- we can treat strings as ints inside our workflow!
But hold on, there is a problem.

700

More on wrapper types

Let's say we give the workflow an input, unwrap it (with let! ) and then immediately rewrap
it (with return ) without doing anything else. What should happen?
let g1 = "99"
let g2 = stringint {
let! i = g1
return i
}
printfn "g1=%s g2=%s" g1 g2

No problem. The input g1 and the output g2 are the same value, as we would expect.
But what about the error case?
let b1 = "xxx"
let b2 = stringint {
let! i = b1
return i
}
printfn "b1=%s b2=%s" b1 b2

In this case we have got some unexpected behavior. The input b1 and the output b2 are
not the same value. We have introduced an inconsistency.
Would this be a problem in practice? I don't know. But I would avoid it and use a different
approach, like options, that are consistent in all cases.

Rules for workflows that use wrapper types

Here's a question? What is the difference between these two code fragments, and should
they behave differently?
// fragment before refactoring
myworkflow {
let wrapped = // some wrapped value
let! unwrapped = wrapped
return unwrapped
}
// refactored fragment
myworkflow {
let wrapped = // some wrapped value
return! wrapped
}

701

More on wrapper types

The answer is no, they should not behave differently. The only difference is that in the
second example, the unwrapped value has been refactored away and the wrapped value is
returned directly.
But as we just saw in the previous section, you can get inconsistencies if you are not careful.
So, any implementation you create should be sure to follow some standard rules, which are:
Rule 1: If you start with an unwrapped value, and then you wrap it (using return ),
then unwrap it (using bind ), you should always get back the original unwrapped
value.
This rule and the next are about not losing information as you wrap and unwrap the values.
Obviously, a sensible thing to ask, and required for refactoring to work as expected.
In code, this would be expressed as something like this:
myworkflow {
let originalUnwrapped = something
// wrap it
let wrapped = myworkflow { return originalUnwrapped }
// unwrap it
let! newUnwrapped = wrapped
// assert they are the same
assertEqual newUnwrapped originalUnwrapped
}

Rule 2: If you start with a wrapped value, and then you unwrap it (using bind ), then
wrap it (using return ), you should always get back the original wrapped value.
This is the rule that the stringInt workflow broke above. As with rule 1, this should
obviously be a requirement.
In code, this would be expressed as something like this:

702

More on wrapper types

myworkflow {
let originalWrapped = something
let newWrapped = myworkflow {
// unwrap it
let! unwrapped = originalWrapped
// wrap it
return unwrapped
}
// assert they are the same
assertEqual newWrapped originalWrapped
}

Rule 3: If you create a child workflow, it must produce the same result as if you had
"inlined" the logic in the main workflow.
This rule is required for composition to behave properly, and again, "extraction" refactoring
will only work correctly if this is true.
In general, you will get this for free if you follow some guidelines (which will be explained in a
later post).
In code, this would be expressed as something like this:
// inlined
let result1 = myworkflow {
let! x = originalWrapped
let! y = f x // some function on x
return! g y // some function on y
}
// using a child workflow ("extraction" refactoring)
let result2 = myworkflow {
let! y = myworkflow {
let! x = originalWrapped
return! f x // some function on x
}
return! g y // some function on y
}
// rule
assertEqual result1 result2

Lists as wrapper types

703

More on wrapper types

I said earlier that types like List<T> or IEnumerable<T> can be used as wrapper types. But
how can this be? There is no one-to-one correspondence between the wrapper type and the
unwrapped type!
This is where the "wrapper type" analogy becomes a bit misleading. Instead, let's go back to
thinking of bind as a way of connecting the output of one expression with the input of
another.
As we have seen, the bind function "unwraps" the type, and applies the continuation
function to the unwrapped value. But there is nothing in the definition that says that there
has to be only one unwrapped value. There is no reason that we can't apply the continuation
function to each item of the list in turn.
In other words, we should be able to write a bind that takes a list and a continuation
function, where the continuation function processes one element at a time, like this:
bind( [1;2;3], fun elem -> // expression using a single element )

And with this concept, we should be able to chain some binds together like this:
let add =
bind( [1;2;3], fun elem1 ->
bind( [10;11;12], fun elem2 ->
elem1 + elem2
))

But we've missed something important. The continuation function passed into bind is
required to have a certain signature. It takes an unwrapped type, but it produces a wrapped
type.
In other words, the continuation function must always create a new list as its result.
bind( [1;2;3], fun elem -> // expression using a single element, returning a list )

And the chained example would have to be written like this, with the elem1 + elem2 result
turned into a list:
let add =
bind( [1;2;3], fun elem1 ->
bind( [10;11;12], fun elem2 ->
[elem1 + elem2] // a list!
))

704

More on wrapper types

So the logic for our bind method now looks like this:
let bind(list,f) =
// 1) for each element in list, apply f
// 2) f will return a list (as required by its signature)
// 3) the result is a list of lists

We have another issue now. Bind itself must produce a wrapped type, which means that
the "list of lists" is no good. We need to turn them back into a simple "one-level" list.
But that is easy enough -- there is a list module function that does just that, called concat .
So putting it together, we have this:
let bind(list,f) =
list
|> List.map f
|> List.concat
let added =
bind( [1;2;3], fun elem1 ->
bind( [10;11;12], fun elem2 ->
// elem1 + elem2 // error.
[elem1 + elem2] // correctly returns a list.
))

Now that we understand how the bind works on its own, we can create a "list workflow".
Bind applies the continuation function to each element of the passed in list, and then

flattens the resulting list of lists into a one-level list. List.collect is a library function
that does exactly that.
Return converts from unwrapped to wrapped. In this case, that just means wrapping a

single element in a list.

type ListWorkflowBuilder() =
member this.Bind(list, f) =
list |> List.collect f
member this.Return(x) =
[x]
let listWorkflow = new ListWorkflowBuilder()

Here is the workflow in use:

705

More on wrapper types

let added =
listWorkflow {
let! i = [1;2;3]
let! j = [10;11;12]
return i+j
}
printfn "added=%A" added
let multiplied =
listWorkflow {
let! i = [1;2;3]
let! j = [10;11;12]
return i*j
}
printfn "multiplied=%A" multiplied

And the results show that every element in the first collection has been combined with every
element in the second collection:
val added : int list = [11; 12; 13; 12; 13; 14; 13; 14; 15]
val multiplied : int list = [10; 11; 12; 20; 22; 24; 30; 33; 36]

That's quite amazing really. We have completely hidden the list enumeration logic, leaving
just the workflow itself.

Syntactic sugar for "for"

If we treat lists and sequences as a special case, we can add some nice syntactic sugar to
replace let! with something a bit more natural.
What we can do is replace the let! with a for..in..do expression:
// let version
let! i = [1;2;3] in [some expression]
// for..in..do version
for i in [1;2;3] do [some expression]

Both variants mean exactly the same thing, they just look different.
To enable the F# compiler to do this, we need to add a For method to our builder class. It
generally has exactly the same implementation as the normal Bind method, but is required
to accept a sequence type.

706

More on wrapper types

type ListWorkflowBuilder() =
member this.Bind(list, f) =
list |> List.collect f
member this.Return(x) =
[x]
member this.For(list, f) =
this.Bind(list, f)
let listWorkflow = new ListWorkflowBuilder()

And here is how it is used:

let multiplied =
listWorkflow {
for i in [1;2;3] do
for j in [10;11;12] do
return i*j
}
printfn "multiplied=%A" multiplied

LINQ and the "list workflow"

Does the for element in collection do look familiar? It is very close to the from element in
collection ... syntax used by LINQ. And indeed LINQ uses basically the same technique

to convert from a query expression syntax like from element in collection ... to actual
method calls behine the scenes.
In F#, as we saw, the bind uses the List.collect function. The equivalent of
List.collect in LINQ is the SelectMany extension method. And once you understand how
SelectMany works, you can implement the same kinds of queries yourself. Jon Skeet has

written a helpful blog post explaining this.

The identity "wrapper type"

So we've seen a number of wrapper types in this post, and have said that every computation
expression must have an associated wrapper type.
But what about the logging example in the previous post? There was no wrapper type there.
There was a let! that did things behind the scenes, but the input type was the same as
the output type. The type was left unchanged.

707

More on wrapper types

The short answer to this is that you can treat any type as its own "wrapper". But there is
another, deeper way to understand this.
Let's step back and consider what a wrapper type definition like List<T> really means.
If you have a type such as List<T> , it is in fact not a "real" type at all. List<int> is a real
type, and List<string> is a real type. But List<T> on its own is incomplete. It is missing
the parameter it needs to become a real type.
One way to think about List<T> is that it is a function, not a type. It is a function in the
abstract world of types, rather than the concrete world of normal values, but just like any
function it maps values to other values, except in this case, the input values are types (say
int or string ) and the output values are other types ( List<int> and List<string> ).

And like any function it takes a parameter, in this case a "type parameter". Which is why the
concept that .NET developers call "generics" is known as "parametric polymorphism" in
computer science terminology.
Once we grasp the concept of functions that generate one type from another type (called
"type constructors"), we can see that what we really mean by a "wrapper type" is just a type
constructor.
But if a "wrapper type" is just a function that maps one type to another type, surely a function
that maps a type to the same type fits into this category? And indeed it does. The "identity"
function for types fits our definition and can be used as a wrapper type for computation
expressions.
Going back to some real code then, we can define the "identity workflow" as the simplest
possible implementation of a workflow builder.
type IdentityBuilder() =
member this.Bind(m, f) = f m
member this.Return(x) = x
member this.ReturnFrom(x) = x
let identity = new IdentityBuilder()
let result = identity {
let! x = 1
let! y = 2
return x + y
}

With this in place, you can see that the logging example discussed earlier is just the identity
workflow with some logging added in.

708

More on wrapper types

Summary
Another long post, and we covered a lot of topics, but I hope that the role of wrapper types is
now clearer. We will see how the wrapper types can be used in practice when we come to
look at common workflows such as the "writer workflow" and the "state workflow" later in this
series.
Here's a summary of the points covered in this post:
A major use of computation expressions is to unwrap and rewrap values that are stored
in some sort of wrapper type.
You can easily compose computation expressions, because the output of a Return can
be fed to the input of a Bind .
Every computation expression must have an associated wrapper type.
Any type with a generic parameter can be used as a wrapper type, even lists.
When creating workflows, you should ensure that your implementation conforms to the
three sensible rules about wrapping and unwrapping and composition.

709

Implementing a builder: Zero and Yield

Having covered bind and continuations, and the use of wrapper types, we're finally ready to
take on the full set of methods associated with "builder" classes.
If you look at the MSDN documentation, you'll see not just Bind and Return , but also
other strangely named methods like Delay and Zero . What are they for? That's what this
and the next few posts will answer.

The plan of action

To demonstrate how to create a builder class, we will create a custom workflow which uses
all of the possible builder methods.
But rather than starting at the top and trying to explain what these methods mean without
context, we'll work from the bottom up, starting with a simple workflow and adding methods
only as needed to solve a problem or an error. In the process, you'll come to understand
how F# processes computation expressions in detail.
The outline of this process is:
Part 1: In this first part, we'll look at what methods are needed for a basic workflow.
We'll introduce Zero , Yield , Combine and For .
Part 2: Next, we'll look at how to delay the execution of your code, so that it is only
evaluated when needed. We'll introduce Delay and Run , and look at lazy
computations.
Part 3: Finally, we'll cover the rest of the methods: While , Using , and exception
handling.

Before we get started

Before we dive into creating the workflow, here are some general comments.

The documentation for computation expressions

First, as you might have noticed, the MSDN documentation for computation expressions is
meagre at best, and although not inaccurate, can be misleading. For example, the
signatures of the builder methods are more flexible than they appear to be, and this can be

710

Implementing a builder: Zero and Yield

used to implement some features that might not be obvious if you work from the
documentation alone. We will show an example of this later.
If you want more detailed documentation, there are two sources I can recommend. For an
detailed overview of the concepts behind computation expressions, a great resource is the
paper "The F# Expression Zoo" by Tomas Petricek and Don Syme. And for the most
accurate up-to-date technical documentation, you should read the F# language specification,
which has a section on computation expressions.

Wrapped and unwrapped types

When you are trying to understand the signatures as documented, remember that what I
have been calling the "unwrapped" type is normally written as 'T and the "wrapped" type is
normally written M<'T> . That is, when you see that the Return method has the signature
'T -> M<'T> it means Return takes an unwrapped type and returns a wrapped type.

As I have in the earlier posts in this series, I will continue to use "unwrapped" and "wrapped"
to describe the relationship between these types, but as we move forward these terms will
be stretched to the breaking point, so I will also start using other terminology, such as
"computation type" instead of "wrapped type". I hope that when we reach this point, the
reason for the change will be clear and understandable.
Also, in my examples, I will generally try to keep things simple by using code such as:
let! x = ...wrapped type value...

But this is actually an oversimplification. To be precise, the "x" can be any pattern not just a
single value, and the "wrapped type" value can, of course, be an expression that evaluates
to a wrapped type. The MSDN documentation uses this more precise approach. It uses
"pattern" and "expression" in the definitions, such as let! pattern = expr in cexpr .
Here are some examples of using patterns and expressions in a maybe computation
expression, where Option is the wrapped type, and the right hand side expressions are
options :

// let! pattern = expr in cexpr

maybe {
let! x,y = Some(1,2)
let! head::tail = Some( [1;2;3] )
// etc
}

711

Implementing a builder: Zero and Yield

Having said this, I will continue to use the oversimplified examples, so as not to add extra
complication to an already complicated topic!

Implementing special methods in the builder class (or not)

The MSDN documentation shows that each special operation (such as for..in , or yield )
is translated into one or more calls to methods in the builder class.
There is not always a one-to-one correspondence, but generally, to support the syntax for a
special operation, you must implement a corresponding method in the builder class,
otherwise the compiler will complain and give you an error.
On the other hand, you do not need to implement every single method if you don't need the
syntax. For example, we have already implemented the maybe workflow quite nicely by only
implementing the two methods Bind and Return . We don't need to implement Delay ,
Use , and so on, if we don't need to use them.

To see what happens if you have not implemented a method, let's try to use the
for..in..do syntax in our maybe workflow like this:

maybe { for i in [1;2;3] do i }

We will get the compiler error:

This control construct may only be used if the computation expression builder defines
a 'For' method

Sometimes you get will errors that might be cryptic unless you know what is going on behind
the scenes. For example, if you forget to put return in your workflow, like this:
maybe { 1 }

You will get the compiler error:

This control construct may only be used if the computation expression builder defines
a 'Zero' method

You might be asking: what is the Zero method? And why do I need it? The answer to that is
coming right up.

Operations with and without '!'

712

Implementing a builder: Zero and Yield

Obviously, many of the special operations come in pairs, with and without a "!" symbol. For
example: let and let! (pronounced "let-bang"), return and return! , yield and
yield! and so on.

The difference is easy to remember when you realize that the operations without a "!" always
have unwrapped types on the right hand side, while the ones with a "!" always have wrapped
types.
So for example, using the maybe workflow, where Option is the wrapped type, we can
compare the different syntaxes:
let x = 1 // 1 is an "unwrapped" type
let! x = (Some 1) // Some 1 is a "wrapped" type
return 1 // 1 is an "unwrapped" type
return! (Some 1) // Some 1 is a "wrapped" type
yield 1 // 1 is an "unwrapped" type
yield! (Some 1) // Some 1 is a "wrapped" type

The "!" versions are particularly important for composition, because the wrapped type can be
the result of another computation expression of the same type.
let! x = maybe {...) // "maybe" returns a "wrapped" type
// bind another workflow of the same type using let!
let! aMaybe = maybe {...) // create a "wrapped" type
return! aMaybe // return it
// bind two child asyncs inside a parent async using let!
let processUri uri = async {
let! html = webClient.AsyncDownloadString(uri)
let! links = extractLinks html
... etc ...
}

Diving in - creating a minimal implementation

of a workflow
Let's start! We'll begin by creating a minimal version of the "maybe" workflow (which we'll
rename as "trace") with every method instrumented, so we can see what is going on. We'll
use this as our testbed throughout this post.
Here's the code for the first version of the trace workflow:

713

Implementing a builder: Zero and Yield

type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Return(x) =
printfn "Returning a unwrapped %A as an option" x
Some x
member this.ReturnFrom(m) =
printfn "Returning an option (%A) directly" m
m
// make an instance of the workflow
let trace = new TraceBuilder()

Nothing new here, I hope. We have already seen all these methods before.
Now let's run some sample code through it:
trace {
return 1
} |> printfn "Result 1: %A"
trace {
return! Some 2
} |> printfn "Result 2: %A"
trace {
let! x = Some 1
let! y = Some 2
return x + y
} |> printfn "Result 3: %A"
trace {
let! x = None
let! y = Some 1
return x + y
} |> printfn "Result 4: %A"

Everything should work as expected, in particular, you should be able to see that the use of
None in the 4th example caused the next two lines ( let! y = ... return x+y ) to be

skipped and the result of the whole expression was None .

714

Implementing a builder: Zero and Yield

Introducing "do!"
Our expression supports let! , but what about do! ?
In normal F#, do is just like let , except that the expression doesn't return anything useful
(namely, a unit value).
Inside a computation expression, do! is very similar. Just as let! passes a wrapped
result to the Bind method, so does do! , except that in the case of do! the "result" is the
unit value, and so a wrapped version of unit is passed to the bind method.
Here is a simple demonstration using the trace workflow:
trace {
do! Some (printfn "...expression that returns unit")
do! Some (printfn "...another expression that returns unit")
let! x = Some (1)
return x
} |> printfn "Result from do: %A"

Here is the output:

...expression that returns unit
Binding with Some(<null>). Continuing
...another expression that returns unit
Binding with Some(<null>). Continuing
Binding with Some(1). Continuing
Returning a unwrapped 1 as an option
Result from do: Some 1

You can verify for yourself that a unit option is being passed to Bind as a result of each
do! .

Introducing "Zero"
What is the smallest computation expression you can get away with? Let's try nothing at all:
trace {
} |> printfn "Result for empty: %A"

We get an error immediately:

715

Implementing a builder: Zero and Yield

This value is not a function and cannot be applied

Fair enough. If you think about it, it doesn't make sense to have nothing at all in a
computation expression. After all, it's purpose is to chain expressions together.
Next, what about a simple expression with no let! or return ?
trace {
printfn "hello world"
} |> printfn "Result for simple expression: %A"

Now we get a different error:

This control construct may only be used if the computation expression builder defines
a 'Zero' method

So why is the Zero method needed now but we haven't needed it before? The answer is
that in this particular case we haven't returned anything explicitly, yet the computation
expression as a whole must return a wrapped value. So what value should it return?
In fact, this situation will occur any time the return value of the computation expression has
not been explicitly given. The same thing happens if you have an if..then expression
without an else clause.
trace {
if false then return 1
} |> printfn "Result for if without else: %A"

In normal F# code, an "if..then" without an "else" would result in a unit value, but in a
computation expression, the particular return value must be a member of the wrapped type,
and the compiler does not know what value this is.
The fix is to tell the compiler what to use -- and that is the purpose of the Zero method.

What value should you use for Zero?

So which value should you use for Zero ? It depends on the kind of workflow you are
creating.
Here are some guidelines that might help:
Does the workflow have a concept of "success" or "failure"? If so, use the "failure"
value for Zero . For example, in our trace workflow, we use None to indicate failure,

716

Implementing a builder: Zero and Yield

and so we can use None as the Zero value.

Does the workflow have a concept of "sequential processing"? That is, in your
workflow you do one step and then another, with some processing behind the scenes.
In normal F# code, an expression that did return anything explicitly would evaluate to
unit. So to parallel this case, your Zero should be the wrapped version of unit. For
example, in a variant on an option-based workflow, we might use Some () to mean
Zero (and by the way, this would always be the same as Return () as well).

Is the workflow primarily concerned with manipulating data structures? If so,

Zero should be the "empty" data structure. For example, in a "list builder" workflow, we

would use the empty list as the Zero value.

The Zero value also has an important role to play when combining wrapped types. So stay
tuned, and we'll revisit Zero in the next post.

A Zero implementation
So now let's extend our testbed class with a Zero method that returns None , and try again.
type TraceBuilder() =
// other members as before
member this.Zero() =
printfn "Zero"
None
// make a new instance
let trace = new TraceBuilder()
// test
trace {
printfn "hello world"
} |> printfn "Result for simple expression: %A"
trace {
if false then return 1
} |> printfn "Result for if without else: %A"

The test code makes it clear that Zero is being called behind the scenes. And None is the
return value for the expression as whole. Note: None may print out as <null> . You can
ignore this.

Do you always need a Zero?

Remember, you not required to have a Zero , but only if it makes sense in the context of the
workflow. For example seq does not allow zero, but async does:

717

Implementing a builder: Zero and Yield

let s = seq {printfn "zero" } // Error

let a = async {printfn "zero" } // OK

Introducing "Yield"
In C#, there is a "yield" statement that, within an iterator, is used to return early and then
picks up where you left off when you come back.
And looking at the docs, there is a "yield" available in F# computation expressions as well.
What does it do? Let's try it and see.
trace {
yield 1
} |> printfn "Result for yield: %A"

And we get the error:

This control construct may only be used if the computation expression builder defines
a 'Yield' method

No surprise there. So what should the implementation of "yield" method look like? The
MSDN documentation says that it has the signature 'T -> M<'T> , which is exactly the same
as the signature for the Return method. It must take an unwrapped value and wrap it.
So let's implement it the same way as Return and retry the test expression.
type TraceBuilder() =
// other members as before
member this.Yield(x) =
printfn "Yield an unwrapped %A as an option" x
Some x
// make a new instance
let trace = new TraceBuilder()
// test
trace {
yield 1
} |> printfn "Result for yield: %A"

This works now, and it seems that it can be used as an exact substitute for return .

718

Implementing a builder: Zero and Yield

There is a also a YieldFrom method that parallels the ReturnFrom method. And it behaves
the same way, allowing you to yield a wrapped value rather than a unwrapped one.
So let's add that to our list of builder methods as well:
type TraceBuilder() =
// other members as before
member this.YieldFrom(m) =
printfn "Yield an option (%A) directly" m
m
// make a new instance
let trace = new TraceBuilder()
// test
trace {
yield! Some 1
} |> printfn "Result for yield!: %A"

At this point you might be wondering: if return and yield are basically the same thing,
why are there two different keywords? The answer is mainly so that you can enforce
appropriate syntax by implementing one but not the other. For example, the seq expression
does allow yield but doesn't allow return , while the async does allow return , but does
not allow yield , as you can see from the snippets below.
let s = seq {yield 1} // OK
let s = seq {return 1} // error
let a = async {return 1} // OK
let a = async {yield 1} // error

In fact, you could create slightly different behavior for return vs. yield , so that, for
example, using return stops the rest of the computation expression from being evaluated,
while yield doesn't.
More generally, of course, yield should be used for sequence/enumeration semantics,
while return is normally used once per expression. (We'll see how yield can be used
multiple times in the next post.)

Revisiting "For"

719

Implementing a builder: Zero and Yield

We talked about the for..in..do syntax in the last post. So now let's revisit the "list builder"
that we discussed earlier and add the extra methods. We already saw how to define Bind
and Return for a list in a previous post, so we just need to implement the additional
methods.
The Zero method just returns an empty list.
The Yield method can be implemented in the same way as Return .
The For method can be implemented the same as Bind .
type ListBuilder() =
member this.Bind(m, f) =
m |> List.collect f
member this.Zero() =
printfn "Zero"
[]
member this.Return(x) =
printfn "Return an unwrapped %A as a list" x
[x]
member this.Yield(x) =
printfn "Yield an unwrapped %A as a list" x
[x]
member this.For(m,f) =
printfn "For %A" m
this.Bind(m,f)
// make an instance of the workflow
let listbuilder = new ListBuilder()

And here is the code using let! :

listbuilder {
let! x = [1..3]
let! y = [10;20;30]
return x + y
} |> printfn "Result: %A"

And here is the equivalent code using for :

listbuilder {
for x in [1..3] do
for y in [10;20;30] do
return x + y
} |> printfn "Result: %A"

720

Implementing a builder: Zero and Yield

You can see that both approaches give the same result.

Summary
In this post, we've seen how to implement the basic methods for a simple computation
expression.
Some points to reiterate:
For simple expressions you don't need to implement all the methods.
Things with bangs have wrapped types on the right hand side.
Things without bangs have unwrapped types on the right hand side.
You need to implement Zero if you want a workflow that doesn't explicitly return a
value.
Yield is basically equivalent to Return , but Yield should be used for

sequence/enumeration semantics.
For is basically equivalent to Bind in simple cases.

In the next post, we'll look at what happens when we need to combine multiple values.

721

Implementing a builder: Combine

In this post we're going to look at returning multiple values from a computation expression
using the Combine method.

The story so far...

So far, our expression builder class looks like this:
type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Return(x) =
printfn "Returning a unwrapped %A as an option" x
Some x
member this.ReturnFrom(m) =
printfn "Returning an option (%A) directly" m
m
member this.Zero() =
printfn "Zero"
None
member this.Yield(x) =
printfn "Yield an unwrapped %A as an option" x
Some x
member this.YieldFrom(m) =
printfn "Yield an option (%A) directly" m
m
// make an instance of the workflow
let trace = new TraceBuilder()

And this class has worked fine so far. But we are about to run into a problem...

A problem with two 'yields'

722

Implementing a builder: Combine

Previously, we saw how yield could be used to return values just like return .
Normally, yield is not used just once, of course, but multiple times in order to return values
at different stages of a process such as an enumeration. So let's try that:
trace {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"

But uh-oh, we get an error message:

This control construct may only be used if the computation expression builder defines
a 'Combine' method.

And if you use return instead of yield , you get the same error.
trace {
return 1
return 2
} |> printfn "Result for return then return: %A"

And this problem occurs in other contexts too. For example, if we want to do something and
then return, like this:
trace {
if true then printfn "hello"
return 1
} |> printfn "Result for if then return: %A"

We get the same error message about a missing 'Combine' method.

Understanding the problem

So what's going on here?
To understand, let's go back to the behind-the-scenes view of the computation expression.
We have seen that return and yield are really just the last step in a series of
continuations, like this:

723

Implementing a builder: Combine

Bind(1,fun x ->
Bind(2,fun y ->
Bind(x + y,fun z ->
Return(z) // or Yield

You can think of return (or yield ) as "resetting" the indentation, if you like. So when we
return/yield and then return/yield again, we are generating code like this:

Bind(1,fun x ->
Bind(2,fun y ->
Bind(x + y,fun z ->
Yield(z)
// start a new expression
Bind(3,fun w ->
Bind(4,fun u ->
Bind(w + u,fun v ->
Yield(v)

But really this can be simplified to:

let value1 = some expression
let value2 = some other expression

In other words, we now have two values in our computation expression. And then the
obvious question is, how should these two values be combined to give a single result for the
computation expression as a whole?
This is a very important point. Return and yield do not generate an early return from a
computation expression. No, the entire computation expression, all the way to the last
curly brace, is always evaluated and results in a single value. Let me repeat that. Every part
of the computation expression is always evaluated -- there is no short circuiting going on. If
we want to short circuit and return early, we have to write our own code to do that (and we'll
see how to do that later).
So, back to the pressing question. We have two expressions resulting in two values: how
should those multiple values be combined into one?

Introducing "Combine"
The answer is by using the Combine method, which takes two wrapped values and
combines them to make another wrapped value. Exactly how this works is up to us.

724

Implementing a builder: Combine

In our case, we are dealing specifically with int options , so one simple implementation
that leaps to mind it just to add the numbers together. Each parameter is an option of
course (the wrapped type), so we need to pick them apart and handle the four possible
cases:
type TraceBuilder() =
// other members as before
member this.Combine (a,b) =
match a,b with
| Some a', Some b' ->
printfn "combining %A and %A" a' b'
Some (a' + b')
| Some a', None ->
printfn "combining %A with None" a'
Some a'
| None, Some b' ->
printfn "combining None with %A" b'
Some b'
| None, None ->
printfn "combining None with None"
None
// make a new instance
let trace = new TraceBuilder()

Running the test code again:

trace {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"

But now we get a different error message:

This control construct may only be used if the computation expression builder defines
a 'Delay' method

The Delay method is a hook that allows you to delay evaluation of a computation
expression until needed -- we'll discuss this in detail very soon; but for now, let's create a
default implementation:

725

Implementing a builder: Combine

type TraceBuilder() =
// other members as before
member this.Delay(f) =
printfn "Delay"
f()
// make a new instance
let trace = new TraceBuilder()

Running the test code again:

trace {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"

And finally we get the code to complete.

Delay
Yield an unwrapped 1 as an option
Delay
Yield an unwrapped 2 as an option
combining 1 and 2
Result for yield then yield: Some 3

The result of the entire workflow is the sum of all the yields, namely Some 3 .
If we have a "failure" in the workflow (e.g. a None ), the second yield doesn't occur and the
overall result is Some 1 instead.
trace {
yield 1
let! x = None
yield 2
} |> printfn "Result for yield then None: %A"

We can have three yields rather than two:

trace {
yield 1
yield 2
yield 3
} |> printfn "Result for yield x 3: %A"

726

Implementing a builder: Combine

The result is what you would expect, Some 6 .

We can even try mixing up yield and return together. Other than the syntax difference,
the overall effect is the same.
trace {
yield 1
return 2
} |> printfn "Result for yield then return: %A"
trace {
return 1
return 2
} |> printfn "Result for return then return: %A"

Using Combine for sequence generation

Adding numbers up is not really the point of yield , although you might perhaps use a
similar idea for constructing concatenated strings, somewhat like StringBuilder .
No, yield is naturally used as part of sequence generation, and now that we understand
Combine , we can extend our "ListBuilder" workflow (from last time) with the required

methods.
The Combine method is just list concatenation.
The Delay method can use a default implementation for now.
Here's the full class:

727

Implementing a builder: Combine

type ListBuilder() =
member this.Bind(m, f) =
m |> List.collect f
member this.Zero() =
printfn "Zero"
[]
member this.Yield(x) =
printfn "Yield an unwrapped %A as a list" x
[x]
member this.YieldFrom(m) =
printfn "Yield a list (%A) directly" m
m
member this.For(m,f) =
printfn "For %A" m
this.Bind(m,f)
member this.Combine (a,b) =
printfn "combining %A and %A" a b
List.concat [a;b]
member this.Delay(f) =
printfn "Delay"
f()
// make an instance of the workflow
let listbuilder = new ListBuilder()

And here it is in use:

listbuilder {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"
listbuilder {
yield 1
yield! [2;3]
} |> printfn "Result for yield then yield! : %A"

And here's a more complicated example with a for loop and some yield s.

728

Implementing a builder: Combine

listbuilder {
for i in ["red";"blue"] do
yield i
for j in ["hat";"tie"] do
yield! [i + " " + j;"-"]
} |> printfn "Result for for..in..do : %A"

And the result is:

["red"; "red hat"; "-"; "red tie"; "-"; "blue"; "blue hat"; "-"; "blue tie"; "-"]

You can see that by combining for..in..do with yield , we are not too far away from the
built-in seq expression syntax (except that seq is lazy, of course).
I would strongly encourage you to play around with this a bit until you are clear on what is
going on behind the scenes. As you can see from the example above, you can use yield
in creative ways to generate all sorts of irregular lists, not just simple ones.
Note: If you're wondering about While , we're going to hold off on it for a bit, until after we
have looked at Delay in an upcoming post.

Order of processing for "combine"

The Combine method only has two parameters. So what happens when you combine more
than two values? For example, here are four values to combine:
listbuilder {
yield 1
yield 2
yield 3
yield 4
} |> printfn "Result for yield x 4: %A"

If you look at the output you can see that the values are combined pair-wise, as you might
expect.
combining [3] and [4]
combining [2] and [3; 4]
combining [1] and [2; 3; 4]
Result for yield x 4: [1; 2; 3; 4]

729

Implementing a builder: Combine

A subtle but important point is that they are combined "backwards", starting from the last
value. First "3" is combined with "4", and the result of that is then combined with "2", and so
on.

Combine for non-sequences

In the second of our earlier problematic examples, we didn't have a sequence; we just had
two separate expressions in a row.
trace {
if true then printfn "hello" //expression 1
return 1 //expression 2
} |> printfn "Result for combine: %A"

How should these expressions be combined?

There are a number of common ways of doing this, depending on the concepts that the
workflow supports.

Implementing combine for workflows with "success" or

"failure"
If the workflow has some concept of "success" or "failure", then a standard approach is:
If the first expression "succeeds" (whatever that means in context), then use that value.
Otherwise use the value of the second expression.
In this case, we also generally use the "failure" value for Zero .
This approach is useful for chaining together a series of "or else" expressions where the first
success "wins" and becomes the overall result.

730

Implementing a builder: Combine

if (do first expression)

or else (do second expression)
or else (do third expression)

For example, for the maybe workflow, it is common to return the first expression if it is
Some , but otherwise the second expression, like this:

type TraceBuilder() =
// other members as before
member this.Zero() =
printfn "Zero"
None // failure
member this.Combine (a,b) =
printfn "Combining %A with %A" a b
match a with
| Some _ -> a // a succeeds -- use it
| None -> b // a fails -- use b instead
// make a new instance
let trace = new TraceBuilder()

Example: Parsing
Let's try a parsing example with this implementation:
type IntOrBool = I of int | B of bool
let parseInt s =
match System.Int32.TryParse(s) with
| true,i -> Some (I i)
| false,_ -> None
let parseBool s =
match System.Boolean.TryParse(s) with
| true,i -> Some (B i)
| false,_ -> None
trace {
return! parseBool "42" // fails
return! parseInt "42"
} |> printfn "Result for parsing: %A"

We get the following result:

Some (I 42)

731

Implementing a builder: Combine

You can see that the first return! expression is None , and ignored. So the overall result is
the second expression, Some (I 42) .
Example: Dictionary lookup
In this example, we'll try looking up the same key in a number of dictionaries, and return
when we find a value:
let map1 = [ ("1","One"); ("2","Two") ] |> Map.ofList
let map2 = [ ("A","Alice"); ("B","Bob") ] |> Map.ofList
trace {
return! map1.TryFind "A"
return! map2.TryFind "A"
} |> printfn "Result for map lookup: %A"

We get the following result:

Result for map lookup: Some "Alice"

You can see that the first lookup is None , and ignored. So the overall result is the second
lookup.
As you can see, this technique is very convenient when doing parsing or evaluating a
sequence of (possibly unsuccessful) operations.

Implementing combine for workflows with sequential steps

If the workflow has the concept of sequential steps, then the overall result is just the value of
the last step, and all the previous steps are evaluated only for their side effects.
In normal F#, this would be written:
do some expression
do some other expression
final expression

Or using the semicolon syntax, just:

some expression; some other expression; final expression

In normal F#, each expression (other than the last) evaluates to the unit value.

732

Implementing a builder: Combine

The equivalent approach for a computation expression is to treat each expression (other
than the last) as a wrapped unit value, and "pass it into" the next expression, and so on, until
you reach the last expression.
This is exactly what bind does, of course, and so the easiest implementation is just to reuse
the Bind method itself. Also, for this approach to work it is important that Zero is the
wrapped unit value.
type TraceBuilder() =
// other members as before
member this.Zero() =
printfn "Zero"
this.Return () // unit not None
member this.Combine (a,b) =
printfn "Combining %A with %A" a b
this.Bind( a, fun ()-> b )
// make a new instance
let trace = new TraceBuilder()

The difference from a normal bind is that the continuation has a unit parameter, and
evaluates to b . This in turn forces a to be of type WrapperType<unit> in general, or unit
option in our case.

Here's an example of sequential processing that works with this implementation of Combine :
trace {
if true then printfn "hello......."
if false then printfn ".......world"
return 1
} |> printfn "Result for sequential combine: %A"

Here's the following trace. Note that the result of the whole expression was the result of the
last expression in the sequence, just like normal F# code.
hello.......
Zero
Returning a unwrapped <null> as an option
Zero
Returning a unwrapped <null> as an option
Returning a unwrapped 1 as an option
Combining Some null with Some 1
Combining Some null with Some 1
Result for sequential combine: Some 1

733

Implementing a builder: Combine

Implementing combine for workflows that build data

structures
Finally, another common pattern for workflows is that they build data structures. In this case,
Combine should merge the two data structures in whatever way is appropriate. And the
Zero method should create an empty data structure, if needed (and if even possible).

In the "list builder" example above, we used exactly this approach. Combine was just list
concatenation and Zero was the empty list.

Guidelines for mixing "Combine" and "Zero"

We have looked at two different implementations for Combine for option types.
The first one used options as "success/failure" indicators, when the first success "won".
In this case Zero was defined as None
The second one was sequential, In this case Zero was defined as Some ()
Both cases worked nicely, but was that luck, or are there are any guidelines for
implementing Combine and Zero correctly?
First, note that Combine does not have to give the same result if the parameters are
swapped. That is, Combine(a,b) need not be the same as Combine(b,a) . The list builder is
a good example of this.
On the other hand there is a useful rule that connects Zero and Combine .
Rule: Combine(a,Zero) should be the same as Combine(Zero,a) which should the same
as just a .
To use an analogy from arithmetic, you can think of Combine like addition (which is not a
bad analogy -- it really is "adding" two values). And Zero is just the number zero, of course!
So the rule above can be expressed as:
Rule: a + 0 is the same as 0 + a is the same as just a , where + means Combine
and 0 means Zero .
If you look at the first Combine implementation ("success/failure") for option types, you'll see
that it does indeed comply with this rule, as does the second implementation ("bind" with
Some() ).

On the other hand, if we had used the "bind" implementation of Combine but left Zero
defined as None , it would not have obeyed the addition rule, which would be a clue that we
had got something wrong.

734

Implementing a builder: Combine

"Combine" without bind

As with all the builder methods, if you don't need them, you don't need to implement them.
So for a workflow that is strongly sequential, you could easily create a builder class with
Combine , Zero , and Yield , say, without having to implement Bind and Return at all.

Here's an example of a minimal implementation that works:

type TraceBuilder() =
member this.ReturnFrom(x) = x
member this.Zero() = Some ()
member this.Combine (a,b) =
a |> Option.bind (fun ()-> b )
member this.Delay(f) = f()
// make an instance of the workflow
let trace = new TraceBuilder()

And here it is in use:

trace {
if true then printfn "hello......."
if false then printfn ".......world"
return! Some 1
} |> printfn "Result for minimal combine: %A"

Similarly, if you have a data-structure oriented workflow, you could just implement Combine
and some other helpers. For example, here is a minimal implementation of our list builder
class:

735

Implementing a builder: Combine

type ListBuilder() =
member this.Yield(x) = [x]
member this.For(m,f) =
m |> List.collect f
member this.Combine (a,b) =
List.concat [a;b]
member this.Delay(f) = f()
// make an instance of the workflow
let listbuilder = new ListBuilder()

And even with the minimal implementation, we can write code like this:
listbuilder {
yield 1
yield 2
} |> printfn "Result: %A"
listbuilder {
for i in [1..5] do yield i + 2
yield 42
} |> printfn "Result: %A"

A standalone "Combine" function

In a previous post, we saw that the "bind" function is often used as standalone function, and
is normally given the operator >>= .
The Combine function too, is often used as a standalone function. Unlike bind, there is no
standard symbol -- it can vary depending on how the combine function works.
A symmetric combination operation is often written as ++ or <+> . And the "left-biased"
combination (that is, only do the second expression if the first one fails) that we used earlier
for options is sometimes written as <++ .
So here is an example of a standalone left-biased combination of options, as used in a
dictionary lookup example.

736

Implementing a builder: Combine

module StandaloneCombine =
let combine a b =
match a with
| Some _ -> a // a succeeds -- use it
| None -> b // a fails -- use b instead
// create an infix version
let ( <++ ) = combine
let map1 = [ ("1","One"); ("2","Two") ] |> Map.ofList
let map2 = [ ("A","Alice"); ("B","Bob") ] |> Map.ofList
let result =
(map1.TryFind "A")
<++ (map1.TryFind "B")
<++ (map2.TryFind "A")
<++ (map2.TryFind "B")
|> printfn "Result of adding options is: %A"

Summary
What have we learned about Combine in this post?
You need to implement Combine (and Delay ) if you need to combine or "add" more
than one wrapped value in a computation expression.
Combine combines values pairwise, from last to first.

There is no universal implementation of Combine that works in all cases -- it needs to

be customized according the particular needs of the workflow.
There is a sensible rule that relates Combine with Zero .
Combine doesn't require Bind to be implemented.
Combine can be exposed as a standalone function

In the next post, we'll add logic to control exactly when the internal expressions get
evaluated, and introduce true short circuiting and lazy evaluation.

737

Implementing a builder: Delay and Run

In the last few posts we have covered all the basic methods (Bind, Return, Zero, and
Combine) needed to create your own computation expression builder. In this post, we'll look
at some of the extra features needed to make the workflow more efficient, by controlling
when expressions get evaluated.

The problem: avoiding unnecessary

evaluations
Let's say that we have created a "maybe" style workflow as before. But this time we want to
use the "return" keyword to return early and stop any more processing being done.
Here is our complete builder class. The key method to look at is Combine , in which we
simply ignore any secondary expressions after the first return.
type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Return(x) =
printfn "Return an unwrapped %A as an option" x
Some x
member this.Zero() =
printfn "Zero"
None
member this.Combine (a,b) =
printfn "Returning early with %A. Ignoring second part: %A" a b
a
member this.Delay(f) =
printfn "Delay"
f()
// make an instance of the workflow
let trace = new TraceBuilder()

738

Implementing a builder: Delay and Run

Let's see how it works by printing something, returning, and then printing something else:
trace {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
} |> printfn "Result for Part1 without Part2: %A"

The debugging output should look something like the following, which I have annotated:
// first expression, up to "return"
Delay
Part 1: about to return 1
Return an unwrapped 1 as an option
// second expression, up to last curly brace.
Delay
Part 2: after return has happened
Zero // zero here because no explicit return was given for this part
// combining the two expressions
Returning early with Some 1. Ignoring second part: <null>
// final result
Result for Part1 without Part2: Some 1

We can see a problem here. The "Part 2: after return" was printed, even though we were
trying to return early.
Why? Well I'll repeat what I said in the last post: return and yield do not generate an early
return from a computation expression. The entire computation expression, all the way to
the last curly brace, is always evaluated and results in a single value.
This is a problem, because you might get unwanted side effects (such as printing a message
in this case) and your code is doing something unnecessary, which might cause
performance problems.
So, how can we avoid evaluating the second part until we need it?

Introducing "Delay"
The answer to the question is straightforward -- simply wrap part 2 of the expression in a
function and only call this function when needed, like this.

739

Implementing a builder: Delay and Run

let part2 =
fun () ->
printfn "Part 2: after return has happened"
// do other stuff
// return Zero
// only evaluate if needed
if needed then
let result = part2()

Using this technique, part 2 of the computation expression can be processed completely, but
because the expression returns a function, nothing actually happens until the function is
called.
But the Combine method will never call it, and so the code inside it does not run at all.
And this is exactly what the Delay method is for. Any result from Return or Yield is
immediately wrapped in a "delay" function like this, and then you can choose whether to run
it or not.
Let's change the builder to implement a delay:
type TraceBuilder() =
// other members as before
member this.Delay(funcToDelay) =
let delayed = fun () ->
printfn "%A - Starting Delayed Fn." funcToDelay
let delayedResult = funcToDelay()
printfn "%A - Finished Delayed Fn. Result is %A" funcToDelay delayedResult
delayedResult // return the result
printfn "%A - Delaying using %A" funcToDelay delayed
delayed // return the new function

As you can see, the Delay method is given a function to execute. Previously, we executed
it immediately. What we're doing now is wrapping this function in another function and
returning the delayed function instead. I have added a number of trace statements before
and after the function is wrapped.
If you compile this code, you can see that the signature of Delay has changed. Before the
change, it returned a concrete value (an option in this case), but now it returns a function.

740

Implementing a builder: Delay and Run

// signature BEFORE the change

member Delay : f:(unit -> 'a) -> 'a
// signature AFTER the change
member Delay : f:(unit -> 'b) -> (unit -> 'b)

By the way, we could have implemented Delay in a much simpler way, without any tracing,
just by returning the same function that was passed in, like this:
member this.Delay(f) =
f

Much more concise! But in this case, I wanted to add some detailed tracing information as
well.
Now let's try again:
trace {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
} |> printfn "Result for Part1 without Part2: %A"

Uh-oh. This time nothing happens at all! What went wrong?

If we look at the output we see this:
Result for Part1 without Part2: <fun:Delay@84-5>

Hmmm. The output of the whole trace expression is now a function, not an option. Why?
Because we created all these delays, but we never "undelayed" them by actually calling the
function!
One way to do this is to assign the output of the computation expression to a function value,
say f , and then evaluate it.
let f = trace {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
}
f() |> printfn "Result for Part1 without Part2: %A"

This works as expected, but is there a way to do this from inside the computation expression
itself? Of course there is!

741

Implementing a builder: Delay and Run

Introducing "Run"
The Run method exists for exactly this reason. It is called as the final step in the process of
evaluating a computation expression, and can be used to undo the delay.
Here's an implementation:
type TraceBuilder() =
// other members as before
member this.Run(funcToRun) =
printfn "%A - Run Start." funcToRun
let runResult = funcToRun()
printfn "%A - Run End. Result is %A" funcToRun runResult
runResult // return the result of running the delayed function

Let's try one more time:

trace {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
} |> printfn "Result for Part1 without Part2: %A"

And the result is exactly what we wanted. The first part is evaluated, but the second part is
not. And the result of the entire computation expression is an option, not a function.

When is delay called?

The way that Delay is inserted into the workflow is straightforward, once you understand it.
The bottom (or innermost) expression is delayed.
If this is combined with a prior expression, the output of Combine is also delayed.
And so on, until the final delay is fed into Run .
Using this knowledge, let's review what happened in the example above:
The first part of the expression is the print statement plus return 1 .
The second part of the expression is the print statement without an explicit return, which
means that Zero() is called
The None from the Zero is fed into Delay , resulting in a "delayed option", that is, a
function that will evaluate to an option when called.
The option from part 1 and the delayed option from part 2 are combined in Combine

742

Implementing a builder: Delay and Run

and the second one is discarded.

The result of the combine is turned into another "delayed option".
Finally, the delayed option is fed to Run , which evaluates it and returns a normal
option.
Here is a diagram that represents this process visually:

If we look at the debug trace for the example above, we can see in detail what happened.
It's a little confusing, so I have annotated it. Also, it helps to remember that working down
this trace is the same as working up from the bottom of the diagram above, because the
outermost code is run first.

743

Implementing a builder: Delay and Run

// delaying the overall expression (the output of Combine)

<fun:clo@160-66> - Delaying using <fun:delayed@141-3>
// running the outermost delayed expression (the output of Combine)
<fun:delayed@141-3> - Run Start.
<fun:clo@160-66> - Starting Delayed Fn.
// the first expression results in Some(1)
Part 1: about to return 1
Return an unwrapped 1 as an option
// the second expression is wrapped in a delay
<fun:clo@162-67> - Delaying using <fun:delayed@141-3>
// the first and second expressions are combined
Combine. Returning early with Some 1. Ignoring <fun:delayed@141-3>
// overall delayed expression (the output of Combine) is complete
<fun:clo@160-66> - Finished Delayed Fn. Result is Some 1
<fun:delayed@141-3> - Run End. Result is Some 1
// the result is now an Option not a function
Result for Part1 without Part2: Some 1

"Delay" changes the signature of "Combine"

When Delay is introduced into the pipeline like this, it has an effect on the signature of
Combine .

When we originally wrote Combine we were expecting it to handle options . But now it is
handling the output of Delay , which is a function.
We can see this if we hard-code the types that Combine expects, with int option type
annotations like this:
member this.Combine (a: int option,b: int option) =
printfn "Returning early with %A. Ignoring %A" a b
a

If this is done, we get an compiler error in the "return" expression:

trace {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
} |> printfn "Result for Part1 without Part2: %A"

744

Implementing a builder: Delay and Run

The error is:

error FS0001: This expression was expected to have type
int option
but here has type
unit -> 'a

In other words, the Combine is being passed a delayed function ( unit -> 'a ), which
doesn't match our explicit signature.
So what happens when we do want to combine the parameters, but they are passed in as a
function instead of as a simple value?
The answer is straightforward: just call the function that was passed in to get the underlying
value.
Let's demonstrate that using the adding example from the previous post.
type TraceBuilder() =
// other members as before
member this.Combine (m,f) =
printfn "Combine. Starting second param %A" f
let y = f()
printfn "Combine. Finished second param %A. Result is %A" f y
match m,y with
| Some a, Some b ->
printfn "combining %A and %A" a b
Some (a + b)
| Some a, None ->
printfn "combining %A with None" a
Some a
| None, Some b ->
printfn "combining None with %A" b
Some b
| None, None ->
printfn "combining None with None"
None

In this new version of Combine , the second parameter is now a function, not an int
option . So to combine them, we must first evaluate the function before doing the

combination logic.
If we test this out:

745

Implementing a builder: Delay and Run

trace {
return 1
return 2
} |> printfn "Result for return then return: %A"

We get the following (annotated) trace:

// entire expression is delayed
<fun:clo@318-69> - Delaying using <fun:delayed@295-6>
// entire expression is run
<fun:delayed@295-6> - Run Start.
// delayed entire expression is run
<fun:clo@318-69> - Starting Delayed Fn.
// first return
Returning a unwrapped 1 as an option
// delaying second return
<fun:clo@319-70> - Delaying using <fun:delayed@295-6>
// combine starts
Combine. Starting second param <fun:delayed@295-6>
// delayed second return is run inside Combine
<fun:clo@319-70> - Starting Delayed Fn.
Returning a unwrapped 2 as an option
<fun:clo@319-70> - Finished Delayed Fn. Result is Some 2
// delayed second return is complete
Combine. Finished second param <fun:delayed@295-6>. Result is Some 2
combining 1 and 2
// combine is complete
<fun:clo@318-69> - Finished Delayed Fn. Result is Some 3
// delayed entire expression is complete
<fun:delayed@295-6> - Run End. Result is Some 3
// Run is complete
// final result is printed
Result for return then return: Some 3

Understanding the type constraints

Up to now, we have used only our "wrapped type" (e.g. int option ) and the delayed
version (e.g. unit -> int option ) in the implementation of our builder.
746

Implementing a builder: Delay and Run

But in fact we can use other types if we like, subject to certain constraints. In fact,
understanding exactly what the type constraints are in a computation expression can clarify
how everything fits together.
For example, we have seen that:
The output of Return is passed into Delay , so they must have compatible types.
The output of Delay is passed into the second parameter of Combine .
The output of Delay is also passed into Run .
But the output of Return does not have to be our "public" wrapped type. It could be an
internally defined type instead.

Similarly, the delayed type does not have to be a simple function, it could be any type that
satisfies the constraints.
So, given a simple set of return expressions, like this:
trace {
return 1
return 2
return 3
} |> printfn "Result for return x 3: %A"

Then a diagram that represents the various types and their flow would look like this:

747

Implementing a builder: Delay and Run

And to prove that this is valid, here is an implementation with distinct types for Internal
and Delayed :
type Internal = Internal of int option
type Delayed = Delayed of (unit -> Internal)
type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Return(x) =
printfn "Returning a unwrapped %A as an option" x
Internal (Some x)

748

Implementing a builder: Delay and Run

member this.ReturnFrom(m) =
printfn "Returning an option (%A) directly" m
Internal m
member this.Zero() =
printfn "Zero"
Internal None
member this.Combine (Internal x, Delayed g) : Internal =
printfn "Combine. Starting %A" g
let (Internal y) = g()
printfn "Combine. Finished %A. Result is %A" g y
let o =
match x,y with
| Some a, Some b ->
printfn "Combining %A and %A" a b
Some (a + b)
| Some a, None ->
printfn "combining %A with None" a
Some a
| None, Some b ->
printfn "combining None with %A" b
Some b
| None, None ->
printfn "combining None with None"
None
// return the new value wrapped in a Internal
Internal o
member this.Delay(funcToDelay) =
let delayed = fun () ->
printfn "%A - Starting Delayed Fn." funcToDelay
let delayedResult = funcToDelay()
printfn "%A - Finished Delayed Fn. Result is %A" funcToDelay delayedResult
delayedResult // return the result
printfn "%A - Delaying using %A" funcToDelay delayed
Delayed delayed // return the new function wrapped in a Delay
member this.Run(Delayed funcToRun) =
printfn "%A - Run Start." funcToRun
let (Internal runResult) = funcToRun()
printfn "%A - Run End. Result is %A" funcToRun runResult
runResult // return the result of running the delayed function
// make an instance of the workflow
let trace = new TraceBuilder()

And the method signatures in the builder class methods look like this:

749

Implementing a builder: Delay and Run

type Internal = | Internal of int option

type Delayed = | Delayed of (unit -> Internal)
type TraceBuilder =
class
new : unit -> TraceBuilder
member Bind : m:'a option * f:('a -> 'b option) -> 'b option
member Combine : Internal * Delayed -> Internal
member Delay : funcToDelay:(unit -> Internal) -> Delayed
member Return : x:int -> Internal
member ReturnFrom : m:int option -> Internal
member Run : Delayed -> int option
member Zero : unit -> Internal
end

Creating this artifical builder is overkill of course, but the signatures clearly show how the
various methods fit together.

Summary
In this post, we've seen that:
You need to implement Delay and Run if you want to delay execution within a
computation expression.
Using Delay changes the signature of Combine .
Delay and Combine can use internal types that are not exposed to clients of the

computation expression.
The next logical step is wanting to delay execution outside a computation expression until
you are ready, and that will be the topic on the next but one post. But first, we'll take a little
detour to discuss method overloads.

750

Implementing a builder: Overloading

In this post, we'll take a detour and look at some tricks you can do with methods in a
computation expression builder.
Ultimately, this detour will lead to a dead end, but I hope the journey might provide some
more insight into good practices for designing your own computation expressions.

An insight: builder methods can be overloaded

At some point, you might have an insight:
The builder methods are just normal class methods, and unlike standalone functions,
methods can support overloading with different parameter types, which means we can
create different implementations of any method, as long as the parameter types are
different.
So then you might get excited about this and how it could be used. But it turns out to be less
useful than you might think. Let's look at some examples.

Overloading "return"
Say that you have a union type. You might consider overloading Return or Yield with
multiple implementations for each union case.
For example, here's a very simple example where Return has two overloads:

751

Implementing a builder: Overloading

type SuccessOrError =
| Success of int
| Error of string
type SuccessOrErrorBuilder() =
member this.Bind(m, f) =
match m with
| Success s -> f s
| Error _ -> m
/// overloaded to accept ints
member this.Return(x:int) =
printfn "Return a success %i" x
Success x
/// overloaded to accept strings
member this.Return(x:string) =
printfn "Return an error %s" x
Error x
// make an instance of the workflow
let successOrError = new SuccessOrErrorBuilder()

And here it is in use:

successOrError {
return 42
} |> printfn "Result for success: %A"
// Result for success: Success 42
successOrError {
return "error for step 1"
} |> printfn "Result for error: %A"
//Result for error: Error "error for step 1"

What's wrong with this, you might think?

Well, first, if we go back to the discussion on wrapper types, we made the point that wrapper
types should be generic. Workflows should be reusable as much as possible -- why tie the
implementation to any particular primitive type?
What that means in this case is that the union type should be resigned to look like this:
type SuccessOrError<'a,'b> =
| Success of 'a
| Error of 'b

752

Implementing a builder: Overloading

But as a consequence of the generics, the Return method can't be overloaded any more!
Second, it's probably not a good idea to expose the internals of the type inside the
expression like this anyway. The concept of "success" and "failure" cases is useful, but a
better way would be to hide the "failure" case and handle it automatically inside Bind , like
this:
type SuccessOrError<'a,'b> =
| Success of 'a
| Error of 'b
type SuccessOrErrorBuilder() =
member this.Bind(m, f) =
match m with
| Success s ->
try
f s
with
| e -> Error e.Message
| Error _ -> m
member this.Return(x) =
Success x
// make an instance of the workflow
let successOrError = new SuccessOrErrorBuilder()

In this approach, Return is only used for success, and the failure cases are hidden.
successOrError {
return 42
} |> printfn "Result for success: %A"
successOrError {
let! x = Success 1
return x/0
} |> printfn "Result for error: %A"

We'll see more of this technique in an upcoming post.

Multiple Combine implementations

Another time when you might be tempted to overload a method is when implementing
Combine .

753

Implementing a builder: Overloading

Let's revisit the Combine method for the trace workflow. If you remember, in the previous
implementation of Combine , we just added the numbers together.
But what if we change our requirements, and say that:
if we yield multiple values in the trace workflow, then we want to combine them into a
list.
A first attempt using combine might look this:
member this.Combine (a,b) =
match a,b with
| Some a', Some b' ->
printfn "combining %A and %A" a' b'
Some [a';b']
| Some a', None ->
printfn "combining %A with None" a'
Some [a']
| None, Some b' ->
printfn "combining None with %A" b'
Some [b']
| None, None ->
printfn "combining None with None"
None

In the Combine method, we unwrap the value from the passed-in option and combine them
into a list wrapped in a Some (e.g. Some [a';b'] ).
For two yields it works as expected:
trace {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"
// Result for yield then yield: Some [1; 2]

And for a yielding a None , it also works as expected:

trace {
yield 1
yield! None
} |> printfn "Result for yield then None: %A"
// Result for yield then None: Some [1]

But what happens if there are three values to combine? Like this:

754

Implementing a builder: Overloading

trace {
yield 1
yield 2
yield 3
} |> printfn "Result for yield x 3: %A"

If we try this, we get a compiler error:

error FS0001: Type mismatch. Expecting a
int option
but given a
'a list option
The type 'int' does not match the type ''a list'

What is the problem?

The answer is that after combining the 2nd and 3rd values ( yield 2; yield 3 ), we get an
option containing a list of ints or int list option . The error happens when we attempt to
combine the first value ( Some 1 ) with the combined value ( Some [2;3] ). That is, we are
passing a int list option as the second parameter of Combine , but the first parameter is
still a normal int option . The compiler is telling you that it wants the second parameter to
be the same type as the first.
But, here's where we might want use our overloading trick. We can create two different
implementations of Combine , with different types for the second parameter, one that takes
an int option and the other taking an int list option .
So here are the two methods, with different parameter types:

755

Implementing a builder: Overloading

/// combine with a list option

member this.Combine (a, listOption) =
match a,listOption with
| Some a', Some list ->
printfn "combining %A and %A" a' list
Some ([a'] @ list)
| Some a', None ->
printfn "combining %A with None" a'
Some [a']
| None, Some list ->
printfn "combining None with %A" list
Some list
| None, None ->
printfn "combining None with None"
None
/// combine with a non-list option
member this.Combine (a,b) =
match a,b with
| Some a', Some b' ->
printfn "combining %A and %A" a' b'
Some [a';b']
| Some a', None ->
printfn "combining %A with None" a'
Some [a']
| None, Some b' ->
printfn "combining None with %A" b'
Some [b']
| None, None ->
printfn "combining None with None"
None

Now if we try combining three results, as before, we get what we expect.

trace {
yield 1
yield 2
yield 3
} |> printfn "Result for yield x 3: %A"
// Result for yield x 3: Some [1; 2; 3]

Unfortunately, this trick has broken some previous code! If you try yielding a None now, you
will get a compiler error.

756

Implementing a builder: Overloading

trace {
yield 1
yield! None
} |> printfn "Result for yield then None: %A"

The error is:

error FS0041: A unique overload for method 'Combine' could not be determined based on
type information prior to this program point. A type annotation may be needed.

But hold on, before you get too annoyed, try thinking like the compiler. If you were the
compiler, and you were given a None , which method would you call?
There is no correct answer, because a None could be passed as the second parameter to
either method. The compiler does not know where this is a None of type int list option
(the first method) or a None of type int option (the second method).
As the compiler reminds us, a type annotation will help, so let's give it one. We'll force the
None to be an int option .
trace {
yield 1
let x:int option = None
yield! x
} |> printfn "Result for yield then None: %A"

This is ugly, of course, but in practice might not happen very often.
More importantly, this is a clue that we have a bad design. Sometimes the computation
expression returns an 'a option and sometimes it returns an 'a list option . We should
be consistent in our design, so that the computation expression always returns the same
type, no matter how many yield s are in it.
That is, if we do want to allow multiple yield s, then we should use 'a list option as the
wrapper type to begin with rather than just a plain option. In this case the Yield method
would create the list option, and the Combine method could be collapsed to a single method
again.
Here's the code for our third version:

757

Implementing a builder: Overloading

type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Zero() =
printfn "Zero"
None
member this.Yield(x) =
printfn "Yield an unwrapped %A as a list option" x
Some [x]
member this.YieldFrom(m) =
printfn "Yield an option (%A) directly" m
m
member this.Combine (a, b) =
match a,b with
| Some a', Some b' ->
printfn "combining %A and %A" a' b'
Some (a' @ b')
| Some a', None ->
printfn "combining %A with None" a'
Some a'
| None, Some b' ->
printfn "combining None with %A" b'
Some b'
| None, None ->
printfn "combining None with None"
None
member this.Delay(f) =
printfn "Delay"
f()
// make an instance of the workflow
let trace = new TraceBuilder()

And now the examples work as expected without any special tricks:

758

Implementing a builder: Overloading

trace {
yield 1
yield 2
} |> printfn "Result for yield then yield: %A"
// Result for yield then yield: Some [1; 2]
trace {
yield 1
yield 2
yield 3
} |> printfn "Result for yield x 3: %A"
// Result for yield x 3: Some [1; 2; 3]
trace {
yield 1
yield! None
} |> printfn "Result for yield then None: %A"
// Result for yield then None: Some [1]

Not only is the code cleaner, but as in the Return example, we have made our code more
generic as well, having gone from a specific type ( int option ) to a more generic type ( 'a
option ).

Overloading "For"
One legitimate case where overloading might be needed is the For method. Some possible
reasons:
You might want to support different kinds of collections (e.g. list and IEnumerable )
You might have a more efficient looping implementation for certain kinds of collections.
You might have a "wrapped" version of a list (e.g. LazyList) and you want support
looping for both unwrapped and wrapped values.
Here's an example of our list builder that has been extended to support sequences as well
as lists:

759

Implementing a builder: Overloading

type ListBuilder() =
member this.Bind(m, f) =
m |> List.collect f
member this.Yield(x) =
printfn "Yield an unwrapped %A as a list" x
[x]
member this.For(m,f) =
printfn "For %A" m
this.Bind(m,f)
member this.For(m:_ seq,f) =
printfn "For %A using seq" m
let m2 = List.ofSeq m
this.Bind(m2,f)
// make an instance of the workflow
let listbuilder = new ListBuilder()

And here is it in use:

listbuilder {
let list = [1..10]
for i in list do yield i
} |> printfn "Result for list: %A"
listbuilder {
let s = seq {1..10}
for i in s do yield i
} |> printfn "Result for seq : %A"

If you comment out the second For method, you will see the "sequence` example will
indeed fail to compile. So the overload is needed.

Summary
So we've seen that methods can be overloaded if needed, but be careful at jumping to this
solution immediately, because having to doing this may be a sign of a weak design.
In the next post, we'll go back to controlling exactly when the expressions get evaluated, this
time using a delay outside the builder.

760

Implementing a builder: Adding laziness

In a previous post, we saw how to avoid unnecessary evaluation of expressions in a
workflow until needed.
But that approach was designed for expressions inside a workflow. What happens if we want
to delay the whole workflow itself until needed.

The problem
Here is the code from our "maybe" builder class. This code is based on the trace builder
from the earlier post, but with all the tracing taken out, so that it is nice and clean.
type MaybeBuilder() =
member this.Bind(m, f) =
Option.bind f m
member this.Return(x) =
Some x
member this.ReturnFrom(x) =
x
member this.Zero() =
None
member this.Combine (a,b) =
match a with
| Some _ -> a // if a is good, skip b
| None -> b() // if a is bad, run b
member this.Delay(f) =
f
member this.Run(f) =
f()
// make an instance of the workflow
let maybe = new MaybeBuilder()

Before moving on, make sure that you understand how this works. If we analyze this using
the terminology of the earlier post, we can see that the types used are:

761

Implementing a builder: Adding laziness

Wrapper type: 'a option

Internal type: 'a option
Delayed type: unit -> 'a option
Now let's check this code and make sure everything works as expected.
maybe {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
} |> printfn "Result for Part1 but not Part2: %A"
// result - second part is NOT evaluated
maybe {
printfn "Part 1: about to return None"
return! None
printfn "Part 2: after None, keep going"
} |> printfn "Result for Part1 and then Part2: %A"
// result - second part IS evaluated

But what happens if we refactor the code into a child workflow, like this:
let childWorkflow =
maybe {printfn "Child workflow"}
maybe {
printfn "Part 1: about to return 1"
return 1
return! childWorkflow
} |> printfn "Result for Part1 but not childWorkflow: %A"

The output shows that the child workflow was evaluated even though it wasn't needed in the
end. This might not be a problem in this case, but in many cases, we may not want this to
happen.
So, how to avoid it?

Wrapping the inner type in a delay

The obvious approach is to wrap the entire result of the builder in a delay function, and then
to "run" the result, we just evaluate the delay function.
So, here's our new wrapper type:

762

Implementing a builder: Adding laziness

type Maybe<'a> = Maybe of (unit -> 'a option)

We've replaced a simple option with a function that evaluates to an option, and then
wrapped that function in a single case union for good measure.
And now we need to change the Run method as well. Previously, it evaluated the delay
function that was passed in to it, but now it should leave it unevaluated and wrap it in our
new wrapper type:
// before
member this.Run(f) =
f()
// after
member this.Run(f) =
Maybe f

I've forgotten to fix up another method -- do you know which one? We'll bump into it soon!
One more thing -- we'll need a way to "run" the result now.
let run (Maybe f) = f()

Let's try out our new type on our previous examples:

let m1 = maybe {
printfn "Part 1: about to return 1"
return 1
printfn "Part 2: after return has happened"
}

Running this, we get something like this:

val m1 : Maybe<int> = Maybe <fun:m1@123-7>

That looks good; nothing else was printed.

And now run it:
run m1 |> printfn "Result for Part1 but not Part2: %A"

and we get the output:

763

Implementing a builder: Adding laziness

Part 1: about to return 1

Result for Part1 but not Part2: Some 1

Perfect. Part 2 did not run.

But we run into a problem with the next example:
let m2 = maybe {
printfn "Part 1: about to return None"
return! None
printfn "Part 2: after None, keep going"
}

Oops! We forgot to fix up ReturnFrom ! As we know, that method takes a wrapped type, and
we have redefined the wrapped type now.
Here's the fix:
member this.ReturnFrom(Maybe f) =
f()

We are going to accept a Maybe from outside, and then immediately run it to get at the
option.
But now we have another problem -- we can't return an explicit None anymore in return!
None , we have to return a Maybe type instead. How are we going to create one of these?

Well, we could create a helper function that constructs one for us. But there is a much
simpler answer: you can create a new Maybe type by using a maybe expression!
let m2 = maybe {
return! maybe {printfn "Part 1: about to return None"}
printfn "Part 2: after None, keep going"
}

This is why the Zero method is useful. With Zero and the builder instance, you can create
new instances of the type even if they don't do anything.
But now we have one more error -- the dreaded "value restriction":
Value restriction. The value 'm2' has been inferred to have generic type

764

Implementing a builder: Adding laziness

The reason why this has happened is that both expressions are returning None . But the
compiler does not know what type None is. The code is using None of type Option<obj>
(presumably because of implicit boxing) yet the compiler knows that the type can be more
generic than that.
There are two fixes. One is to make the type explicit:
let m2_int: Maybe<int> = maybe {
return! maybe {printfn "Part 1: about to return None"}
printfn "Part 2: after None, keep going;"
}

Or we can just return some non-None value instead:

let m2 = maybe {
return! maybe {printfn "Part 1: about to return None"}
printfn "Part 2: after None, keep going;"
return 1
}

Both of these solutions will fix the problem.

Now if we run the example, we see that the result is as expected. The second part is run this
time.
run m2 |> printfn "Result for Part1 and then Part2: %A"

The trace output:

Part 1: about to return None
Part 2: after None, keep going;
Result for Part1 and then Part2: Some 1

Finally, we'll try the child workflow examples again:

let childWorkflow =
maybe {printfn "Child workflow"}
let m3 = maybe {
printfn "Part 1: about to return 1"
return 1
return! childWorkflow
}
run m3 |> printfn "Result for Part1 but not childWorkflow: %A"

765

Implementing a builder: Adding laziness

And now the child workflow is not evaluated, just as we wanted.

And if we do need the child workflow to be evaluated, this works too:
let m4 = maybe {
return! maybe {printfn "Part 1: about to return None"}
return! childWorkflow
}
run m4 |> printfn "Result for Part1 and then childWorkflow: %A"

Reviewing the builder class

Let's look at all the code in the new builder class again:
type Maybe<'a> = Maybe of (unit -> 'a option)
type MaybeBuilder() =
member this.Bind(m, f) =
Option.bind f m
member this.Return(x) =
Some x
member this.ReturnFrom(Maybe f) =
f()
member this.Zero() =
None
member this.Combine (a,b) =
match a with
| Some _' -> a // if a is good, skip b
| None -> b() // if a is bad, run b
member this.Delay(f) =
f
member this.Run(f) =
Maybe f
// make an instance of the workflow
let maybe = new MaybeBuilder()
let run (Maybe f) = f()

If we analyze this new builder using the terminology of the earlier post, we can see that the
types used are:

766

Implementing a builder: Adding laziness

Wrapper type: Maybe<'a>

Internal type: 'a option
Delayed type: unit -> 'a option
Note that in this case it was convenient to use the standard 'a option as the internal type,
because we didn't need to modify Bind or Return at all.
An alternative design might use Maybe<'a> as the internal type as well, which would make
things more consistent, but makes the code harder to read.

True laziness
Let's look at a variant of the last example:
let child_twice: Maybe<unit> = maybe {
let workflow = maybe {printfn "Child workflow"}
return! maybe {printfn "Part 1: about to return None"}
return! workflow
return! workflow
}
run child_twice |> printfn "Result for childWorkflow twice: %A"

What should happen? How many times should the child workflow be run?
The delayed implementation above does ensure that the child workflow is only be evaluated
on demand, but it does not stop it being run twice.
In some situations, you might require that the workflow is guaranteed to only run at most
once, and then cached ("memoized"). This is easy enough to do using the Lazy type that is
built into F#.
The changes we need to make are:
Change Maybe to wrap a Lazy instead of a delay
Change ReturnFrom and run to force the evaluation of the lazy value
Change Run to run the delay from inside a lazy
Here is the new class with the changes:

767

Implementing a builder: Adding laziness

type Maybe<'a> = Maybe of Lazy<'a option>

type MaybeBuilder() =
member this.Bind(m, f) =
Option.bind f m
member this.Return(x) =
Some x
member this.ReturnFrom(Maybe f) =
f.Force()
member this.Zero() =
None
member this.Combine (a,b) =
match a with
| Some _' -> a // if a is good, skip b
| None -> b() // if a is bad, run b
member this.Delay(f) =
f
member this.Run(f) =
Maybe (lazy f())
// make an instance of the workflow
let maybe = new MaybeBuilder()
let run (Maybe f) = f.Force()

And if we run the "child twice` code from above, we get:

Part 1: about to return None
Child workflow
Result for childWorkflow twice: <null>

from which it is clear that the child workflow only ran once.

Summary: Immediate vs. Delayed vs. Lazy

On this page, we've seen three different implementations of the maybe workflow. One that is
always evaluated immediately, one that uses a delay function, and one that uses laziness
with memoization.
So... which approach should you use?

768

Implementing a builder: Adding laziness

There is no single "right" answer. Your choice depends on a number of things:

Is the code in the expression cheap to execute, and without important side-effects? If
so, stick with the first, immediate version. It's simple and easy to understand, and this is
exactly what most implementations of the maybe workflow use.
Is the code in the expression expensive to execute, might the result vary with each call
(e.g. non-deterministic), or are there important side-effects? If so, use the second,
delayed version. This is exactly what most other workflows do, especially those relating
to I/O (such as async ).
F# does not attempt to be a purely functional language, so almost all F# code will fall
into one of these two categories. But, if you need to code in a guaranteed side-effect
free style, or you just want to ensure that expensive code is evaluated at most once,
then use the third, lazy option.
Whatever your choice, do make it clear in the documentation. For example, the delayed vs.
lazy implementations appear exactly the same to the client, but they have very different
semantics, and the client code must be written differently for each case.
Now that we have finished with delays and laziness, we can go back to the builder methods
and finish them off.

769

Implementing a builder: The rest of the standard methods

Implementing a builder: The rest of the

standard methods
We're coming into the home stretch now. There are only a few more builder methods that
need to be covered, and then you will be ready to tackle anything!
These methods are:
While for repetition.
TryWith and TryFinally for handling exceptions.
Use for managing disposables

Remember, as always, that not all methods need to be implemented. If While is not
relevant to you, don't bother with it.
One important note before we get started: all the methods discussed here rely on delays
being used. If you are not using delay functions, then none of the methods will give the
expected results.

Implementing "While"
We all know what "while" means in normal code, but what does it mean in the context of a
computation expression? To understand, we have to revisit the concept of continuations
again.
In previous posts, we saw that a series of expressions is converted into a chain of
continuations like this:
Bind(1,fun x ->
Bind(2,fun y ->
Bind(x + y,fun z ->
Return(z) // or Yield

And this is the key to understanding a "while" loop -- it can be expanded in the same way.
First, some terminology. A while loop has two parts:
There is a test at the top of the "while" loop which is evaluated each time to determine
whether the body should be run. When it evaluates to false, the while loop is "exited". In
computation expressions, the test part is known as the "guard". The test function has
no parameters, and returns a bool, so its signature is unit -> bool , of course.

770

Implementing a builder: The rest of the standard methods

And there is the body of the "while" loop, evaluated each time until the "while" test fails.
In computation expressions, this is a delay function that evaluates to a wrapped value.
Since the body of the while loop is always the same, the same function is evaluated
each time. The body function has no parameters, and returns nothing, and so its
signature is just unit -> wrapped unit .
With this in place, we can create pseudo-code for a while loop using continuations:
// evaluate test function
let bool = guard()
if not bool
then
// exit loop
return what??
else
// evaluate the body function
body()
// back to the top of the while loop
// evaluate test function again
let bool' = guard()
if not bool'
then
// exit loop
return what??
else
// evaluate the body function again
body()
// back to the top of the while loop
// evaluate test function a third time
let bool'' = guard()
if not bool''
then
// exit loop
return what??
else
// evaluate the body function a third time
body()
// etc

One question that is immediately apparent is: what should be returned when the while loop
test fails? Well, we have seen this before with if..then.. , and the answer is of course to
use the Zero value.

771

Implementing a builder: The rest of the standard methods

The next thing is that the body() result is being discarded. Yes, it is a unit function, so there
is no value to return, but even so, in our expressions, we want to be able to hook into this so
we can add behavior behind the scenes. And of course, this calls for using the Bind
function.
So here is a revised version of the pseudo-code, using Zero and Bind :
// evaluate test function
let bool = guard()
if not bool
then
// exit loop
return Zero
else
// evaluate the body function
Bind( body(), fun () ->
// evaluate test function again
let bool' = guard()
if not bool'
then
// exit loop
return Zero
else
// evaluate the body function again
Bind( body(), fun () ->
// evaluate test function a third time
let bool'' = guard()
if not bool''
then
// exit loop
return Zero
else
// evaluate the body function again
Bind( body(), fun () ->
// etc

In this case, the continuation function passed into Bind has a unit parameter, because the
body function does not have a value.

Finally, the pseudo-code can be simplified by collapsing it into a recursive function like this:

772

Implementing a builder: The rest of the standard methods

member this.While(guard, body) =

// evaluate test function
if not (guard())
then
// exit loop
this.Zero()
else
// evaluate the body function
this.Bind( body(), fun () ->
// call recursively
this.While(guard, body))

And indeed, this is the standard "boiler-plate" implementation for While in almost all builder
classes.
It is a subtle but important point that the value of Zero must be chosen properly. In previous
posts, we saw that we could set the value for Zero to be None or Some () depending on
the workflow. For While to work however, the Zero must be set to Some () and not
None , because passing None into Bind will cause the whole thing to aborted early.

Also note that, although this is a recursive function, we didn't need the rec keyword. It is
only needed for standalone functions that are recursive, not methods.

"While" in use
Let's look at it being used in the trace builder. Here's the complete builder class, with the
While method:

773

Implementing a builder: The rest of the standard methods

type TraceBuilder() =
member this.Bind(m, f) =
match m with
| None ->
printfn "Binding with None. Exiting."
| Some a ->
printfn "Binding with Some(%A). Continuing" a
Option.bind f m
member this.Return(x) =
Some x
member this.ReturnFrom(x) =
x
member this.Zero() =
printfn "Zero"
this.Return ()
member this.Delay(f) =
printfn "Delay"
f
member this.Run(f) =
f()
member this.While(guard, body) =
printfn "While: test"
if not (guard())
then
printfn "While: zero"
this.Zero()
else
printfn "While: body"
this.Bind( body(), fun () ->
this.While(guard, body))
// make an instance of the workflow
let trace = new TraceBuilder()

If you look at the signature for While , you will see that the body parameter is unit -> unit
option , that is, a delayed function. As noted above, if you don't implement Delay properly,

you will get unexpected behavior and cryptic compiler errors.

type TraceBuilder =
// other members
member
While : guard:(unit -> bool) * body:(unit -> unit option) -> unit option

774

Implementing a builder: The rest of the standard methods

And here is a simple loop using a mutable value that is incremented each time round.
let mutable i = 1
let test() = i < 5
let inc() = i <- i + 1
let m = trace {
while test() do
printfn "i is %i" i
inc()
}

Handling exceptions with "try..with"

Exception handling is implemented in a similar way.
If we look at a try..with expression for example, it has two parts:
There is the body of the "try", evaluated once. In a computation expressions, this will be
a delayed function that evaluates to a wrapped value. The body function has no
parameters, and so its signature is just unit -> wrapped type .
The "with" part handles the exception. It has an exception as a parameters, and returns
the same type as the "try" part, so its signature is exception -> wrapped type .
With this in place, we can create pseudo-code for the exception handler:
try
let wrapped = delayedBody()
wrapped // return a wrapped value
with
| e -> handlerPart e

And this maps exactly to a standard implementation:

member this.TryWith(body, handler) =
try
printfn "TryWith Body"
this.ReturnFrom(body())
with
e ->
printfn "TryWith Exception handling"
handler e

As you can see, it is common to use pass the returned value through ReturnFrom so that it
gets the same treatment as other wrapped values.

775

Implementing a builder: The rest of the standard methods

Here is an example snippet to test how the handling works:

trace {
try
failwith "bang"
with
| e -> printfn "Exception! %s" e.Message
} |> printfn "Result %A"

Implementing "try..finally"
try..finally is very similar to try..with .

There is the body of the "try", evaluated once. The body function has no parameters,
and so its signature is unit -> wrapped type .
The "finally" part is always called. It has no parameters, and returns a unit, so its
signature is unit -> unit .
Just as with try..with , the standard implementation is obvious.
member this.TryFinally(body, compensation) =
try
printfn "TryFinally Body"
this.ReturnFrom(body())
finally
printfn "TryFinally compensation"
compensation()

Another little snippet:

trace {
try
failwith "bang"
finally
printfn "ok"
} |> printfn "Result %A"

Implementing "using"
The final method to implement is Using . This is the builder method for implementing the
use! keyword.

This is what the MSDN documentation says about use! :

776

Implementing a builder: The rest of the standard methods

{| use! value = expr in cexpr |}

is translated to:
builder.Bind(expr, (fun value -> builder.Using(value, (fun value -> {| cexpr |} ))))

In other words, the use! keyword triggers both a Bind and a Using . First a Bind is done
to unpack the wrapped value, and then the unwrapped disposable is passed into Using to
ensure disposal, with the continuation function as the second parameter.
Implementing this is straightforward. Similar to the other methods, we have a body, or
continuation part, of the "using" expression, which is evaluated once. This body function has
a "disposable" parameter, and so its signature is #IDisposable -> wrapped type .
Of course we want to ensure that the disposable value is always disposed no matter what,
so we need to wrap the call to the body function in a TryFinally .
Here's a standard implementation:
member this.Using(disposable:#System.IDisposable, body) =
let body' = fun () -> body disposable
this.TryFinally(body', fun () ->
match disposable with
| null -> ()
| disp -> disp.Dispose())

Notes:
The parameter to TryFinally is a unit -> wrapped , with a unit as the first parameter,
so we created a delayed version of the body that is passed in.
Disposable is a class, so it could be null , and we have to handle that case specially.
Otherwise we just dispose it in the "finally" continuation.
Here's a demonstration of Using in action. Note that the makeResource makes a wrapped
disposable. If it wasn't wrapped, we wouldn't need the special use! and could just use a
normal use instead.

777

Implementing a builder: The rest of the standard methods

let makeResource name =

Some {
new System.IDisposable with
member this.Dispose() = printfn "Disposing %s" name
}
trace {
use! x = makeResource "hello"
printfn "Disposable in use"
return 1
} |> printfn "Result: %A"

"For" revisited
Finally, we can revisit how For is implemented. In the previous examples, For took a
simple list parameter. But with Using and While under our belts, we can change it to
accept any IEnumerable<_> or sequence.
Here's the standard implementation for For now:
member this.For(sequence:seq<_>, body) =
this.Using(sequence.GetEnumerator(),fun enum ->
this.While(enum.MoveNext,
this.Delay(fun () -> body enum.Current)))
{% endhighlight fsharp %}
As you can see, it is quite different from the previous implementation, in order to ha
ndle a generic ÌEnumerable<_>`.
* We explicitly iterate using an ÌEnumerator<_>`.
* ÌEnumerator<_>` implements ÌDisposable`, so we wrap the enumerator in a Ùsing`.
* We use `While .. MoveNext` to iterate.
* Next, we pass the ènum.Current` into the body function
* Finally, we delay the call to the body function using `Delay`
## Complete code without tracing
Up to now, all the builder methods have been made more complex than necessary by the a
dding of tracing and printing expressions. The tracing is helpful to understand what i
s going on,
but it can obscure the simplicity of the methods.
So as a final step, let's have a look at the complete code for the "trace" builder cla
ss, but this time without any extraneous code at all. Even though the code is cryptic
, the purpose and implementation of each method should now be familiar to you.
```fsharp
type TraceBuilder() =

778

Implementing a builder: The rest of the standard methods

member this.Bind(m, f) =
Option.bind f m
member this.Return(x) = Some x
member this.ReturnFrom(x) = x
member this.Yield(x) = Some x
member this.YieldFrom(x) = x
member this.Zero() = this.Return ()
member this.Delay(f) = f
member this.Run(f) = f()
member this.While(guard, body) =
if not (guard())
then this.Zero()
else this.Bind( body(), fun () ->
this.While(guard, body))
member this.TryWith(body, handler) =
try this.ReturnFrom(body())
with e -> handler e
member this.TryFinally(body, compensation) =
try this.ReturnFrom(body())
finally compensation()
member this.Using(disposable:#System.IDisposable, body) =
let body' = fun () -> body disposable
this.TryFinally(body', fun () ->
match disposable with
| null -> ()
| disp -> disp.Dispose())
member this.For(sequence:seq<_>, body) =
this.Using(sequence.GetEnumerator(),fun enum ->
this.While(enum.MoveNext,
this.Delay(fun () -> body enum.Current)))

After all this discussion, the code seems quite tiny now. And yet this builder implements
every standard method, uses delayed functions. A lot of functionality in a just a few lines!

779

Organizing modules in a project

Before we move on to any coding in the recipe, let's look at the overall structure of a F#
project. In particular: (a) what code should be in which modules and (b) how the modules
should be organized within a project.

How not to do it
A newcomer to F# might be tempted to organize code in classes just like in C#. One class
per file, in alphabetical order. After all, F# supports the same object-oriented features that C#
does, right? So surely the F# code can be organized the same way as C# code?
After a while, this is often followed by the discovery that F# requires files (and code within a
file) to be in dependency order. That is, you cannot use forward references to code that
hasn't been seen by the compiler yet**.
This is followed by general annoyance and swearing. How can F# be so stupid? Surely it
impossible to write any kind of large project!
In this post, we'll look at one simple way to organize your code so that this doesn't happen.
** The and keyword can be used in some cases to allow mutual recursion, but is
discouraged.

The functional approach to layered design

A standard way of thinking about code is to group it into layers: a domain layer, a
presentation layer, and so on, like this:

Each layer contains only the code that is relevant to that layer.

780

Organizing modules in a project

But in practice, it is not that simple, as there are dependencies between each layer. The
domain layer depends on the infrastructure, and the presentation layer depends on the
domain.
And most importantly, the domain layer should not depend on the persistence layer. That is,
it should be "persistence agnostic".
We therefore need to tweak the layer diagram to look more like this (where each arrow
represents a dependency):

And ideally this reorganization would be made even more fine grained, with a separate
"Service Layer", containing application services, domain services, etc. And when we are
finished, the core domain classes are "pure" and have no dependencies on anything else
outside the domain. This is often called a "hexagonal architecture" or "onion architecture".
But this post is not about the subtleties of OO design, so for now, let's just work with the
simpler model.

Separating behavior from types

"It is better to have 100 functions operate on one data structure than 10 functions on 10 data
structures" -- Alan Perlis
In a functional design, it is very important to separate behavior from data. The data types are
simple and "dumb". And then separately, you have a number of functions that act on those
data types.
This is the exact opposite of an object-oriented design, where behavior and data are meant
to be combined. After all, that's exactly what a class is. In a truly object-oriented design in
fact, you should have nothing but behavior -- the data is private and can only be accessed
via methods.

781

Organizing modules in a project

In fact, in OOD, not having enough behavior around a data type is considered a Bad Thing,
and even has a name: the "anemic domain model".
In functional design though, having "dumb data" with transparency is preferred. It is normally
fine for the data to be exposed without being encapsulated. The data is immutable, so it
can't get "damaged" by a misbehaving function. And it turns out that the focus on
transparent data allows for more code that is more flexible and generic.
If you haven't seen it, I highly recommend Rich Hickey's excellent talk on "The Value of
Values", which explains the benefits of this approach.

Type layers and behavior layers

So how does this apply to our layered design from above?
First, we must separate each layer into two distinct parts:
Data Types. Data structures that are used by that layer.
Logic. Functions that are implemented in that layer.
Once we have separated these two elements, our diagram will look like this:

Notice though, that we might have some backwards references (shown by the red arrow).
For example, a function in the domain layer might depend on a persistence-related type,
such as IRepository .
In an OO design, we would add more layers (e.g. application services) to handle this. But in
a functional design, we don't need to -- we can just move the persistence-related types to a
different place in the hierarchy, underneath the domain functions, like this:

782

Organizing modules in a project

In this design, we have now eliminated all cyclic references between layers. All the arrows
point down.
And this without having to create any extra layers or overhead.
Finally, we can translate this layered design into F# files by turning it upside down.
The first file in the project should contain code which has no dependencies. This
represents the functionality at the bottom of the layer diagram. It is generally a set of
types, such the infrastructure or domain types.
The next file depends only on the first file. It would represents the functionality at the
next-to-bottom layer.
And so on. Each file depends only on the previous ones.
So, if we refer back to the use case example discussed in Part 1:

783

Organizing modules in a project

then the corresponding code in an F# project might look something like this:

At the very bottom of the list is the main file, called "main" or "program", which contains the
entry point for the program.
And just above it is the code for the use cases in the application. The code in this file is
where all the functions from all the other modules are "glued together" into a single function
that represents a particular use case or service request. (The nearest equivalent of this in an
OO design are the "application services", which serve roughly the same purpose.)

784

Organizing modules in a project

And then just above that is the "UI layer" and then the "DB layer" and so on, until you get to
the top.
What's nice about this approach is that, if you are a newcomer to a code base, you always
know where to start. The first few files will always be the "bottom layer" of an application and
the last few files will always be the "top layer". No folders needed!

Putting code in modules, not classes

A common question from newcomers to F# is "how should I organize my code if I don't use
classes?"
The answer is: modules. As you know, in an object oriented program, a data structure and
the functions that act on it would be combined in a class. However in functional-style F#, a
data structure and the functions that act on it are contained in modules instead.
There are three common patterns for mixing types and functions together:
having the type declared in the same module as the functions.
having the type declared separately from the functions but in the same file.
having the type declared separately from the functions and in a different file, typically
containing type definitions only.
In the first approach, types are defined inside the module along with their related functions. If
there is only one primary type, it is often given a simple name such as "T" or the name of the
module.
Here's an example:
namespace Example
// declare a module
module Person =
type T = {First:string; Last:string}
// constructor
let create first last =
{First=first; Last=last}
// method that works on the type
let fullName {First=first; Last=last} =
first + " " + last

785

Organizing modules in a project

So the functions are accessed with names like Person.create and Person.fullName while
the type itself is accessed with the name Person.T .
In the second approach, types are declared in the same file, but outside any module:
namespace Example
// declare the type outside the module
type PersonType = {First:string; Last:string}
// declare a module for functions that work on the type
module Person =
// constructor
let create first last =
{First=first; Last=last}
// method that works on the type
let fullName {First=first; Last=last} =
first + " " + last

In this case, the functions are accessed with the same names ( Person.create and
Person.fullName ) while the type itself is accessed with the name such as PersonType .

And finally, here's the third approach. The type is declared in a special "types-only" module
(typically in a different file):
// =========================
// File: DomainTypes.fs
// =========================
namespace Example
// "types-only" module
[<AutoOpen>]
module DomainTypes =
type Person = {First:string; Last:string}
type OtherDomainType = ...
type ThirdDomainType = ...

In this particular case, the AutoOpen attribute has been used to make the types in this
module automatically visible to all the other modules in the project -- making them "global".
And then a different module contains all the functions that work on, say, the Person type.

786

Organizing modules in a project

// =========================
// File: Person.fs
// =========================
namespace Example
// declare a module for functions that work on the type
module Person =
// constructor
let create first last =
{First=first; Last=last}
// method that works on the type
let fullName {First=first; Last=last} =
first + " " + last

Note that in this example, both the type and the module are called Person . This is not
normally a problem in practice, as the compiler can normally figure out what you want.
So, if you write this:
let f (p:Person) = p.First

Then the compiler will understand that you are referring to the Person type.
On the other hand, if you write this:
let g () = Person.create "Alice" "Smith"

Then the compiler will understand that you are referring to the Person module.
For more on modules, see the post on organizing functions.

The organization of the modules

For our recipe we will use a mixture of approaches, with the following guidelines:
Module Guidelines
If a type is shared among multiple modules, then put it in a special types-only module.
For example, if a type is used globally (or to be precise, within a "bounded domain" in
DDD-speak), I would put it in a module called DomainTypes or DomainModel , which
comes early in the compilation order.
If a type is used only in a subsystem, such as a type shared by a number of UI modules,

787

Organizing modules in a project

then I would put it in a module called UITypes , which would come just before the other
UI modules in the compilation order.
If a type is private to a module (or two) then put it in the same module as its related
functions.
For example, a type that was used only for validation would be put in the Validation
module. A type used only for database access would be put in the Database module,
and so on.
Of course, there are many ways to organize types, but these guidelines act as a good
default starting point.

Dude, where are my folders?

A common complaint is that F# projects do not support a folder structure, which supposedly
makes it hard to organize large projects.
If you are doing a pure object-oriented design, then this is a legitimate complaint. But as you
can see from the discussion above, having a linear list of modules is very helpful (if not
strictly necessary) to ensure that the dependencies are maintained correctly. Yes, in theory,
the files could be scattered about and the compiler might be able to figure out the correct
compilation order, but in practice, it's not easy for the compiler to determine this order.
Even more importantly, it's not easy for a human to determine the correct order either, and
so it would make maintenance more painful than it needed to be.
In reality, even for large projects, not having folders is not as much of a problem as you
might think. There are a number of large F# projects which successfully work within this
limitation, such as the F# compiler itself. See the post on cycles and modularity in the wild
for more.

Help, I have mutual dependencies between my types

If you are coming from an OO design, you might run into mutual dependencies between
types, such as this example, which won't compile:
type Location = {name: string; workers: Employee list}
type Employee = {name: string; worksAt: Location}

How can you fix this to make the F# compiler happy?

788

Organizing modules in a project

It's not that hard, but it does requires some more explanation, so I have devoted another
whole post to dealing with cyclic dependencies.

Example code
Let's revisit at the code we have so far, but this time organized into modules.
Each module below would typically become a separate file.
Be aware that this is still a skeleton. Some of the modules are missing, and some of the
modules are almost empty.
This kind of organization would be overkill for a small project, but there will be lots more
code to come!
/// ===========================================
/// Common types and functions shared across multiple projects
/// ===========================================
module CommonLibrary =
// the two-track type
type Result<'TSuccess,'TFailure> =
| Success of 'TSuccess
| Failure of 'TFailure
// convert a single value into a two-track result
let succeed x =
Success x
// convert a single value into a two-track result
let fail x =
Failure x
// appy either a success function or failure function
let either successFunc failureFunc twoTrackInput =
match twoTrackInput with
| Success s -> successFunc s
| Failure f -> failureFunc f

// convert a switch function into a two-track function

let bind f =
either f fail
// pipe a two-track value into a switch function
let (>>=) x f =
bind f x
// compose two switches into another switch

789

Organizing modules in a project

let (>=>) s1 s2 =
s1 >> bind s2
// convert a one-track function into a switch
let switch f =
f >> succeed
// convert a one-track function into a two-track function
let map f =
either (f >> succeed) fail
// convert a dead-end function into a one-track function
let tee f x =
f x; x
// convert a one-track function into a switch with exception handling
let tryCatch f exnHandler x =
try
f x |> succeed
with
| ex -> exnHandler ex |> fail
// convert two one-track functions into a two-track function
let doubleMap successFunc failureFunc =
either (successFunc >> succeed) (failureFunc >> fail)
// add two switches in parallel
let plus addSuccess addFailure switch1 switch2 x =
match (switch1 x),(switch2 x) with
| Success s1,Success s2 -> Success (addSuccess s1 s2)
| Failure f1,Success _ -> Failure f1
| Success _ ,Failure f2 -> Failure f2
| Failure f1,Failure f2 -> Failure (addFailure f1 f2)

/// ===========================================
/// Global types for this project
/// ===========================================
module DomainTypes =
open CommonLibrary
/// The DTO for the request
type Request = {name:string; email:string}
// Many more types coming soon!
/// ===========================================
/// Logging functions
/// ===========================================
module Logger =
open CommonLibrary

790

Organizing modules in a project

open DomainTypes
let log twoTrackInput =
let success x = printfn "DEBUG. Success so far: %A" x; x
let failure x = printfn "ERROR. %A" x; x
doubleMap success failure twoTrackInput
/// ===========================================
/// Validation functions
/// ===========================================
module Validation =
open CommonLibrary
open DomainTypes
let validate1 input =
if input.name = "" then Failure "Name must not be blank"
else Success input
let validate2 input =
if input.name.Length > 50 then Failure "Name must not be longer than 50 chars"
else Success input
let validate3 input =
if input.email = "" then Failure "Email must not be blank"
else Success input
// create a "plus" function for validation functions
let (&&&) v1 v2 =
let addSuccess r1 r2 = r1 // return first
let addFailure s1 s2 = s1 + "; " + s2 // concat
plus addSuccess addFailure v1 v2
let combinedValidation =
validate1
&&& validate2
&&& validate3
let canonicalizeEmail input =
{ input with email = input.email.Trim().ToLower() }
/// ===========================================
/// Database functions
/// ===========================================
module CustomerRepository =
open CommonLibrary
open DomainTypes
let updateDatabase input =
() // dummy dead-end function for now
// new function to handle exceptions

791

Organizing modules in a project

let updateDatebaseStep =
tryCatch (tee updateDatabase) (fun ex -> ex.Message)
/// ===========================================
/// All the use cases or services in one place
/// ===========================================
module UseCases =
open CommonLibrary
open DomainTypes
let handleUpdateRequest =
Validation.combinedValidation
>> map Validation.canonicalizeEmail
>> bind CustomerRepository.updateDatebaseStep
>> Logger.log

Summary
In this post, we looked at organizing code into modules. In the next post in this series, we'll
finally start doing some real coding!
Meanwhile, you can read more on cyclic dependencies in the follow up posts:
Cyclic dependencies are evil.
Refactoring to remove cyclic dependencies.
Cycles and modularity in the wild, which compares some real-world metrics for C# and
F# projects.

792

The "Dependency cycles" Series

One of the most common complaints about F# is that it requires code to be in dependency
order. That is, you cannot use forward references to code that hasn't been seen by the
compiler yet.
In this series, I discuss dependency cycles, why they are bad, and how to get rid of them.
Cyclic dependencies are evil. Cyclic dependencies: Part 1.
Refactoring to remove cyclic dependencies. Cyclic dependencies: Part 2.
Cycles and modularity in the wild. Comparing some real-world metrics of C# and F#
projects.

793

Cyclic dependencies are evil

One of three related posts on module organization and cyclic dependencies.
One of the most common complaints about F# is that it requires code to be in dependency
order. That is, you cannot use forward references to code that hasn't been seen by the
compiler yet.
Here's a typical example:
"The order of .fs files makes it hard to compile... My F# application is just over 50 lines
of code, but it's already more work than it's worth to compile even the tiniest non-trivial
application. Is there a way to make the F# compiler more like the C# compiler, so that
it's not so tightly coupled to the order that files are passed to the compiler?" [fpish.net]
and another:
"After trying to build a slightly above-toy-size project in F#, I came to the conclusion that
with current tools it would be quite difficult to maintain a project of even moderate
complexity." [www.ikriv.com]
and another:
"F# compiler [is] too linear. The F# compiler should handle all type resolution matters
automatically, independent of declaration order" [www.sturmnet.org]
and one more:
"The topic of annoying (and IMHO unnecessary) limitations of the F# project system
was already discussed on this forum. I am talking about the way compilation order is
controlled" [fpish.net]
Well, these complaints are unfounded. You most certainly can build and maintain large
projects using F#. The F# compiler and the core library are two obvious examples.
In fact, most of these problems boil down to "why can't F# be like C#". If you are coming
from C#, you are used to having the compiler connect everything automatically. Having to
deal with dependency relationships explicitly is very annoying -- old-fashioned and
regressive, even.
The aim of this post is to explain (a) why dependency management is important, and (b)
some techniques that can help you deal with it.

794

Cyclic dependencies are evil

Dependencies are bad things...

We all know that dependencies are the bane of our existence. Assembly dependencies,
configuration dependencies, database dependencies, network dependencies -- there's
always something.
So we developers, as a profession, tend to put a lot of effort into making dependencies more
manageable. This goal manifests itself in many disparate ways: the interface segregation
principle, inversion of control and dependency injection; package management with NuGet;
configuration management with puppet/chef; and so on. In some sense all these approaches
are trying to reduce the number of things we have to be aware of, and the number of things
that can break.
This is not a new problem, of course. A large part of the classic book "Large-Scale C++
Software Design" is devoted to dependency management. As John Lakos, the author, put it:
"The maintenance cost of a subsystem can be reduced significantly by avoiding
unnecessary dependencies among components"
The key word here is "unnecessary". What is an "unnecessary" dependency? It depends, of
course. But one particular kind of dependency is almost always unnecessary -- a circular
dependency.

... and circular dependencies are evil

To understand why circular dependencies are evil, let's revisit what we mean by a
"component".
Components are Good Things. Whether you think of them as packages, assemblies,
modules, classes or whatever, their primary purpose is to break up large amounts of code
into smaller and more manageable pieces. In other words, we are applying a divide and
conquer approach to the problem of software development.
But in order to be useful for maintenance, deployment, or whatever, a component shouldn't
just be a random collection of stuff. It should (of course) group only related code together.
In an ideal world, each component would thus be completely independent of any others. But
generally (of course), some dependencies are always necessary.
But, now that we have components with dependencies, we need a way to manage these
dependencies. One standard way to do this is with the "layering" principle. We can have
"high level" layers and "low level" layers, and the critical rule is: each layer should depend
only on layers below it, and never on a layer above it.

795

Cyclic dependencies are evil

You are very familiar with this, I'm sure. Here's a diagram of some simple layers:

But now what happens when you introduce a dependency from the bottom layer to the top
layer, like this?

By having a dependency from the bottom to the top, we have introduced the evil "circular
dependency".
Why is it evil? Because any alternative layering method is now valid!
For example, we could put the bottom layer on top instead, like this:

From a logical point of view, this alternative layering is just the same as the original layering.
Or how about we put the middle layer on top?

796

Cyclic dependencies are evil

Something has gone badly wrong! It's clear that we've really messed things up.
In fact, as soon as you have any kind of circular dependency between components, the only
thing you can do is to put them all into the same layer.

In other words, the circular dependency has completely destroyed our "divide and conquer"
approach, the whole reason for having components in the first place. Rather than having
three components, we now have just one "super component", which is three times bigger
and more complicated than it needed to be.

And that's why circular dependencies are evil.

For more on this subject, see this StackOverflow answer and this article about layering by
Patrick Smacchia (of NDepend).

797

Cyclic dependencies are evil

Circular dependencies in the real world

Let's start by looking at circular dependencies between .NET assemblies. Here are some
war stories from Brian McNamara (my emphasis):
The .Net Framework 2.0 has this problem in spades; System.dll,
System.Configuration.dll, and System.Xml.dll are all hopelessly entangled with one
another. This manifests in a variety of ugly ways. For example, I found a simple [bug] in
the VS debugger that effectively crashes the debuggee when hitting a breakpoint while
trying to loads symbols, caused by the circular dependencies among these assemblies.
Another story: a friend of mine was a developer on the initial versions of Silverlight and
was tasked with trying to trim down these three assemblies, and the first arduous task
was trying to untangle the circular dependencies. "Mutual recursion for free" is very
convenient on a small scale, but it will destroy you on a large scale.
VS2008 shipped a week later than planned, because VS2008 had a dependency on
SQL server, and SQL server had a dependency on VS, and whoops! in the end they
couldn't produce a full product version where everything had the same build number,
and had to scramble to make it work. [fpish.net]
So there is plenty of evidence that circular dependencies between assemblies are bad. In
fact, circular dependencies between assemblies are considered bad enough that Visual
Studio won't even let you create them!
You might say, "Yes, I can understand why circular dependencies are bad for assemblies,
but why bother for code inside an assembly?"
Well, for exactly the same reasons! Layering allows better partitioning, easier testing and
cleaner refactoring. You can see what I mean in a related post on dependency cycles "in the
wild" where I compare C# projects and F# projects. The dependencies in the F# projects are
a lot less spaghetti-like.
Another quote from Brian's (excellent) comment:

798

Cyclic dependencies are evil

I'm evangelizing an unpopular position here, but my experience is that everything in the
world is better when you're forced to consider and manage "dependency order among
software components" at every level of the system. The specific UI/tooling for F# may
not yet be ideal, but I think the principle is right. This is a burden you want. It is more
work. "Unit testing" is also more work, but we've gotten to the point where the
consensus is that work is "worth it" in that it saves you time in the long run. I feel the
same way about 'ordering'. There are dependencies among the classes and methods in
your system. You ignore those dependencies at your own peril. A system that forces
you to consider this dependency graph (roughly, the topological sort of components) is
likely to steer you into developing software with cleaner architectures, better system
layering, and fewer needless dependencies.

Detecting and removing circular dependencies

Ok, we're agreed that circular dependencies are bad. So how do we detect them and then
get rid of them?
Let's start with detection. There are a number of tools to help you detect circular
dependencies in your code.
If you're using C#, you will need a tool like the invaluable NDepend.
And if you are using Java, there are equivalent tools such as JDepend.
But if you are using F#, you're in luck! You get circular dependency detection for free!
"Very funny," you might say, "I already know about F#'s circular dependency prohibition -- it's
driving me nuts! What can I do to fix the problem and make the compiler happy?"
For that, you'll need to read the next post...

799

Refactoring to remove cyclic dependencies

Refactoring to remove cyclic

dependencies
In the previous post, we looked at the concept of dependency cycles, and why they are bad.
In this post, we'll look at some techniques for eliminating them from your code. Having to do
this may seem annoying at first, but really, you'll come to appreciate that in the long run, "it's
not a bug, it's a feature!"

Classifying some common cyclic

dependencies
Let's classify the kinds of dependencies you're likely to run into. I'll look at three common
situations, and for each one, demonstrate some techniques for dealing with them.
First, there is what I will call a "method dependency".
Type A stores a value of type B in a property
Type B references type A in a method signature, but doesn't store a value of type A
Second, there is what I will call a "structural dependency".
Type A stores a value of type B in a property
Type B stores a value of type A in a property
Finally, there is what I will call an "inheritance dependency".
Type A stores a value of type B in a property
Type B inherits from type A
There are, of course, other variants. But if you know how to deal with these, you can use the
same techniques to deal with the others as well.

Three tips on dealing with dependencies in F#

Before we get started, here are three useful tips which apply generally when trying to
untangle dependencies.
Tip 1: Treat F# like F#.

800

Refactoring to remove cyclic dependencies

Recognize that F# is not C#. If you are willing to work with F# using its native idioms, then it
is normally very straightforward to avoid circular dependencies by using a different style of
code organization.
Tip 2: Separate types from behavior.
Since most types in F# are immutable, it is acceptable for them to be "exposed" and
"anemic", even. So in a functional design it is common to separate the types themselves
from the functions that act on them. This approach will often help to clean up dependencies,
as we'll see below.
Tip 3: Parameterize, parameterize, parameterize.
Dependencies can only happen when a specific type is referenced. If you use generic types,
you cannot have a dependency!
And rather than hard coding behavior for a type, why not parameterize it by passing in
functions instead? The List module is a great example of this approach, and I'll show
some examples below as well.

Dealing with a "method dependency"

We'll start with the simplest kind of dependency -- what I will call a "method dependency".
Here is an example.
module MethodDependencyExample =
type Customer(name, observer:CustomerObserver) =
let mutable name = name
member this.Name
with get() = name
and set(value) =
name <- value
observer.OnNameChanged(this)
and CustomerObserver() =
member this.OnNameChanged(c:Customer) =
printfn "Customer name changed to '%s' " c.Name
// test
let observer = new CustomerObserver()
let customer = Customer("Alice",observer)
customer.Name <- "Bob"

801

Refactoring to remove cyclic dependencies

The Customer class has a property/field of type CustomerObserver , but the

CustomerObserver class has a method which takes a Customer as a parameter, causing a

mutual dependency.

Using the "and" keyword

One straightforward way to get the types to compile is to use the and keyword, as I did
above.
The and keyword is designed for just this situation -- it allows you to have two or more
types that refer to each other.
To use it, just replace the second type keyword with and . Note that using and type , as
shown below, is incorrect. Just the single and is all you need.
type Something
and type SomethingElse // wrong
type Something
and SomethingElse // correct

But and has a number of problems, and using it is generally discouraged except as a last
resort.
First, it only works for types declared in the same module. You can't use it across module
boundaries.
Second, it should really only be used for tiny types. If you have 500 lines of code between
the type and the and , then you are doing something very wrong.
type Something
// 500 lines of code
and SomethingElse
// 500 more lines of code

The code snippet shown above is an example of how not to do it.

In other words, don't treat and as a panacea. Overusing it is a symptom that you have not
refactored your code properly.

Introducing parameterization
So, instead of using and , let's see what we can do using parameterization, as mentioned in
the third tip.

802

Refactoring to remove cyclic dependencies

If we think about the example code, do we really need a special CustomerObserver class?
Why have we restricted it to Customer only? Can't we have a more generic observer class?
So why don't we create a INameObserver<'T> interface instead, with the same
OnNameChanged method, but the method (and interface) parameterized to accept any class?

Here's what I mean:

module MethodDependency_ParameterizedInterface =
type INameObserver<'T> =
abstract OnNameChanged : 'T -> unit
type Customer(name, observer:INameObserver<Customer>) =
let mutable name = name
member this.Name
with get() = name
and set(value) =
name <- value
observer.OnNameChanged(this)
type CustomerObserver() =
interface INameObserver<Customer> with
member this.OnNameChanged c =
printfn "Customer name changed to '%s' " c.Name
// test
let observer = new CustomerObserver()
let customer = Customer("Alice", observer)
customer.Name <- "Bob"

In this revised version, the dependency has been broken! No and is needed at all. In fact,
you could even put the types in different projects or assemblies now!
The code is almost identical to the first version, except that the Customer constructor
accepts a interface, and CustomerObserver now implements the same interface. In fact, I
would argue that introducing the interface has actually made the code better than before.
But we don't have to stop there. Now that we have an interface, do we really need to create
a whole class just to implement it? F# has a great feature called object expressions which
allows you to instantiate an interface directly.
Here is the same code again, but this time the CustomerObserver class has been eliminated
completely and the INameObserver created directly.

803

Refactoring to remove cyclic dependencies

module MethodDependency_ParameterizedInterface =
// code as above
// test
let observer2 = {
new INameObserver<Customer> with
member this.OnNameChanged c =
printfn "Customer name changed to '%s' " c.Name
}
let customer2 = Customer("Alice", observer2)
customer2.Name <- "Bob"

This technique will obviously work for more complex interfaces as well, such as that shown
below, where there are two methods:
module MethodDependency_ParameterizedInterface2 =
type ICustomerObserver<'T> =
abstract OnNameChanged : 'T -> unit
abstract OnEmailChanged : 'T -> unit
type Customer(name, email, observer:ICustomerObserver<Customer>) =
let mutable name = name
let mutable email = email
member this.Name
with get() = name
and set(value) =
name <- value
observer.OnNameChanged(this)
member this.Email
with get() = email
and set(value) =
email <- value
observer.OnEmailChanged(this)
// test
let observer2 = {
new ICustomerObserver<Customer> with
member this.OnNameChanged c =
printfn "Customer name changed to '%s' " c.Name
member this.OnEmailChanged c =
printfn "Customer email changed to '%s' " c.Email
}
let customer2 = Customer("Alice", "x@example.com",observer2)
customer2.Name <- "Bob"
customer2.Email <- "y@example.com"

804

Refactoring to remove cyclic dependencies

Using functions instead of parameterization

In many cases, we can go even further and eliminate the interface class as well. Why not
just pass in a simple function that is called when the name changes, like this:
module MethodDependency_ParameterizedClasses_HOF =
type Customer(name, observer) =
let mutable name = name
member this.Name
with get() = name
and set(value) =
name <- value
observer this
// test
let observer(c:Customer) =
printfn "Customer name changed to '%s' " c.Name
let customer = Customer("Alice", observer)
customer.Name <- "Bob"

I think you'll agree that this snippet is "lower ceremony" than either of the previous versions.
The observer is now defined inline as needed, very simply:
let observer(c:Customer) =
printfn "Customer name changed to '%s' " c.Name

True, it only works when the interface being replaced is simple, but even so, this approach
can be used more often than you might think.

A more functional approach: separating types

from functions
As I mentioned above, a more "functional design" would be to separate the types
themselves from the functions that act on those types. Let's see how this might be done in
this case.
Here is a first pass:

805

Refactoring to remove cyclic dependencies

module MethodDependencyExample_SeparateTypes =
module DomainTypes =
type Customer = { name:string; observer:NameChangedObserver }
and NameChangedObserver = Customer -> unit

module Customer =
open DomainTypes
let changeName customer newName =
let newCustomer = {customer with name=newName}
customer.observer newCustomer
newCustomer // return the new customer
module Observer =
open DomainTypes
let printNameChanged customer =
printfn "Customer name changed to '%s' " customer.name
// test
module Test =
open DomainTypes
let observer = Observer.printNameChanged
let customer = {name="Alice"; observer=observer}
Customer.changeName customer "Bob"

In the example above, we now have three modules: one for the types, and one each for the
functions. Obviously, in a real application, there will be a lot more Customer related functions
in the Customer module than just this one!
In this code, though, we still have the mutual dependency between Customer and
CustomerObserver . The type definitions are more compact, so it is not such a problem, but

even so, can we eliminate the and ?

Yes, of course. We can use the same trick as in the previous approach, eliminating the
observer type and embedding a function directly in the Customer data structure, like this:

806

Refactoring to remove cyclic dependencies

module MethodDependency_SeparateTypes2 =
module DomainTypes =
type Customer = { name:string; observer:Customer -> unit}
module Customer =
open DomainTypes
let changeName customer newName =
let newCustomer = {customer with name=newName}
customer.observer newCustomer
newCustomer // return the new customer
module Observer =
open DomainTypes
let printNameChanged customer =
printfn "Customer name changed to '%s' " customer.name
module Test =
open DomainTypes
let observer = Observer.printNameChanged
let customer = {name="Alice"; observer=observer}
Customer.changeName customer "Bob"

Making types dumber

The Customer type still has some behavior embedded in it. In many cases, there is no need
for this. A more functional approach would be to pass a function only when you need it.
So let's remove the observer from the customer type, and pass it as an extra parameter to
the changeName function, like this:
let changeName observer customer newName =
let newCustomer = {customer with name=newName}
observer newCustomer // call the observer with the new customer
newCustomer // return the new customer

Here's the complete code:

807

Refactoring to remove cyclic dependencies

module MethodDependency_SeparateTypes3 =
module DomainTypes =
type Customer = {name:string}
module Customer =
open DomainTypes
let changeName observer customer newName =
let newCustomer = {customer with name=newName}
observer newCustomer // call the observer with the new customer
newCustomer // return the new customer
module Observer =
open DomainTypes
let printNameChanged customer =
printfn "Customer name changed to '%s' " customer.name
module Test =
open DomainTypes
let observer = Observer.printNameChanged
let customer = {name="Alice"}
Customer.changeName observer customer "Bob"

You might be thinking that I have made things more complicated now -- I have to specify the
observer function everywhere I call changeName in my code. Surely this is worse than

before? At least in the OO version, the observer was part of the customer object and I didn't
have to keep passing it in.
Ah, but, you're forgetting the magic of partial application! You can set up a function with the
observer "baked in", and then use that function everywhere, without needing to pass in an
observer every time you use it. Clever!

808

Refactoring to remove cyclic dependencies

module MethodDependency_SeparateTypes3 =
// code as above
module TestWithPartialApplication =
open DomainTypes
let observer = Observer.printNameChanged
// set up this partial application only once (at the top of your module, say)
let changeName = Customer.changeName observer
// then call changeName without needing an observer
let customer = {name="Alice"}
changeName customer "Bob"

But wait... there's more!

Let's look at the changeName function again:
let changeName observer customer newName =
let newCustomer = {customer with name=newName}
observer newCustomer // call the observer with the new customer
newCustomer // return the new customer

It has the following steps:

1. do something to make a result value
2. call the observer with the result value
3. return the result value
This is completely generic logic -- it has nothing to do with customers at all. So we can
rewrite it as a completely generic library function. Our new function will allow any observer
function to "hook into" into the result of any other function, so let's call it hook for now.
let hook2 observer f param1 param2 =
let y = f param1 param2 // do something to make a result value
observer y // call the observer with the result value
y // return the result value

Actually, I called it hook2 because the function f being "hooked into" has two parameters.
I could make another version for functions that have one parameter, like this:

809

Refactoring to remove cyclic dependencies

let hook observer f param1 =

let y = f param1 // do something to make a result value
observer y // call the observer with the result value
y // return the result value

If you have read the railway oriented programming post, you might notice that this is quite
similar to what I called a "dead-end" function. I won't go into more details here, but this is
indeed a common pattern.
Ok, back to the code -- how do we use this generic hook function?
Customer.changeName is the function being hooked into, and it has two parameters, so

we use hook2 .
The observer function is just as before
So, again, we create a partially applied changeName function, but this time we create it by
passing the observer and the hooked function to hook2 , like this:
let observer = Observer.printNameChanged
let changeName = hook2 observer Customer.changeName

Note that the resulting changeName has exactly the same signature as the original
Customer.changeName function, so it can be used interchangably with it anywhere.

let customer = {name="Alice"}

changeName customer "Bob"

Here's the complete code:

810

Refactoring to remove cyclic dependencies

module MethodDependency_SeparateTypes_WithHookFunction =
[<AutoOpen>]
module MyFunctionLibrary =
let hook observer f param1 =
let y = f param1 // do something to make a result value
observer y // call the observer with the result value
y // return the result value
let hook2 observer f param1 param2 =
let y = f param1 param2 // do something to make a result value
observer y // call the observer with the result value
y // return the result value
module DomainTypes =
type Customer = { name:string}
module Customer =
open DomainTypes
let changeName customer newName =
{customer with name=newName}
module Observer =
open DomainTypes
let printNameChanged customer =
printfn "Customer name changed to '%s' " customer.name
module TestWithPartialApplication =
open DomainTypes
// set up this partial application only once (at the top of your module, say)
let observer = Observer.printNameChanged
let changeName = hook2 observer Customer.changeName
// then call changeName without needing an observer
let customer = {name="Alice"}
changeName customer "Bob"

Creating a hook function like this might seem to add extra complication initially, but it has
eliminated yet more code from the main application, and once you have built up a library of
functions like this, you will find uses for them everywhere.
By the way, if it helps you to use OO design terminology, you can think of this approach as a
"Decorator" or "Proxy" pattern.

Dealing with a "structural dependency"

811

Refactoring to remove cyclic dependencies

The second of our classifications is what I am calling a "structural dependency", where each
type stores a value of the other type.
Type A stores a value of type B in a property
Type B stores a value of type A in a property
For this set of examples, consider an Employee who works at a Location . The Employee
contains the Location they work at, and the Location stores a list of Employees who work
there.
Voila -- mutual dependency!
Here is the example in code:
module StructuralDependencyExample =
type Employee(name, location:Location) =
member this.Name = name
member this.Location = location
and Location(name, employees: Employee list) =
member this.Name = name
member this.Employees = employees

Before we get on to refactoring, let's consider how awkward this design is. How can we
initialize an Employee value without having a Location value, and vice versa.
Here's one attempt. We create a location with an empty list of employees, and then create
other employees using that location:
module StructuralDependencyExample =
// code as above
module Test =
let location = new Location("CA",[])
let alice = new Employee("Alice",location)
let bob = new Employee("Bob",location)
location.Employees // empty!
|> List.iter (fun employee ->
printfn "employee %s works at %s" employee.Name employee.Location.Name)

But this code doesn't work as we want. We have to set the list of employees for location
as empty because we can't forward reference the alice and bob values..

812

Refactoring to remove cyclic dependencies

F# will sometimes allow you to use the and keyword in these situation too, for recursive
"lets". Just as with "type", the "and" keyword replaces the "let" keyword. Unlike "type", the
first "let" has to be marked as recursive with let rec .
Let's try it. We will give location a list of alice and bob even though they are not
declared yet.
module UncompilableTest =
let rec location = new Location("NY",[alice;bob])
and alice = new Employee("Alice",location )
and bob = new Employee("Bob",location )

But no, the compiler is not happy about the infinite recursion that we have created. In some
cases, and does indeed work for let definitions, but this is not one of them! And anyway,
just as for types, having to use and for "let" definitions is a clue that you might need to
refactor.
So, really, the only sensible solution is to use mutable structures, and to fix up the location
object after the individual employees have been created, like this:
module StructuralDependencyExample_Mutable =
type Employee(name, location:Location) =
member this.Name = name
member this.Location = location
and Location(name, employees: Employee list) =
let mutable employees = employees
member this.Name = name
member this.Employees = employees
member this.SetEmployees es =
employees <- es
module TestWithMutableData =
let location = new Location("CA",[])
let alice = new Employee("Alice",location)
let bob = new Employee("Bob",location)
// fixup after creation
location.SetEmployees [alice;bob]
location.Employees
|> List.iter (fun employee ->
printfn "employee %s works at %s" employee.Name employee.Location.Name)

So, a lot of trouble just to create some values. This is another reason why mutual
dependencies are a bad idea!

813

Refactoring to remove cyclic dependencies

Parameterizing again
To break the dependency, we can use the parameterization trick again. We can just create a
parameterized vesion of Employee .
module StructuralDependencyExample_ParameterizedClasses =
type ParameterizedEmployee<'Location>(name, location:'Location) =
member this.Name = name
member this.Location = location
type Location(name, employees: ParameterizedEmployee<Location> list) =
let mutable employees = employees
member this.Name = name
member this.Employees = employees
member this.SetEmployees es =
employees <- es
type Employee = ParameterizedEmployee<Location>
module Test =
let location = new Location("CA",[])
let alice = new Employee("Alice",location)
let bob = new Employee("Bob",location)
location.SetEmployees [alice;bob]
location.Employees // non-empty!
|> List.iter (fun employee ->
printfn "employee %s works at %s" employee.Name employee.Location.Name)

Note that we create a type alias for Employee , like this:

type Employee = ParameterizedEmployee<Location>

One nice thing about creating an alias like that is that the original code for creating
employees will continue to work unchanged.
let alice = new Employee("Alice",location)

Parameterizing with behavior dependencies

The code above assumes that the particular class being parameterized over is not
important. But what if there are dependencies on particular properties of the type?
For example, let's say that the Employee class expects a Name property, and the Location
class expects an Age property, like this:

814

Refactoring to remove cyclic dependencies

module StructuralDependency_WithAge =
type Employee(name, age:float, location:Location) =
member this.Name = name
member this.Age = age
member this.Location = location
// expects Name property
member this.LocationName = location.Name
and Location(name, employees: Employee list) =
let mutable employees = employees
member this.Name = name
member this.Employees = employees
member this.SetEmployees es =
employees <- es
// expects Age property
member this.AverageAge =
employees |> List.averageBy (fun e -> e.Age)
module Test =
let location = new Location("CA",[])
let alice = new Employee("Alice",20.0,location)
let bob = new Employee("Bob",30.0,location)
location.SetEmployees [alice;bob]
printfn "Average age is %g" location.AverageAge

How can we possibly parameterize this?

Well, let's try using the same approach as before:
module StructuralDependencyWithAge_ParameterizedError =
type ParameterizedEmployee<'Location>(name, age:float, location:'Location) =
member this.Name = name
member this.Age = age
member this.Location = location
member this.LocationName = location.Name // error
type Location(name, employees: ParameterizedEmployee<Location> list) =
let mutable employees = employees
member this.Name = name
member this.Employees = employees
member this.SetEmployees es =
employees <- es
member this.AverageAge =
employees |> List.averageBy (fun e -> e.Age)

815

Refactoring to remove cyclic dependencies

The Location is happy with ParameterizedEmployee.Age , but location.Name fails to

compile. obviously, because the type parameter is too generic.
One way would be to fix this by creating interfaces such as ILocation and IEmployee , and
that might often be the most sensible approach.
But another way is to let the Location parameter be generic and pass in an additional
function that knows how to handle it. In this case a getLocationName function.
module StructuralDependencyWithAge_ParameterizedCorrect =
type ParameterizedEmployee<'Location>(name, age:float, location:'Location, getLoca
tionName) =
member this.Name = name
member this.Age = age
member this.Location = location
member this.LocationName = getLocationName location // ok
type Location(name, employees: ParameterizedEmployee<Location> list) =
let mutable employees = employees
member this.Name = name
member this.Employees = employees
member this.SetEmployees es =
employees <- es
member this.AverageAge =
employees |> List.averageBy (fun e -> e.Age)

One way of thinking about this is that we are providing the behavior externally, rather than as
part of the type.
To use this then, we need to pass in a function along with the type parameter. This would be
annoying to do all the time, so naturally we will wrap it in a function, like this:
module StructuralDependencyWithAge_ParameterizedCorrect =
// same code as above
// create a helper function to construct Employees
let Employee(name, age, location) =
let getLocationName (l:Location) = l.Name
new ParameterizedEmployee<Location>(name, age, location, getLocationName)

With this in place, the original test code continues to work, almost unchanged (we have to
change new Employee to just Employee ).

816

Refactoring to remove cyclic dependencies

module StructuralDependencyWithAge_ParameterizedCorrect =
// same code as above
module Test =
let location = new Location("CA",[])
let alice = Employee("Alice",20.0,location)
let bob = Employee("Bob",30.0,location)
location.SetEmployees [alice;bob]
location.Employees // non-empty!
|> List.iter (fun employee ->
printfn "employee %s works at %s" employee.Name employee.LocationName)

The functional approach: separating types

from functions again
Now let's apply the functional design approach to this problem, just as we did before.
Again, we'll separate the types themselves from the functions that act on those types.

817

Refactoring to remove cyclic dependencies

module StructuralDependencyExample_SeparateTypes =
module DomainTypes =
type Employee = {name:string; age:float; location:Location}
and Location = {name:string; mutable employees: Employee list}
module Employee =
open DomainTypes
let Name (employee:Employee) = employee.name
let Age (employee:Employee) = employee.age
let Location (employee:Employee) = employee.location
let LocationName (employee:Employee) = employee.location.name
module Location =
open DomainTypes
let Name (location:Location) = location.name
let Employees (location:Location) = location.employees
let AverageAge (location:Location) =
location.employees |> List.averageBy (fun e -> e.age)
module Test =
open DomainTypes
let location = { name="NY"; employees= [] }
let alice = {name="Alice"; age=20.0; location=location }
let bob = {name="Bob"; age=30.0; location=location }
location.employees <- [alice;bob]
Location.Employees location
|> List.iter (fun e ->
printfn "employee %s works at %s" (Employee.Name e) (Employee.LocationName
e) )

Before we go any further, let's remove some unneeded code. One nice thing about using a
record type is that you don't need to define "getters", so the only functions you need in the
modules are functions that manipulate the data, such as AverageAge .

818

Refactoring to remove cyclic dependencies

module StructuralDependencyExample_SeparateTypes2 =
module DomainTypes =
type Employee = {name:string; age:float; location:Location}
and Location = {name:string; mutable employees: Employee list}
module Employee =
open DomainTypes
let LocationName employee = employee.location.name
module Location =
open DomainTypes
let AverageAge location =
location.employees |> List.averageBy (fun e -> e.age)

Parameterizing again
Once again, we can remove the dependency by creating a parameterized version of the
types.
Let's step back and think about the "location" concept. Why does a location have to only
contain Employees? If we make it a bit more generic, we could consider a location as being
a "place" plus "a list of things at that place".
For example, if the things are products, then a place full of products might be a warehouse.
If the things are books, then a place full of books might be a library.
Here are these concepts expressed in code:
module LocationOfThings =
type Location<'Thing> = {name:string; mutable things: 'Thing list}
type Employee = {name:string; age:float; location:Location<Employee> }
type WorkLocation = Location<Employee>
type Product = {SKU:string; price:float }
type Warehouse = Location<Product>
type Book = {title:string; author:string}
type Library = Location<Book>

Of course, these locations are not exactly the same, but there might be something in
common that you can extract into a generic design, especially as there is no behavior
requirement attached to the things they contain.

819

Refactoring to remove cyclic dependencies

So, using the "location of things" design, here is our dependency rewritten to use
parameterized types.
module StructuralDependencyExample_SeparateTypes_Parameterized =
module DomainTypes =
type Location<'Thing> = {name:string; mutable things: 'Thing list}
type Employee = {name:string; age:float; location:Location<Employee> }
module Employee =
open DomainTypes
let LocationName employee = employee.location.name
module Test =
open DomainTypes
let location = { name="NY"; things = [] }
let alice = {name="Alice"; age=20.0; location=location }
let bob = {name="Bob"; age=30.0; location=location }
location.things <- [alice;bob]
let employees = location.things
employees
|> List.iter (fun e ->
printfn "employee %s works at %s" (e.name) (Employee.LocationName e) )
let averageAge =
employees
|> List.averageBy (fun e -> e.age)

In this revised design you will see that the AverageAge function has been completely
removed from the Location module. There is really no need for it, because we can do
these kinds of calculations quite well "inline" without needing the overhead of special
functions.
And if you think about it, if we did need to have such a function pre-defined, it would
probably be more appropriate to put in the Employee module rather than the Location
module. After all, the functionality is much more related to how employees work than how
locations work.
Here's what I mean:
module Employee =
let AverageAgeAtLocation location =
location.things |> List.averageBy (fun e -> e.age)

820

Refactoring to remove cyclic dependencies

This is one advantage of modules over classes; you can mix and match functions with
different types, as long as they are all related to the underlying use cases.

Moving relationships into distinct types

In the examples so far, the "list of things" field in location has had to be mutable. How can
we work with immutable types and still support relationships?
Well one way not to do it is to have the kind of mutual dependency we have seen. In that
design, synchronization (or lack of) is a terrible problem
For example, I could change Alice's location without telling the location she points to,
resulting in an inconsistency. But if I tried to change the contents of the location as well, then
I would also need to update the value of Bob as well. And so on, ad infinitum. A nightmare,
basically.
The correct way to do this with immutable data is steal a leaf from database design, and
extract the relationship into a separate "table" or type in our case. The current relationships
are held in a single master list, and so when changes are made, no synchronization is
needed.
Here is a very crude example, using a simple list of Relationship s.

821

Refactoring to remove cyclic dependencies

module StructuralDependencyExample_Normalized =
module DomainTypes =
type Relationship<'Left,'Right> = 'Left * 'Right
type Location= {name:string}
type Employee = {name:string; age:float }
module Employee =
open DomainTypes
let EmployeesAtLocation location relations =
relations
|> List.filter (fun (loc,empl) -> loc = location)
|> List.map (fun (loc,empl) -> empl)
let AverageAgeAtLocation location relations =
EmployeesAtLocation location relations
|> List.averageBy (fun e -> e.age)
module Test =
open DomainTypes
let location = { Location.name="NY"}
let alice = {name="Alice"; age=20.0; }
let bob = {name="Bob"; age=30.0; }
let relations = [
(location,alice)
(location,bob)
]
relations
|> List.iter (fun (loc,empl) ->
printfn "employee %s works at %s" (empl.name) (loc.name) )

Or course, a more efficient design would use dictionaries/maps, or special in-memory

structures designed for this kind of thing.

Inheritance dependencies
Finally, let's look at an "inheritance dependency".
Type A stores a value of type B in a property
Type B inherits from type A
We'll consider a UI control hierarchy, where every control belongs to a top-level "Form", and
the Form itself is a Control.

822

Refactoring to remove cyclic dependencies

Here's a first pass at an implementation:

module InheritanceDependencyExample =
type Control(name, form:Form) =
member this.Name = name
abstract Form : Form
default this.Form = form
and Form(name) as self =
inherit Control(name, self)
// test
let form = new Form("form") // NullReferenceException!
let button = new Control("button",form)

The thing to note here is that the Form passes itself in as the form value for the Control
constructor.
This code will compile, but will cause a NullReferenceException error at runtime. This kind of
technique will work in C#, but not in F#, because the class initialization logic is done
differently.
Anyway, this is a terrible design. The form shouldn't have to pass itself in to a constructor.
A better design, which also fixes the constructor error, is to make Control an abstract class
instead, and distinguish between non-form child classes (which do take a form in their
constructor) and the Form class itself, which doesn't.
Here's some sample code:

823

Refactoring to remove cyclic dependencies

module InheritanceDependencyExample2 =
[<AbstractClass>]
type Control(name) =
member this.Name = name
abstract Form : Form
and Form(name) =
inherit Control(name)
override this.Form = this
and Button(name,form) =
inherit Control(name)
override this.Form = form
// test
let form = new Form("form")
let button = new Button("button",form)

Our old friend parameterization again

To remove the circular dependency, we can parameterize the classes in the usual way, as
shown below.
module InheritanceDependencyExample_ParameterizedClasses =
[<AbstractClass>]
type Control<'Form>(name) =
member this.Name = name
abstract Form : 'Form
type Form(name) =
inherit Control<Form>(name)
override this.Form = this
type Button(name,form) =
inherit Control<Form>(name)
override this.Form = form

// test
let form = new Form("form")
let button = new Button("button",form)

824

Refactoring to remove cyclic dependencies

A functional version
I will leave a functional design as an exercise for you to do yourself.
If we were going for truly functional design, we probably would not be using inheritance at
all. Instead, we would use composition in conjunction with parameterization.
But that's a big topic, so I'll save it for another day.

Summary
I hope that this post has given you some useful tips on removing dependency cycles. With
these various approaches in hand, any problems with module organization should be able to
be resolved easily.
In the next post in this series, I'll look at dependency cycles "in the wild", by comparing some
real world C# and F# projects.
As we have seen, F# is a very opinionated language! It wants us to use modules instead of
classes and it prohibits dependency cycles. Are these just annoyances, or do they really
make a difference to the way that code is organized? Read on and find out!

825

Cycles and modularity in the wild

(Updated 2013-06-15. See comments at the end of the post)
(Updated 2014-04-12. A follow up post that applies the same analysis to Roslyn)
(Updated 2015-01-23. A much clearer version of this analysis has been done by Evelina
Gabasova. She knows what she is talking about, so I highly recommend you read her post
first!)
This is a follow up post to two earlier posts on module organization and cyclic dependencies.
I thought it would be interesting to look at some real projects written in C# and F#, and see
how they compare in modularity and number of cyclic dependencies.

The plan
My plan was to take ten or so projects written in C# and ten or so projects written in F#, and
somehow compare them.
I didn't want to spend too much time on this, and so rather than trying to analyze the source
files, I thought I would cheat a little and analyze the compiled assemblies, using the
Mono.Cecil library.
This also meant that I could get the binaries directly, using NuGet.
The projects I picked were:
C# projects
Mono.Cecil, which inspects programs and libraries in the ECMA CIL format.
NUnit
SignalR for real-time web functionality.
NancyFx, a web framework
YamlDotNet, for parsing and emitting YAML.
SpecFlow, a BDD tool.
Json.NET.
Entity Framework.
ELMAH, a logging framework for ASP.NET.
NuGet itself.
Moq, a mocking framework.
NDepend, a code analysis tool.

826

Cycles and modularity in the wild

And, to show I'm being fair, a business application that I wrote in C#.
F# projects
Unfortunately, there is not yet a wide variety of F# projects to choose from. I picked the
following:
FSharp.Core, the core F# library.
FSPowerPack.
FsUnit, extensions for NUnit.
Canopy, a wrapper around the Selenium test automation tool.
FsSql, a nice little ADO.NET wrapper.
WebSharper, the web framework.
TickSpec, a BDD tool.
FSharpx, an F# library.
FParsec, a parser library.
FsYaml, a YAML library built on FParsec.
Storm, a tool for testing web services.
Foq, a mocking framework.
Another business application that I wrote, this time in F#.
I did choose SpecFlow and TickSpec as being directly comparable, and also Moq and and
Foq.
But as you can see, most of the F# projects are not directly comparable to the C# ones. For
example, there is no direct F# equivalent to Nancy, or Entity Framework.
Nevertheless, I was hoping that I might observe some sort of pattern by comparing the
projects. And I was right. Read on for the results!

What metrics to use?

I wanted to examine two things: "modularity" and "cyclic dependencies".
First, what should be the unit of "modularity"?
From a coding point of view, we generally work with files (Smalltalk being a notable
exception), and so it makes sense to think of the file as the unit of modularity. A file is used
to group related items together, and if two chunks of code are in different files, they are
somehow not as "related" as if they were in the same file.
In C#, the best practice is to have one class per file. So 20 files means 20 classes.
Sometimes classes have nested classes, but with rare exceptions, the nested class is in the
same file as the parent class. This means that we can ignore them and just use top-level

827

Cycles and modularity in the wild

classes as our unit of modularity, as a proxy for files.

In F#, the best practice is to have one module per file (or sometimes more). So 20 files
means 20 modules. Behind the scenes, modules are turned into static classes, and any
classes defined within the module are turned into nested classes. So again, this means that
we can ignore nested classes and just use top-level classes as our unit of modularity.
The C# and F# compilers generate many "hidden" types, for things such as LINQ, lambdas,
etc. In some cases, I wanted to exclude these, and only include "authored" types, which
have been coded for explicitly. I also excluded the case classes generated by F#
discriminated unions from being "authored" classes as well. That means that a union type
with three cases will be counted as one authored type rather than four.
So my definition of a top-level type is: a type that is not nested and which is not compiler
generated.
The metrics I chose for modularity were:
The number of top-level types as defined above.
The number of authored types as defined above.
The number of all types. This number would include the compiler generated types as
well. Comparing this number to the top-level types gives us some idea of how
representative the top-level types are.
The size of the project. Obviously, there will be more types in a larger project, so we
need to make adjustments based on the size of the project. The size metric I picked
was the number of instructions, rather than the physical size of the file. This eliminates
issues with embedded resources, etc.

Dependencies
Once we have our units of modularity, we can look at dependencies between modules.
For this analysis, I only want to include dependencies between types in the same assembly.
In other words, dependencies on system types such as String or List do not count as a
dependency.
Let's say we have a top-level type A and another top-level type B . Then I say that a
dependency exists from A to B if:
Type A or any of its nested types inherits from (or implements) type B or any of its
nested types.
Type A or any of its nested types has a field, property or method that references type
B or any of its nested types as a parameter or return value. This includes private

members as well -- after all, it is still a dependency.

828

Cycles and modularity in the wild

Type A or any of its nested types has a method implementation that references type
B or any of its nested types.

This might not be a perfect definition. But it is good enough for my purposes.
In addition to all dependencies, I thought it might be useful to look at "public" or "published"
dependencies. A public dependency from A to B exists if:
Type A or any of its nested types inherits from (or implements) type B or any of its
nested types.
Type A or any of its nested types has a public field, property or method that references
type B or any of its nested types as a parameter or return value.
Finally, a public dependency is only counted if the source type itself is public.
The metrics I chose for dependencies were:
The total number of dependencies. This is simply the sum of all dependencies of all
types. Again, there will be more dependencies in a larger project, but we will also take
the size of the project into account.
The number of types that have more than X dependencies. This gives us an idea of
how many types are "too" complex.

Cyclic dependencies
Given this definition of dependency, then, a cyclic dependency occurs when two different
top-level types depend on each other.
Note what not included in this definition. If a nested type in a module depends on another
nested type in the same module, then that is not a cyclic dependency.
If there is a cyclic dependency, then there is a set of modules that are all linked together. For
example, if A depends on B , B depends on C , and then say, C depends on A , then
A , B and C are linked together. In graph theory, this is called a strongly connected

component.
The metrics I chose for cyclic dependencies were:
The number of cycles. That is, the number of strongly connected components which
had more than one module in them.
The size of the largest component. This gives us an idea of how complex the
dependencies are.
I analyzed cyclic dependencies for all dependencies and also for public dependencies only.

829

Cycles and modularity in the wild

Doing the experiment

First, I downloaded each of the project binaries using NuGet. Then I wrote a little F# script
that did the following steps for each assembly:
1. Analyzed the assembly using Mono.Cecil and extracted all the types, including the
nested types
2. For each type, extracted the public and implementation references to other types,
divided into internal (same assembly) and external (different assembly).
3. Created a list of the "top level" types.
4. Created a dependency list from each top level type to other top level types, based on
the lower level dependencies.
This dependency list was then used to extract various statistics, shown below. I also
rendered the dependency graphs to SVG format (using graphViz).
For cycle detection, I used the QuickGraph library to extract the strongly connected
components, and then did some more processing and rendering.
If you want the gory details, here is a link to the script that I used, and here is the raw data.
It is important to recognize that this is not a proper statistical study, just a quick analysis.
However the results are quite interesting, as we shall see.

Modularity
Let's look at the modularity first.
Here are the modularity-related results for the C# projects:

830

Cycles and modularity in the wild

Project

Code
size

Toplevel
types

Authored
types

All
types

Code/Top

Code/Auth

269521

514

565

876

524

477

jsonDotNet

148829

215

232

283

692

642

nancy

143445

339

366

560

423

392

cecil

101121

240

245

247

421

413

nuget

114856

216

237

381

532

485

signalR

65513

192

229

311

341

286

nunit

45023

173

195

197

260

231

specFlow

46065

242

287

331

190

161

elmah

43855

116

140

141

378

313

yamlDotNet

23499

336

322

fparsecCS

57474

1402

625

moq

133189

397

420

533

335

317

ndepend

478508

734

828

843

652

578

ndependPlat

151625

185

205

820

740

personalCS

422147

195

278

346

2165

1519

TOTAL

2244670

3869

4392

5420

580

511

And here are the results for the F# projects:

831

Cycles and modularity in the wild

Project

Code
size

Toplevel
types

Authored
types

All
types

Code/Top

Code/Auth

fsxCore

339596

173

328

2024

1963

1035

fsCore

226830

154

313

1186

1473

725

fsPowerPack

117581

150

410

1264

784

storm

73595

405

1098

1051

fParsec

67252

245

8407

2802

websharper

47391

128

285

911

370

tickSpec

30797

170

906

629

websharperHtml

14787

822

528

canopy

15105

103

2518

944

fsYaml

15191

160

2170

1381

fsSql

15434

162

1187

857

fsUnit

1848

924

616

foq

26957

103

770

562

personalFS

118893

146

655

3963

814

TOTAL

1111257

692

1332

5987

1606

834

The columns are:

Code size is the number of CIL instructions from all methods, as reported by Cecil.
Top-level types is the total number of top-level types in the assembly, using the
definition above.
Authored types is the total number of types in the assembly, including nested types,
enums, and so on, but excluding compiler generated types.
All types is the total number of types in the assembly, including compiler generated
types.
I have extended these core metrics with some extra calculated columns:
Code/Top is the number of CIL instructions per top level type / module. This is a
measure of how much code is associated with each unit of modularity. Generally, more
is better, because you don't want to have to deal with multiple files if you don't have too.
On the other hand, there is a trade off. Too many lines of code in a file makes reading
the code impossible. In both C# and F#, good practice is not to have more than 5001000 lines of code per file, and with a few exceptions, that seems to be the case in the
source code that I looked at.

832

Cycles and modularity in the wild

Code/Auth is the number of CIL instructions per authored type. This is a measure of
how "big" each authored type is.
Code/All is the number of CIL instructions per type. This is a measure of how "big" each
type is.
Auth/Top is the ratio of all authored types to the top-level-types. It is a rough measure
of how many authored types are in each unit of modularity.
All/Top is the ratio of all types to the top-level-types. It is a rough measure of how many
types are in each unit of modularity.

Analysis
The first thing I noticed is that, with a few exceptions, the code size is bigger for the C#
projects than for the F# projects. Partly that is because I picked bigger projects, of course.
But even for a somewhat comparable project like SpecFlow vs. TickSpec, the SpecFlow
code size is bigger. It may well be that SpecFlow does a lot more than TickSpec, of course,
but it also may be a result of using more generic code in F#. There is not enough information
to know either way right now -- it would be interesting to do a true side by side comparison.
Next, the number of top-level types. I said earlier that this should correspond to the number
of files in a project. Does it?
I didn't get all the sources for all the projects to do a thorough check, but I did a couple of
spot checks. For example, for Nancy, there are 339 top level classes, which implies that
there should be about 339 files. In fact, there are actually 322 .cs files, so not a bad
estimate.
On the other hand, for SpecFlow there are 242 top level types, but only 171 .cs files, so a bit
of an overestimate there. And for Cecil, the same thing: 240 top level classes but only 128
.cs files.
For the FSharpX project, there are 173 top level classes, which implies there should be
about 173 files. In fact, there are actually only 78 .fs files, so it is a serious over-estimate by
a factor of more than 2. And if we look at Storm, there are 67 top level classes. In fact, there
are actually only 35 .fs files, so again it is an over-estimate by a factor of 2.
So it looks like the number of top level classes is always an over-estimate of the number of
files, but much more so for F# than for C#. It would be worth doing some more detailed
analysis in this area.

Ratio of code size to number of top-level types

833

Cycles and modularity in the wild

The "Code/Top" ratio is consistently bigger for F# code than for C# code. Overall, the
average top-level type in C# is converted into 580 instructions. But for F# that number is
1606 instructions, about three times as many.
I expect that this is because F# code is more concise than C# code. I would guess that 500
lines of F# code in a single module would create many more CIL instructions than 500 lines
of C# code in a class.
If we visually plot "Code size" vs. "Top-level types", we get this chart:

What's surprising to me is how distinct the F# and C# projects are in this chart. The C#
projects seem to have a consistent ratio of about 1-2 top-level types per 1000 instructions,
even across different project sizes. And the F# projects are consistent too, having a ratio of
about 0.6 top-level types per 1000 instructions.
In fact, the number of top level types in F# projects seems to taper off as projects get larger,
rather than increasing linearly like the C# projects.

834

Cycles and modularity in the wild

The message I get from this chart is that, for a given size of project, an F# implementation
will have fewer modules, and presumably less complexity as a result.
You probably noticed that there are two anomalies. Two C# projects are out of place -- the
one at the 50K mark is FParsecCS and the one at the 425K mark is my business
application.
I am fairly certain that this because both these implementations have some rather large C#
classes in them, which helps the code ratio. Probably a necessarily evil for a parser, but in
the case of my business application, I know that it is due to cruft accumulating over the
years, and there are some massive classes that ought to be refactored into smaller ones. So
a metric like this is probably a bad sign for a C# code base.

Ratio of code size to number of all types

On the other hand, if we compare the ratio of code to all types, including compiler generated
ones, we get a very different result.
Here's the corresponding chart of "Code size" vs. "All types":

835

Cycles and modularity in the wild

This is surprisingly linear for F#. The total number of types (including compiler generated
ones) seems to depend closely on the size of the project. On the other hand, the number of
types for C# seems to vary a lot.
The average "size" of a type is somewhat smaller for F# code than for C# code. The
average type in C# is converted into about 400 instructions. But for F# that number is about
180 instructions.
I'm not sure why this is. Is it because the F# types are more fine-grained, or could it be
because the F# compiler generates many more little types than the C# compiler? Without
doing a more subtle analysis, I can't tell.

Ratio of top-level types to authored types

Having compared the type counts to the code size, let's now compare them to each other:

836

Cycles and modularity in the wild

Again, there is a significant difference. For each unit of modularity in C# there are an
average of 1.1 authored types. But in F# the average is 1.9, and for some projects a lot more
than that.
Of course, creating nested types is trivial in F#, and quite uncommon in C#, so you could
argue that this is not a fair comparison. But surely the ability to create a dozen types in as
many lines of F# has some effect on the quality of the design? This is harder to do in C#, but
there is nothing to stop you. So might this not mean that there is a temptation in C# to not be
as fine-grained as you could potentially be?
The project with the highest ratio (4.9) is my F# business application. I believe that this is
due to this being only F# project in this list which is designed around a specific business
domain, I created many "little" types to model the domain accurately, using the concepts
described here. For other projects created using DDD principles, I would expect to see this
same high number.

837

Cycles and modularity in the wild

Dependencies
Now let's look at the dependency relationships between the top level classes.
Here are the results for the C# projects:

Dep/Top

One
or
more
dep.

Three
or
more
dep.

Five
or
more
dep.

Ten
or
more
dep.

Diagram

2354

4.6

76%

51%

32%

13%

svg

215

913

4.2

69%

42%

30%

14%

svg

nancy

339

1132

3.3

78%

41%

22%

svg

cecil

240

1145

4.8

73%

43%

23%

13%

svg

nuget

216

833

3.9

71%

43%

26%

12%

svg

signalR

192

641

3.3

66%

34%

19%

10%

svg

nunit

173

499

2.9

75%

39%

13%

svg

specFlow

242

578

2.4

64%

25%

17%

svg

elmah

116

300

2.6

72%

28%

22%

svg

yamlDotNet

228

3.3

83%

30%

11%

svg

fparsecCS

1.6

59%

29%

svg

moq

397

1100

2.8

63%

29%

17%

svg

ndepend

734

2426

3.3

67%

37%

25%

10%

svg

ndependPlat

185

404

2.2

67%

24%

11%

svg

personalCS

195

532

2.7

69%

29%

19%

TOTAL

3869

13149

3.4

70%

37%

22%

Top
Level
Types

Total
Dep.
Count

514

jsonDotNet

Project

And here are the results for the F# projects:

838

Cycles and modularity in the wild

Dep/Top

One
or
more
dep.

Three
or
more
dep.

Five
or
more
dep.

Ten
or
more
dep.

0.4

30%

154

287

1.9

55%

26%

14%

fsPowerPack

0.7

38%

13%

storm

195

2.9

72%

40%

18%

fParsec

1.1

63%

25%

websharper

0.3

31%

tickSpec

1.4

50%

15%

websharperHtml

2.1

78%

39%

canopy

1.3

50%

33%

fsYaml

1.4

71%

14%

fsSql

1.1

54%

fsUnit

0.0

foq

1.9

66%

29%

11%

personalFS

111

3.7

93%

60%

27%

TOTAL

692

947

1.4

49%

19%

Top
Level
Types

Total
Dep.
Count

fsxCore

173

fsCore

Project

The columns are:

Top-level types is the total number of top-level types in the assembly, as before.
Total dep. count is the total number of dependencies between top level types.
Dep/Top is the number of dependencies per top level type / module only. This is a
measure of how many dependencies the average top level type/module has.
One or more dep is the number of top level types that have dependencies on one or
more other top level types.
Three or more dep. Similar to above, but with dependencies on three or more other top
level types.
Five or more dep. Similar to above.
Ten or more dep. Similar to above. Top level types with this many dependencies will be
harder to understand and maintain. So this is measure of how complex the project is.
The diagram column contains a link to a SVG file, generated from the dependencies, and
also the DOT file that was used to generate the SVG. See below for a discussion of these
diagrams. (Note that I can't expose the internals of my applications, so I will just give the
metrics)

839

Cycles and modularity in the wild

Analysis
These results are very interesting. For C#, the number of total dependencies increases with
project size. Each top-level type depends on 3-4 others, on average.
On the other hand, the number of total dependencies in an F# project does not seem to vary
too much with project size at all. Each F# module depends on no more than 1-2 others, on
average. And the largest project (FSharpX) has a lower ratio than many of the smaller
projects. My business app and the Storm project are the only exceptions.
Here's a chart of the relationship between code size and the number of dependencies:

The disparity between C# and F# projects is very clear. The C# dependencies seem to grow
linearly with project size, while the F# dependencies seem to be flat.

Distribution of dependencies

840

Cycles and modularity in the wild

The average number of dependencies per top level type is interesting, but it doesn't help us
understand the variability. Are there many modules with lots of dependencies? Or does each
one just have a few?
This might make a difference in maintainability, perhaps. I would assume that a module with
only one or two dependencies would be easier to understand in the context of the
application that one with tens of dependencies.
Rather than doing a sophisticated statistical analysis, I thought I would keep it simple and
just count how many top level types had one or more dependencies, three or more
dependencies, and so on.
Here are the same results, displayed visually:

841

Cycles and modularity in the wild

So, what can we deduce from these numbers?

First, in the F# projects, more than half of the modules have no outside dependencies at
all. This is a bit surprising, but I think it is due to the heavy use of generics compared
with C# projects.
Second, the modules in the F# projects consistently have fewer dependencies than the
classes in the C# projects.
Finally, in the F# projects, modules with a high number of dependencies are quite rare -less than 2% overall. But in the C# projects, 9% of classes have more than 10
dependencies on other classes.
The worst offender in the F# group is my very own F# application, which is even worse than
my C# application with respect to these metrics. Again, it might be due to heavy use of nongenerics in the form of domain-specific types, or it might just be that the code needs more
refactoring!

The dependency diagrams

It might be useful to look at the dependency diagrams now. These are SVG files, so you
should be able to view them in your browser.
Note that most of these diagrams are very big -- so after you open them you will need to
zoom out quite a bit in order to see anything!
Let's start by comparing the diagrams for SpecFlow and TickSpec.

842

Cycles and modularity in the wild

Here's the one for SpecFlow:

Here's the one for TickSpec:

843

Cycles and modularity in the wild

Each diagram lists all the top-level types found in the project. If there is a dependency from
one type to another, it is shown by an arrow. The dependencies point from left to right where
possible, so any arrows going from right to left implies that there is a cyclic dependency.
The layout is done automatically by graphviz, but in general, the types are organized into
columns or "ranks". For example, the SpecFlow diagram has 12 ranks, and the TickSpec
diagram has five.
As you can see, there are generally a lot of tangled lines in a typical dependency diagram!
How tangled the diagram looks is a sort of visual measure of the code complexity. For
instance, if I was tasked to maintain the SpecFlow project, I wouldn't really feel comfortable
until I had understood all the relationships between the classes. And the more complex the
project, the longer it takes to come up to speed.

OO vs functional design revealed?

The TickSpec diagram is a lot simpler than the SpecFlow one. Is that because TickSpec
perhaps doesn't do as much as SpecFlow?
The answer is no, I don't think that it has anything to do with the size of the feature set at all,
but rather because the code is organized differently.
Looking at the SpecFlow classes (dotfile), we can see it follows good OOD and TDD
practices by creating interfaces. There's a TestRunnerManager and an ITestRunnerManager ,
for example. And there are many other patterns that commonly crop up in OOD: "listener"
classes and interfaces, "provider" classes and interfaces, "comparer" classes and interfaces,
and so on.
But if we look at the TickSpec modules (dotfile) there are no interfaces at all. And no
"listeners", "providers" or "comparers" either. There might well be a need for such things in
the code, but either they are not exposed outside their module, or more likely, the role they
play is fulfilled by functions rather than types.
I'm not picking on the SpecFlow code, by the way. It seems well designed, and is a very
useful library, but I think it does highlight some of the differences between OO design and
functional design.

Moq compared with Foq

Let's also compare the diagrams for Moq and Foq. These two projects do roughly the same
thing, so the code should be comparable.
As before, the project written in F# has a much smaller dependency diagram.

844

Cycles and modularity in the wild

Looking at the Moq classes (dotfile), we can see it includes the "Castle" library, which I didn't
eliminate from the analysis. Out of the 249 classes with dependencies, only 66 are Moq
specific. If we had considered only the classes in the Moq namespace, we might have had a
cleaner diagram.
On the other hand, looking at the Foq modules (dotfile) there are only 23 modules with
dependencies, fewer even than just the Moq classes alone.
So something is very different with code organization in F#.

FParsec compared with FParsecCS

The FParsec project is an interesting natural experiment. The project has two assemblies,
roughly the same size, but one is written in C# and the other in F#.
It is a bit unfair to compare them directly, because the C# code is designed for parsing fast,
while the F# code is more high level. But... I'm going to be unfair and compare them anyway!
Here are the diagrams for the F# assembly "FParsec" and C# assembly "FParsecCS".
They are both nice and clear. Lovely code!
What's not clear from the diagram is that my methodology is being unfair to the C#
assembly.
For example, the C# diagram shows that there are dependencies between Operator ,
OperatorType , InfixOperator and so on. But in fact, looking at the source code, these

classes are all in the same physical file. In F#, they would all be in the same module, and
their relationships would not count as public dependencies. So the C# code is being
penalized in a way.
Even so, looking at the source code, the C# code has 20 source files compared to F#'s 8, so
there is still some difference in complexity.

What counts as a dependency?

In defence of my method though, the only thing that is keeping these FParsec C# classes
together in the same file is good coding practice; it is not enforced by the C# compiler.
Another maintainer could come along and unwittingly separate them into different files,
which really would increase the complexity. In F# you could not do that so easily, and
certainly not accidentally.
So it depends on what you mean by "module", and "dependency". In my view, a module
contains things that really are "joined at the hip" and shouldn't easily be decoupled. Hence
dependencies within a module don't count, while dependencies between modules do.

845

Cycles and modularity in the wild

Another way to think about it is that F# encourages high coupling in some areas (modules)
in exchange for low coupling in others. In C#, the only kind of strict coupling available is
class-based. Anything looser, such as using namespaces, has to be enforced using good
practices or a tool such as NDepend.
Whether the F# approach is better or worse depends on your preference. It does make
certain kinds of refactoring harder as a result.

Cyclic dependencies
Finally, we can turn our attention to the oh-so-evil cyclic dependencies. (If you want to know
why they are bad, read this post ).
Here are the cyclic dependency results for the C# projects.
Toplevel
types

Cycle
count

Partic.

Partic.%

Max
comp.
size

Cycle
count
(public)

Partic.
(public)

514

123

24%

jsonDotNet

215

41%

nancy

339

10%

cecil

240

125

52%

123

nuget

216

11%

signalR

192

nunit

173

46%

specFlow

242

elmah

116

yamlDotNet

fparsecCS

15%

moq

397

13%

ndepend

734

11%

ndependPlat

185

personalCS

195

17%

TOTAL

3869

683

18%

Project

186

And here are the results for the F# projects:

846

Cycles and modularity in the wild

Toplevel
types

Cycle
count

Partic.

Partic.%

Max
comp.
size

Cycle
count
(public)

Partic.
(public)

fsxCore

173

fsCore

154

fsPowerPack

storm

fParsec

websharper

tickSpec

websharperHtml

canopy

fsYaml

fsSql

fsUnit

foq

personalFS

TOTAL

692

Project

The columns are:

Top-level types is the total number of top-level types in the assembly, as before.
Cycle count is the number of cycles altogether. Ideally it would be zero. But larger is
not necessarily worse. Better to have 10 small cycles than one giant one, I think.
Partic.. The number of top level types that participate in any cycle.
Partic.%. The number of top level types that participate in any cycle, as a percent of all
types.
Max comp. size is the number of top level types in the largest cyclic component. This is
a measure of how complex the cycle is. If there are only two mutually dependent types,
then the cycle is a lot less complex than, say, 123 mutually dependent types.
... (public) columns have the same definitions, but using only public dependencies. I
thought it would be interesting to see what effect it would have to limit the analysis to
public dependencies only.
The diagram column contains a link to a SVG file, generated from the dependencies in
the cycles only, and also the DOT file that was used to generate the SVG. See below for
an analysis.

847

Cycles and modularity in the wild

Analysis
If we are looking for cycles in the F# code, we will be sorely disappointed. Only two of the F#
projects have cycles at all, and those are tiny. For example in FSharp.Core there is a mutual
dependency between two types right next to each other in the same file, here.
On the other hand, almost all the C# projects have one or more cycles. Entity Framework
has the most cycles, involving 24% of the classes, and Cecil has the worst participation rate,
with over half of the classes being involved in a cycle.
Even NDepend has cycles, although to be fair, there may be good reasons for this. First
NDepend focuses on removing cycles between namespaces, not classes so much, and
second, it's possible that the cycles are between types declared in the same source file. As
a result, my method may penalize well-organized C# code somewhat (as noted in the
FParsec vs. FParsecCS discussion above).

Why the difference between C# and F#?

848

Cycles and modularity in the wild

In C#, there is nothing stopping you from creating cycles -- a perfect example of
accidental complexity. In fact, you have to make a special effort to avoid them.
In F#, of course, it is the other way around. You can't easily create cycles at all.

My business applications compared

One more comparison. As part of my day job, I have written a number of business
applications in C#, and more recently, in F#. Unlike the other projects listed here, they are
very focused on addressing a particular business need, with lots of domain specific code,
custom business rules, special cases, and so on.
Both projects were produced under deadline, with changing requirements and all the usual
real world constraints that stop you writing ideal code. Like most developers in my position, I
would love a chance to tidy them up and refactor them, but they do work, the business is
happy, and I have to move on to new things.
Anyway, let's see how they stack up to each other. I can't reveal any details of the code
other than the metrics, but I think that should be enough to be useful.
Taking the C# project first:
It has 195 top level types, about 1 for every 2K of code. Comparing this with other C#
projects, there should be many more top level types than this. And in fact, I know that
this is true. As with many projects (this one is 6 years old) it is lower risk to just add a
method to an existing class rather than refactoring it, especially under deadline.
Keeping old code stable is always a higher priority than making it beautiful! The result is
that classes grow too large over time.
The flip side of having large classes is that there many fewer cross-class dependencies!
It has some of the better scores among the C# projects. So it goes to show that
dependencies aren't the only metric. There has to be a balance.
In terms of cyclic dependencies, it's pretty typical for a C# project. There are a number
of them (11) but the largest involves only 8 classes.
Now let's look at my F# project:
It has 30 modules, about 1 for every 4K of code. Comparing this with other F# projects,
it's not excessive, but perhaps a bit of refactoring is in order.
As an aside, in my experience with maintaining this code, I have noticed that, unlike
C# code, I don't feel that I have to add cruft to existing modules when feature
requests come in. Instead, I find that in many cases, the faster and lower risk way
of making changes is simply to create a new module and put all the code for a new
feature in there. Because the modules have no state, a function can live anywhere -

849

Cycles and modularity in the wild

- it is not forced to live in the same class. Over time this approach may create its
own problems too (COBOL anyone?) but right now, I find it a breath of fresh air.
The metrics show that there are an unusually large number of "authored" types per
module (4.9). As I noted above, I think this is a result of having fine-grained DDD-style
design. The code per authored type is in line with the other F# projects, so that implies
they are not too big or small.
Also, as I noted earlier, the inter-module dependencies are the worst of any F# project. I
know that there are some API/service functions that depend on almost all the other
modules, but this could be a clue that they might need refactoring.
However, unlike C# code, I know exactly where to find these problem modules. I
can be fairly certain that all these modules are in the top layer of my application and
will thus appear at the bottom of the module list in Visual Studio. How can I be so
sure? Because...
In terms of cyclic dependencies, it's pretty typical for a F# project. There aren't any.

Summary
I started this analysis from curiosity -- was there any meaningful difference in the
organization of C# and F# projects?
I was quite surprised that the distinction was so clear. Given these metrics, you could
certainly predict which language the assembly was written in.
Project complexity. For a given number of instructions, a C# project is likely to have
many more top level types (and hence files) than an F# one -- more than double, it
seems.
Fine-grained types. For a given number of modules, a C# project is likely to have
fewer authored types than an F# one, implying that the types are not as fine-grained as
they could be.
Dependencies. In a C# project, the number of dependencies between classes
increases linearly with the size of the project. In an F# project, the number of
dependencies is much smaller and stays relatively flat.
Cycles. In a C# project, cycles occur easily unless care is taken to avoid them. In an F#
project, cycles are extremely rare, and if present, are very small.
Perhaps this has do with the competency of the programmer, rather than a difference
between languages? Well, first of all, I think that the quality of the C# projects is quite good
on the whole -- I certainly wouldn't claim that I could write better code! And, in two cases in
particular, the C# and F# projects were written by the same person, and differences were
still apparent, so I don't think this argument holds up.

850

Cycles and modularity in the wild

Future work
This approach of using just the binaries might have gone as far as it can go. For a more
accurate analysis, we would need to use metrics from the source code as well (or maybe the
pdb file).
For example, a high "instructions per type" metric is good if it corresponds to small source
files (concise code), but not if it corresponds to large ones (bloated classes). Similarly, my
definition of modularity used top-level types rather than source files, which penalized C#
somewhat over F#.
So, I don't claim that this analysis is perfect (and I hope haven't made a terrible mistake in
the analysis code!) but I think that it could be a useful starting point for further investigation.

Update 2013-06-15
This post caused quite a bit of interest. Based on feedback, I made the following changes:
Assemblies profiled
Added Foq and Moq (at the request of Phil Trelford).
Added the C# component of FParsec (at the request of Dave Thomas and others).
Added two NDepend assemblies.
Added two of my own projects, one C# and one F#.
As you can see, adding seven new data points (five C# and two F# projects) didn't change
the overall analysis.
Algorithm changes
Made definition of "authored" type stricter. Excluded types with
"GeneratedCodeAttribute" and F# types that are subtypes of a sum type. This had an
effect on the F# projects and reduced the "Auth/Top" ratio somewhat.
Text changes
Rewrote some of the analysis.
Removed the unfair comparison of YamlDotNet with FParsec.
Added a comparison of the C# component and F# components of FParsec.
Added a comparison of Moq and Foq.
Added a comparison of my own two projects.
The orginal post is still available here

851

Cycles and modularity in the wild

852

The "Porting from C#" Series

Do you want to port C# code to F#? In this series of posts we'll look at various approaches
to this, and the design decisions and trade-offs involved.
Porting from C# to F#: Introduction. Three approaches to porting existing C# code to F#.
Getting started with direct porting. F# equivalents to C#.

853

Porting from C# to F#: Introduction

NOTE: Before reading this series, I suggest that you read the following series as a
prerequisite: "thinking functionally", "expressions and syntax", and "understanding F# types".
For many developers, the next step after learning a new language might be to port some
existing code over to it, so that they can get a good feel for the differences between the two
languages.
As we pointed out earlier, functional languages are very different from imperative languages,
and so trying to do a direct port of imperative code to a functional language is often not
possible, and even if a crude port is done successfully, the ported code will probably not be
using the functional model to its best advantage.
Of course, F# is a multi-paradigm language, and includes support for object-oriented and
imperative techniques, but even so, a direct port will generally not be the best way to write
the corresponding F# code.
So, in this series, we'll look at various approaches to porting existing C# code to F#.

Levels of porting sophistication

If you recall the diagram from an earlier post, there are four key concepts that differentiate
F# from C#.
Function-oriented rather than object-oriented
Expressions rather than statements
Algebraic types for creating domain models
Pattern matching for flow of control

854

Porting from C# to F#: Introduction

And, as explained in that post and its sequels, these aspects are not just academic, but offer
concrete benefits to you as a developer.
So I have divided the porting process into three levels of sophistication (for lack of a better
term), which represent how well the ported code exploits these benefits.

Basic Level: Direct port

At this first level, the F# code is a direct port (where possible) of the C# code. Classes and
methods are used instead of modules and functions, and values are frequently mutated.

Intermediate Level: Functional code

At the next level, the F# code has been refactored to be fully functional.
Classes and methods have been replaced by modules and functions, and values are
generally immutable.
Higher order functions are used to replace interfaces and inheritance.
Pattern matching is used extensively for control flow.
Loops have been replaced with list functions such as "map" or recursion.

855

Porting from C# to F#: Introduction

There are two different paths that can get you to this level.
The first path is to do a basic direct port to F#, and then refactor the F# code.
The second path is to convert the existing imperative code to functional code while
staying in C#, and only then port the functional C# code to functional F# code!
The second option might seem clumsy, but for real code it will probably be both faster and
more comfortable. Faster because you can use a tool such as Resharper to do the
refactoring, and more comfortable because you are working in C# until the final port. This
approach also makes it clear that the hard part is not the actual port from C# to F#, but the
conversion of imperative code to functional code!

Advanced Level: Types represent the domain

At this final level, not only is the code functional, but the design itself has been changed to
exploit the power of algebraic data types (especially union types).
The domain will have been encoded into types such that illegal states are not even
representable, and correctness is enforced at compile time. For a concrete demonstration of
the power of this approach, see the shopping cart example in the "why use F#" series and
the whole "Designing with types" series.
This level can only be done in F#, and is not really practical in C#.

Porting diagram
Here is a diagram to help you visualize the various porting paths described above.

856

Porting from C# to F#: Introduction

The approach for this series

To see how these three levels work in practice, we'll apply them to some worked examples:
The first example is a simple system for creating and scoring a ten-pin bowling game,
based on the code from the well known "bowling game kata" described by "Uncle" Bob
Martin. The original C# code has only one class and about 70 lines of code, but even
so, it demonstrates a number of important principles.
Next, we'll look at some shopping cart code, based on this example.
The final example is code that represents states for a subway turnstile system, also
based on an example from Bob Martin. This example demonstrates how the union types
in F# can represent a state transition model more easily than the OO approach.
But first, before we get started on the detailed examples, we'll go back to basics and do
some simple porting of some code snippets. That will be the topic of the next post.

857

Getting started with direct porting

Before we get started on the detailed examples, we'll go back to basics and do some simple
porting of trivial examples.
In this post and the next, we'll look at the nearest F# equivalents to common C# statements
and keywords, to guide you when doing direct ports.

Basic syntax conversion guidelines

Before starting a port, you need to understand how F# syntax is different from C# syntax.
This section presents some general guidelines for converting from one to another. (For a
quick overview of F# syntax as a whole, see "F# syntax in 60 seconds")

Curly braces and indentation

C# uses curly braces to indicate the start and end of a block of code. F# generally just uses
indentation.
Curly braces are used in F#, but not for blocks of code. Instead, you will see them used:
For definitions and usage of "record" types.
In conjunction with computation expressions, such as seq and async . In general, you
will not be using these expressions for basic ports anyway.
For details on the indentation rules, see this post.

Semicolons
Unlike C#'s semicolon, F# does not require any kind of line or statement terminator.

Commas
F# does not use commas for separating parameters or list elements, so remember not to
use commas when porting!
For separating list elements, use semicolons rather than commas.
// C# example
var list = new int[] { 1,2,3}

858

Getting started with direct porting

// F# example
let list = [1;2;3] // semicolons

For separating parameters for native F# functions, use white space.

// C# example
int myFunc(int x, int y, int z) { ... function body ...}

// F# example
let myFunc (x:int) (y:int) (z:int) :int = ... function body ...
let myFunc x y z = ... function body ...

Commas are generally only used for tuples, or for separating parameters when calling .NET
library functions. (See this post for more on tuples vs multiple parameters)

Defining variables, functions and types

In F#, definitions of both variables and functions use the form:
let someName = // the definition

Definitions for all types (classes, structures, interfaces, etc.) use the form:
type someName = // the definition

The use of the = sign is an important difference between F# and C#. Where C# uses curly
braces, F# uses the = and then the following block of code must be indented.

Mutable values
In F#, values are immutable by default. If you are doing a direct imperative port, you
probably need to make some of the values mutable, using the mutable keyword. Then to
assign to the values, use the <- operator, not the equals sign.
// C# example
var variableName = 42
variableName = variableName + 1

859

Getting started with direct porting

// F# example
let mutable variableName = 42
variableName <- variableName + 1

Assignment vs. testing for equality

In C#, the equals sign is used for assignment, and the double equals == is used for testing
equality.
However in F#, the equals sign is used for testing equality, and is also used to initially bind
values to other values when declared,
let mutable variableName = 42 // Bound to 42 on declaration
variableName <- variableName + 1 // Mutated (reassigned)
variableName = variableName + 1 // Comparison not assignment!

To test for inequality, use SQL-style <> rather than !=

let variableName = 42 // Bound to 42 on declaration
variableName <> 43 // Comparison will return true.
variableName != 43 // Error FS0020.

If you accidentally use != you will probably get an error FS0020.

Conversion example #1
With these basic guidelines in place, let's look at some real code examples, and do a direct
port for them.
This first example has some very simple code, which we will port line by line. Here's the C#
code.

860

Getting started with direct porting

using System;
using System.Collections.Generic;
namespace PortingToFsharp
{
public class Squarer
{
public int Square(int input)
{
var result = input * input;
return result;
}
public void PrintSquare(int input)
{
var result = this.Square(input);
Console.WriteLine("Input={0}. Result={1}",
input, result);
}
}

Converting "using" and "namespace"

These keywords are straightforward:
using becomes open
namespace with curly braces becomes just namespace .

Unlike C#, F# files do not generally declare namespaces unless they need to interop with
other .NET code. The filename itself acts as a default namespace.
Note that the namespace, if used, must come before anything else, such as "open". This the
opposite order from most C# code.

Converting the class

To declare a simple class, use:
type myClassName() =
... code ...

Note that there are parentheses after the class name. These are required for class
definitions.
More complicated class definitions will be shown in the next example, and you read the
complete discussion of classes.

861

Getting started with direct porting

Converting function/method signatures

For function/method signatures:
Parentheses are not needed around the parameter list
Whitespace is used to separate the parameters, not commas
Rather than curly braces, an equals sign signals the start of the function body
The parameters don't normally need types but if you do need them:
The type name comes after the value or parameter
The parameter name and type are separated by colons
When specifying types for parameters, you should probably wrap the pair in
parentheses to avoid unexpected behavior.
The return type for the function as a whole is prefixed by a colon, and comes after
all the other parameters
Here's a C# function signature:
int Square(int input) { ... code ...}

and here's the corresponding F# function signature with explicit types:

let Square (input:int) :int = ... code ...

However, because F# can normally infer the parameter and return types, you rarely need to
specify them explicitly.
Here's a more typical F# signature, with inferred types:
let Square input = ... code ...

void
The void keyword in C# is generally not needed, but if required, would be converted to
unit

So the C# code:
void PrintSquare(int input) { ... code ...}

could be converted to the F# code:

862

Getting started with direct porting

let PrintSquare (input:int) :unit = ... code ...

but again, the specific types are rarely needed, and so the F# version is just:
let PrintSquare input = ... code ...

Converting function/method bodies

In a function body, you are likely to have a combination of:
Variable declarations and assignments
Function calls
Control flow statements
Return values
We'll have a quick look at porting each of these in turn, except for control flow, which we'll
discuss later.

Converting variable declarations

Almost always, you can use let on its own, just like var in C#:
// C# variable declaration
var result = input * input;

// F# value declaration
let result = input * input

Unlike C#, you must always assign ("bind") something to an F# value as part of its
declaration.
// C# example
int unassignedVariable; //valid

// F# example
let unassignedVariable // not valid

As noted above, if you need to change the value after its declaration, you must use the
"mutable" keyword.

863

Getting started with direct porting

If you need to specify a type for a value, the type name comes after the value or parameter,
preceded by a colon.
// C# example
int variableName = 42;

// F# example
let variableName:int = 42

Converting function calls

When calling a native F# function, there is no need for parentheses or commas. In other
words, the same rules apply for calling a function as when defining it.
Here's C# code for defining a function, then calling it:
// define a method/function
int Square(int input) { ... code ...}
// call it
var result = Square(input);

However, because F# can normally infer the parameter and return types, you rarely need to
specify them explicitly So here's typical F# code for defining a function and then calling it:
// define a function
let Square input = ... code ...
// call it
let result = Square input

Return values
In C#, you use the return keyword. But in F#, the last value in the block is automatically
the "return" value.
Here's the C# code returning the result variable.
public int Square(int input)
{
var result = input * input;
return result; //explicit "return" keyword
}

864

Getting started with direct porting

And here's the F# equivalent.

let Square input =
let result = input * input
result // implicit "return" value

This is because F# is expression-based. Everything is an expression, and the value of a

block expression as a whole is just the value of the last expression in the block.
For more details on expression-oriented code, see "expressions vs statements".

Printing to the console

To print output in C#, you generally use Console.WriteLine or similar. In F#, you generally
use printf or similar, which is typesafe. (More details on using "printf" family).

The complete port of example #1

Putting it all together, here is the complete direct port of example #1 to F#.
The C# code again:
using System;
using System.Collections.Generic;
namespace PortingToFsharp
{
public class Squarer
{
public int Square(int input)
{
var result = input * input;
return result;
}
public void PrintSquare(int input)
{
var result = this.Square(input);
Console.WriteLine("Input={0}. Result={1}",
input, result);
}
}

And the equivalent F# code:

865

Getting started with direct porting

namespace PortingToFsharp
open System
open System.Collections.Generic
type Squarer() =
let Square input =
let result = input * input
result
let PrintSquare input =
let result = Square input
printf "Input=%i. Result=%i" input result

866

The "Designing with types" Series

In this series, we'll look at some of the ways we can use types as part of the design process.
In particular, the thoughtful use of types can make a design more transparent and improve
correctness at the same time.
This series will be focused on the "micro level" of design. That is, working at the lowest level
of individual types and functions. Higher level design approaches, and the associated
decisions about using functional or object-oriented style, will be discussed in another series.
Many of the suggestions are also feasable in C# or Java, but the lightweight nature of F#
types means that it is much more likely that we will do this kind of refactoring.
Designing with types: Introduction. Making design more transparent and improving
correctness.
Designing with types: Single case union types. Adding meaning to primitive types.
Designing with types: Making illegal states unrepresentable. Encoding business logic in
types.
Designing with types: Discovering new concepts. Gaining deeper insight into the
domain.
Designing with types: Making state explicit. Using state machines to ensure
correctness.
Designing with types: Constrained strings. Adding more semantic information to a
primitive type.
Designing with types: Non-string types. Working with integers and dates safely.
Designing with types: Conclusion. A before and after comparison.

867

Designing with types: Introduction

A basic example
For demonstration of the various uses of types, I'll work with a very straightforward example,
namely a Contact type, such as the one below.
type Contact =
{
FirstName: string;
MiddleInitial: string;
LastName: string;
EmailAddress: string;
//true if ownership of email address is confirmed
IsEmailVerified: bool;
Address1: string;
Address2: string;
City: string;
State: string;
Zip: string;
//true if validated against address service
IsAddressValid: bool;
}

This seems very obvious -- I'm sure we have all seen something like this many times. So
what can we do with it? How can we refactor this to make the most of the type system?

Creating "atomic" types

868

Designing with types: Introduction

The first thing to do is to look at the usage pattern of data access and updates. For example,
would be it be likely that Zip is updated without also updating Address1 at the same time?
On the other hand, it might be common that a transaction updates EmailAddress but not
FirstName .

This leads to the first guideline:

Guideline: Use records or tuples to group together data that are required to be
consistent (that is "atomic") but don't needlessly group together data that is not related.
In this case, it is fairly obvious that the three name values are a set, the address values are
a set, and the email is also a set.
We have also some extra flags here, such as IsAddressValid and IsEmailVerified . Should
these be part of the related set or not? Certainly yes for now, because the flags are
dependent on the related values.
For example, if the EmailAddress changes, then IsEmailVerified probably needs to be
reset to false at the same time.
For PostalAddress , it seems clear that the core "address" part is a useful common type,
without the IsAddressValid flag. On the other hand, the IsAddressValid is associated with
the address, and will be updated when it changes.
So it seems that we should create two types. One is a generic PostalAddress and the other
is an address in the context of a contact, which we can call PostalContactInfo , say.
type PostalAddress =
{
Address1: string;
Address2: string;
City: string;
State: string;
Zip: string;
}
type PostalContactInfo =
{
Address: PostalAddress;
IsAddressValid: bool;
}

Finally, we can use the option type to signal that certain values, such as MiddleInitial , are
indeed optional.

869

Designing with types: Introduction

type PersonalName =
{
FirstName: string;
// use "option" to signal optionality
MiddleInitial: string option;
LastName: string;
}

Summary
With all these changes, we now have the following code:
type PersonalName =
{
FirstName: string;
// use "option" to signal optionality
MiddleInitial: string option;
LastName: string;
}
type EmailContactInfo =
{
EmailAddress: string;
IsEmailVerified: bool;
}
type PostalAddress =
{
Address1: string;
Address2: string;
City: string;
State: string;
Zip: string;
}
type PostalContactInfo =
{
Address: PostalAddress;
IsAddressValid: bool;
}
type Contact =
{
Name: PersonalName;
EmailContactInfo: EmailContactInfo;
PostalContactInfo: PostalContactInfo;
}

870

Designing with types: Introduction

We haven't written a single function yet, but already the code represents the domain better.
However, this is just the beginning of what we can do.
Next up, using single case unions to add semantic meaning to primitive types.

871

Single case union types

Designing with types: Single case union

types
At the end of the previous post, we had values for email addresses, zip codes, etc., defined
like this:

EmailAddress: string;
State: string;
Zip: string;

These are all defined as simple strings. But really, are they just strings? Is an email address
interchangable with a zip code or a state abbreviation?
In a domain driven design, they are indeed distinct things, not just strings. So we would
ideally like to have lots of separate types for them so that they cannot accidentally be mixed
up.
This has been known as good practice for a long time, but in languages like C# and Java it
can be painful to create hundred of tiny types like this, leading to the so called "primitive
obsession" code smell.
But F# there is no excuse! It is trivial to create simple wrapper types.

Wrapping primitive types

The simplest way to create a separate type is to wrap the underlying string type inside
another type.
We can do it using single case union types, like so:
type EmailAddress = EmailAddress of string
type ZipCode = ZipCode of string
type StateCode = StateCode of string

or alternatively, we could use record types with one field, like this:
type EmailAddress = { EmailAddress: string }
type ZipCode = { ZipCode: string }
type StateCode = { StateCode: string}

872

Single case union types

Both approaches can be used to create wrapper types around a string or other primitive
type, so which way is better?
The answer is generally the single case discriminated union. It is much easier to "wrap" and
"unwrap", as the "union case" is actually a proper constructor function in its own right.
Unwrapping can be done using inline pattern matching.
Here's some examples of how an EmailAddress type might be constructed and
deconstructed:
type EmailAddress = EmailAddress of string
// using the constructor as a function
"a" |> EmailAddress
["a"; "b"; "c"] |> List.map EmailAddress
// inline deconstruction
let a' = "a" |> EmailAddress
let (EmailAddress a'') = a'
let addresses =
["a"; "b"; "c"]
|> List.map EmailAddress
let addresses' =
addresses
|> List.map (fun (EmailAddress e) -> e)

You can't do this as easily using record types.

So, let's refactor the code again to use these union types. It now looks like this:

873

Single case union types

type PersonalName =
{
FirstName: string;
MiddleInitial: string option;
LastName: string;
}
type EmailAddress = EmailAddress of string
type EmailContactInfo =
{
EmailAddress: EmailAddress;
IsEmailVerified: bool;
}
type ZipCode = ZipCode of string
type StateCode = StateCode of string
type PostalAddress =
{
Address1: string;
Address2: string;
City: string;
State: StateCode;
Zip: ZipCode;
}
type PostalContactInfo =
{
Address: PostalAddress;
IsAddressValid: bool;
}
type Contact =
{
Name: PersonalName;
EmailContactInfo: EmailContactInfo;
PostalContactInfo: PostalContactInfo;
}

Another nice thing about the union type is that the implementation can be encapsulated with
module signatures, as we'll discuss below.

Naming the "case" of a single case union

In the examples above we used the same name for the case as we did for the type:

874

Single case union types

type EmailAddress = EmailAddress of string

type ZipCode = ZipCode of string
type StateCode = StateCode of string

This might seem confusing initially, but really they are in different scopes, so there is no
naming collision. One is a type, and one is a constructor function with the same name.
So if you see a function signature like this:
val f: string -> EmailAddress

this refers to things in the world of types, so EmailAddress refers to the type.
On the other hand, if you see some code like this:
let x = EmailAddress y

this refers to things in the world of values, so EmailAddress refers to the constructor
function.

Constructing single case unions

For values that have special meaning, such as email addresses and zip codes, generally
only certain values are allowed. Not every string is an acceptable email or zip code.
This implies that we will need to do validation at some point, and what better point than at
construction time? After all, once the value is constructed, it is immutable, so there is no
worry that someone might modify it later.
Here's how we might extend the above module with some constructor functions:

875

Single case union types

... types as above ...

let CreateEmailAddress (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Some (EmailAddress s)
else None
let CreateStateCode (s:string) =
let s' = s.ToUpper()
let stateCodes = ["AZ";"CA";"NY"] //etc
if stateCodes |> List.exists ((=) s')
then Some (StateCode s')
else None

We can test the constructors now:

CreateStateCode "CA"
CreateStateCode "XX"
CreateEmailAddress "a@example.com"
CreateEmailAddress "example.com"

Handling invalid input in a constructor

With these kinds of constructor functions, one immediate challenge is the question of how to
handle invalid input. For example, what should happen if I pass in "abc" to the email address
constructor?
There are a number of ways to deal with it.
First, you could throw an exception. I find this ugly and unimaginative, so I'm rejecting this
one out of hand!
Next, you could return an option type, with None meaning that the input wasn't valid. This is
what the constructor functions above do.
This is generally the easiest approach. It has the advantage that the caller has to explicitly
handle the case when the value is not valid.
For example, the caller's code for the example above might look like:
match (CreateEmailAddress "a@example.com") with
| Some email -> ... do something with email
| None -> ... ignore?

876

Single case union types

The disadvantage is that with complex validations, it might not be obvious what went wrong.
Was the email too long, or missing a '@' sign, or an invalid domain? We can't tell.
If you do need more detail, you might want to return a type which contains a more detailed
explanation in the error case.
The following example uses a CreationResult type to indicate the error in the failure case.
type EmailAddress = EmailAddress of string
type CreationResult<'T> = Success of 'T | Error of string
let CreateEmailAddress2 (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Success (EmailAddress s)
else Error "Email address must contain an @ sign"
// test
CreateEmailAddress2 "example.com"

Finally, the most general approach uses continuations. That is, you pass in two functions,
one for the success case (that takes the newly constructed email as parameter), and
another for the failure case (that takes the error string as parameter).
type EmailAddress = EmailAddress of string
let CreateEmailAddressWithContinuations success failure (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then success (EmailAddress s)
else failure "Email address must contain an @ sign"

The success function takes the email as a parameter and the error function takes a string.
Both functions must return the same type, but the type is up to you.
Here is a simple example -- both functions do a printf, and return nothing (i.e. unit).
let success (EmailAddress s) = printfn "success creating email %s" s
let failure msg = printfn "error creating email: %s" msg
CreateEmailAddressWithContinuations success failure "example.com"
CreateEmailAddressWithContinuations success failure "x@example.com"

With continuations, you can easily reproduce any of the other approaches. Here's the way to
create options, for example. In this case both functions return an EmailAddress option .

877

Single case union types

let success e = Some e

let failure _ = None
CreateEmailAddressWithContinuations success failure "example.com"
CreateEmailAddressWithContinuations success failure "x@example.com"

And here is the way to throw exceptions in the error case:

let success e = e
let failure _ = failwith "bad email address"
CreateEmailAddressWithContinuations success failure "example.com"
CreateEmailAddressWithContinuations success failure "x@example.com"

This code seems quite cumbersome, but in practice you would probably create a local
partially applied function that you use instead of the long-winded one.
// setup a partially applied function
let success e = Some e
let failure _ = None
let createEmail = CreateEmailAddressWithContinuations success failure
// use the partially applied function
createEmail "x@example.com"
createEmail "example.com"

Creating modules for wrapper types

These simple wrapper types are starting to get more complicated now that we are adding
validations, and we will probably discover other functions that we want to associate with the
type.
So it is probably a good idea to create a module for each wrapper type, and put the type and
its associated functions there.

878

Single case union types

module EmailAddress =
type T = EmailAddress of string
// wrap
let create (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Some (EmailAddress s)
else None
// unwrap
let value (EmailAddress e) = e

The users of the type would then use the module functions to create and unwrap the type.
For example:

// create email addresses

let address1 = EmailAddress.create "x@example.com"
let address2 = EmailAddress.create "example.com"
// unwrap an email address
match address1 with
| Some e -> EmailAddress.value e |> printfn "the value is %s"
| None -> ()

Forcing use of the constructor

One issue is that you cannot force callers to use the constructor. Someone could just bypass
the validation and create the type directly.
In practice, that tends not to be a problem. One simple techinique is to use naming
conventions to indicate a "private" type, and provide "wrap" and "unwrap" functions so that
the clients never need to interact with the type directly.
Here's an example:

879

Single case union types

module EmailAddress =
// private type
type _T = EmailAddress of string
// wrap
let create (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Some (EmailAddress s)
else None
// unwrap
let value (EmailAddress e) = e

Of course the type is not really private in this case, but you are encouraging the callers to
always use the "published" functions.
If you really want to encapsulate the internals of the type and force callers to use a
constructor function, you can use module signatures.
Here's a signature file for the email address example:
// FILE: EmailAddress.fsi
module EmailAddress
// encapsulated type
type T
// wrap
val create : string -> T option
// unwrap
val value : T -> string

(Note that module signatures only work in compiled projects, not in interactive scripts, so to
test this, you will need to create three files in an F# project, with the filenames as shown
here.)
Here's the implementation file:

880

Single case union types

// FILE: EmailAddress.fs
module EmailAddress
// encapsulated type
type T = EmailAddress of string
// wrap
let create (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Some (EmailAddress s)
else None
// unwrap
let value (EmailAddress e) = e

And here's a client:

// FILE: EmailAddressClient.fs
module EmailAddressClient
open EmailAddress
// code works when using the published functions
let address1 = EmailAddress.create "x@example.com"
let address2 = EmailAddress.create "example.com"
// code that uses the internals of the type fails to compile
let address3 = T.EmailAddress "bad email"

The type EmailAddress.T exported by the module signature is opaque, so clients cannot
access the internals.
As you can see, this approach enforces the use of the constructor. Trying to create the type
directly ( T.EmailAddress "bad email" ) causes a compile error.

When to "wrap" single case unions

Now that we have the wrapper type, when should we construct them?
Generally you only need to at service boundaries (for example, boundaries in a hexagonal
architecture)

881

Single case union types

In this approach, wrapping is done in the UI layer, or when loading from a persistence layer,
and once the wrapped type is created, it is passed in to the domain layer and manipulated
"whole", as an opaque type. It is suprisingly uncommon that you actually need the wrapped
contents directly when working in the domain itself.
As part of the construction, it is critical that the caller uses the provided constructor rather
than doing its own validation logic. This ensures that "bad" values can never enter the
domain.
For example, here is some code that shows the UI doing its own validation:
let processFormSubmit () =
let s = uiTextBox.Text
if (s.Length < 50)
then // set email on domain object
else // show validation error message

A better way is to let the constructor do it, as shown earlier.

let processFormSubmit () =
let emailOpt = uiTextBox.Text |> EmailAddress.create
match emailOpt with
| Some email -> // set email on domain object
| None -> // show validation error message

When to "unwrap" single case unions

And when is unwrapping needed? Again, generally only at service boundaries. For example,
when you are persisting an email to a database, or binding to a UI element or view model.
One tip to avoid explicit unwrapping is to use the continuation approach again, passing in a
function that will be applied to the wrapped value.
That is, rather than calling the "unwrap" function explicitly:
address |> EmailAddress.value |> printfn "the value is %s"

You would pass in a function which gets applied to the inner value, like this:
address |> EmailAddress.apply (printfn "the value is %s")

Putting this together, we now have the complete EmailAddress module.

882

Single case union types

module EmailAddress =
type _T = EmailAddress of string
// create with continuation
let createWithCont success failure (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then success (EmailAddress s)
else failure "Email address must contain an @ sign"
// create directly
let create s =
let success e = Some e
let failure _ = None
createWithCont success failure s
// unwrap with continuation
let apply f (EmailAddress e) = f e
// unwrap directly
let value e = apply id e

The create and value functions are not strictly necessary, but are added for the
convenience of callers.

The code so far

Let's refactor the Contact code now, with the new wrapper types and modules added.
module EmailAddress =
type T = EmailAddress of string
// create with continuation
let createWithCont success failure (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then success (EmailAddress s)
else failure "Email address must contain an @ sign"
// create directly
let create s =
let success e = Some e
let failure _ = None
createWithCont success failure s
// unwrap with continuation
let apply f (EmailAddress e) = f e

883

Single case union types

// unwrap directly
let value e = apply id e
module ZipCode =
type T = ZipCode of string
// create with continuation
let createWithCont success failure (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\d{5}$")
then success (ZipCode s)
else failure "Zip code must be 5 digits"
// create directly
let create s =
let success e = Some e
let failure _ = None
createWithCont success failure s
// unwrap with continuation
let apply f (ZipCode e) = f e
// unwrap directly
let value e = apply id e
module StateCode =
type T = StateCode of string
// create with continuation
let createWithCont success failure (s:string) =
let s' = s.ToUpper()
let stateCodes = ["AZ";"CA";"NY"] //etc
if stateCodes |> List.exists ((=) s')
then success (StateCode s')
else failure "State is not in list"
// create directly
let create s =
let success e = Some e
let failure _ = None
createWithCont success failure s
// unwrap with continuation
let apply f (StateCode e) = f e
// unwrap directly
let value e = apply id e
type PersonalName =
{
FirstName: string;
MiddleInitial: string option;

884

Single case union types

LastName: string;
}
type EmailContactInfo =
{
EmailAddress: EmailAddress.T;
IsEmailVerified: bool;
}
type PostalAddress =
{
Address1: string;
Address2: string;
City: string;
State: StateCode.T;
Zip: ZipCode.T;
}
type PostalContactInfo =
{
Address: PostalAddress;
IsAddressValid: bool;
}
type Contact =
{
Name: PersonalName;
EmailContactInfo: EmailContactInfo;
PostalContactInfo: PostalContactInfo;
}

By the way, notice that we now have quite a lot of duplicate code in the three wrapper type
modules. What would be a good way of getting rid of it, or at least making it cleaner?

Summary
To sum up the use of discriminated unions, here are some guidelines:
Do use single case discriminated unions to create types that represent the domain
accurately.
If the wrapped value needs validation, then provide constructors that do the validation
and enforce their use.
Be clear what happens when validation fails. In simple cases, return option types. In
more complex cases, let the caller pass in handlers for success and failure.
If the wrapped value has many associated functions, consider moving it into its own
module.
If you need to enforce encapsulation, use signature files.

885

Single case union types

We're still not done with refactoring. We can alter the design of types to enforce business
rules at compile time -- making illegal states unrepresentable.

Update
Many people have asked for more information on how to ensure that constrained types such
as EmailAddress are only created through a special constructor that does the validation. So
I have created a gist here that has some detailed examples of other ways of doing it.

886

Making illegal states unrepresentable

Designing with types: Making illegal states

unrepresentable
In this post, we look at a key benefit of F#, which using the type system to "make illegal
states unrepresentable" (a phrase borrowed from Yaron Minsky).
Let's look at our Contact type. Thanks to the previous refactoring, it is quite simple:
type Contact =
{
Name: Name;
EmailContactInfo: EmailContactInfo;
PostalContactInfo: PostalContactInfo;
}

Now let's say that we have the following simple business rule: "A contact must have an email
or a postal address". Does our type conform to this rule?
The answer is no. The business rule implies that a contact might have an email address but
no postal address, or vice versa. But as it stands, our type requires that a contact must
always have both pieces of information.
The answer seems obvious -- make the addresses optional, like this:
type Contact =
{
Name: PersonalName;
EmailContactInfo: EmailContactInfo option;
PostalContactInfo: PostalContactInfo option;
}

But now we have gone too far the other way. In this design, it would be possible for a contact
to have neither type of address at all. But the business rule says that at least one piece of
information must be present.
What's the solution?

Making illegal states unrepresentable

If we think about the business rule carefully, we realize that there are three possibilities:

887

Making illegal states unrepresentable

A contact only has an email address

A contact only has a postal address
A contact has both a email address and a postal address
Once it is put like this, the solution becomes obvious -- use a union type with a case for each
possibility.
type ContactInfo =
| EmailOnly of EmailContactInfo
| PostOnly of PostalContactInfo
| EmailAndPost of EmailContactInfo * PostalContactInfo
type Contact =
{
Name: Name;
ContactInfo: ContactInfo;
}

This design meets the requirements perfectly. All three cases are explictly represented, and
the fourth possible case (with no email or postal address at all) is not allowed.
Note that for the "email and post" case, I just used a tuple type for now. It's perfectly
adequate for what we need.

Constructing a ContactInfo
Now let's see how we might use this in practice. We'll start by creating a new contact:
let contactFromEmail name emailStr =
let emailOpt = EmailAddress.create emailStr
// handle cases when email is valid or invalid
match emailOpt with
| Some email ->
let emailContactInfo =
{EmailAddress=email; IsEmailVerified=false}
let contactInfo = EmailOnly emailContactInfo
Some {Name=name; ContactInfo=contactInfo}
| None -> None
let name = {FirstName = "A"; MiddleInitial=None; LastName="Smith"}
let contactOpt = contactFromEmail name "abc@example.com"

In this code, we have created a simple helper function contactFromEmail to create a new
contact by passing in a name and email. However, the email might not be valid, so the
function has to handle both cases, which it doesn't by returning a Contact option , not a
Contact

888

Making illegal states unrepresentable

Updating a ContactInfo
Now if we need to add a postal address to an existing ContactInfo , we have no choice but
to handle all three possible cases:
If a contact previously only had an email address, it now has both an email address and
a postal address, so return a contact using the EmailAndPost case.
If a contact previously only had a postal address, return a contact using the PostOnly
case, replacing the existing address.
If a contact previously had both an email address and a postal address, return a contact
with using the EmailAndPost case, replacing the existing address.
So here's a helper method that updates the postal address. You can see how it explicitly
handles each case.
let updatePostalAddress contact newPostalAddress =
let {Name=name; ContactInfo=contactInfo} = contact
let newContactInfo =
match contactInfo with
| EmailOnly email ->
EmailAndPost (email,newPostalAddress)
| PostOnly _ -> // ignore existing address
PostOnly newPostalAddress
| EmailAndPost (email,_) -> // ignore existing address
EmailAndPost (email,newPostalAddress)
// make a new contact
{Name=name; ContactInfo=newContactInfo}

And here is the code in use:

let contact = contactOpt.Value // see warning about option.Value below
let newPostalAddress =
let state = StateCode.create "CA"
let zip = ZipCode.create "97210"
{
Address =
{
Address1= "123 Main";
Address2="";
City="Beverly Hills";
State=state.Value; // see warning about option.Value below
Zip=zip.Value; // see warning about option.Value below
};
IsAddressValid=false
}
let newContact = updatePostalAddress contact newPostalAddress

889

Making illegal states unrepresentable

WARNING: I am using option.Value to extract the contents of an option in this code. This is
ok when playing around interactively but is extremely bad practice in production code! You
should always use matching to handle both cases of an option.

Why bother to make these complicated types?

At this point, you might be saying that we have made things unnecessarily complicated. I
would answer with these points:
First, the business logic is complicated. There is no easy way to avoid it. If your code is not
this complicated, you are not handling all the cases properly.
Second, if the logic is represented by types, it is automatically self documenting. You can
look at the union cases below and immediate see what the business rule is. You do not have
to spend any time trying to analyze any other code.
type ContactInfo =
| EmailOnly of EmailContactInfo
| PostOnly of PostalContactInfo
| EmailAndPost of EmailContactInfo * PostalContactInfo

Finally, if the logic is represented by a type, any changes to the business rules will
immediately create breaking changes, which is a generally a good thing.
In the next post, we'll dig deeper into the last point. As you try to represent business logic
using types, you may suddenly find that can gain a whole new insight into the domain.

890

Discovering new concepts

Designing with types: Discovering new

concepts
In the last post, we looked at how we could represent a business rule using types.
The rule was: "A contact must have an email or a postal address".
And the type we designed was:
type ContactInfo =
| EmailOnly of EmailContactInfo
| PostOnly of PostalContactInfo
| EmailAndPost of EmailContactInfo * PostalContactInfo

Now let's say that the business decides that phone numbers need to be supported as well.
The new business rule is: "A contact must have at least one of the following: an email, a
postal address, a home phone, or a work phone".
How can we represent this now?
A little thought reveals that there are 15 possible combinations of these four contact
methods. Surely we don't want to create a union case with 15 choices? Is there a better
way?
Let's hold that thought and look at a different but related problem.

Forcing breaking changes when requirements

change
Here's the problem. Say that you have a contact structure which contains a list of email
addresses and also a list of postal addresses, like so:
type ContactInformation =
{
EmailAddresses : EmailContactInfo list;
PostalAddresses : PostalContactInfo list
}

And, also let's say that you have created a printReport function that loops through the
information and prints it out in a report:

891

Discovering new concepts

// mock code
let printEmail emailAddress =
printfn "Email Address is %s" emailAddress
// mock code
let printPostalAddress postalAddress =
printfn "Postal Address is %s" postalAddress
let printReport contactInfo =
let {
EmailAddresses = emailAddresses;
PostalAddresses = postalAddresses;
} = contactInfo
for email in emailAddresses do
printEmail email
for postalAddress in postalAddresses do
printPostalAddress postalAddress

Crude, but simple and understandable.

Now if the new business rule comes into effect, we might decide to change the structure to
have some new lists for the phone numbers. The updated structure will now look something
like this:
type PhoneContactInfo = string // dummy for now
type ContactInformation =
{
EmailAddresses : EmailContactInfo list;
PostalAddresses : PostalContactInfo list;
HomePhones : PhoneContactInfo list;
WorkPhones : PhoneContactInfo list;
}

If you make this change, you also want to make sure that all the functions that process the
contact infomation are updated to handle the new phone cases as well.
Certainly, you will be forced to fix any pattern matches that break. But in many cases, you
would not be forced to handle the new cases.
For example, here's printReport updated to work with the new lists:

892

Discovering new concepts

let printReport contactInfo =

let {
EmailAddresses = emailAddresses;
PostalAddresses = postalAddresses;
} = contactInfo
for email in emailAddresses do
printEmail email
for postalAddress in postalAddresses do
printPostalAddress postalAddress

Can you see the deliberate mistake? Yes, I forgot to change the function to handle the
phones. The new fields in the record have not caused the code to break at all. There is no
guarantee that you will remember to handle the new cases. It would be all too easy to forget.
Again, we have the challenge: can we design types such that these situations cannot easily
happen?

Deeper insight into the domain

If you think about this example a bit more deeply, you will realize that we have missed the
forest for the trees.
Our initial concept was: "to contact a customer, there will be a list of possible emails, and a
list of possible addresses, etc".
But really, this is all wrong. A much better concept is: "To contact a customer, there will be a
list of contact methods. Each contact method could be an email OR a postal address OR a
phone number".
This is a key insight into how the domain should be modelled. It creates a whole new type, a
"ContactMethod", which resolves our problems in one stroke.
We can immediately refactor the types to use this new concept:
type ContactMethod =
| Email of EmailContactInfo
| PostalAddress of PostalContactInfo
| HomePhone of PhoneContactInfo
| WorkPhone of PhoneContactInfo
type ContactInformation =
{
ContactMethods : ContactMethod list;
}

893

Discovering new concepts

And the reporting code must now be changed to handle the new type as well:
// mock code
let printContactMethod cm =
match cm with
| Email emailAddress ->
printfn "Email Address is %s" emailAddress
| PostalAddress postalAddress ->
printfn "Postal Address is %s" postalAddress
| HomePhone phoneNumber ->
printfn "Home Phone is %s" phoneNumber
| WorkPhone phoneNumber ->
printfn "Work Phone is %s" phoneNumber
let printReport contactInfo =
let {
ContactMethods=methods;
} = contactInfo
methods
|> List.iter printContactMethod

These changes have a number of benefits.

First, from a modelling point of view, the new types represent the domain much better, and
are more adaptable to changing requitements.
And from a development point of view, changing the type to be a union means that any new
cases that we add (or remove) will break the code in a very obvious way, and it will be much
harder to accidentally forget to handle all the cases.

Back to the business rule with 15 possible

combinations
So now back to the original example. We left it thinking that, in order to encode the business
rule, we might have to create 15 possible combinations of various contact methods.
But the new insight from the reporting problem also affects our understanding of the
business rule.
With the "contact method" concept in our heads, we can rephase the requirement as: "A
customer must have at least one contact method. A contact method could be an email OR a
postal addresses OR a phone number".
So let's redesign the Contact type to have a list of contact methods:

894

Discovering new concepts

type Contact =
{
Name: PersonalName;
ContactMethods: ContactMethod list;
}

But this is still not quite right. The list could be empty. How can we enforce the rule that there
must be at least one contact method?
The simplest way is to create a new field that is required, like this:
type Contact =
{
Name: PersonalName;
PrimaryContactMethod: ContactMethod;
SecondaryContactMethods: ContactMethod list;
}

In this design, the PrimaryContactMethod is required, and the secondary contact methods
are optional, which is exactly what the business rule requires!
And this refactoring too, has given us some insight. It may be that the concepts of "primary"
and "secondary" contact methods might, in turn, clarify code in other areas, creating a
cascading change of insight and refactoring.

Summary
In this post, we've seen how using types to model business rules can actually help you to
understand the domain at a deeper level.
In the Domain Driven Design book, Eric Evans devotes a whole section and two chapters in
particular (chapters 8 and 9) to discussing the importance of refactoring towards deeper
insight. The example in this post is simple in comparison, but I hope that it shows that how
an insight like this can help improve both the model and the code correctness.
In the next post, we'll see how types can help with representing fine-grained states.

895

Making state explicit

Designing with types: Making state explicit

In this post we will look at making implicit states explicit by using state machines, and then
modelling these state machines with union types.

Background
In an earlier post in this series, we looked at single case unions as a wrapper for types such
as email addresses.
module EmailAddress =
type T = EmailAddress of string
let create (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then Some (EmailAddress s)
else None

This code assumes that either an address is valid or it is not. If it is not, we reject it
altogether and return None instead of a valid value.
But there are degrees of validity. For example, what happens if we want to keep an invalid
email address around rather than just rejecting it? In this case, as usual, we want to use the
type system to make sure that we don't get a valid address mixed up with an invalid
address.
The obvious way to do this is with a union type:
module EmailAddress =
type T =
| ValidEmailAddress of string
| InvalidEmailAddress of string
let create (s:string) =
if System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
then ValidEmailAddress s // change result type
else InvalidEmailAddress s // change result type
// test
let valid = create "abc@example.com"
let invalid = create "example.com"

896

Making state explicit

and with these types we can ensure that only valid emails get sent:
let sendMessageTo t =
match t with
| ValidEmailAddress email ->
// send email
| InvalidEmailAddress _ ->
// ignore

So far, so good. This kind of design should be obvious to you by now.

But this approach is more widely applicable than you might think. In many situations, there
are similar "states" that are not made explicit, and handled with flags, enums, or conditional
logic in code.

State machines
In the example above, the "valid" and "invalid" cases are mutually incompatible. That is, a
valid email can never become invalid, and vice versa.
But in many cases, it is possible to go from one case to another, triggered by some kind of
event. At which point we have a "state machine", where each case represents a "state", and
moving from one state to another is a "transition".
Some examples:
A email address might have states "Unverified" and "Verified", where you can transition
from the "Unverified" state to the "Verified" state by asking the user to click on a link in a

confirmation email.
A shopping cart might have states "Empty", "Active" and "Paid", where you can
transition from the "Empty" state to the "Active" state by adding an item to the cart, and
to the "Paid" state by paying.

897

Making state explicit

A game such as chess might have states "WhiteToPlay", "BlackToPlay" and

"GameOver", where you can transition from the "WhiteToPlay" state to the
"BlackToPlay" state by White making a non-game-ending move, or transition to the
"GameOver" state by playing a checkmate move.

In each of these cases, we have a set of states, a set of transitions, and events that can
trigger a transition.
State machines are often represented by a table, like this one for a shopping cart:
Current
State

Event>

Add Item

Remove Item

Pay

Empty

new state =
Active

n/a

Active

new state =
Active

new state = Active or

Empty,
depending on the number
of items

new state =
Paid

n/a

Paid

With a table like this, you can quickly see exactly what should happen for each event when
the system is in a given state.

Why use state machines?

There are a number of benefits to using state machines in these cases:
Each state can have different allowable behavior.
In the verified email example, there is probably a business rule that says that you can only
send password resets to verified email addresses, not to unverified addresses. And in the
shopping cart example, only an active cart can be paid for, and a paid cart cannot be added
to.
All the states are explicitly documented.

898

Making state explicit

It is all too easy to have important states that are implicit but never documented.
For example, the "empty cart" has different behavior from the "active cart" but it would be
rare to see this documented explicitly in code.
It is a design tool that forces you to think about every possibility that could occur.
A common cause of errors is that certain edge cases are not handled, but a state machine
forces all cases to be thought about.
For example, what should happen if we try to verify an already verified email? What happens
if we try to remove an item from an empty shopping cart? What happens if white tries to play
when the state is "BlackToPlay"? And so on.

How to implement simple state machines in F#

You are probably familiar with complex state machines, such as those used in language
parsers and regular expressions. Those kinds of state machines are generated from rule
sets or grammars, and are quite complicated.
The kinds of state machines that I'm talking about are much, much simpler. Just a few cases
at the most, with a small number of transitions, so we don't need to use complex generators.
So what is the best way implement these simple state machines?
Typically, each state will have its own type, to store the data that is relevant to that state (if
any), and then the entire set of states will be represented by a union class.
Here's an example using the shopping cart state machine:
type ActiveCartData = { UnpaidItems: string list }
type PaidCartData = { PaidItems: string list; Payment: float }
type ShoppingCart =
| EmptyCart // no data
| ActiveCart of ActiveCartData
| PaidCart of PaidCartData

Note that the EmptyCart state has no data, so no special type is needed.
Each event is then represented by a function that accepts the entire state machine (the
union type) and returns a new version of the state machine (again, the union type).
Here's an example using two of the shopping cart events:

899

Making state explicit

let addItem cart item =

match cart with
| EmptyCart ->
// create a new active cart with one item
ActiveCart {UnpaidItems=[item]}
| ActiveCart {UnpaidItems=existingItems} ->
// create a new ActiveCart with the item added
ActiveCart {UnpaidItems = item :: existingItems}
| PaidCart _ ->
// ignore
cart
let makePayment cart payment =
match cart with
| EmptyCart ->
// ignore
cart
| ActiveCart {UnpaidItems=existingItems} ->
// create a new PaidCart with the payment
PaidCart {PaidItems = existingItems; Payment=payment}
| PaidCart _ ->
// ignore
cart

You can see that from the caller's point of view, the set of states is treated as "one thing" for
general manipulation (the ShoppingCart type), but when processing the events internally,
each state is treated separately.

Designing event handling functions

Guideline: Event handling functions should always accept and return the entire state
machine
You might ask: why do we have to pass in the whole shopping cart to the event-handling
functions? For example, the makePayment event only has relevance when the cart is in the
Active state, so why not just explicitly pass it the ActiveCart type, like this:
let makePayment2 activeCart payment =
let {UnpaidItems=existingItems} = activeCart
{PaidItems = existingItems; Payment=payment}

Let's compare the function signatures:

900

Making state explicit

// the original function

val makePayment : ShoppingCart -> float -> ShoppingCart
// the new more specific function
val makePayment2 : ActiveCartData -> float -> PaidCartData

You will see that the original makePayment function takes a cart and results in a cart, while
the new function takes an ActiveCartData and results in a PaidCartData , which seems to
be much more relevant.
But if you did this, how would you handle the same event when the cart was in a different
state, such as empty or paid? Someone has to handle the event for all three possible states
somewhere, and it is much better to encapsulate this business logic inside the function than
to be at the mercy of the caller.

Working with "raw" states

Occasionally you do genuinely need to treat one of the states as a separate entity in its own
right and use it independently. Because each state is a type as well, this is normally
straightforward.
For example, if I need to report on all paid carts, I can pass it a list of PaidCartData .
let paymentReport paidCarts =
let printOneLine {Payment=payment} =
printfn "Paid %f for items" payment
paidCarts |> List.iter printOneLine

By using a list of PaidCartData as the parameter rather than ShoppingCart itself, I ensure
that I cannot accidentally report on unpaid carts.
If you do this, it should be in a supporting function to the event handlers, never the event
handlers themselves.

Using explicit states to replace boolean flags

Let's look at how we can apply this approach to a real example now.
In the Contact example from an earlier post we had a flag that was used to indicate
whether a customer had verified their email address. The type looked like this:

901

Making state explicit

type EmailContactInfo =
{
EmailAddress: EmailAddress.T;
IsEmailVerified: bool;
}

Any time you see a flag like this, chances are you are dealing with state. In this case, the
boolean is used to indicate that we have two states: "Unverified" and "Verified".
As mentioned above, there will probably be various business rules associated with what is
permissible in each state. For example, here are two:
Business rule: "Verification emails should only be sent to customers who have
unverified email addresses"
Business rule: "Password reset emails should only be sent to customers who have
verified email addresses"
As before, we can use types to ensure that code conforms to these rules.
Let's rewrite the EmailContactInfo type using a state machine. We'll put it in an module as
well.
We'll start by defining the two states.
For the "Unverified" state, the only data we need to keep is the email address.
For the "Verified" state, we might want to keep some extra data in addition to the email
address, such as the date it was verified, the number of recent password resets, on so
on. This data is not relevant (and should not even be visible) to the "Unverified" state.
module EmailContactInfo =
open System
// placeholder
type EmailAddress = string
// UnverifiedData = just the email
type UnverifiedData = EmailAddress
// VerifiedData = email plus the time it was verified
type VerifiedData = EmailAddress * DateTime
// set of states
type T =
| UnverifiedState of UnverifiedData
| VerifiedState of VerifiedData

902

Making state explicit

Note that for the UnverifiedData type I just used a type alias. No need for anything more
complicated right now, but using a type alias makes the purpose explicit and helps with
refactoring.
Now let's handle the construction of a new state machine, and then the events.
Construction always results in an unverified email, so that is easy.
There is only one event that transitions from one state to another: the "verified" event.
module EmailContactInfo =
// types as above
let create email =
// unverified on creation
UnverifiedState email
// handle the "verified" event
let verified emailContactInfo dateVerified =
match emailContactInfo with
| UnverifiedState email ->
// construct a new info in the verified state
VerifiedState (email, dateVerified)
| VerifiedState _ ->
// ignore
emailContactInfo

Note that, as discussed here, every branch of the match must return the same type, so when
ignoring the verified state we must still return something, such as the object that was passed
in.
Finally, we can write the two utility functions sendVerificationEmail and sendPasswordReset .

903

Making state explicit

module EmailContactInfo =
// types and functions as above
let sendVerificationEmail emailContactInfo =
match emailContactInfo with
| UnverifiedState email ->
// send email
printfn "sending email"
| VerifiedState _ ->
// do nothing
()
let sendPasswordReset emailContactInfo =
match emailContactInfo with
| UnverifiedState email ->
// ignore
()
| VerifiedState _ ->
// ignore
printfn "sending password reset"

Using explicit cases to replace case/switch

statements
Sometimes it is not just a simple boolean flag that is used to indicate state. In C# and Java it
is common to use a int or an enum to represent a set of states.
For example, here's a simple state diagram of a package status for a delivery system, where
a package has three possible states:

There are some obvious business rules that come out of this diagram:
Rule: "You can't put a package on a truck if it is already out for delivery"
Rule: "You can't sign for a package that is already delivered"
and so on.

904

Making state explicit

Now, without using union types, we might represent this design by using an enum to
represent the state, like this:
open System
type PackageStatus =
| Undelivered
| OutForDelivery
| Delivered
type Package =
{
PackageId: int;
PackageStatus: PackageStatus;
DeliveryDate: DateTime;
DeliverySignature: string;
}

And then the code to handle the "putOnTruck" and "signedFor" events might look like this:
let putOnTruck package =
{package with PackageStatus=OutForDelivery}
let signedFor package signature =
let {PackageStatus=packageStatus} = package
if (packageStatus = Undelivered)
then
failwith "package not out for delivery"
else if (packageStatus = OutForDelivery)
then
{package with
PackageStatus=OutForDelivery;
DeliveryDate = DateTime.UtcNow;
DeliverySignature=signature;
}
else
failwith "package already delivered"

This code has some subtle bugs in it.

When handling the "putOnTruck" event, what should happen in the case that the status
is already OutForDelivery or Delivered . The code is not explicit about it.
When handling the "signedFor" event, we do handle the other states, but the last else
branch assumes that we only have three states, and therefore doesn't bother to be
explicit about testing for it. This code would be incorrect if we ever added a new status.
Finally, because the DeliveryDate and DeliverySignature are in the basic structure, it
would be possible to set them accidentally, even though the status was not Delivered .

905

Making state explicit

But as usual, the idiomatic and more type-safe F# approach is to use an overall union type
rather than embed a status value inside a data structure.
open System
type UndeliveredData =
{
PackageId: int;
}
type OutForDeliveryData =
{
PackageId: int;
}
type DeliveredData =
{
PackageId: int;
DeliveryDate: DateTime;
DeliverySignature: string;
}
type Package =
| Undelivered of UndeliveredData
| OutForDelivery of OutForDeliveryData
| Delivered of DeliveredData

And then the event handlers must handle every case.

let putOnTruck package =
match package with
| Undelivered {PackageId=id} ->
OutForDelivery {PackageId=id}
| OutForDelivery _ ->
failwith "package already out"
| Delivered _ ->
failwith "package already delivered"
let signedFor package signature =
match package with
| Undelivered _ ->
failwith "package not out"
| OutForDelivery {PackageId=id} ->
Delivered {
PackageId=id;
DeliveryDate = DateTime.UtcNow;
DeliverySignature=signature;
}
| Delivered _ ->
failwith "package already delivered"

906

Making state explicit

Note: I am using failWith to handle the errors. In a production system, this code should be
replaced by client driven error handlers. See the discussion of handling constructor errors in
the post about single case DUs for some ideas.

Using explicit cases to replace implicit

conditional code
Finally, there are often cases where a system has states, but they are implicit in conditional
code.
For example, here is a type that represents an order.
open System
type Order =
{
OrderId: int;
PlacedDate: DateTime;
PaidDate: DateTime option;
PaidAmount: float option;
ShippedDate: DateTime option;
ShippingMethod: string option;
ReturnedDate: DateTime option;
ReturnedReason: string option;
}

You can guess that Orders can be "new", "paid", "shipped" or "returned", and have
timestamps and extra information for each transition, but this is not made explicit in the
structure.
The option types are a clue that this type is trying to do too much. At least F# forces you to
use options -- in C# or Java these might just be nulls, and you would have no idea from the
type definition whether they were required or not.
And now let's look at the kind of ugly code that might test these option types to see what
state the order is in.
Again, there is some important business logic that depends on the state of the order, but
nowhere is it explicitly documented what the various states and transitions are.

907

Making state explicit

let makePayment order payment =

if (order.PaidDate.IsSome)
then failwith "order is already paid"
//return an updated order with payment info
{order with
PaidDate=Some DateTime.UtcNow
PaidAmount=Some payment
}
let shipOrder order shippingMethod =
if (order.ShippedDate.IsSome)
then failwith "order is already shipped"
//return an updated order with shipping info
{order with
ShippedDate=Some DateTime.UtcNow
ShippingMethod=Some shippingMethod
}

Note: I added IsSome to test for option values being present as a direct port of the way that
a C# program would test for null . But IsSome is both ugly and dangerous. Don't use it!
Here is a better approach using types that makes the states explicit.

908

Making state explicit

open System
type InitialOrderData =
{
OrderId: int;
PlacedDate: DateTime;
}
type PaidOrderData =
{
Date: DateTime;
Amount: float;
}
type ShippedOrderData =
{
Date: DateTime;
Method: string;
}
type ReturnedOrderData =
{
Date: DateTime;
Reason: string;
}
type Order =
| Unpaid of InitialOrderData
| Paid of InitialOrderData * PaidOrderData
| Shipped of InitialOrderData * PaidOrderData * ShippedOrderData
| Returned of InitialOrderData * PaidOrderData * ShippedOrderData * ReturnedOrderD
ata

And here are the event handling methods:

909

Making state explicit

let makePayment order payment =

match order with
| Unpaid i ->
let p = {Date=DateTime.UtcNow; Amount=payment}
// return the Paid order
Paid (i,p)
| _ ->
printfn "order is already paid"
order
let shipOrder order shippingMethod =
match order with
| Paid (i,p) ->
let s = {Date=DateTime.UtcNow; Method=shippingMethod}
// return the Shipped order
Shipped (i,p,s)
| Unpaid _ ->
printfn "order is not paid for"
order
| _ ->
printfn "order is already shipped"
order

Note: Here I am using printfn to handle the errors. In a production system, do use a
different approach.

When not to use this approach

As with any technique we learn, we have to be careful of treating it like a golden hammer.
This approach does add complexity, so before you start using it, be sure that benefits will
outweigh the costs.
To recap, here are the conditions where using simple state machines might be benficial:
You have a set of mutually exclusive states with transitions between them.
The transitions are triggered by external events.
The states are exhaustive. That is, there are no other choices and you must always
handle all cases.
Each state might have associated data that should not be accessable when the system
is in another state.
There are static business rules that apply to the states.
Let's look at some examples where these guidelines don't apply.
States are not important in the domain.

910

Making state explicit

Consider a blog authoring application. Typically, each blog post can be in a state such as
"Draft", "Published", etc. And there are obviously transitions between these states driven by
events (such as clicking a "publish" button).
But is it worth creating a state machine for this? Generally, I would say not.
Yes, there are state transitions, but is there really any change in logic because of this? From
the authoring point of view, most blogging apps don't have any restrictions based on the
state. You can author a draft post in exactly the same way as you author a published post.
The only part of the system that does care about the state is the display engine, and that
filters out the drafts in the database layer before it ever gets to the domain.
Since there is no special domain logic that cares about the state, it is probably unnecessary.
State transitions occur outside the application
In a customer management application, it is common to classify customers as "prospects",
"active", "inactive", etc.

In the application, these states have business meaning and should be represented by the
type system (such as a union type). But the state transitions generally do not occur within
the application itself. For example, we might classify a customer as inactive if they haven't
ordered anything for 6 months. And then this rule might be applied to customer records in a
database by a nightly batch job, or when the customer record is loaded from the database.
But from our application's point of view, the transitions do not happen within the application,
and so we do not need to create a special state machine.
Dynamic business rules
The last bullet point in the list above refers to "static" business rules. By this I mean that the
rules change slowly enough that they should be embedded into the code itself.
On the other hand, if the rules are dynamic and change frequently, it is probably not worth
going to the trouble of creating static types.
In these situations, you should consider using active patterns, or even a proper rules engine.

911

Making state explicit

Summary
In this post, we've seen that if you have data structures with explicit flags ("IsVerified") or
status fields ("OrderStatus"), or implicit state (clued by an excessive number of nullable or
option types), it is worth considering using a simple state machine to model the domain
objects. In most cases the extra complexity is compensated for by the explicit documention
of the states and the elimination of errors due to not handling all possible cases.

912

Constrained strings

Designing with types: Constrained strings

In a previous post, I talked about avoiding using plain primitive strings for email addresses,
zip codes, states, etc. By wrapping them in a single case union, we could (a) force the types
to be distinct and (b) add validation rules.
In this post, we'll look at whether we can extend that concept to an even more fine grained
level.

When is a string not a string?

Let's look a simple PersonalName type.
type PersonalName =
{
FirstName: string;
LastName: string;
}

The type says that the first name is a string . But really, is that all it is? Are there any other
constraints that we might need to add to it?
Well, OK, it must not be null. But that is assumed in F#.
What about the length of the string? Is it acceptable to have a name which is 64K characters
long? If not, then is there some maximum length allowed?
And can a name contain linefeed characters or tabs? Can it start or end with whitespace?
Once you put it this way, there are quite a lot of constraints even for a "generic" string. Here
are some of the obvious ones:
What is its maximum length?
Can it cross over multiple lines?
Can it have leading or trailing whitespace?
Can it contain non-printing characters?

Should these constraints be part of the domain

model?
913

Constrained strings

So we might acknowledge that some constraints exist, but should they really be part of the
domain model (and the corresponding types derived from it)? For example, the constraint
that a last name is limited to 100 characters -- surely that is specific to a particular
implementation and not part of the domain at all.
I would answer that there is a difference between a logical model and a physical model. In a
logical model some of these constraints might not be relevant, but in a physical model they
most certainly are. And when we are writing code, we are always dealing with a physical
model anyway.
Another reason for incorporating the constraints into the model is that often the model is
shared across many separate applications. For example, a personal name may be created
in a e-commerce application, which writes it into a database table and then puts it on a
message queue to be picked up by a CRM application, which in turn calls an email
templating service, and so on.
It is important that all these applications and services have the same idea of what a personal
name is, including the length and other constraints. If the model does not make the
constraints explicit, then it is easy to have a mismatch when moving across service
boundaries.
For example, have you ever written code that checks the length of a string before writing it to
a database?
void SaveToDatabase(PersonalName personalName)
{
var first = personalName.First;
if (first.Length > 50)
{
// ensure string is not too long
first = first.Substring(0,50);
}
//save to database
}

If the string is too long at this point, what should you do? Silently truncate it? Throw an
exception?
A better answer is to avoid the problem altogether if you can. By the time the string gets to
the database layer it is too late -- the database layer should not be making these kinds of
decisions.
The problem should be dealt with when the string was first created, not when it is used. In
other words, it should have been part of the validation of the string.

914

Constrained strings

But how can we trust that the validation has been done correctly for all possible paths? I
think you can guess the answer...

Modeling constrained strings with types

The answer, of course, is to create wrapper types which have the constraints built into the
type.
So let's knock up a quick prototype using the single case union technique we used before.
module String100 =
type T = String100 of string
let create (s:string) =
if s <> null && s.Length <= 100
then Some (String100 s)
else None
let apply f (String100 s) = f s
let value s = apply id s
module String50 =
type T = String50 of string
let create (s:string) =
if s <> null && s.Length <= 50
then Some (String50 s)
else None
let apply f (String50 s) = f s
let value s = apply id s
module String2 =
type T = String2 of string
let create (s:string) =
if s <> null && s.Length <= 2
then Some (String2 s)
else None
let apply f (String2 s) = f s
let value s = apply id s

Note that we immediately have to deal with the case when the validation fails by using an
option type as the result. It makes creation more painful, but we can't avoid it if we want the
benefits later.
For example, here is a good string and a bad string of length 2.

915

Constrained strings

let s2good = String2.create "CA"

let s2bad = String2.create "California"
match s2bad with
| Some s2 -> // update domain object
| None -> // handle error

In order to use the String2 value we are forced to check whether it is Some or None at the
time of creation.

Problems with this design

One problem is that we have a lot of duplicated code. In practice a typical domain only has a
few dozen string types, so there won't be that much wasted code. But still, we can probably
do better.
Another more serious problem is that comparisons become harder. A String50 is a different
type from a String100 so that they cannot be compared directly.
let s50 = String50.create "John"
let s100 = String100.create "Smith"
let s50' = s50.Value
let s100' = s100.Value
let areEqual = (s50' = s100') // compiler error

This kind of thing will make working with dictionaries and lists harder.

Refactoring
At this point we can exploit F#'s support for interfaces, and create a common interface that
all wrapped strings have to support, and also some standard functions:

916

Constrained strings

module WrappedString =
/// An interface that all wrapped strings support
type IWrappedString =
abstract Value : string
/// Create a wrapped value option
/// 1) canonicalize the input first
/// 2) If the validation succeeds, return Some of the given constructor
/// 3) If the validation fails, return None
/// Null values are never valid.
let create canonicalize isValid ctor (s:string) =
if s = null
then None
else
let s' = canonicalize s
if isValid s'
then Some (ctor s')
else None
/// Apply the given function to the wrapped value
let apply f (s:IWrappedString) =
s.Value |> f
/// Get the wrapped value
let value s = apply id s
/// Equality test
let equals left right =
(value left) = (value right)
/// Comparison
let compareTo left right =
(value left).CompareTo (value right)

The key function is create , which takes a constructor function and creates new values
using it only when the validation passes.
With this in place it is a lot easier to define new types:

917

Constrained strings

module WrappedString =
// ... code from above ...
/// Canonicalizes a string before construction
/// * converts all whitespace to a space char
/// * trims both ends
let singleLineTrimmed s =
System.Text.RegularExpressions.Regex.Replace(s,"\s"," ").Trim()
/// A validation function based on length
let lengthValidator len (s:string) =
s.Length <= len
/// A string of length 100
type String100 = String100 of string with
interface IWrappedString with
member this.Value = let (String100 s) = this in s
/// A constructor for strings of length 100
let string100 = create singleLineTrimmed (lengthValidator 100) String100
/// Converts a wrapped string to a string of length 100
let convertTo100 s = apply string100 s
/// A string of length 50
type String50 = String50 of string with
interface IWrappedString with
member this.Value = let (String50 s) = this in s
/// A constructor for strings of length 50
let string50 = create singleLineTrimmed (lengthValidator 50) String50
/// Converts a wrapped string to a string of length 50
let convertTo50 s = apply string50 s

For each type of string now, we just have to:

create a type (e.g. String100 )
an implementation of IWrappedString for that type
and a public constructor (e.g. string100 ) for that type.
(In the sample above I have also thrown in a useful convertTo to convert from one type to
another.)
The type is a simple wrapped type as we have seen before.
The implementation of the Value method of the IWrappedString could have been written
using multiple lines, like this:

918

Constrained strings

member this.Value =
let (String100 s) = this
s

But I chose to use a one liner shortcut:

member this.Value = let (String100 s) = this in s

The constructor function is also very simple. The canonicalize function is

singleLineTrimmed , the validator function checks the length, and the constructor is the
String100 function (the function associated with the single case, not to be confused with

the type of the same name).

let string100 = create singleLineTrimmed (lengthValidator 100) String100

If you want to have other types with different constraints, you can easily add them. For
example you might want to have a Text1000 type that supports multiple lines and
embedded tabs and is not trimmed.
module WrappedString =
// ... code from above ...
/// A multiline text of length 1000
type Text1000 = Text1000 of string with
interface IWrappedString with
member this.Value = let (Text1000 s) = this in s
/// A constructor for multiline strings of length 1000
let text1000 = create id (lengthValidator 1000) Text1000

Playing with the WrappedString module

We can now play with the module interactively to see how it works:

919

Constrained strings

let s50 = WrappedString.string50 "abc" |> Option.get

printfn "s50 is %A" s50
let bad = WrappedString.string50 null
printfn "bad is %A" bad
let s100 = WrappedString.string100 "abc" |> Option.get
printfn "s100 is %A" s100
// equality using module function is true
printfn "s50 is equal to s100 using module equals? %b" (WrappedString.equals s50 s100)
// equality using Object method is false
printfn "s50 is equal to s100 using Object.Equals? %b" (s50.Equals s100)
// direct equality does not compile
printfn "s50 is equal to s100? %b" (s50 = s100) // compiler error

When we need to interact with types such as maps that use raw strings, it is easy to
compose new helper functions.
For example, here are some helpers to work with maps:
module WrappedString =
// ... code from above ...
/// map helpers
let mapAdd k v map =
Map.add (value k) v map
let mapContainsKey k map =
Map.containsKey (value k) map
let mapTryFind k map =
Map.tryFind (value k) map

And here is how these helpers might be used in practice:

let abc = WrappedString.string50 "abc" |> Option.get
let def = WrappedString.string100 "def" |> Option.get
let map =
Map.empty
|> WrappedString.mapAdd abc "value for abc"
|> WrappedString.mapAdd def "value for def"
printfn "Found abc in map? %A" (WrappedString.mapTryFind abc map)
let xyz = WrappedString.string100 "xyz" |> Option.get
printfn "Found xyz in map? %A" (WrappedString.mapTryFind xyz map)

920

Constrained strings

So overall, this "WrappedString" module allows us to create nicely typed strings without
interfering too much. Now let's use it in a real situation.

Using the new string types in the domain

Now we have our types, we can change the definition of the PersonalName type to use them.
module PersonalName =
open WrappedString
type T =
{
FirstName: String50;
LastName: String100;
}
/// create a new value
let create first last =
match (string50 first),(string100 last) with
| Some f, Some l ->
Some {
FirstName = f;
LastName = l;
}
| _ ->
None

We have created a module for the type and added a creation function that converts a pair of
strings into a PersonalName .
Note that we have to decide what to do if either of the input strings are invalid. Again, we
cannot postpone the issue till later, we have to deal with it at construction time.
In this case we use the simple approach of creating an option type with None to indicate
failure.
Here it is in use:
let name = PersonalName.create "John" "Smith"

We can also provide additional helper functions in the module.

Let's say, for example, that we want to create a fullname function that will return the first
and last names joined together.
Again, more decisions to make.

921

Constrained strings

Should we return a raw string or a wrapped string? The advantage of the latter is that
the callers know exactly how long the string will be, and it will be compatible with other
similar types.
If we do return a wrapped string (say a String100 ), then how do we handle the the
case when the combined length is too long? (It could be up to 151 chars, based on the
length of the first and last name types.). We could either return an option, or force a
truncation if the combined length is too long.
Here's code that demonstrates all three options.
module PersonalName =
// ... code from above ...
/// concat the first and last names together
/// and return a raw string
let fullNameRaw personalName =
let f = personalName.FirstName |> value
let l = personalName.LastName |> value
f + " " + l
/// concat the first and last names together
/// and return None if too long
let fullNameOption personalName =
personalName |> fullNameRaw |> string100
/// concat the first and last names together
/// and truncate if too long
let fullNameTruncated personalName =
// helper function
let left n (s:string) =
if (s.Length > n)
then s.Substring(0,n)
else s
personalName
|> fullNameRaw // concat
|> left 100 // truncate
|> string100 // wrap
|> Option.get // this will always be ok

Which particular approach you take to implementing fullName is up to you. But it

demonstrates a key point about this style of type-oriented design: these decisions have to be
taken up front, when creating the code. You cannot postpone them till later.
This can be very annoying at times, but overall I think it is a good thing.

922

Constrained strings

Revisiting the email address and zip code

types
We can use this WrappedString module to reimplement the EmailAddress and ZipCode
types.
module EmailAddress =
type T = EmailAddress of string with
interface WrappedString.IWrappedString with
member this.Value = let (EmailAddress s) = this in s
let create =
let canonicalize = WrappedString.singleLineTrimmed
let isValid s =
(WrappedString.lengthValidator 100 s) &&
System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
WrappedString.create canonicalize isValid EmailAddress
/// Converts any wrapped string to an EmailAddress
let convert s = WrappedString.apply create s
module ZipCode =
type T = ZipCode of string with
interface WrappedString.IWrappedString with
member this.Value = let (ZipCode s) = this in s
let create =
let canonicalize = WrappedString.singleLineTrimmed
let isValid s =
System.Text.RegularExpressions.Regex.IsMatch(s,@"^\d{5}$")
WrappedString.create canonicalize isValid ZipCode
/// Converts any wrapped string to a ZipCode
let convert s = WrappedString.apply create s

Other uses of wrapped strings

This approach to wrapping strings can also be used for other scenarios where you don't
want to mix string types together accidentally.
One case that leaps to mind is ensuring safe quoting and unquoting of strings in web
applications.

923

Constrained strings

For example, let's say that you want to output a string to HTML. Should the string be
escaped or not?
If it is already escaped, you want to leave it alone but if it is not, you do want to escape it.
This can be a tricky problem. Joel Spolsky discusses using a naming convention here, but of
course, in F#, we want a type-based solution instead.
A type-based solution will probably use a type for "safe" (already escaped) HTML strings
( HtmlString say), and one for safe Javascript strings ( JsString ), one for safe SQL strings
( SqlString ), etc. Then these strings can be mixed and matched safely without accidentally
causing security issues.
I won't create a solution here (and you will probably be using something like Razor anyway),
but if you are interested you can read about a Haskell approach here and a port of that to
F#.

924

Non-string types

Designing with types: Non-string types

In this series we've seen a lot of uses of single case discriminated unions to wrap strings.
There is no reason why you cannot use this technique with other primitive types, such as
numbers and dates. Let's look a few examples.

Single case unions

In many cases, we want to avoid accidentally mixing up different kinds of integers. Two
domain objects may have the same representation (using integers) but they should never be
confused.
For example, you may have an OrderId and a CustomerId , both of which are stored as
ints. But they are not really ints. You cannot add 42 to a CustomerId , for example. And
CustomerId(42) is not equal to OrderId(42) . In fact, they should not even be allowed to be

compared at all.
Types to the rescue, of course.
type CustomerId = CustomerId of int
type OrderId = OrderId of int
let custId = CustomerId 42
let orderId = OrderId 42
// compiler error
printfn "cust is equal to order? %b" (custId = orderId)

Similarly, you might want avoid mixing up semantically different date values by wrapping
them in a type. ( DateTimeKind is an attempt at this, but not always reliable.)
type LocalDttm = LocalDttm of System.DateTime
type UtcDttm = UtcDttm of System.DateTime

With these types we can ensure that we always pass the right kind of datetime as
parameters. Plus, it acts as documentation as well.

925

Non-string types

let SetOrderDate (d:LocalDttm) =

() // do something
let SetAuditTimestamp (d:UtcDttm) =
() // do something

Constraints on integers
Just as we had validation and constraints on types such as String50 and ZipCode , we can
use the same approach when we need to have constraints on integers.
For example, an inventory management system or a shopping cart may require that certain
types of number are always positive. You might ensure this by creating a NonNegativeInt
type.
module NonNegativeInt =
type T = NonNegativeInt of int
let create i =
if (i >= 0 )
then Some (NonNegativeInt i)
else None
module InventoryManager =
// example of NonNegativeInt in use
let SetStockQuantity (i:NonNegativeInt.T) =
//set stock
()

Embedding business rules in the type

Just as we wondered earlier whether first names could ever be 64K characters long, can you
really add 999999 items to your shopping cart?

Is it worth trying to avoid this issue by using constrained types? Let's look at some real code.
Here is a very simple shopping cart manager using a standard int type for the quantity.
The quantity is incremented or decremented when the related buttons are clicked. Can you
find the obvious bug?

926

Non-string types

module ShoppingCartWithBug =
let mutable itemQty = 1 // don't do this at home!
let incrementClicked() =
itemQty <- itemQty + 1
let decrementClicked() =
itemQty <- itemQty - 1

If you can't quickly find the bug, perhaps you should consider making any constraints more
explicit.
Here is the same simple shopping cart manager using a typed quantity instead. Can you find
the bug now? (Tip: paste the code into a F# script file and run it)
module ShoppingCartQty =
type T = ShoppingCartQty of int
let initialValue = ShoppingCartQty 1
let create i =
if (i > 0 && i < 100)
then Some (ShoppingCartQty i)
else None
let increment t = create (t + 1)
let decrement t = create (t - 1)
module ShoppingCartWithTypedQty =
let mutable itemQty = ShoppingCartQty.initialValue
let incrementClicked() =
itemQty <- ShoppingCartQty.increment itemQty
let decrementClicked() =
itemQty <- ShoppingCartQty.decrement itemQty

You might think this is overkill for such a trivial problem. But if you want to avoid being in the
DailyWTF, it might be worth considering.

Constraints on dates

927

Non-string types

Not all systems can handle all possible dates. Some systems can only store dates going
back to 1/1/1980, and some systems can only go into the future up to 2038 (I like to use
1/1/2038 as a max date to avoid US/UK issues with month/day order).
As with integers, it might be useful to have constraints on the valid dates built into the type,
so that any out of bound issues are dealt with at construction time rather than later on.
type SafeDate = SafeDate of System.DateTime
let create dttm =
let min = new System.DateTime(1980,1,1)
let max = new System.DateTime(2038,1,1)
if dttm < min || dttm > max
then None
else Some (SafeDate dttm)

Union types vs. units of measure

You might be asking at this point: What about units of measure? Aren't they meant to be
used for this purpose?
Yes and no. Units of measure can indeed be used to avoid mixing up numeric values of
different type, and are much more powerful than the single case unions we've been using.
On the other hand, units of measure are not encapsulated and cannot have constraints.
Anyone can create a int with unit of measure <kg> say, and there is no min or max value.
In many cases, both approaches will work fine. For example, there are many parts of the
.NET library that use timeouts, but sometimes the timeouts are set in seconds, and
sometimes in milliseconds. I often have trouble remembering which is which. I definitely
don't want to accidentally use a 1000 second timeout when I really meant a 1000 millisecond
timeout.
To avoid this scenario, I often like to create separate types for seconds and milliseconds.
Here's a type based approach using single case unions:

928

Non-string types

type TimeoutSecs = TimeoutSecs of int

type TimeoutMs = TimeoutMs of int
let toMs (TimeoutSecs secs) =
TimeoutMs (secs * 1000)
let toSecs (TimeoutMs ms) =
TimeoutSecs (ms / 1000)
/// sleep for a certain number of milliseconds
let sleep (TimeoutMs ms) =
System.Threading.Thread.Sleep ms
/// timeout after a certain number of seconds
let commandTimeout (TimeoutSecs s) (cmd:System.Data.IDbCommand) =
cmd.CommandTimeout <- s

And here's the same thing using units of measure:

[<Measure>] type sec
[<Measure>] type ms
let toMs (secs:int<sec>) =
secs * 1000<ms/sec>
let toSecs (ms:int<ms>) =
ms / 1000<ms/sec>
/// sleep for a certain number of milliseconds
let sleep (ms:int<ms>) =
System.Threading.Thread.Sleep (ms * 1<_>)
/// timeout after a certain number of seconds
let commandTimeout (s:int<sec>) (cmd:System.Data.IDbCommand) =
cmd.CommandTimeout <- (s * 1<_>)

Which approach is better?

If you are doing lots of arithmetic on them (adding, multiplying, etc) then the units of
measure approach is much more convenient, but otherwise there is not much to choose
between them.

929

Designing with types: Conclusion

In this series, we've looked at some of the ways we can use types as part of the design
process, including:
Breaking large structures down into small "atomic" components.
Using single case unions to add semantic meaning and validation to key domain types
such EmailAddress and ZipCode .
Ensuring that the type system can only represent valid data ("making illegal states
unrepresentable").
Using types as an analysis tool to uncover hidden requirements
Replacing flags and enums with simple state machines
Replacing primitive strings with types that guarantee various constraints
For this final post, let's see them all applied together.

The "before" code

Here's the original example we started off with in the first post in the series:
type Contact =
{
FirstName: string;
MiddleInitial: string;
LastName: string;
EmailAddress: string;
//true if ownership of email address is confirmed
IsEmailVerified: bool;
Address1: string;
Address2: string;
City: string;
State: string;
Zip: string;
//true if validated against address service
IsAddressValid: bool;
}

And how does that compare to the final result after applying all the techniques above?

930

Designing with types: Conclusion

The "after" code

First, let's start with the types that are not application specific. These types could probably
be reused in many applications.
// ========================================
// WrappedString
// ========================================
/// Common code for wrapped strings
module WrappedString =
/// An interface that all wrapped strings support
type IWrappedString =
abstract Value : string
/// Create a wrapped value option
/// 1) canonicalize the input first
/// 2) If the validation succeeds, return Some of the given constructor
/// 3) If the validation fails, return None
/// Null values are never valid.
let create canonicalize isValid ctor (s:string) =
if s = null
then None
else
let s' = canonicalize s
if isValid s'
then Some (ctor s')
else None
/// Apply the given function to the wrapped value
let apply f (s:IWrappedString) =
s.Value |> f
/// Get the wrapped value
let value s = apply id s
/// Equality
let equals left right =
(value left) = (value right)
/// Comparison
let compareTo left right =
(value left).CompareTo (value right)
/// Canonicalizes a string before construction
/// * converts all whitespace to a space char
/// * trims both ends
let singleLineTrimmed s =
System.Text.RegularExpressions.Regex.Replace(s,"\s"," ").Trim()

931

Designing with types: Conclusion

/// A validation function based on length

let lengthValidator len (s:string) =
s.Length <= len
/// A string of length 100
type String100 = String100 of string with
interface IWrappedString with
member this.Value = let (String100 s) = this in s
/// A constructor for strings of length 100
let string100 = create singleLineTrimmed (lengthValidator 100) String100
/// Converts a wrapped string to a string of length 100
let convertTo100 s = apply string100 s
/// A string of length 50
type String50 = String50 of string with
interface IWrappedString with
member this.Value = let (String50 s) = this in s
/// A constructor for strings of length 50
let string50 = create singleLineTrimmed (lengthValidator 50) String50
/// Converts a wrapped string to a string of length 50
let convertTo50 s = apply string50 s
/// map helpers
let mapAdd k v map =
Map.add (value k) v map
let mapContainsKey k map =
Map.containsKey (value k) map
let mapTryFind k map =
Map.tryFind (value k) map
// ========================================
// Email address (not application specific)
// ========================================
module EmailAddress =
type T = EmailAddress of string with
interface WrappedString.IWrappedString with
member this.Value = let (EmailAddress s) = this in s
let create =
let canonicalize = WrappedString.singleLineTrimmed
let isValid s =
(WrappedString.lengthValidator 100 s) &&
System.Text.RegularExpressions.Regex.IsMatch(s,@"^\S+@\S+\.\S+$")
WrappedString.create canonicalize isValid EmailAddress

932

Designing with types: Conclusion

/// Converts any wrapped string to an EmailAddress

let convert s = WrappedString.apply create s
// ========================================
// ZipCode (not application specific)
// ========================================
module ZipCode =
type T = ZipCode of string with
interface WrappedString.IWrappedString with
member this.Value = let (ZipCode s) = this in s
let create =
let canonicalize = WrappedString.singleLineTrimmed
let isValid s =
System.Text.RegularExpressions.Regex.IsMatch(s,@"^\d{5}$")
WrappedString.create canonicalize isValid ZipCode
/// Converts any wrapped string to a ZipCode
let convert s = WrappedString.apply create s
// ========================================
// StateCode (not application specific)
// ========================================
module StateCode =
type T = StateCode of string with
interface WrappedString.IWrappedString with
member this.Value = let (StateCode s) = this in s
let create =
let canonicalize = WrappedString.singleLineTrimmed
let stateCodes = ["AZ";"CA";"NY"] //etc
let isValid s =
stateCodes |> List.exists ((=) s)
WrappedString.create canonicalize isValid StateCode
/// Converts any wrapped string to a StateCode
let convert s = WrappedString.apply create s
// ========================================
// PostalAddress (not application specific)
// ========================================
module PostalAddress =
type USPostalAddress =
{
Address1: WrappedString.String50;
Address2: WrappedString.String50;

933

Designing with types: Conclusion

City: WrappedString.String50;
State: StateCode.T;
Zip: ZipCode.T;
}
type UKPostalAddress =
{
Address1: WrappedString.String50;
Address2: WrappedString.String50;
Town: WrappedString.String50;
PostCode: WrappedString.String50; // todo
}
type GenericPostalAddress =
{
Address1: WrappedString.String50;
Address2: WrappedString.String50;
Address3: WrappedString.String50;
Address4: WrappedString.String50;
Address5: WrappedString.String50;
}
type T =
| USPostalAddress of USPostalAddress
| UKPostalAddress of UKPostalAddress
| GenericPostalAddress of GenericPostalAddress
// ========================================
// PersonalName (not application specific)
// ========================================
module PersonalName =
open WrappedString
type T =
{
FirstName: String50;
MiddleName: String50 option;
LastName: String100;
}
/// create a new value
let create first middle last =
match (string50 first),(string100 last) with
| Some f, Some l ->
Some {
FirstName = f;
MiddleName = (string50 middle)
LastName = l;
}
| _ ->
None

934

Designing with types: Conclusion

/// concat the names together

/// and return a raw string
let fullNameRaw personalName =
let f = personalName.FirstName |> value
let l = personalName.LastName |> value
let names =
match personalName.MiddleName with
| None -> [| f; l |]
| Some middle -> [| f; (value middle); l |]
System.String.Join(" ", names)
/// concat the names together
/// and return None if too long
let fullNameOption personalName =
personalName |> fullNameRaw |> string100
/// concat the names together
/// and truncate if too long
let fullNameTruncated personalName =
// helper function
let left n (s:string) =
if (s.Length > n)
then s.Substring(0,n)
else s
personalName
|> fullNameRaw // concat
|> left 100 // truncate
|> string100 // wrap
|> Option.get // this will always be ok

And now the application specific types.

// ========================================
// EmailContactInfo -- state machine
// ========================================
module EmailContactInfo =
open System
// UnverifiedData = just the EmailAddress
type UnverifiedData = EmailAddress.T
// VerifiedData = EmailAddress plus the time it was verified
type VerifiedData = EmailAddress.T * DateTime
// set of states
type T =
| UnverifiedState of UnverifiedData
| VerifiedState of VerifiedData

935

Designing with types: Conclusion

let create email =

// unverified on creation
UnverifiedState email
// handle the "verified" event
let verified emailContactInfo dateVerified =
match emailContactInfo with
| UnverifiedState email ->
// construct a new info in the verified state
VerifiedState (email, dateVerified)
| VerifiedState _ ->
// ignore
emailContactInfo
let sendVerificationEmail emailContactInfo =
match emailContactInfo with
| UnverifiedState email ->
// send email
printfn "sending email"
| VerifiedState _ ->
// do nothing
()
let sendPasswordReset emailContactInfo =
match emailContactInfo with
| UnverifiedState email ->
// ignore
()
| VerifiedState _ ->
// ignore
printfn "sending password reset"
// ========================================
// PostalContactInfo -- state machine
// ========================================
module PostalContactInfo =
open System
// InvalidData = just the PostalAddress
type InvalidData = PostalAddress.T
// ValidData = PostalAddress plus the time it was verified
type ValidData = PostalAddress.T * DateTime
// set of states
type T =
| InvalidState of InvalidData
| ValidState of ValidData
let create address =
// invalid on creation

936

Designing with types: Conclusion

InvalidState address
// handle the "validated" event
let validated postalContactInfo dateValidated =
match postalContactInfo with
| InvalidState address ->
// construct a new info in the valid state
ValidState (address, dateValidated)
| ValidState _ ->
// ignore
postalContactInfo
let contactValidationService postalContactInfo =
let dateIsTooLongAgo (d:DateTime) =
d < DateTime.Today.AddYears(-1)
match postalContactInfo with
| InvalidState address ->
printfn "contacting the address validation service"
| ValidState (address,date) when date |> dateIsTooLongAgo ->
printfn "last checked a long time ago."
printfn "contacting the address validation service again"
| ValidState _ ->
printfn "recently checked. Doing nothing."
// ========================================
// ContactMethod and Contact
// ========================================
type ContactMethod =
| Email of EmailContactInfo.T
| PostalAddress of PostalContactInfo.T
type Contact =
{
Name: PersonalName.T;
PrimaryContactMethod: ContactMethod;
SecondaryContactMethods: ContactMethod list;
}

Conclusion
Phew! The new code is much, much longer than the original code. Granted, it has a lot of
supporting functions that were not needed in the original version, but even so it seems like a
lot of extra work. So was it worth it?
I think the answer is yes. Here are some of the reasons why:
The new code is more explicit

937

Designing with types: Conclusion

If we look at the original example, there was no atomicity between fields, no validation rules,
no length constraints, nothing to stop you updating flags in the wrong order, and so on.
The data structure was "dumb" and all the business rules were implicit in the application
code. Chances are that the application would have lots of subtle bugs that might not even
show up in unit tests. (Are you sure the application reset the IsEmailVerified flag to false in
every place the email address was updated?)
On the other hand, the new code is extremely explicit about every little detail. If I stripped
away everything but the types themselves, you would have a very good idea of what the
business rules and domain constraints were.
The new code won't let you postpone error handling
Writing code that works with the new types means that you are forced to handle every
possible thing that could go wrong, from dealing with a name that is too long, to failing to
supply a contact method. And you have to do this up front at construction time. You can't
postpone it till later.
Writing such error handling code can be annoying and tedious, but on the other hand, it
pretty much writes itself. There is really only one way to write code that actually compiles
with these types.
The new code is more likely to be correct
The huge benefit of the new code is that it is probably bug free. Without even writing any unit
tests, I can be quite confident that a first name will never be truncated when written to a
varchar(50) in a database, and that I can never accidentally send out a verification email

twice.
And in terms of the code itself, many of the things that you as a developer have to remember
to deal with (or forget to deal with) are completely absent. No null checks, no casting, no
worrying about what the default should be in a switch statement. And if you like to use
cyclomatic complexity as a code quality metric, you might note that there are only three if
statements in the entire 350 odd lines.
A word of warning...
Finally, beware! Getting comfortable with this style of type-based design will have an
insidious effect on you. You will start to develop paranoia whenever you see code that isn't
typed strictly enough. (How long should an email address be, exactly?) and you will be
unable to write the simplest python script without getting anxious. When this happens, you
will have been fully inducted into the cult. Welcome!

938

Designing with types: Conclusion

If you liked this series, here is a slide deck that covers many of the same topics. There is a
video as well (here)

Domain Driven Design with the F# type System -- F#unctional Londoners 2014 from
my slideshare page

939

Algebraic type sizes and domain modelling

Algebraic type sizes and domain

modelling
In this post, we'll look at how to calculate the "size", or cardinality, of an algebraic type, and
see how this knowledge can help us with design decisions.

Getting started
I'm going to define the "size" of a type by thinking of it as a set, and counting the number of
possible elements.
For example, there are two possible booleans, so the size of the Boolean type is two.
Is there a type with size one? Yes -- the unit type only has one value: () .
Is there a type with size zero? That is, is there a type that has no values at all? Not in F#, but
in Haskell there is. It is called Void .
What about a type like this:
type ThreeState =
| Checked
| Unchecked
| Unknown

What is its size? There are three possible values, so the size is three.
What about a type like this:
type Direction =
| North
| East
| South
| West

Obviously, four.
I think you get the idea!

Calculating the size of compound types

940

Algebraic type sizes and domain modelling

Let's look at calculating the sizes of compound types now. If you remember from the
understanding F# types series, there are two kinds of algebraic types: "product" types such
as tuples and records, and "sum" types, called discriminated unions in F#.
For example, let's say that we have a Speed as well as a Direction , and we combine them
into a record type called Velocity :
type Speed =
| Slow
| Fast
type Velocity = {
direction: Direction
speed: Speed
}

What is the size of Velocity ?

Here's every possible value:
{direction=North; speed=Slow}; {direction=North; speed=Fast}
{direction=East; speed=Slow}; {direction=East; speed=Fast}
{direction=South; speed=Slow}; {direction=South; speed=Fast}
{direction=West; speed=Slow}; {direction=West; speed=Fast}

There are eight possible values, one for every possible combination of the two Speed
values and the four Direction values.
We can generalize this into a rule:
RULE: The size of a product type is the product of the sizes of the component
types.
That is, given a record type like this:
type RecordType = {
a : TypeA
b : TypeB }

The size is calculated like this:

size(RecordType) = size(TypeA) * size(TypeB)

And similarly for a tuple:

941

Algebraic type sizes and domain modelling

type TupleType = TypeA * TypeB

The size is:

size(TupleType) = size(TypeA) * size(TypeB)

Sum types
Sum types can be analyzed the same way. Given a type Movement defined like this:
type Movement =
| Moving of Direction
| NotMoving

We can write out and count all the possibilities:

Moving North
Moving East
Moving South
Moving West
NotMoving

Again, we can write out and count all the possibilities:

942

Algebraic type sizes and domain modelling

YouSay Yes
ISay No
YouSay Stop
ISay GoGoGo
YouSay Goodbye
ISay Hello

There are three possible values in the YouSay case, and three possible values in the ISay
case, making six in all.
Again, we can make a general rule.
RULE: The size of a sum or union type is the sum of the sizes of the component
types.
That is, given a union type like this:
type SumType =
| CaseA of TypeA
| CaseB of TypeB

The size is calculated like this:

size(SumType) = size(TypeA) + size(TypeB)

Working with generic types

What happens if we throw generic types into the mix?
For example, what is the size of a type like this:
type Optional<'a> =
| Something of 'a
| Nothing

Well, the first thing to say is that Optional<'a> is not a type but a type constructor.
Optional<string> is a type. Optional<int> is a type, but Optional<'a> isn't.

Nevertheless, we can still calculate its size by noting that size(Optional<string>) is just
size(string) + 1 , size(Optional<int>) is just size(int) + 1 , and so on.

So we can say:

943

Algebraic type sizes and domain modelling

size(Optional<'a>) = size('a) + 1

Similarly, for a type with two generics like this:

type Either<'a,'b> =
| Left of 'a
| Right of 'b

we can say that its size can be calculated using the size of the generic components (using
the "sum rule" above):
size(Either<'a,'b>) = size('a) + size('b)

Recursive types
What about a recursive type? Let's look at the simplest one, a linked list.
A linked list is either empty, or it has a cell with a tuple: a head and a tail. The head is an 'a
and the tail is another list. Here's the definition:
type LinkedList<'a> =
| Empty
| Node of head:'a * tail:LinkedList<'a>

To calculate the size, let's assign some names to the various components:
let S = size(LinkedList<'a>)
let N = size('a)

Now we can write:

S =
1 // Size of "Empty" case
+ // Union type
N * S // Size of "Cell" case using tuple size calculation

Let's play with this formula a bit. We start with:

S = 1 + (N * S)

944

Algebraic type sizes and domain modelling

and let's substitute the last S with the formula to get:

S = 1 + (N * (1 + (N * S)))

If we clean this up, we get:

S = 1 + N + (N^2 * S)

(where N^2 means "N squared")

Let's substitute the last S with the formula again:
S = 1 + N + (N^2 * (1 + (N * S)))

and clean up again:

S = 1 + N + N^2 + (N^3 * S)

You can see where this is going! The formula for S can be expanded out indefinitely to be:
S = 1 + N + N^2 + N^3 + N^4 + N^5 + ...

How can we interpret this? Well, we can say that a list is a union of the following cases:
an empty list(size = 1)
a one element list (size = N)
a two element list (size = N x N)
a three element list (size = N x N x N)
and so on.
And this formula has captured that.
As an aside, you can calculate S directly using the formula S = 1/(1-N) , which means
that a list of Direction (size=4) has size "-1/3". Hmmm, that's strange! It reminds me of this
"-1/12" video.

Calculating the size of functions

What about functions? Can they be sized?
Yes, all we need to do is write down every possible implementation and count them. Easy!

945

Algebraic type sizes and domain modelling

For example, say that we have a function SuitColor that maps a card Suit to a Color ,
red or black.
type Suit = Heart | Spade | Diamond | Club
type Color = Red | Black
type SuitColor = Suit -> Color

One implementation would be to return red, no matter what suit was provided:
(Heart -> Red); (Spade -> Red); (Diamond -> Red); (Club -> Red)

Another implementation would be to return red for all suits except Club :
(Heart -> Red); (Spade -> Red); (Diamond -> Red); (Club -> Black)

In fact we can write down all 16 possible implementations of this function:

(Heart -> Red); (Spade -> Red); (Diamond -> Red); (Club -> Red)
(Heart -> Red); (Spade -> Red); (Diamond -> Red); (Club -> Black)
(Heart -> Red); (Spade -> Red); (Diamond -> Black); (Club -> Red)
(Heart -> Red); (Spade -> Red); (Diamond -> Black); (Club -> Black)
(Heart -> Red); (Spade -> Black); (Diamond -> Red); (Club -> Red)
(Heart -> Red); (Spade -> Black); (Diamond -> Red); (Club -> Black) // the right one!
(Heart -> Red); (Spade -> Black); (Diamond -> Black); (Club -> Red)
(Heart -> Red); (Spade -> Black); (Diamond -> Black); (Club -> Black)
(Heart -> Black); (Spade -> Red); (Diamond -> Red); (Club -> Red)
(Heart -> Black); (Spade -> Red); (Diamond -> Red); (Club -> Black)
(Heart -> Black); (Spade -> Red); (Diamond -> Black); (Club -> Red)
(Heart -> Black); (Spade -> Red); (Diamond -> Black); (Club -> Black)
(Heart -> Black); (Spade -> Black); (Diamond -> Red); (Club -> Red)
(Heart -> Black); (Spade -> Black); (Diamond -> Red); (Club -> Black)
(Heart -> Black); (Spade -> Black); (Diamond -> Black); (Club -> Red)
(Heart -> Black); (Spade -> Black); (Diamond -> Black); (Club -> Black)

Another way to think of it is that we can define a record type where each value represents a
particular implementation: which color do we return for a Heart input, which color do we
return for a Spade input, and so on.
The type definition for the implementations of SuitColor would therefore look like this:

946

Algebraic type sizes and domain modelling

type SuitColorImplementation = {
Heart : Color
Spade : Color
Diamond : Color
Club : Color }

What is the size of this record type?

size(SuitColorImplementation) = size(Color) * size(Color) * size(Color) * size(Color)

There are four size(Color) here. In other words, there is one size(Color) for every input,
so we could write this as:
size(SuitColorImplementation) = size(Color) to the power of size(Suit)

In general, then, given a function type:

type Function<'input,'output> = 'input -> 'output

The size of the function is size(output type) to the power of size(input type) :
size(Function) = size(output) ^ size(input)

Lets codify that into a rule too:

RULE: The size of a function type is size(output type) to the power of size(input
type) .

Converting between types

All right, that is all very interesting, but is it useful?
Yes, I think it is. I think that understanding sizes of types like this helps us design
conversions from one type to another, which is something we do a lot of!
Let's say that we have a union type and a record type, both representing a yes/no answer:

947

Algebraic type sizes and domain modelling

type YesNoUnion =
| Yes
| No
type YesNoRecord = {
isYes: bool }

How can we map between them?

They both have size=2, so we should be able to map each value in one type to the other,
and vice versa:
let toUnion yesNoRecord =
if yesNoRecord.isYes then
Yes
else
No
let toRecord yesNoUnion =
match yesNoUnion with
| Yes -> {isYes = true}
| No -> {isYes = false}

This is what you might call a "lossless" conversion. If you round-trip the conversion, you can
recover the original value. Mathematicians would call this an isomorphism (from the Greek
"equal shape").
What about another example? Here's a type with three cases, yes, no, and maybe.
type YesNoMaybe =
| Yes
| No
| Maybe

Can we losslessly convert this to this type?

type YesNoOption = { maybeIsYes: bool option }

Well, what is the size of an option ? One plus the size of the inner type, which in this case
is a bool . So size(YesNoOption) is also three.
Here are the conversion functions:

948

Algebraic type sizes and domain modelling

let toYesNoMaybe yesNoOption =

match yesNoOption.maybeIsYes with
| None -> Maybe
| Some b -> if b then Yes else No
let toYesNoOption yesNoMaybe =
match yesNoMaybe with
| Yes -> {maybeIsYes = Some true}
| No -> {maybeIsYes = Some false}
| Maybe -> {maybeIsYes = None}

So we can make a rule:

RULE: If two types have the same size, you can create a pair of lossless
conversion functions
Let's try it out. Here's a Nibble type and a TwoNibbles type:
type Nibble = {
bit1: bool
bit2: bool
bit3: bool
bit4: bool }
type TwoNibbles = {
high: Nibble
low: Nibble }

Can we convert TwoNibbles to a byte and back?

The size of Nibble is 2 x 2 x 2 x 2 = 16 (using the product size rule), and the size of
TwoNibbles is size(Nibble) x size(Nibble), or 16 x 16 , which is 256.

So yes, we can convert from TwoNibbles to a byte and back.

Lossy conversions
What happens if the types are different sizes?
If the target type is "larger" than the source type, then you can always map without loss, but
if the target type is "smaller" than the source type, you have a problem.
For example, the int type is smaller than the string type. You can convert an int to a
string accurately, but you can't convert a string to an int easily.

949

Algebraic type sizes and domain modelling

If you do want to map a string to an int, then some of the non-integer strings will have to be
mapped to a special, non-integer value in the target type:

In other words we know from the sizes that the target type can't just be an int type, it must
be an int + 1 type. In other words, an Option type!
Interestingly, the Int32.TryParse function in the BCL returns two values, a success/failure
bool and the parsed result as an int . In other words, a tuple bool * int .

The size of that tuple is 2 x int , many more values that are really needed. Option types
ftw!
Now let's say we are converting from a string to a Direction . Some strings are valid, but
most of them are not. But this time, instead of having one invalid case, let's also say that we
want to distinguish between empty inputs, inputs that are too long, and other invalid inputs.

We can't model the target with an Option any more, so let's design a custom type that
contains all seven cases:

950

Algebraic type sizes and domain modelling

What is the size of StringToDirection_V2 ?

There are 4 choices of Direction in the Success case, and three choices of
ConversionFailure in the Failure case, so the total size is seven, just as in the first

version.
In other words, both of these designs are equivalent and we can use either one.
Personally, I prefer version 2, but if we had version 1 in our legacy code, the good news is
that we can losslessly convert from version 1 to version 2 and back again. Which in turn
means that we can safely refactor to version 2 if we need to.

Designing the core domain

Knowing that different types can be losslessly converted allows you to tweak your domain
designs as needed.
For example, this type:

951

Algebraic type sizes and domain modelling

type Something_V1 =
| CaseA1 of TypeX * TypeY
| CaseA2 of TypeX * TypeZ

can be losslessly converted to this one:

type Inner =
| CaseA1 of TypeY
| CaseA2 of TypeZ
type Something_V2 =
TypeX * Inner

or this one:
type Something_V3 = {
x: TypeX
inner: Inner }

Here's a real example:

You have a website where some users are registered and some are not.
For all users, you have a session id
For registered users only, you have extra information
We could model that requirement like this:
module Customer_V1 =
type UserInfo = {name:string} //etc
type SessionId = SessionId of int
type WebsiteUser =
| RegisteredUser of SessionId * UserInfo
| GuestUser of SessionId

or alternatively, we can pull the common SessionId up to a higher level like this:

952

Algebraic type sizes and domain modelling

module Customer_V2 =
type UserInfo = {name:string} //etc
type SessionId = SessionId of int
type WebsiteUserInfo =
| RegisteredUser of UserInfo
| GuestUser
type WebsiteUser = {
sessionId : SessionId
info: WebsiteUserInfo }

Which is better? In one sense, they are both the "same", but obviously the best design
depends on the usage pattern.
If you care more about the type of user than the session id, then version 1 is better.
If you are constantly looking at the session id without caring about the type of user, then
version 2 is better.
The nice thing about knowing that they are isomorphic is that you can define both types if
you like, use them in different contexts, and losslessly map between them as needed.

Interfacing with the outside world

We have all these nice domain types like Direction or WebsiteUser but at some point we
need to interface with the outside world -- store them in a database, receive them as JSON,
etc.
The problem is that the outside world does not have a nice type system! Everything tends to
be primitives: strings, ints and bools.
Going from our domain to the outside world means going from types with a "small" set of
values to types with a "large" set of values, which we can do straightforwardly. But coming in
from the outside world into our domain means going from a "large" set of values to a "small"
set of values, which requires validation and error cases.
For example, a domain type might look like this:
type DomainCustomer = {
Name: String50
Email: EmailAddress
Age: PositiveIntegerLessThan130 }

953

Algebraic type sizes and domain modelling

The values are constrained: max 50 chars for the name, a validated email, an age which is
between 1 and 129.
On the other hand, the DTO type might look like this:
type CustomerDTO = {
Name: string
Email: string
Age: int }

The values are unconstrained: any string for the name, a unvalidated email, an age that can
be any of 2^32 different values, including negative ones.
This means that we cannot create a CustomerDTO to DomainCustomer mapping. We have to
have at least one other value ( DomainCustomer + 1 ) to map the invalid inputs onto, and
preferably more to document the various errors.
This leads naturally to the Success/Failure model as described in my functional error
handling talk,
The final version of the mapping would then be from a CustomerDTO to a
SuccessFailure<DomainCustomer> or similar.

So that leads to the final rule:

RULE: Trust no one. If you import data from an external source, be sure to handle
invalid input.
If we take this rule seriously, it has some knock on effects, such as:
Never try to deserialize directly to a domain type (e.g. no ORMs), only to DTO types.
Always validate every record you read from a database or other "trusted" source.
You might think that having everything wrapped in a Success/Failure type can get
annoying, and this is true (!), but there are ways to make this easier. See this post for
example.

Further reading
The "algebra" of algebraic data types is well known. There is a good recent summary in "The
algebra (and calculus!) of algebraic data types" and a series by Chris Taylor.
And after I wrote this, I was pointed to two similar posts:
One by Tomas Petricek with almost the same content!

954

Algebraic type sizes and domain modelling

One by Bartosz Milewski in his series on category theory.

As some of those posts mention, you can do strange things with these type formulas, such
as differentiate them!
If you like academic papers, you can read the original discussion of derivatives in "The
Derivative of a Regular Type is its Type of One-Hole Contexts"(PDF) by Conor McBride from
2001, and a follow up in "Differentiating Data Structures"(PDF) [Abbott, Altenkirch, Ghani,
and McBride, 2005].

Summary
This might not be the most exciting topic in the world, but I've found this approach both
interesting and useful, and I wanted to share it with you.
Let me know what you think. Thanks for reading!

955

Thirteen ways of looking at a turtle

This post is part of the F# Advent Calendar in English 2015 project. Check out all the
other great posts there! And special thanks to Sergey Tihon for organizing this.
I was discussing how to implement a simple turtle graphics system some time ago, and it
struck me that, because the turtle requirements are so simple and so well known, it would
make a great basis for demonstrating a range of different techniques.
So, in this two part mega-post, I'll stretch the turtle model to the limit while demonstrating
things like: partial application, validation with Success/Failure results, the concept of "lifting",
agents with message queues, dependency injection, the State monad, event sourcing,
stream processing, and finally a custom interpreter!
Without further ado then, I hereby present thirteen different ways of implementing a turtle:
Way 1. A basic object-oriented approach, in which we create a class with mutable state.
Way 2. A basic functional approach, in which we create a module of functions with
immutable state.
Way 3. An API with a object-oriented core, in which we create an object-oriented API
that calls a stateful core class.
Way 4. An API with a functional core, in which we create an stateful API that uses
stateless core functions.
Way 5. An API in front of an agent, in which we create an API that uses a message
queue to communicate with an agent.
Way 6. Dependency injection using interfaces, in which we decouple the
implementation from the API using an interface or record of functions.
Way 7. Dependency injection using functions, in which we decouple the implementation
from the API by passing a function parameter.
Way 8. Batch processing using a state monad, in which we create a special "turtle
workflow" computation expression to track state for us.
Way 9. Batch processing using command objects, in which we create a type to
represent a turtle command, and then process a list of commands all at once.
Interlude: Conscious decoupling with data types. A few notes on using data vs.
interfaces for decoupling.
Way 10. Event sourcing, in which state is built from a list of past events.
Way 11. Functional Retroactive Programming (stream processing), in which business
logic is based on reacting to earlier events.
Episode V: The Turtle Strikes Back, in which the turtle API changes so that some
commands may fail.

956

Thirteen ways of looking at a turtle

Way 12. Monadic control flow, in which we make decisions in the turtle workflow based
on results from earlier commands.
Way 13. A turtle interpreter, in which we completely decouple turtle programming from
turtle implementation, and nearly encounter the free monad.
Review of all the techniques used.
and 2 bonus ways for the extended edition:
Way 14. Abstract Data Turtle, in which we encapsulate the details of a turtle
implementation by using an Abstract Data Type.
Way 15. Capability-based Turtle, in which we control what turtle functions are available
to a client, based on the current state of the turtle.
All source code for this post is available on github.

The requirements for a Turtle

A turtle supports four instructions:
Move some distance in the current direction.
Turn a certain number of degrees clockwise or anticlockwise.
Put the pen down or up. When the pen is down, moving the turtle draws a line.
Set the pen color (one of black, blue or red).
These requirements lead naturally to some kind of "turtle interface" like this:
Move aDistance
Turn anAngle
PenUp
PenDown
SetColor aColor

All of the following implementations will be based on this interface or some variant of it.
Note that the turtle must convert these instructions to drawing lines on a canvas or other
graphics context. So the implementation will probably need to keep track of the turtle
position and current state somehow.

Common code

957

Thirteen ways of looking at a turtle

Before we start implementing, let's get some common code out of the way.
First, we'll need some types to represent distances, angles, the pen state, and the pen
colors.
/// An alias for a float
type Distance = float
/// Use a unit of measure to make it clear that the angle is in degrees, not radians
type [<Measure>] Degrees
/// An alias for a float of Degrees
type Angle = float<Degrees>
/// Enumeration of available pen states
type PenState = Up | Down
/// Enumeration of available pen colors
type PenColor = Black | Red | Blue

and we'll also need a type to represent the position of the turtle:
/// A structure to store the (x,y) coordinates
type Position = {x:float; y:float}

We'll also need a helper function to calculate a new position based on moving a certain
distance at a certain angle:
// round a float to two places to make it easier to read
let round2 (flt:float) = Math.Round(flt,2)
/// calculate a new position from the current position given an angle and a distance
let calcNewPosition (distance:Distance) (angle:Angle) currentPos =
// Convert degrees to radians with 180.0 degrees = 1 pi radian
let angleInRads = angle * (Math.PI/180.0) * 1.0<1/Degrees>
// current pos
let x0 = currentPos.x
let y0 = currentPos.y
// new pos
let x1 = x0 + (distance * cos angleInRads)
let y1 = y0 + (distance * sin angleInRads)
// return a new Position
{x=round2 x1; y=round2 y1}

Let's also define the initial state of a turtle:

958

Thirteen ways of looking at a turtle

/// Default initial state

let initialPosition,initialColor,initialPenState =
{x=0.0; y=0.0}, Black, Down

And a helper that pretends to draw a line on a canvas:

let dummyDrawLine log oldPos newPos color =
// for now just log it
log (sprintf "...Draw line from (%0.1f,%0.1f) to (%0.1f,%0.1f) using %A" oldPos.x
oldPos.y newPos.x newPos.y color)

Now we're ready for the first implementation!

1. Basic OO -- A class with mutable state

In this first design, we will use an object-oriented approach and represent the turtle with a
simple class.
The state will be stored in local fields ( currentPosition , currentAngle , etc) that are
mutable.
We will inject a logging function log so that we can monitor what happens.

And here's the complete code, which should be self-explanatory:

959

Thirteen ways of looking at a turtle

type Turtle(log) =
let mutable currentPosition = initialPosition
let mutable currentAngle = 0.0<Degrees>
let mutable currentColor = initialColor
let mutable currentPenState = initialPenState
member this.Move(distance) =
log (sprintf "Move %0.1f" distance)
// calculate new position
let newPosition = calcNewPosition distance currentAngle currentPosition
// draw line if needed
if currentPenState = Down then
dummyDrawLine log currentPosition newPosition currentColor
// update the state
currentPosition <- newPosition
member this.Turn(angle) =
log (sprintf "Turn %0.1f" angle)
// calculate new angle
let newAngle = (currentAngle + angle) % 360.0<Degrees>
// update the state
currentAngle <- newAngle
member this.PenUp() =
log "Pen up"
currentPenState <- Up
member this.PenDown() =
log "Pen down"
currentPenState <- Down
member this.SetColor(color) =
log (sprintf "SetColor %A" color)
currentColor <- color

Calling the turtle object

The client code instantiates the turtle and talks to it directly:

960

Thirteen ways of looking at a turtle

/// Function to log a message

let log message =
printfn "%s" message
let drawTriangle() =
let turtle = Turtle(log)
turtle.Move 100.0
turtle.Turn 120.0<Degrees>
turtle.Move 100.0
turtle.Turn 120.0<Degrees>
turtle.Move 100.0
turtle.Turn 120.0<Degrees>
// back home at (0,0) with angle 0

The logged output of drawTriangle() is:

Move 100.0
...Draw line from (0.0,0.0) to (100.0,0.0) using Black
Turn 120.0
Move 100.0
...Draw line from (100.0,0.0) to (50.0,86.6) using Black
Turn 120.0
Move 100.0
...Draw line from (50.0,86.6) to (0.0,0.0) using Black
Turn 120.0

Similarly, here's the code to draw a polygon:

let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let angleDegrees = angle * 1.0<Degrees>
let turtle = Turtle(log)
// define a function that draws one side
let drawOneSide() =
turtle.Move 100.0
turtle.Turn angleDegrees
// repeat for all sides
for i in [1..n] do
drawOneSide()

Note that drawOneSide() does not return anything -- all the code is imperative and stateful.
Compare this to the code in the next example, which takes a pure functional approach.

Advantages and disadvantages

961

Thirteen ways of looking at a turtle

So what are the advantages and disadvantages of this simple approach?

Advantages
It's very easy to implement and understand.
Disadvantages
The stateful code is harder to test. We have to put an object into a known state state
before testing, which is simple in this case, but can be long-winded and error-prone for
more complex objects.
The client is coupled to a particular implementation. No interfaces here! We'll look at
using interfaces shortly.
The source code for this version is available here (turtle class) and here (client).

2: Basic FP - A module of functions with

immutable state
The next design will use a pure, functional approach. An immutable TurtleState is defined,
and then the various turtle functions accept a state as input and return a new state as
output.
In this approach then, the client is responsible for keeping track of the current state and
passing it into the next function call.

Here's the definition of TurtleState and the values for the initial state:

962

Thirteen ways of looking at a turtle

module Turtle =
type TurtleState = {
position : Position
angle : float<Degrees>
color : PenColor
penState : PenState
}
let initialTurtleState = {
position = initialPosition
angle = 0.0<Degrees>
color = initialColor
penState = initialPenState
}

And here are the "api" functions, all of which take a state parameter and return a new state:
module Turtle =
// [state type snipped]
let move log distance state =
log (sprintf "Move %0.1f" distance)
// calculate new position
let newPosition = calcNewPosition distance state.angle state.position
// draw line if needed
if state.penState = Down then
dummyDrawLine log state.position newPosition state.color
// update the state
{state with position = newPosition}
let turn log angle state =
log (sprintf "Turn %0.1f" angle)
// calculate new angle
let newAngle = (state.angle + angle) % 360.0<Degrees>
// update the state
{state with angle = newAngle}
let penUp log state =
log "Pen up"
{state with penState = Up}
let penDown log state =
log "Pen down"
{state with penState = Down}
let setColor log color state =
log (sprintf "SetColor %A" color)
{state with color = color}

963

Thirteen ways of looking at a turtle

Note that the state is always the last parameter -- this makes it easier to use the "piping"
idiom.

Using the turtle functions

The client now has to pass in both the log function and the state to every function, every
time!
We can eliminate the need to pass in the log function by using partial application to create
new versions of the functions with the logger baked in:
/// Function to log a message
let log message =
printfn "%s" message
// versions with log baked in (via partial application)
let move = Turtle.move log
let turn = Turtle.turn log
let penDown = Turtle.penDown log
let penUp = Turtle.penUp log
let setColor = Turtle.setColor log

When it comes to drawing a polygon, it's a little more complicated, as we have to "fold" the
state through the repetitions for each side:

964

Thirteen ways of looking at a turtle

let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let angleDegrees = angle * 1.0<Degrees>
// define a function that draws one side
let oneSide state sideNumber =
state
|> move 100.0
|> turn angleDegrees
// repeat for all sides
[1..n]
|> List.fold oneSide Turtle.initialTurtleState

Advantages and disadvantages

What are the advantages and disadvantages of this purely functional approach?
Advantages
Again, it's very easy to implement and understand.
The stateless functions are easier to test. We always provide the current state as input,
so there is no setup needed to get an object into a known state.
Because there is no global state, the functions are modular and can be reused in other
contexts (as we'll see later in this post).
Disadvantages
As before, the client is coupled to a particular implementation.
The client has to keep track of the state (but some solutions to make this easier are
shown later in this post).
The source code for this version is available here (turtle functions) and here (client).

3: An API with a object-oriented core

Let's hide the client from the implementation using an API!
In this case, the API will be string based, with text commands such as "move 100" or "turn
90" . The API must validate these commands and turn them into method calls on the turtle

(we'll use the OO approach of a stateful Turtle class again).

965

Thirteen ways of looking at a turtle

If the command is not valid, the API must indicate that to the client. Since we are using an
OO approach, we'll do this by throwing a TurtleApiException containing a string, like this.
exception TurtleApiException of string

Next we need some functions that validate the command text:

// convert the distance parameter to a float, or throw an exception
let validateDistance distanceStr =
try
float distanceStr
with
| ex ->
let msg = sprintf "Invalid distance '%s' [%s]" distanceStr ex.Message
raise (TurtleApiException msg)
// convert the angle parameter to a float<Degrees>, or throw an exception
let validateAngle angleStr =
try
(float angleStr) * 1.0<Degrees>
with
| ex ->
let msg = sprintf "Invalid angle '%s' [%s]" angleStr ex.Message
raise (TurtleApiException msg)
// convert the color parameter to a PenColor, or throw an exception
let validateColor colorStr =
match colorStr with
| "Black" -> Black
| "Blue" -> Blue
| "Red" -> Red
| _ ->
let msg = sprintf "Color '%s' is not recognized" colorStr
raise (TurtleApiException msg)

With these in place, we can create the API.

The logic for parsing the command text is to split the command text into tokens and then
match the first token to "move" , "turn" , etc.
Here's the code:

966

Thirteen ways of looking at a turtle

type TurtleApi() =
let turtle = Turtle(log)
member this.Exec (commandStr:string) =
let tokens = commandStr.Split(' ') |> List.ofArray |> List.map trimString
match tokens with
| [ "Move"; distanceStr ] ->
let distance = validateDistance distanceStr
turtle.Move distance
| [ "Turn"; angleStr ] ->
let angle = validateAngle angleStr
turtle.Turn angle
| [ "Pen"; "Up" ] ->
turtle.PenUp()
| [ "Pen"; "Down" ] ->
turtle.PenDown()
| [ "SetColor"; colorStr ] ->
let color = validateColor colorStr
turtle.SetColor color
| _ ->
let msg = sprintf "Instruction '%s' is not recognized" commandStr
raise (TurtleApiException msg)

Using the API

Here's how drawPolygon is implemented using the TurtleApi class:
let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let api = TurtleApi()
// define a function that draws one side
let drawOneSide() =
api.Exec "Move 100.0"
api.Exec (sprintf "Turn %f" angle)
// repeat for all sides
for i in [1..n] do
drawOneSide()

You can see that the code is quite similar to the earlier OO version, with the direct call
turtle.Move 100.0 being replaced with the indirect API call api.Exec "Move 100.0" .

Now if we trigger an error with a bad command such as api.Exec "Move bad" , like this:

967

Thirteen ways of looking at a turtle

let triggerError() =
let api = TurtleApi()
api.Exec "Move bad"

then the expected exception is thrown:

Exception of type 'TurtleApiException' was thrown.

Advantages and disadvantages

What are the advantages and disadvantages of an API layer like this?
The turtle implementation is now hidden from the client.
An API at a service boundary supports validation and can be extended to support
monitoring, internal routing, load balancing, etc.
Disadvantages
The API is coupled to a particular implementation, even though the client isn't.
The system is very stateful. Even though the client does not know about the
implementation behind the API, the client is still indirectly coupled to the inner core via
shared state which in turn can make testing harder.
The source code for this version is available here.

4: An API with a functional core

An alternative approach for this scenario is to use a hybrid design, where the core of the
application consists of pure functions, while the boundaries are imperative and stateful.
This approach has been named "Functional Core/Imperative Shell" by Gary Bernhardt.
Applied to our API example, the API layer uses only pure turtle functions, but the API layer
manages the state (rather than the client) by storing a mutable turtle state.
Also, to be more functional, the API will not throw exceptions if the command text is not
valid, but instead will return a Result value with Success and Failure cases, where the
Failure case is used for any errors. (See my talk on the functional approach to error

handling for a more in depth discussion of this technique).

968

Thirteen ways of looking at a turtle

Let's start by implementing the API class. This time it contains a mutable turtle state:
type TurtleApi() =
let mutable state = initialTurtleState
/// Update the mutable state value
let updateState newState =
state <- newState

The validation functions no longer throw an exception, but return Success or Failure :
let validateDistance distanceStr =
try
Success (float distanceStr)
with
| ex ->
Failure (InvalidDistance distanceStr)

The error cases are documented in their own type:

type ErrorMessage =
| InvalidDistance of string
| InvalidAngle of string
| InvalidColor of string
| InvalidCommand of string

Now because the validation functions now return a Result<Distance> rather than a "raw"
distance, the move function needs to be lifted to the world of Results , as does the current
state.
There are three functions that we will use when working with Result s: returnR , mapR and
lift2R .
returnR transforms a "normal" value into a value in the world of Results:

969

Thirteen ways of looking at a turtle

mapR transforms a "normal" one-parameter function into a one-parameter function in

the world of Results:

lift2R transforms a "normal" two-parameter function into a two-parameter function in

the world of Results:

As an example, with these helper functions, we can turn the normal move function into a
function in the world of Results:
The distance parameter is already in Result world
The state parameter is lifted into Result world using returnR
The move function is lifted into Result world using lift2R

970

Thirteen ways of looking at a turtle

// lift current state to Result

let stateR = returnR state
// get the distance as a Result
let distanceR = validateDistance distanceStr
// call "move" lifted to the world of Results
lift2R move distanceR stateR

(For more details on lifting functions to Result world, see the post on "lifting" in general )
Here's the complete code for Exec :

971

Thirteen ways of looking at a turtle

/// Execute the command string, and return a Result

/// Exec : commandStr:string -> Result<unit,ErrorMessage>
member this.Exec (commandStr:string) =
let tokens = commandStr.Split(' ') |> List.ofArray |> List.map trimString
// lift current state to Result
let stateR = returnR state
// calculate the new state
let newStateR =
match tokens with
| [ "Move"; distanceStr ] ->
// get the distance as a Result
let distanceR = validateDistance distanceStr
// call "move" lifted to the world of Results
lift2R move distanceR stateR
| [ "Turn"; angleStr ] ->
let angleR = validateAngle angleStr
lift2R turn angleR stateR
| [ "Pen"; "Up" ] ->
returnR (penUp state)
| [ "Pen"; "Down" ] ->
returnR (penDown state)
| [ "SetColor"; colorStr ] ->
let colorR = validateColor colorStr
lift2R setColor colorR stateR
| _ ->
Failure (InvalidCommand commandStr)
// Lift `updateState` into the world of Results and
// call it with the new state.
mapR updateState newStateR
// Return the final result (output of updateState)

Using the API

The API returns a Result , so the client can no longer call each function in sequence, as we
need to handle any errors coming from a call and abandon the rest of the steps.
To make our lives easier, we'll use a result computation expression (or workflow) to chain
the calls and preserve the imperative "feel" of the OO version.

972

Thirteen ways of looking at a turtle

let drawTriangle() =
let api = TurtleApi()
result {
do! api.Exec "Move 100"
do! api.Exec "Turn 120"
do! api.Exec "Move 100"
do! api.Exec "Turn 120"
do! api.Exec "Move 100"
do! api.Exec "Turn 120"
}

The source code for the result computation expression is available here.
Similarly, for the drawPolygon code, we can create a helper to draw one side and then call it
n times inside a result expression.

let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let api = TurtleApi()
// define a function that draws one side
let drawOneSide() = result {
do! api.Exec "Move 100.0"
do! api.Exec (sprintf "Turn %f" angle)
}
// repeat for all sides
result {
for i in [1..n] do
do! drawOneSide()
}

The code looks imperative, but is actually purely functional, as the returned Result values
are being handled transparently by the result workflow.

Advantages and disadvantages

Advantages
The same as for the OO version of an API -- the turtle implementation is hidden from
the client, validation can be done, etc.
The only stateful part of the system is at the boundary. The core is stateless which
makes testing easier.
Disadvantages
The API is still coupled to a particular implementation.

973

Thirteen ways of looking at a turtle

The source code for this version is available here (api helper functions) and here (API and
client).

5: An API in front of an agent

In this design, an API layer communicates with a TurtleAgent via a message queue and
the client talks to the API layer as before.

There are no mutables in the API (or anywhere). The TurtleAgent manages state by
storing the current state as a parameter in the recursive message processing loop.
Now because the TurtleAgent has a typed message queue, where all messages are the
same type, we must combine all possible commands into a single discriminated union type
( TurtleCommand ).
type TurtleCommand =
| Move of Distance
| Turn of Angle
| PenUp
| PenDown
| SetColor of PenColor

The agent implementation is similar to the previous ones, but rather than exposing the turtle
functions directly, we now do pattern matching on the incoming command to decide which
function to call:

974

Thirteen ways of looking at a turtle

type TurtleAgent() =
/// Function to log a message
let log message =
printfn "%s" message
// logged versions
let move = Turtle.move log
let turn = Turtle.turn log
let penDown = Turtle.penDown log
let penUp = Turtle.penUp log
let setColor = Turtle.setColor log
let mailboxProc = MailboxProcessor.Start(fun inbox ->
let rec loop turtleState = async {
// read a command message from teh queue
let! command = inbox.Receive()
// create a new state from handling the message
let newState =
match command with
| Move distance ->
move distance turtleState
| Turn angle ->
turn angle turtleState
| PenUp ->
penUp turtleState
| PenDown ->
penDown turtleState
| SetColor color ->
setColor color turtleState
return! loop newState
}
loop Turtle.initialTurtleState )
// expose the queue externally
member this.Post(command) =
mailboxProc.Post command

Sending a command to the Agent

The API calls the agent by constructing a TurtleCommand and posting it to the agent's
queue.
This time, rather than using the previous approach of "lifting" the move command:
let stateR = returnR state
let distanceR = validateDistance distanceStr
lift2R move distanceR stateR

975

Thirteen ways of looking at a turtle

we'll use the result computation expression instead, so the code above would have
looked like this:
result {
let! distance = validateDistance distanceStr
move distance state
}

In the agent implementation, we are not calling a move command, but instead creating the
Move case of the Command type, so the code looks like:

result {
let! distance = validateDistance distanceStr
let command = Move distance
turtleAgent.Post command
}

Here's the complete code:

976

Thirteen ways of looking at a turtle

member this.Exec (commandStr:string) =

Advantages and disadvantages of the Agent approach

Advantages
A great way to protect mutable state without using locks.
The API is decoupled from a particular implementation via the message queue. The
TurtleCommand acts as a sort of protocol that decouples the two ends of the queue.

The turtle agent is naturally asynchronous.

Agents can easily be scaled horizontally.

977

Thirteen ways of looking at a turtle

Disadvantages
Agents are stateful and have the same problem as stateful objects:
It is harder to reason about your code.
Testing is harder.
It is all too easy to create a web of complex dependencies between actors.
A robust implementation for agents can get quite complex, as you may need support for
supervisors, heartbeats, back pressure, etc.
The source code for this version is available here .

6: Dependency injection using interfaces

All the implementations so far have been tied to a specific implementation of the turtle
functions, with the exception of the Agent version, where the API communicated indirectly
via a queue.
So let's now look at some ways of decoupling the API from the implementation.

Designing an interface, object-oriented style

We'll start with the classic OO way of decoupling implementations: using interfaces.
Applying that approach to the turtle domain, we can see that our API layer will need to
communicate with a ITurtle interface rather than a specific turtle implementation. The
client injects the turtle implementation later, via the API's constructor.
Here's the interface definition:
type ITurtle =
abstract Move : Distance -> unit
abstract Turn : Angle -> unit
abstract PenUp : unit -> unit
abstract PenDown : unit -> unit
abstract SetColor : PenColor -> unit

Note that there are a lot of unit s in these functions. A unit in a function signature implies
side effects, and indeed the TurtleState is not used anywhere, as this is a OO-based
approach where the mutable state is encapsulated in the object.

978

Thirteen ways of looking at a turtle

Next, we need to change the API layer to use the interface by injecting it in the constructor
for TurtleApi . Other than that, the rest of the API code is unchanged, as shown by the
snippet below:
type TurtleApi(turtle: ITurtle) =
// other code
member this.Exec (commandStr:string) =
let tokens = commandStr.Split(' ') |> List.ofArray |> List.map trimString
match tokens with
| [ "Move"; distanceStr ] ->
let distance = validateDistance distanceStr
turtle.Move distance
| [ "Turn"; angleStr ] ->
let angle = validateAngle angleStr
turtle.Turn angle
// etc

Creating some implementations of an OO interface

Now let's create and test some implementations.
The first implementation will be called normalSize and will be the original one. The second
will be called halfSize and will reduce all the distances by half.
For normalSize we could go back and retrofit the orginal Turtle class to support the
ITurtle interface. But I hate having to change working code! Instead, we can create a

"proxy" wrapper around the orginal Turtle class, where the proxy implements the new
interface.
In some languages, creating proxy wrappers can be long-winded, but in F# you can use
object expressions to implement an interface quickly:
let normalSize() =
let log = printfn "%s"
let turtle = Turtle(log)
// return an interface wrapped around the Turtle
{new ITurtle with
member this.Move dist = turtle.Move dist
member this.Turn angle = turtle.Turn angle
member this.PenUp() = turtle.PenUp()
member this.PenDown() = turtle.PenDown()
member this.SetColor color = turtle.SetColor color
}

979

Thirteen ways of looking at a turtle

And to create the halfSize version, we do the same thing, but intercept the calls to Move
and halve the distance parameter:
let halfSize() =
let normalSize = normalSize()
// return a decorated interface
{new ITurtle with
member this.Move dist = normalSize.Move (dist/2.0) // halved!!
member this.Turn angle = normalSize.Turn angle
member this.PenUp() = normalSize.PenUp()
member this.PenDown() = normalSize.PenDown()
member this.SetColor color = normalSize.SetColor color
}

This is actually the "decorator" pattern at work: we're wrapping normalSize in a proxy with
an identical interface, then changing the behavior for some of the methods, while passing
others though untouched.

Injecting dependencies, OO style

Now let's look at the client code that injects the dependencies into the API.
First, some code to draw a triangle, where a TurtleApi is passed in:
let drawTriangle(api:TurtleApi) =
api.Exec "Move 100"
api.Exec "Turn 120"
api.Exec "Move 100"
api.Exec "Turn 120"
api.Exec "Move 100"
api.Exec "Turn 120"

And now let's try drawing the triangle by instantiating the API object with the normal
interface:
let iTurtle = normalSize() // an ITurtle type
let api = TurtleApi(iTurtle)
drawTriangle(api)

Obviously, in a real system, the dependency injection would occur away from the call site,
using an IoC container or similar.
If we run it, the output of drawTriangle is just as before:

980

Thirteen ways of looking at a turtle

And now with the half-size interface..

let iTurtle = halfSize()
let api = TurtleApi(iTurtle)
drawTriangle(api)

...the output is, as we hoped, half the size!

Move 50.0
...Draw line from (0.0,0.0) to (50.0,0.0) using Black
Turn 120.0
Move 50.0
...Draw line from (50.0,0.0) to (25.0,43.3) using Black
Turn 120.0
Move 50.0
...Draw line from (25.0,43.3) to (0.0,0.0) using Black
Turn 120.0

Designing an interface, functional style

In a pure FP world, OO-style interfaces do not exist. However, you can emulate them by
using a record containing functions, with one function for each method in the interface.
So let's create a alternative version of dependency injection, where this time the API layer
will use a record of functions rather than an interface.
A record of functions is a normal record, but the types of the fields are function types. Here's
the definition we'll use:

981

Thirteen ways of looking at a turtle

type TurtleFunctions = {
move : Distance -> TurtleState -> TurtleState
turn : Angle -> TurtleState -> TurtleState
penUp : TurtleState -> TurtleState
penDown : TurtleState -> TurtleState
setColor : PenColor -> TurtleState -> TurtleState
}

Note that there are no unit s in these function signatures, unlike the OO version. Instead,
the TurtleState is explicitly passed in and returned.
Also note that there is no logging either. The logging method will be baked in to the functions
when the record is created.
The TurtleApi constructor now takes a TurtleFunctions record rather than an ITurtle ,
but as these functions are pure, the API needs to manage the state again with a mutable
field.
type TurtleApi(turtleFunctions: TurtleFunctions) =
let mutable state = initialTurtleState

The implementation of the main Exec method is very similar to what we have seen before,
with these differences:
The function is fetched from the record (e.g. turtleFunctions.move ).
All the activity takes place in a result computation expression so that the result of the
validations can be used.
Here's the code:

982

Thirteen ways of looking at a turtle

member this.Exec (commandStr:string) =

let tokens = commandStr.Split(' ') |> List.ofArray |> List.map trimString
// return Success of unit, or Failure
match tokens with
| [ "Move"; distanceStr ] -> result {
let! distance = validateDistance distanceStr
let newState = turtleFunctions.move distance state
updateState newState
}
| [ "Turn"; angleStr ] -> result {
let! angle = validateAngle angleStr
let newState = turtleFunctions.turn angle state
updateState newState
}
// etc

Creating some implementations of a "record of functions"

Noe let's create some implementations.
Again, we'll have a normalSize implementation and a halfSize implementation.
For normalSize we just need to use the functions from the original Turtle module, with the
logging baked in using partial application:
let normalSize() =
let log = printfn "%s"
// return a record of functions
{
move = Turtle.move log
turn = Turtle.turn log
penUp = Turtle.penUp log
penDown = Turtle.penDown log
setColor = Turtle.setColor log
}

And to create the halfSize version, we clone the record, and change just the move
function:
let halfSize() =
let normalSize = normalSize()
// return a reduced turtle
{ normalSize with
move = fun dist -> normalSize.move (dist/2.0)
}

983

Thirteen ways of looking at a turtle

What's nice about cloning records rather than proxying interfaces is that we don't have to
reimplement every function in the record, just the ones we care about.

Injecting dependencies again

The client code that injects the dependencies into the API is implemented just as you
expect. The API is a class with a constructor, and so the record of functions can be passed
into the constructor in exactly the same way that the ITurtle interface was:
let turtleFns = normalSize() // a TurtleFunctions type
let api = TurtleApi(turtleFns)
drawTriangle(api)

As you can see, the client code in the ITurtle version and TurtleFunctions version looks
identical! If it wasn't for the different types, you could not tell them apart.

Advantages and disadvantages of using interfaces

The OO-style interface and the FP-style "record of functions" are very similar, although the
FP functions are stateless, unlike the OO interface.
Advantages
The API is decoupled from a particular implementation via the interface.
For the FP "record of functions" approach (compared to OO interfaces):
Records of functions can be cloned more easily than interfaces.
The functions are stateless
Disadvantages
Interfaces are more monolithic than individual functions and can easily grow to include
too many unrelated methods, breaking the Interface Segregation Principle if care is not
taken.
Interfaces are not composable (unlike individual functions).
For more on the problems with this approach, see this Stack Overflow answer by Mark
Seemann.
For the OO interface approach in particular:
You may have to modify existing classes when refactoring to an interface.
For the FP "record of functions" approach:
Less tooling support, and poor interop, compared to OO interfaces.
The source code for these versions is available here (interface) and here (record of
functions).

984

Thirteen ways of looking at a turtle

7: Dependency injection using functions

The two main disadvantages of the "interface" approach is that interfaces are not
composable, and they break the "pass in only the dependencies you need" rule, which is a
key part of functional design.
In a true functional approach, we would pass in functions. That is, the API layer
communicates via one or more functions that are passed in as parameters to the API call.
These functions are typically partially applied so that the call site is decoupled from the
"injection".
No interface is passed to the constructor as generally there is no constructor! (I'm only using
a API class here to wrap the mutable turtle state.)
In the approach in this section, I'll show two alternatives which use function passing to inject
dependencies:
In the first approach, each dependency (turtle function) is passed separately.
In the second approach, only one function is passed in. So to determine which specific
turtle function is used, a discriminated union type is defined.

Approach 1 - passing in each dependency as a separate

function
The simplest way to manage dependencies is always just to pass in all dependencies as
parameters to the function that needs them.
In our case, the Exec method is the only function that needs to control the turtle, so we can
pass them in there directly:
member this.Exec move turn penUp penDown setColor (commandStr:string) =
...

To stress that point again: in this approach dependencies are always passed "just in time", to
the function that needs them. No dependencies are used in the constructor and then used
later.
Here's a bigger snippet of the Exec method using those functions:

985

Thirteen ways of looking at a turtle

member this.Exec move turn penUp penDown setColor (commandStr:string) =

...
// return Success of unit, or Failure
match tokens with
| [ "Move"; distanceStr ] -> result {
let! distance = validateDistance distanceStr
let newState = move distance state // use `move` function that was passed in
updateState newState
}
| [ "Turn"; angleStr ] -> result {
let! angle = validateAngle angleStr
let newState = turn angle state // use `turn` function that was passed in
updateState newState
}
...

Using partial application to bake in an implementation

To create a normal or half-size version of Exec , we just pass in different functions:
let log = printfn "%s"
let move = Turtle.move log
let turn = Turtle.turn log
let penUp = Turtle.penUp log
let penDown = Turtle.penDown log
let setColor = Turtle.setColor log
let normalSize() =
let api = TurtleApi()
// partially apply the functions
api.Exec move turn penUp penDown setColor
// the return value is a function:
// string -> Result<unit,ErrorMessage>
let halfSize() =
let moveHalf dist = move (dist/2.0)
let api = TurtleApi()
// partially apply the functions
api.Exec moveHalf turn penUp penDown setColor
// the return value is a function:
// string -> Result<unit,ErrorMessage>

In both cases we are returning a function of type string -> Result<unit,ErrorMessage> .

Using a purely functional API

986

Thirteen ways of looking at a turtle

So now when we want to draw something, we need only pass in any function of type string
-> Result<unit,ErrorMessage> . The TurtleApi is no longer needed or mentioned!

// the API type is just a function

type ApiFunction = string -> Result<unit,ErrorMessage>
let drawTriangle(api:ApiFunction) =
result {
do! api "Move 100"
do! api "Turn 120"
do! api "Move 100"
do! api "Turn 120"
do! api "Move 100"
do! api "Turn 120"
}

And here is how the API would be used:

let apiFn = normalSize() // string -> Result<unit,ErrorMessage>
drawTriangle(apiFn)
let apiFn = halfSize()
drawTriangle(apiFn)

So, although we did have mutable state in the TurtleApi , the final "published" api is a
function that hides that fact.
This approach of having the api be a single function makes it very easy to mock for testing!
let mockApi s =
printfn "[MockAPI] %s" s
Success ()
drawTriangle(mockApi)

Approach 2 - passing a single function that handles all

commands
In the version above, we passed in 5 separate functions!
Generally, when you are passing in more than three or four parameters, that implies that
your design needs tweaking. You shouldn't really need that many, if the functions are truly
independent.
But in our case, the five functions are not independent -- they come as a set -- so how can
we pass them in together without using a "record of functions" approach?
987

Thirteen ways of looking at a turtle

The trick is to pass in just one function! But how can one function handle five different
actions? Easy - by using a discriminated union to represent the possible commands.
We've seen this done before in the agent example, so let's revisit that type again:
type TurtleCommand =
| Move of Distance
| Turn of Angle
| PenUp
| PenDown
| SetColor of PenColor

All we need now is a function that handles each case of that type.
Befor we do that though, let's look at the changes to the Exec method implementation:
member this.Exec turtleFn (commandStr:string) =
...
// return Success of unit, or Failure
match tokens with
| [ "Move"; distanceStr ] -> result {
let! distance = validateDistance distanceStr
let command = Move distance // create a Command object
let newState = turtleFn command state
updateState newState
}
| [ "Turn"; angleStr ] -> result {
let! angle = validateAngle angleStr
let command = Turn angle // create a Command object
let newState = turtleFn command state
updateState newState
}
...

Note that a command object is being created and then the turtleFn parameter is being
called with it.
And by the way, this code is very similar to the agent implementation, which used
turtleAgent.Post command rather than newState = turtleFn command state :

Using partial application to bake in an implementation

Let's create the two implementations using this approach:

988

Thirteen ways of looking at a turtle

let log = printfn "%s"

let move = Turtle.move log
let turn = Turtle.turn log
let penUp = Turtle.penUp log
let penDown = Turtle.penDown log
let setColor = Turtle.setColor log
let normalSize() =
let turtleFn = function
| Move dist -> move dist
| Turn angle -> turn angle
| PenUp -> penUp
| PenDown -> penDown
| SetColor color -> setColor color
// partially apply the function to the API
let api = TurtleApi()
api.Exec turtleFn
// the return value is a function:
// string -> Result<unit,ErrorMessage>
let halfSize() =
let turtleFn = function
| Move dist -> move (dist/2.0)
| Turn angle -> turn angle
| PenUp -> penUp
| PenDown -> penDown
| SetColor color -> setColor color
// partially apply the function to the API
let api = TurtleApi()
api.Exec turtleFn
// the return value is a function:
// string -> Result<unit,ErrorMessage>

As before, in both cases we are returning a function of type string ->

Result<unit,ErrorMessage> ,. which we can pass into the drawTriangle function we defined

earlier:
let api = normalSize()
drawTriangle(api)
let api = halfSize()
drawTriangle(api)

Advantages and disadvantages of using functions

Advantages

989

Thirteen ways of looking at a turtle

The API is decoupled from a particular implementation via parameterization.

Because dependencies are passed in at the point of use ("in your face") rather than in a
constructor ("out of sight"), the tendency for dependencies to multiply is greatly reduced.
Any function parameter is automatically a "one method interface" so no retrofitting is
needed.
Regular partial application can be used to bake in parameters for "dependency
injection". No special pattern or IoC container is needed.
Disadvantages
If the number of dependent functions is too great (say more than four) passing them all
in as separate parameters can become awkward (hence, the second approach).
The discriminated union type can be trickier to work with than an interface.
The source code for these versions is available here (five function params) and here (one
function param).

8: Batch processing using a state monad

In the next two sections, we'll switch from "interactive" mode, where instructions are
processed one at a time, to "batch" mode, where a whole series of instructions are grouped
together and then run as one unit.
In the first design, we'll go back to the model where the client uses the Turtle functions
directly.
Just as before, the client must keep track of the current state and pass it into the next
function call, but this time we'll keep the state out of sight by using a so-called "state monad"
to thread the state through the various instructions. As a result, there are no mutables
anywhere!
This won't be a generalized state monad, but a simplified one just for this demonstration. I'll
call it the turtle workflow.
(For more on the state monad see my "monadster" talk and post and post on parser
combinators )

990

Thirteen ways of looking at a turtle

Defining the turtle workflow

The core turtle functions that we defined at the very beginning follow the same "shape" as
many other state-transforming functions, an input plus the turtle state, and the output plus
the turtle state.

(It's true that, so far. we have not had any useable output from the turtle functions, but in a
later example we will see this output being used to make decisions.)
There is a standard way to deal with these kinds of functions -- the "state monad".
Let's look at how this is built.
First, note that, thanks to currying, we can recast a function in this shape into two separate
one-parameter functions: processing the input generates another function that in turn has
the state as the parameter:

We can then think of a turtle function as something that takes an input and returns a new
function, like this:

In our case, using TurtleState as the state, the returned function will look like this:
TurtleState -> 'a * TurtleState

Finally, to make it easier to work with, we can treat the returned function as a thing in its own
right, give it a name such as TurtleStateComputation :

991

Thirteen ways of looking at a turtle

In the implementation, we would typically wrap the function with a single case discriminated
union like this:
type TurtleStateComputation<'a> =
TurtleStateComputation of (Turtle.TurtleState -> 'a * Turtle.TurtleState)

So that is the basic idea behind the "state monad". However, it's important to realize that a
state monad consists of more than just this type -- you also need some functions ("return"
and "bind") that obey some sensible laws.
I won't define the returnT and bindT functions here, but you can see their definitions in
the full source.
We need some additional helper functions too. (I'm going to add a T for Turtle suffix to all
the functions).
In particular, we need a way to feed some state into the TurtleStateComputation to "run" it:
let runT turtle state =
// pattern match against the turtle
// to extract the inner function
let (TurtleStateComputation innerFn) = turtle
// run the inner function with the passed in state
innerFn state

Finally, we can create a turtle workflow, which is a computation expression that makes it
easier to work with the TurtleStateComputation type:
// define a computation expression builder
type TurtleBuilder() =
member this.Return(x) = returnT x
member this.Bind(x,f) = bindT f x
// create an instance of the computation expression builder
let turtle = TurtleBuilder()

Using the Turtle workflow

To use the turtle workflow, we first need to create "lifted" or "monadic" versions of the
turtle functions:

992

Thirteen ways of looking at a turtle

let move dist =

toUnitComputation (Turtle.move log dist)
// val move : Distance -> TurtleStateComputation<unit>
let turn angle =
toUnitComputation (Turtle.turn log angle)
// val turn : Angle -> TurtleStateComputation<unit>
let penDown =
toUnitComputation (Turtle.penDown log)
// val penDown : TurtleStateComputation<unit>
let penUp =
toUnitComputation (Turtle.penUp log)
// val penUp : TurtleStateComputation<unit>
let setColor color =
toUnitComputation (Turtle.setColor log color)
// val setColor : PenColor -> TurtleStateComputation<unit>

The toUnitComputation helper function does the lifting. Don't worry about how it works, but
the effect is that the original version of the move function ( Distance -> TurtleState ->
TurtleState ) is reborn as a function returning a TurtleStateComputation ( Distance ->
TurtleStateComputation<unit> )

Once we have these "monadic" versions, we can use them inside the turtle workflow like
this:
let drawTriangle() =
// define a set of instructions
let t = turtle {
do! move 100.0
do! turn 120.0<Degrees>
do! move 100.0
do! turn 120.0<Degrees>
do! move 100.0
do! turn 120.0<Degrees>
}
// finally, run them using the initial state as input
runT t initialTurtleState

The first part of drawTriangle chains together six instructions, but importantly, does not run
them. Only when the runT function is used at the end are the instructions actually
executed.
The drawPolygon example is a little more complicated. First we define a workflow for
drawing one side:
993

Thirteen ways of looking at a turtle

let oneSide = turtle {

do! move 100.0
do! turn angleDegrees
}

But then we need a way of combining all the sides into a single workflow. There are a couple
of ways of doing this. I'll go with creating a pairwise combiner chain and then using
reduce to combine all the sides into one operation.

// chain two turtle operations in sequence

let chain f g = turtle {
do! f
do! g
}
// create a list of operations, one for each side
let sides = List.replicate n oneSide
// chain all the sides into one operation
let all = sides |> List.reduce chain

Here's the complete code for drawPolygon :

let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let angleDegrees = angle * 1.0<Degrees>
// define a function that draws one side
let oneSide = turtle {
do! move 100.0
do! turn angleDegrees
}
// chain two turtle operations in sequence
let chain f g = turtle {
do! f
do! g
}
// create a list of operations, one for each side
let sides = List.replicate n oneSide
// chain all the sides into one operation
let all = sides |> List.reduce chain
// finally, run them using the initial state
runT all initialTurtleState

994

Thirteen ways of looking at a turtle

Advantages and disadvantages of the turtle workflow

Advantages
The client code is similar to imperative code, but preserves immutability.
The workflows are composable -- you can define two workflows and then combine them
to create another workflow.
Disadvantages
Coupled to a particular implementation of the turtle functions.
More complex than tracking state explicitly.
Stacks of nested monads/workflows are hard to work with.
As an example of that last point, let's say we have a seq containing a result workflow
containing a turtle workflow and we want to invert them so that the turtle workflow is
on the outside. How would you do that? It's not obvious!
The source code for this version is available here.

9: Batch processing using command objects

Another batch-oriented approach is to reuse the TurtleCommand type in a new way. Instead
of calling functions immediately, the client creates a list of commands that will be run as a
group.
When you "run" the list of commands, you can just execute each one in turn using the
standard Turtle library functions, using fold to thread the state through the sequence.

And since all the commands are run at once, this approach means that there is no state that
needs to be persisted between calls by the client.
Here's the TurtleCommand definition again:

995

Thirteen ways of looking at a turtle

To process a sequence of commands, we will need to fold over them, threading the state
through, so we need a function that applies a single command to a state and returns a new
state:
/// Apply a command to the turtle state and return the new state
let applyCommand state command =
match command with
| Move distance ->
move distance state
| Turn angle ->
turn angle state
| PenUp ->
penUp state
| PenDown ->
penDown state
| SetColor color ->
setColor color state

And then, to run all the commands, we just use fold :

/// Run list of commands in one go
let run aListOfCommands =
aListOfCommands
|> List.fold applyCommand Turtle.initialTurtleState

Running a batch of Commands

To draw a triangle, say, we just create a list of the commands and then run them:

996

Thirteen ways of looking at a turtle

let drawTriangle() =
// create the list of commands
let commands = [
Move 100.0
Turn 120.0<Degrees>
Move 100.0
Turn 120.0<Degrees>
Move 100.0
Turn 120.0<Degrees>
]
// run them
run commands

Now, since the commands are just a collection, we can easily build bigger collections from
smaller ones.
Here's an example for drawPolygon , where drawOneSide returns a collection of commands,
and that collection is duplicated for each side:
let drawPolygon n =
let angle = 180.0 - (360.0/float n)
let angleDegrees = angle * 1.0<Degrees>
// define a function that draws one side
let drawOneSide sideNumber = [
Move 100.0
Turn angleDegrees
]
// repeat for all sides
let commands =
[1..n] |> List.collect drawOneSide
// run the commands
run commands

Advantages and disadvantages of batch commands

Advantages
Simpler to construct and use than workflows or monads.
Only one function is coupled to a particular implementation. The rest of the client is
decoupled.
Disadvantages
Batch oriented only.

997

Thirteen ways of looking at a turtle

Only suitable when control flow is not based on the response from a previous
command. If you do need to respond to the result of each command, consider using the
"interpreter" approach discussed later.
The source code for this version is available here.

Interlude: Conscious decoupling with data

types
In three of the examples so far (the agent, functional dependency injection and batch
processing) we have used a Command type -- a discriminated union containing a case for
each API call. We'll also see something similar used for the event sourcing and interpreter
approaches in the next post.
This is not an accident. One of the differences between object-oriented design and
functional design is that OO design focuses on behavior, while functional design focuses on
data transformation.
As a result, their approach to decoupling differs too. OO designs prefer to provide
decoupling by sharing bundles of encapsulated behavior ("interfaces") while functional
designs prefer to provide decoupling by agreeing on a common data type, sometimes called
a "protocol" (although I prefer to reserve that word for message exchange patterns).
Once that common data type is agreed upon, any function that emits that type can be
connected to any function that consumes that type using regular function composition.
You can also think of the two approaches as analogous to the choice between RPC or
message-oriented APIs in web services, and just as message-based designs have many
advantages over RPC, so the data-based decoupling has similar advantages over the
behavior-based decoupling.
Some advantages of decoupling using data include:
Using a shared data type means that composition is trivial. It is harder to compose
behavior-based interfaces.
Every function is already "decoupled", as it were, and so there is no need to retrofit
existing functions when refactoring. At worst you might need to convert one data type to
another, but that is easily accomplished using... moar functions and moar function
composition!
Data structures are easy to serialize to remote services if and when you need to split
your code into physically separate services.

998

Thirteen ways of looking at a turtle

Data structures are easy to evolve safely. For example, if I added a sixth turtle action, or
removed an action, or changed the parameters of an action, the discriminated union
type would change and all clients of the shared type would fail to compile until the sixth
turtle action is accounted for, etc. On the other hand, if you didn't want existing code to
break, you can use a versioning-friendly data serialization format like protobuf. Neither
of these options are as easy when interfaces are used.

Summary
The meme is spreading.
The turtle must be paddling.
-- "Thirteen ways of looking at a turtle", by Wallace D Coriacea
Hello? Anyone still there? Thanks for making it this far!
So, time for a break! In the next post, we'll cover the remaining four ways of looking at a
turtle.
The source code for this post is available on github.

999

Thirteen ways of looking at a turtle (part 2)

This post is part of the F# Advent Calendar in English 2015 project. Check out all the
other great posts there! And special thanks to Sergey Tihon for organizing this.
In this two-part mega-post, I'm stretching the simple turtle graphics model to the limit while
demonstrating partial application, validation, the concept of "lifting", agents with message
queues, dependency injection, the State monad, event sourcing, stream processing, and an
interpreter!
In the previous post, we covered the first nine ways of looking at a turtle. In this post, we'll
look at the remaining four.
As a reminder, here are the thirteen ways:
Way 1. A basic object-oriented approach, in which we create a class with mutable state.
Way 2. A basic functional approach, in which we create a module of functions with
immutable state.
Way 3. An API with a object-oriented core, in which we create an object-oriented API
that calls a stateful core class.
Way 4. An API with a functional core, in which we create an stateful API that uses
stateless core functions.
Way 5. An API in front of an agent, in which we create an API that uses a message
queue to communicate with an agent.
Way 6. Dependency injection using interfaces, in which we decouple the
implementation from the API using an interface or record of functions.
Way 7. Dependency injection using functions, in which we decouple the implementation
from the API by passing a function parameter.
Way 8. Batch processing using a state monad, in which we create a special "turtle
workflow" computation expression to track state for us.
Way 9. Batch processing using command objects, in which we create a type to
represent a turtle command, and then process a list of commands all at once.
Interlude: Conscious decoupling with data types. A few notes on using data vs.
interfaces for decoupling.
Way 10. Event sourcing, in which state is built from a list of past events.
Way 11. Functional Retroactive Programming (stream processing), in which business
logic is based on reacting to earlier events.
Episode V: The Turtle Strikes Back, in which the turtle API changes so that some
commands may fail.
Way 12. Monadic control flow, in which we make decisions in the turtle workflow based

1000

Thirteen ways of looking at a turtle (part 2)

on results from earlier commands.

Way 13. A turtle interpreter, in which we completely decouple turtle programming from
turtle implementation, and nearly encounter the free monad.
Review of all the techniques used.
and 2 bonus ways for the extended edition:
Way 14. Abstract Data Turtle, in which we encapsulate the details of a turtle
implementation by using an Abstract Data Type.
Way 15. Capability-based Turtle, in which we control what turtle functions are available
to a client, based on the current state of the turtle.
It's turtles all the way down!
All source code for this post is available on github.

10: Event sourcing -- Building state from a list

of past events
In this design, we build on the "command" concept used in the Agent (way 5) and Batch
(way 9) approaches, but replacing "commands" with "events" as the method of updating
state.
The way that it works is:
The client sends a Command to a CommandHandler .
Before processing a Command , the CommandHandler first rebuilds the current state from
scratch using the past events associated with that particular turtle.
The CommandHandler then validates the command and decides what to do based on the
current (rebuilt) state. It generates a (possibly empty) list of events.
The generated events are stored in an EventStore for the next command to use.

1001

Thirteen ways of looking at a turtle (part 2)

In this way, neither the client nor the command handler needs to track state. Only the
EventStore is mutable.

The Command and Event types

We will start by defining the types relating to our event sourcing system. First, the types
related to commands:
type TurtleId = System.Guid
/// A desired action on a turtle
type TurtleCommandAction =
| Move of Distance
| Turn of Angle
| PenUp
| PenDown
| SetColor of PenColor
/// A command representing a desired action addressed to a specific turtle
type TurtleCommand = {
turtleId : TurtleId
action : TurtleCommandAction
}

Note that the command is addressed to a particular turtle using a TurtleId .

Next, we will define two kinds of events that can be generated from a command:
A StateChangedEvent which represents what changed in the state
A MovedEvent which represents the start and end positions of a turtle movement.

1002

Thirteen ways of looking at a turtle (part 2)

/// An event representing a state change that happened

type StateChangedEvent =
| Moved of Distance
| Turned of Angle
| PenWentUp
| PenWentDown
| ColorChanged of PenColor
/// An event representing a move that happened
/// This can be easily translated into a line-drawing activity on a canvas
type MovedEvent = {
startPos : Position
endPos : Position
penColor : PenColor option
}
/// A union of all possible events
type TurtleEvent =
| StateChangedEvent of StateChangedEvent
| MovedEvent of MovedEvent

It is an important part of event sourcing that all events are labeled in the past tense: Moved
and Turned rather than Move and Turn . The event are facts -- they have happened in the
past.

The Command handler

The next step is to define the functions that convert a command into events.
We will need:
A (private) applyEvent function that updates the state from a previous event.
A (private) eventsFromCommand function that determines what events to generate, based
on the command and the state.
A public commandHandler function that handles the command, reads the events from the
event store and calls the other two functions.
Here's applyEvent . You can see that it is very similar to the applyCommand function that we
saw in the previous batch-processing example.

1003

Thirteen ways of looking at a turtle (part 2)

/// Apply an event to the current state and return the new state of the turtle
let applyEvent log oldState event =
match event with
| Moved distance ->
Turtle.move log distance oldState
| Turned angle ->
Turtle.turn log angle oldState
| PenWentUp ->
Turtle.penUp log oldState
| PenWentDown ->
Turtle.penDown log oldState
| ColorChanged color ->
Turtle.setColor log color oldState

The eventsFromCommand function contains the key logic for validating the command and
creating events.
In this particular design, the command is always valid, so at least one event is returned.
The StateChangedEvent is created from the TurtleCommand in a direct one-to-one map
of the cases.
The MovedEvent is only created from the TurtleCommand if the turtle has changed
position.

1004

Thirteen ways of looking at a turtle (part 2)

// Determine what events to generate, based on the command and the state.
let eventsFromCommand log command stateBeforeCommand =
// ------------------------- // create the StateChangedEvent from the TurtleCommand
let stateChangedEvent =
match command.action with
| Move dist -> Moved dist
| Turn angle -> Turned angle
| PenUp -> PenWentUp
| PenDown -> PenWentDown
| SetColor color -> ColorChanged color
// ------------------------- // calculate the current state from the new event
let stateAfterCommand =
applyEvent log stateBeforeCommand stateChangedEvent
// ------------------------- // create the MovedEvent
let startPos = stateBeforeCommand.position
let endPos = stateAfterCommand.position
let penColor =
if stateBeforeCommand.penState=Down then
Some stateBeforeCommand.color
else
None
let movedEvent = {
startPos = startPos
endPos = endPos
penColor = penColor
}
// ------------------------- // return the list of events
if startPos <> endPos then
// if the turtle has moved, return both the stateChangedEvent and the movedEve
nt
// lifted into the common TurtleEvent type
[ StateChangedEvent stateChangedEvent; MovedEvent movedEvent]
else
// if the turtle has not moved, return just the stateChangedEvent
[ StateChangedEvent stateChangedEvent]

Finally, the commandHandler is the public interface. It is passed in some dependencies as

parameters: a logging function, a function to retrieve the historical events from the event
store, and a function to save the newly generated events into the event store.

1005

Thirteen ways of looking at a turtle (part 2)

/// The type representing a function that gets the StateChangedEvents for a turtle id
/// The oldest events are first
type GetStateChangedEventsForId =
TurtleId -> StateChangedEvent list
/// The type representing a function that saves a TurtleEvent
type SaveTurtleEvent =
TurtleId -> TurtleEvent -> unit
/// main function : process a command
let commandHandler
(log:string -> unit)
(getEvents:GetStateChangedEventsForId)
(saveEvent:SaveTurtleEvent)
(command:TurtleCommand) =
/// First load all the events from the event store
let eventHistory =
getEvents command.turtleId
/// Then, recreate the state before the command
let stateBeforeCommand =
let nolog = ignore // no logging when recreating state
eventHistory
|> List.fold (applyEvent nolog) Turtle.initialTurtleState
/// Construct the events from the command and the stateBeforeCommand
/// Do use the supplied logger for this bit
let events = eventsFromCommand log command stateBeforeCommand
// store the events in the event store
events |> List.iter (saveEvent command.turtleId)

Calling the command handler

Now we are ready to send events to the command handler.
First we need some helper functions that create commands:
// Command versions of standard actions
let turtleId = System.Guid.NewGuid()
let move dist = {turtleId=turtleId; action=Move dist}
let turn angle = {turtleId=turtleId; action=Turn angle}
let penDown = {turtleId=turtleId; action=PenDown}
let penUp = {turtleId=turtleId; action=PenUp}
let setColor color = {turtleId=turtleId; action=SetColor color}

And then we can draw a figure by sending the various commands to the command handler:

1006

Thirteen ways of looking at a turtle (part 2)

let drawTriangle() =
let handler = makeCommandHandler()
handler (move 100.0)
handler (turn 120.0<Degrees>)
handler (move 100.0)
handler (turn 120.0<Degrees>)
handler (move 100.0)
handler (turn 120.0<Degrees>)

NOTE: I have not shown how to create the command handler or event store, see the code
for full details.

Advantages and disadvantages of event sourcing

Advantages
All code is stateless, hence easy to test.
Supports replay of events.
Disadvantages
Can be more complex to implement than a CRUD approach (or at least, less support
from tools and libraries).
If care is not taken, the command handler can get overly complex and evolve into
implementing too much business logic.
The source code for this version is available here.

11: Functional Retroactive Programming

(stream processing)
In the event sourcing example above, all the domain logic (in our case, just tracing the state)
is embedded in the command handler. One drawback of this is that, as the application
evolves, the logic in the command handler can become very complex.
A way to avoid this is to combine "functional reactive programming" with event sourcing to
create a design where the domain logic is performed on the "read-side", by listening to
events ("signals") emitted from the event store.
In this approach, the "write-side" follows the same pattern as the event-sourcing example. A
client sends a Command to a commandHandler , which converts that to a list of events and
stores them in an EventStore .

1007

Thirteen ways of looking at a turtle (part 2)

However the commandHandler only does the minimal amount of work, such as updating
state, and does NOT do any complex domain logic. The complex logic is performed by one
or more downstream "processors" (also sometimes called "aggregators") that subscribe to
the event stream.

You can even think of these events as "commands" to the processors, and of course, the
processors can generate new events for another processor to consume, so this approach
can be extended into an architectural style where an application consists of a set of
command handlers linked by an event store.
This techique is often called "stream processing". However, Jessica Kerr once called this
approach "Functional Retroactive Programming" -- I like that, so I'm going to steal that
name!

Implementing the design

For this implementation, the commandHandler function is the same as in the event sourcing
example, except that no work (just logging!) is done at all. The command handler only
rebuilds state and generates events. How the events are used for business logic is no longer
in its scope.
The new stuff comes in creating the processors.

1008

Thirteen ways of looking at a turtle (part 2)

However, before we can create a processor, we need some helper functions that can filter
the event store feed to only include turtle specific events, and of those only
StateChangedEvent s or MovedEvent s.

// filter to choose only TurtleEvents

let turtleFilter ev =
match box ev with
| :? TurtleEvent as tev -> Some tev
| _ -> None
// filter to choose only MovedEvents from TurtleEvents
let moveFilter = function
| MovedEvent ev -> Some ev
| _ -> None
// filter to choose only StateChangedEvent from TurtleEvents
let stateChangedEventFilter = function
| StateChangedEvent ev -> Some ev
| _ -> None

Now let's create a processor that listens for movement events and moves a physical turtle
when the virtual turtle is moved.
We will make the input to the processor be an IObservable -- an event stream -- so that it is
not coupled to any specific source such as the EventStore . We will connect the EventStore
"save" event to this processor when the application is configured.
/// Physically move the turtle
let physicalTurtleProcessor (eventStream:IObservable<Guid*obj>) =
// the function that handles the input from the observable
let subscriberFn (ev:MovedEvent) =
let colorText =
match ev.penColor with
| Some color -> sprintf "line of color %A" color
| None -> "no line"
printfn "[turtle ]: Moved from (%0.2f,%0.2f) to (%0.2f,%0.2f) with %s"
ev.startPos.x ev.startPos.y ev.endPos.x ev.endPos.y colorText
// start with all events
eventStream
// filter the stream on just TurtleEvents
|> Observable.choose (function (id,ev) -> turtleFilter ev)
// filter on just MovedEvents
|> Observable.choose moveFilter
// handle these
|> Observable.subscribe subscriberFn

1009

Thirteen ways of looking at a turtle (part 2)

In this case we are just printing the movement -- I'll leave the building of an actual Lego
Mindstorms turtle as an exercise for the reader!
Let's also create a processor that draws lines on a graphics display:
/// Draw lines on a graphics device
let graphicsProcessor (eventStream:IObservable<Guid*obj>) =
// the function that handles the input from the observable
let subscriberFn (ev:MovedEvent) =
match ev.penColor with
| Some color ->
printfn "[graphics]: Draw line from (%0.2f,%0.2f) to (%0.2f,%0.2f) with co
lor %A"
ev.startPos.x ev.startPos.y ev.endPos.x ev.endPos.y color
| None ->
() // do nothing
// start with all events
eventStream
// filter the stream on just TurtleEvents
|> Observable.choose (function (id,ev) -> turtleFilter ev)
// filter on just MovedEvents
|> Observable.choose moveFilter
// handle these
|> Observable.subscribe subscriberFn

And finally, let's create a processor that accumulates the total distance moved so that we
can keep track of how much ink has been used, say.

1010

Thirteen ways of looking at a turtle (part 2)

/// Listen for "moved" events and aggregate them to keep

/// track of the total ink used
let inkUsedProcessor (eventStream:IObservable<Guid*obj>) =
// Accumulate the total distance moved so far when a new event happens
let accumulate distanceSoFar (ev:StateChangedEvent) =
match ev with
| Moved dist ->
distanceSoFar + dist
| _ ->
distanceSoFar
// the function that handles the input from the observable
let subscriberFn distanceSoFar =
printfn "[ink used]: %0.2f" distanceSoFar
// start with all events
eventStream
// filter the stream on just TurtleEvents
|> Observable.choose (function (id,ev) -> turtleFilter ev)
// filter on just StateChangedEvent
|> Observable.choose stateChangedEventFilter
// accumulate total distance
|> Observable.scan accumulate 0.0
// handle these
|> Observable.subscribe subscriberFn

This processor uses Observable.scan to accumulate the events into a single value -- the
total distance travelled.

Processors in practice
Let's try these out!
For example, here is drawTriangle :

1011

Thirteen ways of looking at a turtle (part 2)

let drawTriangle() =
// clear older events
eventStore.Clear turtleId
// create an event stream from an IEvent
let eventStream = eventStore.SaveEvent :> IObservable<Guid*obj>
// register the processors
use physicalTurtleProcessor = EventProcessors.physicalTurtleProcessor eventStream
use graphicsProcessor = EventProcessors.graphicsProcessor eventStream
use inkUsedProcessor = EventProcessors.inkUsedProcessor eventStream
let handler = makeCommandHandler
handler (move 100.0)
handler (turn 120.0<Degrees>)
handler (move 100.0)
handler (turn 120.0<Degrees>)
handler (move 100.0)
handler (turn 120.0<Degrees>)

Note that eventStore.SaveEvent is cast into an IObservable<Guid*obj> (that is, an event

stream) before being passed to the processors as a parameter.
drawTriangle generates this output:

[ink used]: 100.00

[turtle ]: Moved from (0.00,0.00) to (100.00,0.00) with line of color Black
[graphics]: Draw line from (0.00,0.00) to (100.00,0.00) with color Black
[ink used]: 100.00
[ink used]: 200.00
[turtle ]: Moved from (100.00,0.00) to (50.00,86.60) with line of color Black
[graphics]: Draw line from (100.00,0.00) to (50.00,86.60) with color Black
[ink used]: 200.00
[ink used]: 300.00
[turtle ]: Moved from (50.00,86.60) to (0.00,0.00) with line of color Black
[graphics]: Draw line from (50.00,86.60) to (0.00,0.00) with color Black
[ink used]: 300.00

You can see that all the processors are handling events successfully.
The turtle is moving, the graphics processor is drawing lines, and the ink used processor has
correctly calculated the total distance moved as 300 units.
Note, though, that the ink used processor is emitting output on every state change (such as
turning), rather than only when actual movement happens.
We can fix this by putting a pair (previousDistance, currentDistance) in the stream, and
then filtering out those events where the values are the same.

1012

Thirteen ways of looking at a turtle (part 2)

Here's the new inkUsedProcessor code, with the following changes:

The accumulate function now emits a pair.
There is a new filter changedDistanceOnly .
/// Listen for "moved" events and aggregate them to keep
/// track of the total distance moved
/// NEW! No duplicate events!
let inkUsedProcessor (eventStream:IObservable<Guid*obj>) =
// Accumulate the total distance moved so far when a new event happens
let accumulate (prevDist,currDist) (ev:StateChangedEvent) =
let newDist =
match ev with
| Moved dist ->
currDist + dist
| _ ->
currDist
(currDist, newDist)
// convert unchanged events to None so they can be filtered out with "choose"
let changedDistanceOnly (currDist, newDist) =
if currDist <> newDist then
Some newDist
else
None
// the function that handles the input from the observable
let subscriberFn distanceSoFar =
printfn "[ink used]: %0.2f" distanceSoFar
// start with all events
eventStream
// filter the stream on just TurtleEvents
|> Observable.choose (function (id,ev) -> turtleFilter ev)
// filter on just StateChangedEvent
|> Observable.choose stateChangedEventFilter
// NEW! accumulate total distance as pairs
|> Observable.scan accumulate (0.0,0.0)
// NEW! filter out when distance has not changed
|> Observable.choose changedDistanceOnly
// handle these
|> Observable.subscribe subscriberFn

With these changes, the output of drawTriangle looks like this:

1013

Thirteen ways of looking at a turtle (part 2)

[ink used]: 100.00

[turtle ]: Moved from (0.00,0.00) to (100.00,0.00) with line of color Black
[graphics]: Draw line from (0.00,0.00) to (100.00,0.00) with color Black
[ink used]: 200.00
[turtle ]: Moved from (100.00,0.00) to (50.00,86.60) with line of color Black
[graphics]: Draw line from (100.00,0.00) to (50.00,86.60) with color Black
[ink used]: 300.00
[turtle ]: Moved from (50.00,86.60) to (0.00,0.00) with line of color Black
[graphics]: Draw line from (50.00,86.60) to (0.00,0.00) with color Black

and there are no longer any duplicate messages from the inkUsedProcessor .

Advantages and disadvantages of stream processing

Advantages
Same advantages as event-sourcing.
Decouples stateful logic from other non-intrinsic logic.
Easy to add and remove domain logic without affecting the core command handler.
Disadvantages
More complex to implement.
The source code for this version is available here.

Episode V: The Turtle Strikes Back

So far, we have not had to make decisions based on the turtle's state. So, for the two final
approaches, we will change the turtle API so that some commands may fail.
For example, we might say that the turtle must move within a limited arena, and a move
instruction may cause the turtle to hit the barrier. In this case, the move instruction can
return a choice of MovedOk or HitBarrier .
Or let's say that there is only a limited amount of colored ink. In this case, trying to set the
color may return an "out of ink" response.
So let's update the turtle functions with these cases. First the new response types for move
and setColor :

1014

Thirteen ways of looking at a turtle (part 2)

type MoveResponse =
| MoveOk
| HitABarrier
type SetColorResponse =
| ColorOk
| OutOfInk

We will need a bounds checker to see if the turtle is in the arena. Say that if the position tries
to go outside the square (0,0,100,100), the response is HitABarrier :
// if the position is outside the square (0,0,100,100)
// then constrain the position and return HitABarrier
let checkPosition position =
let isOutOfBounds p =
p > 100.0 || p < 0.0
let bringInsideBounds p =
max (min p 100.0) 0.0
if isOutOfBounds position.x || isOutOfBounds position.y then
let newPos = {
x = bringInsideBounds position.x
y = bringInsideBounds position.y }
HitABarrier,newPos
else
MoveOk,position

And finally, the move function needs an extra line to check the new position:
let move log distance state =
let newPosition = ...
// adjust the new position if out of bounds
let moveResult, newPosition = checkPosition newPosition
...

Here's the complete move function:

1015

Thirteen ways of looking at a turtle (part 2)

let move log distance state =

log (sprintf "Move %0.1f" distance)
// calculate new position
let newPosition = calcNewPosition distance state.angle state.position
// adjust the new position if out of bounds
let moveResult, newPosition = checkPosition newPosition
// draw line if needed
if state.penState = Down then
dummyDrawLine log state.position newPosition state.color
// return the new state and the Move result
let newState = {state with position = newPosition}
(moveResult,newState)

We will make similar changes for the setColor function too, returning OutOfInk if we
attempt to set the color to Red .
let setColor log color state =
let colorResult =
if color = Red then OutOfInk else ColorOk
log (sprintf "SetColor %A" color)
// return the new state and the SetColor result
let newState = {state with color = color}
(colorResult,newState)

With the new versions of the turtle functions available, we have to create implementations
that can respond to the error cases. That will be done in the next two examples.
The source code for the new turtle functions is available here.

12: Monadic control flow

In this approach, we will reuse the turtle workflow from way 8. This time though, we will
make decisions for the next command based on the result of the previous one.
Before we do that though, let's look at what effect the change to move will have on our
code. Let's say that we want to move forwards a few times using move 40.0 , say.
If we write the code using do! as we did before, we get a nasty compiler error:

1016

Thirteen ways of looking at a turtle (part 2)

let drawShape() =
// define a set of instructions
let t = turtle {
do! move 60.0
// error FS0001:
// This expression was expected to have type
// Turtle.MoveResponse
// but here has type
// unit
do! move 60.0
}
// etc

Instead, we need to use let! and assign the response to something.

In the following code, we assign the response to a value and then ignore it!
let drawShapeWithoutResponding() =
// define a set of instructions
let t = turtle {
let! response = move 60.0
let! response = move 60.0
let! response = move 60.0
return ()
}
// finally, run the monad using the initial state
runT t initialTurtleState

The code does compile and work, but if we run it the output shows that, by the third call, we
are banging our turtle against the wall (at 100,0) and not moving anywhere.
Move 60.0
...Draw line from (0.0,0.0) to (60.0,0.0) using Black
Move 60.0
...Draw line from (60.0,0.0) to (100.0,0.0) using Black
Move 60.0
...Draw line from (100.0,0.0) to (100.0,0.0) using Black

Making decisions based on a response

Let's say that our response to a move that returns HitABarrier is to turn 90 degrees and
wait for the next command. Not the cleverest algorithm, but it will do for demonstration
purposes!

1017

Thirteen ways of looking at a turtle (part 2)

Let's design a function to implement this. The input will be a MoveResponse , but what will the
output be? We want to encode the turn action somehow, but the raw turn function needs
state input that we don't have. So instead let's return a turtle workflow that represents the
instruction we want to do, when the state becomes available (in the run command).
So here is the code:
let handleMoveResponse moveResponse = turtle {
match moveResponse with
| Turtle.MoveOk ->
() // do nothing
| Turtle.HitABarrier ->
// turn 90 before trying again
printfn "Oops -- hit a barrier -- turning"
do! turn 90.0<Degrees>
}

The type signature looks like this:

val handleMoveResponse : MoveResponse -> TurtleStateComputation<unit>

which means that it is a monadic (or "diagonal") function -- one that starts in the normal
world and ends in the TurtleStateComputation world.
These are exactly the functions that we can use "bind" with, or within computation
expressions, let! or do! .
Now we can add this handleMoveResponse step after move in the turtle workflow:
let drawShape() =
// define a set of instructions
let t = turtle {
let! response = move 60.0
do! handleMoveResponse response
let! response = move 60.0
do! handleMoveResponse response
let! response = move 60.0
do! handleMoveResponse response
}
// finally, run the monad using the initial state
runT t initialTurtleState

And the result of running it is:

1018

Thirteen ways of looking at a turtle (part 2)

Move 60.0
...Draw line from (0.0,0.0) to (60.0,0.0) using Black
Move 60.0
...Draw line from (60.0,0.0) to (100.0,0.0) using Black
Oops -- hit a barrier -- turning
Turn 90.0
Move 60.0
...Draw line from (100.0,0.0) to (100.0,60.0) using Black

You can see that the move response worked. When the turtle hit the edge at (100,0) it
turned 90 degrees and the next move succeeded (from (100,0) to (100,60)).
So there you go! This code demonstrates how you can make decisions inside the turtle
workflow while the state is being passed around behind the scenes.

Advantages and disadvantages

Advantages
Computation expressions allow the code to focus on the logic while taking care of the
"plumbing" -- in this case, the turtle state.
Disadvantages
Still coupled to a particular implementation of the turtle functions.
Computation expressions can be complex to implement and how they work is not
obvious for beginners.
The source code for this version is available here.

13: A turtle interpreter

For our final approach, we'll look at a way to completely decouple the programming of the
turtle from its interpretation.
This is similar to the batch processing using command objects approach, but is enhanced to
support responding to the output of a command.

Designing an interpreter

1019

Thirteen ways of looking at a turtle (part 2)

The approach we will take is to design an "interpreter" for a set of turtle commands, where
the client provides the commands to the turtle, and responds to outputs from the turtle, but
the actual turtle functions are provided later by a particular implementation.
In other words, we have a chain of interleaved commands and turtle functions that look like
this:

So how can we model this design in code?

The problem is that we cannot be sure that the response correctly matches the command.
For example, if I send a Move command, I expect to get a MoveResponse , and never a
SetColorResponse . But this implementation doesn't enforce that!

We want to make illegal states unrepresentable -- how can we do that?

The trick is to combine the request and response in pairs. That is, for a Move command,
there is an associated function which is given a MoveResponse as input, and similarly for
each other combination. Commands that have no response can be considered as returning
unit for now.

1020

Thirteen ways of looking at a turtle (part 2)

Move command => pair of (Move command parameters), (function MoveResponse -> something
)
Turn command => pair of (Turn command parameters), (function unit -> something)
etc

The way this works is that:

The client creates a command, say Move 100 , and also provides the additional function
that handles the response.
The turtle implementation for the Move command (inside the interpreter) processes the
input (a Distance ) and then generates a MoveResponse .
The interpreter then takes this MoveResponse and calls the associated function in the
pair, as supplied by the client.
By associating the Move command with a function in this way, we can guarantee that the
internal turtle implementation must accept a distance and return a MoveResponse , just as
we want.
The next question is: what is the something that is the output? It is the output after the client
has handled the response -- that is, another command/response chain!
So we can model the whole chain of pairs as a recursive structure:

Or in code:
type TurtleProgram =
// (input params) (response)
| Move of Distance * (MoveResponse -> TurtleProgram)
| Turn of Angle * (unit -> TurtleProgram)
| PenUp of (* none *) (unit -> TurtleProgram)
| PenDown of (* none *) (unit -> TurtleProgram)
| SetColor of PenColor * (SetColorResponse -> TurtleProgram)

I've renamed the type from TurtleCommand to TurtleProgram because it is no longer just a
command, but is now a complete chain of commands and associated response handlers.
There's a problem though! Every step needs yet another TurtleProgram to follow -- so when
will it stop? We need some way of saying that there is no next command.
To solve this issue, we will add a special Stop case to the program type:

1021

Thirteen ways of looking at a turtle (part 2)

type TurtleProgram =
// (input params) (response)
| Stop
| Move of Distance * (MoveResponse -> TurtleProgram)
| Turn of Angle * (unit -> TurtleProgram)
| PenUp of (* none *) (unit -> TurtleProgram)
| PenDown of (* none *) (unit -> TurtleProgram)
| SetColor of PenColor * (SetColorResponse -> TurtleProgram)

Note that there is no mention of TurtleState in this structure. How the turtle state is
managed is internal to the interpreter, and is not part of the "instruction set", as it were.
TurtleProgram is an example of an Abstract Syntax Tree (AST) -- a structure that

represents a program to interpreted (or compiled).

Testing the interpreter

Let's create a little program using this model. Here's our old friend drawTriangle :
let drawTriangle =
Move (100.0, fun response ->
Turn (120.0<Degrees>, fun () ->
Move (100.0, fun response ->
Turn (120.0<Degrees>, fun () ->
Move (100.0, fun response ->
Turn (120.0<Degrees>, fun () ->
Stop))))))

This program is a data structure containing only client commands and responses -- there are
no actual turtle functions in it anywhere! And yes, it is really ugly right now, but we will fix that
shortly.
Now the next step is to interpret this data structure.
Let's create an interpreter that calls the real turtle functions. How would we implement the
Move case, say?

Well, just as described above:

Get the distance and associated function from the Move case
Call the real turtle function with the distance and current turtle state, to get a
MoveResult and a new turtle state.

Get the next step in the program by passing the MoveResult to the associated function
Finally call the interpreter again (recursively) with the new program and new turtle state.

1022

Thirteen ways of looking at a turtle (part 2)

let rec interpretAsTurtle state program =

...
match program with
| Move (dist,next) ->
let result,newState = Turtle.move log dist state
let nextProgram = next result // compute the next step
interpretAsTurtle newState nextProgram
...

You can see that the updated turtle state is passed as a parameter to the next recursive call,
and so no mutable field is needed.
Here's the full code for interpretAsTurtle :
let rec interpretAsTurtle state program =
let log = printfn "%s"
match program with
| Stop ->
state
| Move (dist,next) ->
let result,newState = Turtle.move log dist state
let nextProgram = next result // compute the next step
interpretAsTurtle newState nextProgram
| Turn (angle,next) ->
let newState = Turtle.turn log angle state
let nextProgram = next() // compute the next step
interpretAsTurtle newState nextProgram
| PenUp next ->
let newState = Turtle.penUp log state
let nextProgram = next()
interpretAsTurtle newState nextProgram
| PenDown next ->
let newState = Turtle.penDown log state
let nextProgram = next()
interpretAsTurtle newState nextProgram
| SetColor (color,next) ->
let result,newState = Turtle.setColor log color state
let nextProgram = next result
interpretAsTurtle newState nextProgram

Let's run it:

let program = drawTriangle
let interpret = interpretAsTurtle // choose an interpreter
let initialState = Turtle.initialTurtleState
interpret initialState program |> ignore

1023

Thirteen ways of looking at a turtle (part 2)

and the output is exactly what we have seen before:

But unlike all the previous approaches we can take exactly the same program and interpret it
in a new way. We don't need to set up any kind of dependency injection, we just need to use
a different interpreter.
So let's create another interpreter that aggregates the distance travelled, without caring
about the turtle state:
let rec interpretAsDistance distanceSoFar program =
let recurse = interpretAsDistance
let log = printfn "%s"
match program with
| Stop ->
distanceSoFar
| Move (dist,next) ->
let newDistanceSoFar = distanceSoFar + dist
let result = Turtle.MoveOk // hard-code result
let nextProgram = next result
recurse newDistanceSoFar nextProgram
| Turn (angle,next) ->
// no change in distanceSoFar
let nextProgram = next()
recurse distanceSoFar nextProgram
| PenUp next ->
// no change in distanceSoFar
let nextProgram = next()
recurse distanceSoFar nextProgram
| PenDown next ->
// no change in distanceSoFar
let nextProgram = next()
recurse distanceSoFar nextProgram
| SetColor (color,next) ->
// no change in distanceSoFar
let result = Turtle.ColorOk // hard-code result
let nextProgram = next result
recurse distanceSoFar nextProgram

1024

Thirteen ways of looking at a turtle (part 2)

In this case, I've aliased interpretAsDistance as recurse locally to make it obvious what
kind of recursion is happening.
Let's run the same program with this new interpreter:
let program = drawTriangle // same program
let interpret = interpretAsDistance // choose an interpreter
let initialState = 0.0
interpret initialState program |> printfn "Total distance moved is %0.1f"

and the output is again exactly what we expect:

Total distance moved is 300.0

Creating a "turtle program" workflow

That code for creating a program to interpret was pretty ugly! Can we create a computation
expression to make it look nicer?
Well, in order to create a computation expression, we need return and bind functions,
and those require that the TurtleProgram type be generic.
No problem! Let's make TurtleProgram generic then:
type TurtleProgram<'a> =
| Stop of 'a
| Move of Distance * (MoveResponse -> TurtleProgram<'a>)
| Turn of Angle * (unit -> TurtleProgram<'a>)
| PenUp of (unit -> TurtleProgram<'a>)
| PenDown of (unit -> TurtleProgram<'a>)
| SetColor of PenColor * (SetColorResponse -> TurtleProgram<'a>)

Note that the Stop case has a value of type 'a associated with it now. This is needed so
that we can implement return properly:
let returnT x =
Stop x

The bind function is more complicated to implement. Don't worry about how it works right
now -- the important thing is that the types match up and it compiles!

1025

Thirteen ways of looking at a turtle (part 2)

let rec bindT f inst =

match inst with
| Stop x ->
f x
| Move(dist,next) ->
(*
Move(dist,fun moveResponse -> (bindT f)(next moveResponse))
*)
// "next >> bindT f" is a shorter version of function response
Move(dist,next >> bindT f)
| Turn(angle,next) ->
Turn(angle,next >> bindT f)
| PenUp(next) ->
PenUp(next >> bindT f)
| PenDown(next) ->
PenDown(next >> bindT f)
| SetColor(color,next) ->
SetColor(color,next >> bindT f)

With bind and return in place, we can create a computation expression:

// define a computation expression builder
type TurtleProgramBuilder() =
member this.Return(x) = returnT x
member this.Bind(x,f) = bindT f x
member this.Zero(x) = returnT ()
// create an instance of the computation expression builder
let turtleProgram = TurtleProgramBuilder()

We can now create a workflow that handles MoveResponse s just as in the monadic control
flow example (way 12) earlier.

1026

Thirteen ways of looking at a turtle (part 2)

// helper functions
let stop = fun x -> Stop x
let move dist = Move (dist, stop)
let turn angle = Turn (angle, stop)
let penUp = PenUp stop
let penDown = PenDown stop
let setColor color = SetColor (color,stop)
let handleMoveResponse log moveResponse = turtleProgram {
match moveResponse with
| Turtle.MoveOk ->
()
| Turtle.HitABarrier ->
// turn 90 before trying again
log "Oops -- hit a barrier -- turning"
let! x = turn 90.0<Degrees>
()
}
// example
let drawTwoLines log = turtleProgram {
let! response = move 60.0
do! handleMoveResponse log response
let! response = move 60.0
do! handleMoveResponse log response
}

Let's interpret this using the real turtle functions (assuming that the interpretAsTurtle
function has been modified to handle the new generic structure):
let log = printfn "%s"
let program = drawTwoLines log
let interpret = interpretAsTurtle
let initialState = Turtle.initialTurtleState
interpret initialState program |> ignore

The output shows that the MoveResponse is indeed being handled correctly when the barrier
is encountered:
Move 60.0
...Draw line from (0.0,0.0) to (60.0,0.0) using Black
Move 60.0
...Draw line from (60.0,0.0) to (100.0,0.0) using Black
Oops -- hit a barrier -- turning
Turn 90.0

Refactoring the TurtleProgram type into two parts

1027

Thirteen ways of looking at a turtle (part 2)

This approach works fine, but it bothers me that there is a special Stop case in the
TurtleProgram type. It would nice if we could somehow just focus on the five turtle actions

and ignore it.

As it turns out, there is a way to do this. In Haskell and Scalaz it would be called a "free
monad", but since F# doesn't support typeclasses, I'll just call it the "free monad pattern" that
you can use to solve the problem. There's a little bit of boilerplate that you have to write, but
not much.
The trick is to separate the api cases and "stop"/"keep going" logic into two separate types,
like this:
/// Create a type to represent each instruction
type TurtleInstruction<'next> =
| Move of Distance * (MoveResponse -> 'next)
| Turn of Angle * 'next
| PenUp of 'next
| PenDown of 'next
| SetColor of PenColor * (SetColorResponse -> 'next)
/// Create a type to represent the Turtle Program
type TurtleProgram<'a> =
| Stop of 'a
| KeepGoing of TurtleInstruction<TurtleProgram<'a>>

Note that I've also changed the responses for Turn , PenUp and PenDown to be single
values rather than a unit function. Move and SetColor remain as functions though.
In this new "free monad" approach, the only custom code we need to write is a simple map
function for the api type, in this case TurtleInstruction :
let mapInstr f inst =
match inst with
| Move(dist,next) -> Move(dist,next >> f)
| Turn(angle,next) -> Turn(angle,f next)
| PenUp(next) -> PenUp(f next)
| PenDown(next) -> PenDown(f next)
| SetColor(color,next) -> SetColor(color,next >> f)

The rest of the code ( return , bind , and the computation expression) is always
implemented exactly the same way, regardless of the particular api. That is, more boilerplate
is needed but less thinking is required!
The interpreters need to change in order to handle the new cases. Here's a snippet of the
new version of interpretAsTurtle :

1028

Thirteen ways of looking at a turtle (part 2)

let rec interpretAsTurtle log state program =

let recurse = interpretAsTurtle log
match program with
| Stop a ->
state
| KeepGoing (Move (dist,next)) ->
let result,newState = Turtle.move log dist state
let nextProgram = next result // compute next program
recurse newState nextProgram
| KeepGoing (Turn (angle,next)) ->
let newState = Turtle.turn log angle state
let nextProgram = next // use next program directly
recurse newState nextProgram

And we also need to adjust the helper functions when creating a workflow. You can see
below that we now have slightly more complicated code like KeepGoing (Move (dist, Stop))
instead of the simpler code in the original interpreter.
// helper functions
let stop = Stop()
let move dist = KeepGoing (Move (dist, Stop)) // "Stop" is a function
let turn angle = KeepGoing (Turn (angle, stop)) // "stop" is a value
let penUp = KeepGoing (PenUp stop)
let penDown = KeepGoing (PenDown stop)
let setColor color = KeepGoing (SetColor (color,Stop))
let handleMoveResponse log moveResponse = turtleProgram {
... // as before
// example
let drawTwoLines log = turtleProgram {
let! response = move 60.0
do! handleMoveResponse log response
let! response = move 60.0
do! handleMoveResponse log response
}

But with those changes, we are done, and the code works just as before.

Advantages and disadvantages of the interpreter pattern

Advantages
Decoupling. An abstract syntax tree completely decouples the program flow from the
implementation and allows lots of flexibility.
Optimization. Abstract syntax trees can be manipulated and changed before running

1029

Thirteen ways of looking at a turtle (part 2)

them, in order to do optimizations or other transformations. As an example, for the turtle

program, we could process the tree and collapse all contiguous sequences of Turn
into a single Turn operation. This is a simple optimization which saves on the number
of times we need to communicate with a physical turtle. Twitter's Stitch library does
something like this, but obviously, in a more sophisticated way. This video has a good
explanation.
Minimal code for a lot of power. The "free monad" approach to creating abstract syntax
trees allows you to focus on the API and ignore the Stop/KeepGoing logic, and also
means that only a minimal amount of code needs to be customized. For more on the
free monad, start with this excellent video and then see this post and this one.
Disadvantages
Complex to understand.
Only works well if there are a limited set of operations to perform.
Can be inefficient if the ASTs get too large.
The source code for this version is available here (original version) and here ("free monad"
version).

Review of techniques used

In this post, we looked at thirteen different ways to implement a turtle API, using a wide
variety of different techniques. Let's quickly run down all the techniques that were used:
Pure, stateless functions. As seen in all of the FP-oriented examples. All these are
very easy to test and mock.
Partial application. As first seen in the simplest FP example (way 2), when the turtle
functions had the logging function applied so that the main flow could use piping, and
thereafter used extensively, particularly in the "dependency injection using functions
approach" (way 7).
Object expressions to implement an interface without creating a class, as seen in way
6.
The Result type (a.k.a the Either monad). Used in all the functional API examples (e.g.
way 4) to return an error rather than throw an exception.
Applicative "lifting" (e.g. lift2 ) to lift normal functions to the world of Result s,
again in way 4 and others.
Lots of different ways of managing state:
mutable fields (way 1)
managing state explicitly and piping it though a series of functions (way 2)

1030

Thirteen ways of looking at a turtle (part 2)

having state only at the edge (the functional core/imperative shell in way 4)
hiding state in an agent (way 5)
threading state behind the scenes in a state monad (the turtle workflow in ways
8 and 12)
avoiding state altogether by using batches of commands (way 9) or batches of
events (way 10) or an interpreter (way 13)
Wrapping a function in a type. Used in way 8 to manage state (the State monad) and
in way 13 to store responses.
Computation expressions, lots of them! We created and used three:
result for working with errors
turtle for managing turtle state
turtleProgram for building an AST in the interpreter approach (way 13).

Chaining of monadic functions in the result and turtle workflows. The underlying
functions are monadic ("diagonal") and would not normally compose properly, but inside
a workflow, they can be sequenced easily and transparently.
Representing behavior as a data structure in the "functional dependency injection"
example (way 7) so that a single function could be passed in rather than a whole
interface.
Decoupling using a data-centric protocol as seen in the agent, batch command,
event sourcing, and interpreter examples.
Lock free and async processing using an agent (way 5).
The separation of "building" a computation vs. "running" it, as seen in the turtle
workflows (ways 8 and 12) and the turtleProgram workflow (way 13: interpreter).
Use of event sourcing to rebuild state from scratch rather than maintaining mutable
state in memory, as seen in the event sourcing (way 10) and FRP (way 11) examples.
Use of event streams and FRP (way 11) to break business logic into small,
independent, and decoupled processors rather than having a monolithic object.
I hope it's clear that examining these thirteen ways is just a fun exercise, and I'm not
suggesting that you immediately convert all your code to use stream processors and
interpreters! And, especially if you are working with people who are new to functional
programming, I would tend to stick with the earlier (and simpler) approaches unless there is
a clear benefit in exchange for the extra complexity.

Summary

1031

Thirteen ways of looking at a turtle (part 2)

When the tortoise crawled out of sight,

It marked the edge
Of one of many circles.
-- "Thirteen ways of looking at a turtle", by Wallace D Coriacea
I hope you enjoyed this post. I certainly enjoyed writing it. As usual, it ended up much longer
than I intended, so I hope that the effort of reading it was worth it to you!
If you like this kind of comparative approach, and want more, check out the posts by Yan
Cui, who is doing something similar on his blog.
Enjoy the rest of the F# Advent Calendar. Happy Holidays!
The source code for this post is available on github.

1032

Thirteen ways of looking at a turtle - addendum

Thirteen ways of looking at a turtle addendum

In this, the third part of my two-part mega-post, I'm continuing to stretch the simple turtle
graphics model to the breaking point.
In the first and second post, I described thirteen different ways of looking at a turtle graphics
implementation.
Unfortunately, after I published them, I realized that there were some other ways that I had
forgotten to mention. So in this post, you'll get to see two BONUS ways.
Way 14. Abstract Data Turtle, in which we encapsulate the details of a turtle
implementation by using an Abstract Data Type.
Way 15. Capability-based Turtle, in which we control what turtle functions are available
to a client, based on the current state of the turtle.
As a reminder, here were the previous thirteen ways:
Way 1. A basic object-oriented approach, in which we create a class with mutable state.
Way 2. A basic functional approach, in which we create a module of functions with
immutable state.
Way 3. An API with a object-oriented core, in which we create an object-oriented API
that calls a stateful core class.
Way 4. An API with a functional core, in which we create an stateful API that uses
stateless core functions.
Way 5. An API in front of an agent, in which we create an API that uses a message
queue to communicate with an agent.
Way 6. Dependency injection using interfaces, in which we decouple the
implementation from the API using an interface or record of functions.
Way 7. Dependency injection using functions, in which we decouple the implementation
from the API by passing a function parameter.
Way 8. Batch processing using a state monad, in which we create a special "turtle
workflow" computation expression to track state for us.
Way 9. Batch processing using command objects, in which we create a type to
represent a turtle command, and then process a list of commands all at once.
Interlude: Conscious decoupling with data types. A few notes on using data vs.
interfaces for decoupling.
Way 10. Event sourcing, in which state is built from a list of past events.
Way 11. Functional Retroactive Programming (stream processing), in which business

1033

Thirteen ways of looking at a turtle - addendum

logic is based on reacting to earlier events.

Episode V: The Turtle Strikes Back, in which the turtle API changes so that some
commands may fail.
Way 12. Monadic control flow, in which we make decisions in the turtle workflow based
on results from earlier commands.
Way 13. A turtle interpreter, in which we completely decouple turtle programming from
turtle implementation, and nearly encounter the free monad.
Review of all the techniques used.
All source code for this post is available on github.

14: Abstract Data Turtle

In this design, we use the concept of an abstract data type to encapsulate the operations on
a turtle.
That is, a "turtle" is defined as an opaque type along with a corresponding set of operations,
in the same way that standard F# types such as List , Set and Map are defined.
That is, we have number of functions that work on the type, but we are not allowed to see
"inside" the type itself.
In a sense, you can think of it as a third alternative to the OO approach in way 1 and the
functional approach in way 2.
In the OO implementation, the details of the internals are nicely encapsulated, and
access is only via methods. The downside of the OO class is that it is mutable.
In the FP implementation, the TurtleState is immutable, but the downside is that the
internals of the state are public, and some clients may have accessed these fields, so if
we ever change the design of TurtleState , these clients may break.
The abstract data type implementation combines the best of both worlds: the turtle state is
immutable, as in the original FP way, but no client can access it, as in the OO way.
The design for this (and for any abstract type) is as follows:
The turtle state type itself is public, but its constructor and fields are private.
The functions in the associated Turtle module can see inside the turtle state type (and
so are unchanged from the FP design).
Because the turtle state constructor is private, we need a constructor function in the
Turtle module.

The client can not see inside the turtle state type, and so must rely entirely on the

1034

Thirteen ways of looking at a turtle - addendum

Turtle module functions.

That's all there is to it. We only need to add some privacy modifiers to the earlier FP version
and we are done!

The implementation
First, we are going to put both the turtle state type and the Turtle module inside a common
module called AdtTurtle . This allows the turtle state to be accessible to the functions in the
AdtTurtle.Turtle module, while being inaccessible outside the AdtTurtle .

Next, the turtle state type is going to be called Turtle now, rather than TurtleState ,
because we are treating it almost as an object.
Finally, the associated module Turtle (that contains the functions) is going have some
special attributes:
RequireQualifiedAccess means the module name must be used when accessing the

functions (just like List module)

ModuleSuffix is needed so the that module can have the same name as the state type.

This would not be required for generic types (e.g if we had Turtle<'a> instead).
module AdtTurtle =
/// A private structure representing the turtle
type Turtle = private {
position : Position
angle : float<Degrees>
color : PenColor
penState : PenState
}
/// Functions for manipulating a turtle
/// "RequireQualifiedAccess" means the module name *must*
/// be used (just like List module)
/// "ModuleSuffix" is needed so the that module can
/// have the same name as the state type
[<RequireQualifiedAccess>]
[<CompilationRepresentation (CompilationRepresentationFlags.ModuleSuffix)>]
module Turtle =

An alternative way to avoid collisions is to have the state type have a different case, or a
different name with a lowercase alias, like this:

1035

Thirteen ways of looking at a turtle - addendum

type TurtleState = { ... }

type turtle = TurtleState
module Turtle =
let something (t:turtle) = t

No matter how the naming is done, we will need a way to construct a new Turtle .
If there are no parameters to the constructor, and the state is immutable, then we just need
an initial value rather than a function (like Set.empty say).
Otherwise we can define a function called make (or create or similar):
[<RequireQualifiedAccess>]
[<CompilationRepresentation (CompilationRepresentationFlags.ModuleSuffix)>]
module Turtle =
/// return a new turtle with the specified color
let make(initialColor) = {
position = initialPosition
angle = 0.0<Degrees>
color = initialColor
penState = initialPenState
}

The rest of the turtle module functions are unchanged from their implementation in way 2.

An ADT client
Let's look the client now.
First, let's check that the state really is private. If we try to create a state explicitly, as shown
below, we get a compiler error:
let initialTurtle = {
position = initialPosition
angle = 0.0<Degrees>
color = initialColor
penState = initialPenState
}
// Compiler error FS1093:
// The union cases or fields of the type 'Turtle'
// are not accessible from this code location

If we use the constructor and then try to directly access a field directly (such as position ),
we again get a compiler error:

1036

Thirteen ways of looking at a turtle - addendum

let turtle = Turtle.make(Red)

printfn "%A" turtle.position
// Compiler error FS1093:
// The union cases or fields of the type 'Turtle'
// are not accessible from this code location

But if we stick to the functions in the Turtle module, we can safely create a state value and
then call functions on it, just as we did before:
// versions with log baked in (via partial application)
let move = Turtle.move log
let turn = Turtle.turn log
// etc
let drawTriangle() =
Turtle.make(Red)
|> move 100.0
|> turn 120.0<Degrees>
|> move 100.0
|> turn 120.0<Degrees>
|> move 100.0
|> turn 120.0<Degrees>

Advantages and disadvantages of ADTs

Advantages
All code is stateless, hence easy to test.
The encapsulation of the state means that the focus is always fully on the behavior and
properties of the type.
Clients can never have a dependency on a particular implementation, which means that
implementations can be changed safely.
You can even swap implementations (e.g. by shadowing, or linking to a different
assembly) for testing, performance, etc.
Disadvantages
The client has to manage the current turtle state.
The client has no control over the implementation (e.g. by using dependency injection).
For more on ADTs in F#, see this talk and thread by Bryan Edds.
The source code for this version is available here.

1037

Thirteen ways of looking at a turtle - addendum

15: Capability-based Turtle

In the "monadic control flow" approach (way 12) we handled responses from the turtle telling
us that it had hit a barrier.
But even though we had hit a barrier, nothing was stopping us from calling the move
operation over and over again!
Now imagine that, once we had hit the barrier, the move operation was no longer available
to us. We couldn't abuse it because it would be no longer there!
To make this work, we shouldn't provide an API, but instead, after each call, return a list of
functions that the client can call to do the next step. The functions would normally include
the usual suspects of move , turn , penUp , etc., but when we hit a barrier, move would be
dropped from that list. Simple, but effective.
This technique is closely related to an authorization and security technique called capabilitybased security. If you are interested in learning more, I have a whole series of posts devoted
to it.

Designing a capability-based Turtle

The first thing is to define the record of functions that will be returned after each call:
type MoveResponse =
| MoveOk
| HitABarrier
type SetColorResponse =
| ColorOk
| OutOfInk
type TurtleFunctions = {
move : MoveFn option
turn : TurnFn
penUp : PenUpDownFn
penDown : PenUpDownFn
setBlack : SetColorFn option
setBlue : SetColorFn option
setRed : SetColorFn option
}
and MoveFn = Distance -> (MoveResponse * TurtleFunctions)
and TurnFn = Angle -> TurtleFunctions
and PenUpDownFn = unit -> TurtleFunctions
and SetColorFn = unit -> (SetColorResponse * TurtleFunctions)

Let's look at these declarations in detail.

1038

Thirteen ways of looking at a turtle - addendum

First, there is no TurtleState anywhere. The published turtle functions will encapsulate the
state for us. Similarly there is no log function.
Next, the record of functions TurtleFunctions defines a field for each function in the API
( move , turn , etc.):
The move function is optional, meaning that it might not be available.
The turn , penUp and penDown functions are always available.
The setColor operation has been broken out into three separate functions, one for
each color, because you might not be able to use red ink, but still be able to use blue
ink. To indicate that these functions might not be available, option is used again.
We have also declared type aliases for each function to make them easier to work. Writing
MoveFn is easier than writing Distance -> (MoveResponse * TurtleFunctions) everywhere!

Note that, since these definitions are mutually recursive, I was forced to use the and
keyword.
Finally, note the difference between the signature of MoveFn in this design and the signature
of move in the earlier design of way 12.
Earlier version:
val move :
Log -> Distance -> TurtleState -> (MoveResponse * TurtleState)

New version:
val move :
Distance -> (MoveResponse * TurtleFunctions)

On the input side, the Log and TurtleState parameters are gone, and on the output side,
the TurtleState has been replaced with TurtleFunctions .
This means that somehow, the output of every API function must be changed to be a
TurtleFunctions record.

Implementing the turtle operations

In order to decide whether we can indeed move, or use a particular color, we first need to
augment the TurtleState type to track these factors:

1039

Thirteen ways of looking at a turtle - addendum

type Log = string -> unit

type private TurtleState = {
position : Position
angle : float<Degrees>
color : PenColor
penState : PenState
canMove : bool // new!
availableInk: Set<PenColor> // new!
logger : Log // new!
}

This has been enhanced with

canMove , which if false means that we are at a barrier and should not return a valid
move function.
availableInk contains a set of colors. If a color is not in this set, then we should not

return a valid setColorXXX function for that color.

Finally, we've added the log function into the state so that we don't have to pass it
explicitly to each operation. It will get set once, when the turtle is created.
The TurtleState is getting a bit ugly now, but that's alright, because it's private! The clients
will never even see it.
With this augmented state available, we can change move . First we'll make it private, and
second we'll set the canMove flag (using moveResult <> HitABarrier ) before returning a new
state:
/// Function is private! Only accessible to the client via the TurtleFunctions record
let private move log distance state =
log (sprintf "Move %0.1f" distance)
// calculate new position
let newPosition = calcNewPosition distance state.angle state.position
// adjust the new position if out of bounds
let moveResult, newPosition = checkPosition newPosition
// draw line if needed
if state.penState = Down then
dummyDrawLine log state.position newPosition state.color
// return the new state and the Move result
let newState = {
state with
position = newPosition
canMove = (moveResult <> HitABarrier) // NEW!
}
(moveResult,newState)

1040

Thirteen ways of looking at a turtle - addendum

We need some way of changing canMove back to true! So let's assume that if you turn, you
can move again.
Let's add that logic to the turn function then:
let private turn log angle state =
log (sprintf "Turn %0.1f" angle)
// calculate new angle
let newAngle = (state.angle + angle) % 360.0<Degrees>
// NEW!! assume you can always move after turning
let canMove = true
// update the state
{state with angle = newAngle; canMove = canMove}

The penUp and penDown functions are unchanged, other than being made private.
And for the last operation, setColor , we'll remove the ink from the availability set as soon
as it is used just once!
let private setColor log color state =
let colorResult =
if color = Red then OutOfInk else ColorOk
log (sprintf "SetColor %A" color)
// NEW! remove color ink from available inks
let newAvailableInk = state.availableInk |> Set.remove color
// return the new state and the SetColor result
let newState = {state with color = color; availableInk = newAvailableInk}
(colorResult,newState)

Finally we need a function that can create a TurtleFunctions record from the TurtleState .
I'll call it createTurtleFunctions .
Here's the complete code, and I'll discuss it in detail below:
/// Create the TurtleFunctions structure associated with a TurtleState
let rec private createTurtleFunctions state =
let ctf = createTurtleFunctions // alias
// create the move function,
// if the turtle can't move, return None
let move =
// the inner function
let f dist =
let resp, newState = move state.logger dist state
(resp, ctf newState)

1041

Thirteen ways of looking at a turtle - addendum

// return Some of the inner function

// if the turtle can move, or None
if state.canMove then
Some f
else
None
// create the turn function
let turn angle =
let newState = turn state.logger angle state
ctf newState
// create the pen state functions
let penDown() =
let newState = penDown state.logger state
ctf newState
let penUp() =
let newState = penUp state.logger state
ctf newState
// create the set color functions
let setColor color =
// the inner function
let f() =
let resp, newState = setColor state.logger color state
(resp, ctf newState)
// return Some of the inner function
// if that color is available, or None
if state.availableInk |> Set.contains color then
Some f
else
None
let setBlack = setColor Black
let setBlue = setColor Blue
let setRed = setColor Red
// return the structure
{
move = move
turn = turn
penUp = penUp
penDown = penDown
setBlack = setBlack
setBlue = setBlue
setRed = setRed
}

Let's look at how this works.

1042

Thirteen ways of looking at a turtle - addendum

First, note that this function needs the rec keyword attached, as it refers to itself. I've
added a shorter alias ( ctf ) for it as well.
Next, new versions of each of the API functions are created. For example, a new turn
function is defined like this:
let turn angle =
let newState = turn state.logger angle state
ctf newState

This calls the original turn function with the logger and state, and then uses the recursive
call ( ctf ) to convert the new state into the record of functions.
For an optional function like move , it is a bit more complicated. An inner function f is
defined, using the orginal move , and then either f is returned as Some , or None is
returned, depending on whether the state.canMove flag is set:
// create the move function,
// if the turtle can't move, return None
let move =
// the inner function
let f dist =
let resp, newState = move state.logger dist state
(resp, ctf newState)
// return Some of the inner function
// if the turtle can move, or None
if state.canMove then
Some f
else
None

Similarly, for setColor , an inner function f is defined and then returned or not depending
on whether the color parameter is in the state.availableInk collection:
let setColor color =
// the inner function
let f() =
let resp, newState = setColor state.logger color state
(resp, ctf newState)
// return Some of the inner function
// if that color is available, or None
if state.availableInk |> Set.contains color then
Some f
else
None

1043

Thirteen ways of looking at a turtle - addendum

Finally, all these functions are added to the record:

// return the structure
{
move = move
turn = turn
penUp = penUp
penDown = penDown
setBlack = setBlack
setBlue = setBlue
setRed = setRed
}

And that's how you build a TurtleFunctions record!

We need one more thing: a constructor to create some initial value of the TurtleFunctions ,
since we no longer have direct access to the API. This is now the ONLY public function
available to the client!
/// Return the initial turtle.
/// This is the ONLY public function!
let make(initialColor, log) =
let state = {
position = initialPosition
angle = 0.0<Degrees>
color = initialColor
penState = initialPenState
canMove = true
availableInk = [Black; Blue; Red] |> Set.ofList
logger = log
}
createTurtleFunctions state

This function bakes in the log function, creates a new state, and then calls
createTurtleFunctions to return a TurtleFunction record for the client to use.

Implementing a client of the capability-based turtle

Let's try using this now. First, let's try to do move 60 and then move 60 again. The second
move should take us to the boundary (at 100), and so at that point the move function should
no longer be available.
First, we create the TurtleFunctions record with Turtle.make . Then we can't just move
immediately, we have to test to see if the move function is available first:

1044

Thirteen ways of looking at a turtle - addendum

let testBoundary() =
let turtleFns = Turtle.make(Red,log)
match turtleFns.move with
| None ->
log "Error: Can't do move 1"
| Some moveFn ->
...

In the last case, the moveFn is available, so we can call it with a distance of 60.
The output of the function is a pair: a MoveResponse type and a new TurtleFunctions
record.
We'll ignore the MoveResponse and check the TurtleFunctions record again to see if we can
do the next move:
let testBoundary() =
let turtleFns = Turtle.make(Red,log)
match turtleFns.move with
| None ->
log "Error: Can't do move 1"
| Some moveFn ->
let (moveResp,turtleFns) = moveFn 60.0
match turtleFns.move with
| None ->
log "Error: Can't do move 2"
| Some moveFn ->
...

And finally, one more time:

let testBoundary() =
let turtleFns = Turtle.make(Red,log)
match turtleFns.move with
| None ->
log "Error: Can't do move 1"
| Some moveFn ->
let (moveResp,turtleFns) = moveFn 60.0
match turtleFns.move with
| None ->
log "Error: Can't do move 2"
| Some moveFn ->
let (moveResp,turtleFns) = moveFn 60.0
match turtleFns.move with
| None ->
log "Error: Can't do move 3"
| Some moveFn ->
log "Success"

1045

Thirteen ways of looking at a turtle - addendum

If we run this, we get the output:

Move 60.0
...Draw line from (0.0,0.0) to (60.0,0.0) using Red
Move 60.0
...Draw line from (60.0,0.0) to (100.0,0.0) using Red
Error: Can't do move 3

Which shows that indeed, the concept is working!

That nested option matching is really ugly, so let's whip up a quick maybe workflow to make
it look nicer:
type MaybeBuilder() =
member this.Return(x) = Some x
member this.Bind(x,f) = Option.bind f x
member this.Zero() = Some()
let maybe = MaybeBuilder()

And a logging function that we can use inside the workflow:

/// A function that logs and returns Some(),
/// for use in the "maybe" workflow
let logO message =
printfn "%s" message
Some ()

Now we can try setting some colors using the maybe workflow:

1046

Thirteen ways of looking at a turtle - addendum

let testInk() =
maybe {
// create a turtle
let turtleFns = Turtle.make(Black,log)
// attempt to get the "setRed" function
let! setRedFn = turtleFns.setRed
// if so, use it
let (resp,turtleFns) = setRedFn()
// attempt to get the "move" function
let! moveFn = turtleFns.move
// if so, move a distance of 60 with the red ink
let (resp,turtleFns) = moveFn 60.0
// check if the "setRed" function is still available
do! match turtleFns.setRed with
| None ->
logO "Error: Can no longer use Red ink"
| Some _ ->
logO "Success: Can still use Red ink"
// check if the "setBlue" function is still available
do! match turtleFns.setBlue with
| None ->
logO "Error: Can no longer use Blue ink"
| Some _ ->
logO "Success: Can still use Blue ink"
} |> ignore

The output of this is:

SetColor Red
Move 60.0
...Draw line from (0.0,0.0) to (60.0,0.0) using Red
Error: Can no longer use Red ink
Success: Can still use Blue ink

Actually, using a maybe workflow is not a very good idea, because the first failure exits the
workflow! You'd want to come up with something a bit better for real code, but I hope that
you get the idea.

Advantages and disadvantages of a capability-based

approach

1047

Thirteen ways of looking at a turtle - addendum

Advantages
Prevents clients from abusing the API.
Allows APIs to evolve (and devolve) without affecting clients. For example, I could
transition to a monochrome-only turtle by hard-coding None for each color function in
the record of functions, after which I could safely remove the setColor implementation.
During this process no client would break! This is similar to the HATEAOS approach for
RESTful web services.
Clients are decoupled from a particular implementation because the record of functions
acts as an interface.
Disadvantages
Complex to implement.
The client's logic is much more convoluted as it can never be sure that a function will be
available! It has to check every time.
The API is not easily serializable, unlike some of the data-oriented APIs.
For more on capability-based security, see my posts or watch my "Enterprise Tic-Tac-Toe"
video.
The source code for this version is available here.

Summary
I was of three minds,
Like a finger tree
In which there are three immutable turtles.
-- "Thirteen ways of looking at a turtle", by Wallace D Coriacea
I feel better now that I've got these two extra ways out of my system! Thanks for reading!
The source code for this post is available on github.
.

1048

How to design and code a complete program

How to design and code a complete

program
"I think I understand functional programming at the micro level, and I have written toy
programs, but how do I actually go about writing a complete application, with real
data, real error handling, and so on?"
This is a very common question, so I thought that in this series of posts I'd describe a recipe
for doing exactly this, covering design, validation, error handling, persistence, dependency
management, code organization, and so on.
Some comments and caveats first:
I'll focus on just a single use case rather than a whole application. I hope that it will be
obvious how to extend the code as needed.
This will deliberately be a very simple data-flow oriented recipe with no special tricks or
advanced techniques. But if you are just getting started, I think it is useful to have some
straightforward steps you can follow to get a predictable result. I don't claim that this is
the one true way of doing this. Different scenarios will need different recipes, and of
course, as you get more expert, you may find this recipe too simplistic and limited.
To help ease the transition from object-oriented design, I will try to use familiar concepts
such as "patterns", "services", "dependency injection", and so on, and explain how they
map to functional concepts.
This recipe is also deliberately somewhat imperative, that is, it uses an explicit step-bystep workflow. I hope this approach will ease the transition from OO to FP.
To keep things simple (and usable from a simple F# script), I'll mock the entire
infrastructure and avoid the UI directly.

Overview
Here's an overview of what I plan to cover in this series:
Converting a use-case into a function. In this first post, we'll examine a simple use
case and see how it might be implemented using a functional approach.
Connecting smaller functions together. In the next post, we'll discuss a simple
metaphor for combining smaller functions into bigger functions.
Type driven design and failure types. In the third post, we'll build the types needed for
the use case, and discuss the use of special error types for the failure path.
Configuration and dependency management. In this post, we'll talk about how to

1049

How to design and code a complete program

wire up all the functions.

Validation. In this post, we'll discuss various ways of implementing validation, and
converting from the unsafe outside world to the warm fuzzy world of type safety.
Infrastructure. In this post, we'll discuss various infrastructure components, such as
logging, working with external code, and so on.
The domain layer. In this post, we'll discuss how domain driven design works in a
functional environment.
The presentation layer. In this post, we'll discuss how to convey results and errors
back to the UI.
Dealing with changing requirements. In this post, we'll discuss how to deal with
changing requirements and the effect this has on the code.

Getting started
Let's pick a very simple use case, namely updating some customer information via a web
service.
So here are the basic requirements:
A user submits some data (userid, name and email).
We check to see that the name and email are valid.
The appropriate user record in a database is updated with the new name and email.
If the email has changed, send a verification email to that address.
Display the result of the operation to the user.
This is a typical data centric use case. There is some sort of request that triggers the use
case, and then the request data "flows" through the system, being processed by each step
in turn. This kind of scenario is common in enterprise software, which is why I am using it as
an example.
Here's a diagram of the various components:

1050

How to design and code a complete program

But this describes the "happy path" only. Reality is never so simple! What happens if the
userid is not found in the database, or the email address is not valid, or the database has an
error?
Let's update the diagram to show all the things that could go wrong.

At each step in the use case, various things could cause errors, as shown. Explaining how to
handle these errors in an elegant way will be one of the goals of this series.

Thinking functionally
So now that we understand the steps in the use case, how do we design a solution using a
functional approach?

1051

How to design and code a complete program

First of all, we have to address a mismatch between the original use case and functional
thinking.
In the use case, we typically think of a request/response model. The request is sent, and the
response comes back. If something goes wrong, the flow is short-circuited and response is
returned "early".
Here's a diagram showing what I mean, based on a simplified version of the use case:

But in the functional model, a function is a black box with an input and an output, like this:

How can we adapt the use case to fit this model?

Forward flow only

First, you must recognize that functional data flow is forward only. You cannot do a U-turn or
return early.
In our case, that means that all the errors must be sent forward to the end, as an alternative
path to the happy path.

Once we have done that, we can convert the whole flow into a single "black box" function
like this:

1052

How to design and code a complete program

But of course, if you look inside the big function, it is made up of ("composed from" in
functional-speak) smaller functions, one for each step, joined in a pipeline.

Error handling
In that last diagram, there is one success output and three error outputs. This is a problem,
as functions can have only one output, not four!
How can we handle this?
The answer is to use a union type, with one case to represent each of the different possible
outputs. And then the function as a whole would indeed only have a single output.
Here's an example of a possible type definition for the output:
type UseCaseResult =
| Success
| ValidationError
| UpdateError
| SmtpError

And here's the diagram reworked to show a single output with four different cases
embedded in it:

Simplifying the error handling

This does solve the problem, but having one error case for each step in the flow is brittle and
not very reusable. Can we do better?
Yes! All we really need is two cases. One for the happy path, and one for all other error
paths, like this:

1053

How to design and code a complete program

type UseCaseResult =
| Success
| Failure

This type is very generic and will work with any workflow! In fact, you'll soon see that we can
create a nice library of useful functions that will work with this type, and which can be reused
for all sorts of scenarios.
One more thing though -- as it stands there is no data in the result at all, just a
success/failure status. We need to tweak it a bit so that it can contain an actual success or
failure object. We will specify the success type and failure type using generics (a.k.a. type
parameters).
Here's the final, completely generic and reusable version:
type Result<'TSuccess,'TFailure> =
| Success of 'TSuccess
| Failure of 'TFailure

In fact, there is already a type almost exactly like this defined in the F# library. It's called
Choice. For clarity though, I will continue to use the Result type defined above for this and
the next post. When we come to some more serious coding, we'll revisit this.
So, now, showing the individual steps again, we can see that we will have to combine the
errors from each step onto to a single "failure" path.

How to do this will be the topic of the next post.

Summary and guidelines

So far then, we have the following guidelines for the recipe:
Guidelines
Each use case will be equivalent to a single function

1054

How to design and code a complete program

The use case function will return a union type with two cases: Success and Failure .
The use case function will be built from a series of smaller functions, each representing
one step in a data flow.
The errors from each step will be combined into a single error path.

1055

A functional approach to error handling (Railway oriented programming)

Railway oriented programming

UPDATE: Slides and video from a more comprehensive presentation available here (and if
you understand the Either monad, read this first!)
In the previous post, we saw how a use case could be broken into steps, and all the errors
shunted off onto a separate error track, like this:

In this post, we'll look at various ways of connecting these step functions into a single unit.
The detailed internal design of the functions will be described in a later post.

Designing a function that represents a step

Let's have a closer look at these steps. For example, consider the validation function. How
would it work? Some data goes in, but what comes out?
Well, there are two possible cases: either the data is valid (the happy path), or something is
wrong, in which case we go onto the failure path and bypass the rest of the steps, like this:

But as before, this would not be a valid function. A function can only have one output, so we
must use the Result type we defined last time:
type Result<'TSuccess,'TFailure> =
| Success of 'TSuccess
| Failure of 'TFailure

And the diagram now looks like this:

1056

A functional approach to error handling (Railway oriented programming)

To show you how this works in practice, here is an example of what an actual validation
function might look like:
type Request = {name:string; email:string}
let validateInput input =
if input.name = "" then Failure "Name must not be blank"
else if input.email = "" then Failure "Email must not be blank"
else Success input // happy path

If you look at the type of the function, the compiler has deduced that it takes a Request and
spits out a Result as output, with a Request for the success case and a string for the
failure case:
validateInput : Request -> Result<Request,string>

We can analyze the other steps in the flow in the same way. We will find that each one will
have the same "shape" -- some sort of input and then this Success/Failure output.
A pre-emptive apology: Having just said that a function can't have two outputs, I may
occasionally refer to them hereafter as "two output" functions! Of course, what I mean is that
the shape of the function output has two cases.

Railway oriented programming

So we have a lot of these "one input -> Success/Failure output" functions -- how do we
connect them together?
What we want to do is connect the Success output of one to the input of the next, but
somehow bypass the second function in case of a Failure output. This diagram gives the
general idea:

There is a great analogy for doing this -- something you are probably already familiar with.
Railways!
Railways have switches ("points" in the UK) for directing trains onto a different track. We can
think of these "Success/Failure" functions as railway switches, like this:

1057

A functional approach to error handling (Railway oriented programming)

And here we have two in a row.

How do we combine them so that both failure tracks are connected? It's obvious -- like this!

And if we have a whole series of switches, we will end up with a two track system, looking
something like this:

The top track is the happy path, and the bottom track is the failure path.
Now stepping back and looking at the big picture, we can see that we will have a series of
black box functions that appear to be straddling a two-track railway, each function
processing data and passing it down the track to the next function:

But if we look inside the functions, we can see that there is actually a switch inside each
one, for shunting bad data onto the failure track:

Note that once we get on the failure path, we never (normally) get back onto the happy path.
We just bypass the rest of the functions until we reach the end.

1058

A functional approach to error handling (Railway oriented programming)

Basic composition
Before we discuss how to "glue" the step functions together, let's review how composition
works.
Imagine that a standard function is a black box (a tunnel, say) sitting on a one-track railway.
It has one input and one output.
If we want to connect a series of these one-track functions, we can use the left-to-right
composition operator, with the symbol >> .

The same composition operation also works with two-track functions as well:

The only constraint on composition is that the output type of the left-hand function has to
match the input type of the right-hand function.
In our railway analogy, this means that you can connect one-track output to one-track input,
or two-track output to two-track input, but you can't directly connect two-track output to onetrack input.

Converting switches to two-track inputs

So now we have run into a problem.
The function for each step is going to be a switch, with one input track. But the overall flow
requires a two-track system, with each function straddling both tracks, meaning that each
function must have a two-track input (the Result output by the previous function), not just a
simple one-track input ( Request ).
How can we insert the switches into the two track system?

1059

A functional approach to error handling (Railway oriented programming)

The answer is simple. We can create an "adapter" function that has a "hole" or "slot" for a
switch function and converts it into a proper two-track function. Here's an illustration:

And here's what the actual code looks like. I'm going to name the adapter function bind ,
which is the standard name for it.
let bind switchFunction =
fun twoTrackInput ->
match twoTrackInput with
| Success s -> switchFunction s
| Failure f -> Failure f

The bind function takes a switch function as a parameter and returns a new function. The
new function takes a two-track input (which is of type Result ) and then checks each case.
If the input is a Success it calls the switchFunction with the value. But if the input is a
Failure , then the switch function is bypassed.

Compile it and then look at the function signature:

val bind : ('a -> Result<'b,'c>) -> Result<'a,'c> -> Result<'b,'c>

One way of interpreting this signature is that the bind function has one parameter, a switch
function ( 'a -> Result<..> ) and it returns a fully two-track function ( Result<..> ->
Result<..> ) as output.

To be even more specific:

The parameter ( switchFunction ) of bind takes some type 'a and emits a Result of
type 'b (for the success track) and 'c (for the failure track)
The returned function itself has a parameter ( twoTrackInput ) which is a Result of type
'a (for success) and 'c (for failure). The type 'a has to be the same as what the
switchFunction is expecting on its one track.

The output of the returned function is another Result , this time of type 'b (for
success) and 'c (for failure) -- the same type as the switch function output.
If you think about it, this type signature is exactly what we would expect.

1060

A functional approach to error handling (Railway oriented programming)

Note that this function is completely generic -- it will work with any switch function and any
types. All it cares about is the "shape" of the switchFunction , not the actual types involved.

Other ways of writing the bind function

Just as an aside, there are some other ways of writing functions like this.
One way is to use an explicit second parameter for the twoTrackInput rather than defining
an internal function, like this:
let bind switchFunction twoTrackInput =
match twoTrackInput with
| Success s -> switchFunction s
| Failure f -> Failure f

This is exactly the same as the first definition. And if you are wondering how a two
parameter function can be exactly the same as a one parameter function, you need to read
the post on currying!
Yet another way of writing it is to replace the match..with syntax with the more concise
function keyword, like this:

let bind switchFunction =

function
| Success s -> switchFunction s
| Failure f -> Failure f

You might see all three styles in other code, but I personally prefer to use the second style
( let bind switchFunction twoTrackInput = ), because I think that having explicit parameters
makes the code more readable for non-experts.

Example: Combining some validation functions

Let's write a little bit of code now, to test the concepts.
Let's start with what we already have defined. Request , Result and bind :

1061

A functional approach to error handling (Railway oriented programming)

type Result<'TSuccess,'TFailure> =
| Success of 'TSuccess
| Failure of 'TFailure
type Request = {name:string; email:string}
let bind switchFunction twoTrackInput =
match twoTrackInput with
| Success s -> switchFunction s
| Failure f -> Failure f

Next we'll create three validation functions, each of which is a "switch" function, with the goal
of combining them into one bigger function:
let validate1 input =
if input.name = "" then Failure "Name must not be blank"
else Success input
let validate2 input =
if input.name.Length > 50 then Failure "Name must not be longer than 50 chars"
else Success input
let validate3 input =
if input.email = "" then Failure "Email must not be blank"
else Success input

Now to combine them, we apply bind to each validation function to create a new
alternative function that is two-tracked.
Then we can connect the two-tracked functions using standard function composition, like
this:
/// glue the three validation functions together
let combinedValidation =
// convert from switch to two-track input
let validate2' = bind validate2
let validate3' = bind validate3
// connect the two-tracks together
validate1 >> validate2' >> validate3'

The functions validate2' and validate3' are new functions that take two-track input. If
you look at their signatures you will see that they take a Result and return a Result . But
note that validate1 does not need to be converted to two track input. Its input is left as
one-track, and its output is two-track already, as needed for composition to work.

1062

A functional approach to error handling (Railway oriented programming)

Here's a diagram showing the Validate1 switch (unbound) and the Validate2 and
Validate3 switches, together with the Validate2' and Validate3' adapters.

We could have also "inlined" the bind , like this:

let combinedValidation =
// connect the two-tracks together
validate1
>> bind validate2
>> bind validate3

Let's test it with two bad inputs and a good input:

// test 1
let input1 = {name=""; email=""}
combinedValidation input1
|> printfn "Result1=%A"
// ==> Result1=Failure "Name must not be blank"
// test 2
let input2 = {name="Alice"; email=""}
combinedValidation input2
|> printfn "Result2=%A"
// ==> Result2=Failure "Email must not be blank"
// test 3
let input3 = {name="Alice"; email="good"}
combinedValidation input3
|> printfn "Result3=%A"
// ==> Result3=Success {name = "Alice"; email = "good";}

I would encourage you to try it for yourself and play around with the validation functions and
test input.
You might be wondering if there is a way to run all three validations in parallel, rather than
serially, so that you can get back all the validation errors at once. Yes, there is a way, which
I'll explain later in this post.

Bind as a piping operation

1063

A functional approach to error handling (Railway oriented programming)

While we are discussing the bind function, there is a common symbol for it, >>= , which is
used to pipe values into switch functions.
Here's the definition, which switches around the two parameters to make them easier to
chain together:
/// create an infix operator
let (>>=) twoTrackInput switchFunction =
bind switchFunction twoTrackInput

One way to remember the symbol is to think of it as the composition symbol, >> , followed
by a two-track railway symbol, = .
When used like this, the >>= operator is sort of like a pipe ( |> ) but for switch functions.
In a normal pipe, the left hand side is a one-track value, and the right hand value is a normal
function. But in a "bind pipe" operation, the left hand side is a two-track value, and the right
hand value is a switch function.
Here it is in use to create another implementation of the combinedValidation function.
let combinedValidation x =
x
|> validate1 // normal pipe because validate1 has a one-track input
// but validate1 results in a two track output...
>>= validate2 // ... so use "bind pipe". Again the result is a two track output
>>= validate3 // ... so use "bind pipe" again.

The difference between this implementation and the previous one is that this definition is
data-oriented rather than function-oriented. It has an explicit parameter for the initial data
value, namely x . x is passed to the first function, and then the output of that is passed to
the second function, and so on.
In the previous implementation (repeated below), there was no data parameter at all! The
focus was on the functions themselves, not the data that flows through them.
let combinedValidation =
validate1
>> bind validate2
>> bind validate3

An alternative to bind

1064

A functional approach to error handling (Railway oriented programming)

Another way to combine switches is not by adapting them to a two track input, but simply by
joining them directly together to make a new, bigger switch.
In other words, this:

becomes this:

But if you think about it, this combined track is actually just another switch! You can see this
if you cover up the middle bit. There's one input and two outputs:

So what we have really done is a form of composition for switches, like this:

Because each composition results in just another switch, we can always add another switch
again, resulting in an even bigger thing that is still a switch, and so on.
Here's the code for switch composition. The standard symbol used is >=> , a bit like the
normal composition symbol, but with a railway track between the angles.
let (>=>) switch1 switch2 x =
match switch1 x with
| Success s -> switch2 s
| Failure f -> Failure f

Again, the actual implementation is very straightforward. Pass the single track input x
through the first switch. On success, pass the result into the second switch, otherwise
bypass the second switch completely.

1065

A functional approach to error handling (Railway oriented programming)

Now we can rewrite the combinedValidation function to use switch composition rather than
bind:
let combinedValidation =
validate1
>=> validate2
>=> validate3

This one is the simplest yet, I think. It's very easy to extend of course, if we have a fourth
validation function, we can just append it to the end.

Bind vs. switch composition

We have two different concepts that at first glance seem quite similar. What's the difference?
To recap:
Bind has one switch function parameter. It is an adapter that converts the switch
function into a fully two-track function (with two-track input and two-track output).
Switch composition has two switch function parameters. It combines them in series to
make another switch function.
So why would you use bind rather than switch composition? It depends on the context. If you
have an existing two-track system, and you need to insert a switch, then you have to use
bind as an adapter to convert the switch into something that takes two-track input.

On the other hand, if your entire data flow consists of a chain of switches, then switch
composition can be simpler.

1066

A functional approach to error handling (Railway oriented programming)

Switch composition in terms of bind

As it happens, switch composition can be written in terms of bind. If you connect the first
switch with a bind-adapted second switch, you get the same thing as switch composition:
Here are two separate switches:

And then here are the switches composed together to make a new bigger switch:

And here's the same thing done by using bind on the second switch:

Here's the switch composition operator rewritten using this way of thinking:
let (>=>) switch1 switch2 =
switch1 >> (bind switch2)

This implementation of switch composition is much simpler than the first one, but also more
abstract. Whether it is easier to comprehend for a beginner is another matter! I find that if
you think of functions as things in their own right, rather than just as conduits for data, this
approach becomes easier to understand.

1067

A functional approach to error handling (Railway oriented programming)

Converting simple functions to the railwayoriented programming model

Once you get the hang of it, you can fit all sorts of other things into this model.
For example, let's say we have a function that is not a switch, just a regular function. And
say that we want to insert it into our flow.
Here's a real example - say that we want to trim and lowercase the email address after the
validation is complete. Here's some code to do this:
let canonicalizeEmail input =
{ input with email = input.email.Trim().ToLower() }

This code takes a (single-track) Request and returns a (single-track) Request .

How can we insert this after the validation steps but before the update step?
Well, if we can turn this simple function into a switch function, then we can use the switch
composition we just talked about above.
In other words, we need an adapter block. It the same concept that we used for bind ,
except that this time our adapter block will have a slot for one-track function, and the overall
"shape" of the adapter block is a switch.

The code to do this is trivial. All we need to do is take the output of the one track function
and turn it into a two-track result. In this case, the result will always be Success.
// convert a normal function into a switch
let switch f x =
f x |> Success

In railway terms, we have added a bit of failure track. Taken as a whole, it looks like a switch
function (one-track input, two-track output), but of course, the failure track is just a dummy
and the switch never actually gets used.

1068

A functional approach to error handling (Railway oriented programming)

Once switch is available, we can easily append the canonicalizeEmail function to the end
of the chain. Since we are beginning to extend it, let's rename the function to usecase .
let usecase =
validate1
>=> validate2
>=> validate3
>=> switch canonicalizeEmail

Try testing it to see what happens:

Creating two-track functions from one-track

functions
In the previous example, we took a one-track function and created a switch from it. This
enabled us to use switch composition with it.
Sometimes though, you want to use the two-track model directly, in which case you want to
turn a one-track function into a two-track function directly.

1069

A functional approach to error handling (Railway oriented programming)

Again, we just need an adapter block with a slot for the simple function. We typically call this
adapter map .

And again, the actual implementation is very straightforward. If the two-track input is
Success , call the function, and turn its output into Success. On the other hand, if the two-

track input is Failure bypass the function completely.

Here's the code:
// convert a normal function into a two-track function
let map oneTrackFunction twoTrackInput =
match twoTrackInput with
| Success s -> Success (oneTrackFunction s)
| Failure f -> Failure f

And here it is in use with canonicalizeEmail :

let usecase =
validate1
>=> validate2
>=> validate3
>> map canonicalizeEmail // normal composition

Note that normal composition is now used because map canonicalizeEmail is a fully twotrack function and can be connected to the output of the validate3 switch directly.
In other words, for one-track functions, >=> switch is exactly the same as >> map . Your
choice.

1070

A functional approach to error handling (Railway oriented programming)

Converting dead-end functions to two-track

functions
Another function we will often want to work with is a "dead-end" function -- a function that
accepts input but has no useful output.
For example, consider a function that updates a database record. It is useful only for its sideeffects -- it doesn't normally return anything.
How can we incorporate this kind of function into the flow?
What we need to do is:
Save a copy of the input.
Call the function and ignore its output, if any.
Return the original input for passing on to the next function in the chain.
From a railway point of view, this is equivalent to creating a dead-end siding, like this.

To make this work, we need another adapter function, like switch , except that this time it
has a slot for one-track dead-end function, and converts it into a single-track pass through
function, with a one-track output.

Here's the code, which I will call tee , after the UNIX tee command:
let tee f x =
f x |> ignore
x

1071

A functional approach to error handling (Railway oriented programming)

Once we have converted the dead-end function to a simple one-track pass through function,
we can then use it in the data flow by converting it using switch or map as described
above.
Here's the code in use with the "switch composition" style:
// a dead-end function
let updateDatabase input =
() // dummy dead-end function for now
let usecase =
validate1
>=> validate2
>=> validate3
>=> switch canonicalizeEmail
>=> switch (tee updateDatabase)

Or alternatively, rather than using switch and then connecting with >=> , we can use map
and connect with >> .
Here's a variant implementation which is exactly the same but uses the "two-track" style with
normal composition
let usecase =
validate1
>> bind validate2
>> bind validate3
>> map canonicalizeEmail
>> map (tee updateDatabase)

Handling exceptions
Our dead end database update might not return anything, but that doesn't mean that it might
not throw an exception. Rather than crashing, we want to catch that exception and turn it
into a failure.
The code is similar to the switch function, except that it catches exceptions. I'll call it
tryCatch :

let tryCatch f x =
try
f x |> Success
with
| ex -> Failure ex.Message

1072

A functional approach to error handling (Railway oriented programming)

And here is a modified version of the data flow, using tryCatch rather than switch for the
update database code.
let usecase =
validate1
>=> validate2
>=> validate3
>=> switch canonicalizeEmail
>=> tryCatch (tee updateDatabase)

Functions with two-track input

All the functions we have seen so far have only one input, because they always just work
with data travelling along the happy path.
Sometimes though, you do need a function that handles both tracks. For example, a logging
function that logs errors as well as successes.
As we have done previously, we will create an adapter block, but this time it will have slots
for two separate one-track functions.

Here's the code:

let doubleMap successFunc failureFunc twoTrackInput =
match twoTrackInput with
| Success s -> Success (successFunc s)
| Failure f -> Failure (failureFunc f)

As an aside, we can use this function to create a simpler version of map , using id for the
failure function:
let map successFunc =
doubleMap successFunc id

1073

A functional approach to error handling (Railway oriented programming)

Let's use doubleMap to insert some logging into the data flow:
let log twoTrackInput =
let success x = printfn "DEBUG. Success so far: %A" x; x
let failure x = printfn "ERROR. %A" x; x
doubleMap success failure twoTrackInput
let usecase =
validate1
>=> validate2
>=> validate3
>=> switch canonicalizeEmail
>=> tryCatch (tee updateDatabase)
>> log

Here's some test code, with the results:

Converting a single value to a two-track value

For completeness, we should also create simple functions that turn a single simple value
into a two-track value, either success or failure.
let succeed x =
Success x
let fail x =
Failure x

Right now these are trivial, just calling the constructor of the Result type, but when we get
down to some proper coding we'll see that by using these rather than the union case
constructor directly, we can isolate ourselves from changes behind the scenes.

1074

A functional approach to error handling (Railway oriented programming)

Combining functions in parallel

So far, we have combined functions in series. But with something like validation, we might
want to run multiple switches in parallel, and combine the results, like this:

To make this easier, we can reuse the same trick that we did for switch composition. Rather
than doing many at once, if we just focus on a single pair, and "add" them to make a new
switch, we can then easily chain the "addition" together so that we can add as many as we
want. In other words, we just need to implement this:

So, what is the logic for adding two switches in parallel?

First, take the input and apply it to each switch.
Next look at the outputs of both switches, and if both are successful, the overall result is
Success .

If either output is a failure, then the overall result is Failure as well.

Here's the function, which I will call plus :

1075

A functional approach to error handling (Railway oriented programming)

let plus switch1 switch2 x =

match (switch1 x),(switch2 x) with
| Success s1,Success s2 -> Success (s1 + s2)
| Failure f1,Success _ -> Failure f1
| Success _ ,Failure f2 -> Failure f2
| Failure f1,Failure f2 -> Failure (f1 + f2)

But we now have a new problem. What do we do with two successes, or two failures? How
do we combine the inner values?
I used s1 + s2 and f1 + f2 in the example above, but that implies that there is some sort
of + operator we can use. That may be true for strings and ints, but it is not true in general.
The method of combining values might change in different contexts, so rather than trying to
solve it once and for all, let's punt by letting the caller pass in the functions that are needed.
Here's a rewritten version:
let plus addSuccess addFailure switch1 switch2 x =
match (switch1 x),(switch2 x) with
| Success s1,Success s2 -> Success (addSuccess s1 s2)
| Failure f1,Success _ -> Failure f1
| Success _ ,Failure f2 -> Failure f2
| Failure f1,Failure f2 -> Failure (addFailure f1 f2)

I have put these new functions first in the parameter list, to aid partial application.

An implementation for parallel validation

Now let's create a implementation of "plus" for the validation functions.
When both functions succeed, they will return the request unchanged, so the
addSuccess function can return either parameter.

When both functions fail, they will return different strings, so the addFailure function
should concatenate them.
For validation then, the "plus" operation that we want is like an "AND" function. Only if both
parts are "true" is the result "true".
That naturally leads to wanting to use && as the operator symbol. Unfortunately, && is
reserved, but we can use &&& , like this:

1076

A functional approach to error handling (Railway oriented programming)

// create a "plus" function for validation functions

let (&&&) v1 v2 =
let addSuccess r1 r2 = r1 // return first
let addFailure s1 s2 = s1 + "; " + s2 // concat
plus addSuccess addFailure v1 v2

And now using &&& , we can create a single validation function that combines the three
smaller validations:
let combinedValidation =
validate1
&&& validate2
&&& validate3

Now let's try it with the same tests we had earlier:

// test 1
let input1 = {name=""; email=""}
combinedValidation input1
|> printfn "Result1=%A"
// ==> Result1=Failure "Name must not be blank; Email must not be blank"
// test 2
let input2 = {name="Alice"; email=""}
combinedValidation input2
|> printfn "Result2=%A"
// ==> Result2=Failure "Email must not be blank"
// test 3
let input3 = {name="Alice"; email="good"}
combinedValidation input3
|> printfn "Result3=%A"
// ==> Result3=Success {name = "Alice"; email = "good";}

The first test now has two validation errors combined into a single string, just as we wanted.
Next, we can tidy up the main dataflow function by using the usecase function now instead
of the three separate validation functions we had before:
let usecase =
combinedValidation
>=> switch canonicalizeEmail
>=> tryCatch (tee updateDatabase)

And if we test that now, we can see that a success flows all the way to the end and that the
email is lowercased and trimmed:

1077

A functional approach to error handling (Railway oriented programming)

// test 4
let input4 = {name="Alice"; email="UPPERCASE "}
usecase input4
|> printfn "Result4=%A"
// ==> Result4=Success {name = "Alice"; email = "uppercase";}

You might be asking, can we create a way of OR-ing validation functions as well? That is,
the overall result is valid if either part is valid? The answer is yes, of course. Try it! I suggest
that you use the symbol ||| for this.

Dynamic injection of functions

Another thing we might want to do is add or remove functions into the flow dynamically,
based on configuration settings, or even the content of the data.
The simplest way to do this is to create a two-track function to be injected into the stream,
and replace it with the id function if not needed.
Here's the idea:
let injectableFunction =
if config.debug then debugLogger else id

Let's try it with some real code:

type Config = {debug:bool}
let debugLogger twoTrackInput =
let success x = printfn "DEBUG. Success so far: %A" x; x
let failure = id // don't log here
doubleMap success failure twoTrackInput
let injectableLogger config =
if config.debug then debugLogger else id
let usecase config =
combinedValidation
>> map canonicalizeEmail
>> injectableLogger config

And here is it in use:

1078

A functional approach to error handling (Railway oriented programming)

let input = {name="Alice"; email="good"}

let releaseConfig = {debug=false}
input
|> usecase releaseConfig
|> ignore
// no output
let debugConfig = {debug=true}
input
|> usecase debugConfig
|> ignore
// debug output
// DEBUG. Success so far: {name = "Alice"; email = "good";}

The railway track functions: A toolkit

Let's step back and review what we have done so far.
Using railway track as a metaphor, we have created a number of useful building blocks that
will work with any data-flow style application.
We can classify our functions roughly like this:
"constructors" are used to create new track.
"adapters" convert one kind of track into another.
"combiners" link sections of track together to make a bigger piece of track.
These functions form what can be loosely called a combinator library, that is, a group of
functions that are designed to work with a type (here represented by railway track), with the
design goal that bigger pieces can be built by adapting and combining smaller pieces.
Functions like bind , map , plus , etc., crop up in all sorts of functional programming
scenarios, and so you can think of them as functional patterns -- similar to, but not the same
as, the OO patterns such as "visitor", "singleton", "facade", etc.
Here they all are together:

1079

A functional approach to error handling (Railway oriented programming)

Concept

Description

succeed

A constructor that takes a one-track value and creates a two-track value

on the Success branch. In other contexts, this might also be called
return or pure .

fail

A constructor that takes a one-track value and creates a two-track value

on the Failure branch.

bind

An adapter that takes a switch function and creates a new function that
accepts two-track values as input.

>>=

An infix version of bind for piping two-track values into switch functions.

Normal composition. A combiner that takes two normal functions and

creates a new function by connecting them in series.

>=>

Switch composition. A combiner that takes two switch functions and

creates a new switch function by connecting them in series.

switch

An adapter that takes a normal one-track function and turns it into a

switch function. (Also known as a "lift" in some contexts.)

map

An adapter that takes a normal one-track function and turns it into a twotrack function. (Also known as a "lift" in some contexts.)

tee

An adapter that takes a dead-end function and turns it into a one-track

function that can be used in a data flow. (Also known as tap .)

tryCatch

An adapter that takes a normal one-track function and turns it into a

switch function, but also catches exceptions.

doubleMap

An adapter that takes two one-track functions and turns them into a
single two-track function. (Also known as bimap .)

plus

A combiner that takes two switch functions and creates a new switch
function by joining them in "parallel" and "adding" the results. (Also
known as ++ and in other contexts.)

&&&

The "plus" combiner tweaked specifically for the validation functions,

modelled on a binary AND.

The railway track functions: complete code

Here is the complete code for all the functions in one place.
I have made some minor tweaks from the original code presented above:
Most functions are now defined in terms of a core function called either .
tryCatch has been given an extra parameter for the exception handler.

// the two-track type

type Result<'TSuccess,'TFailure> =
| Success of 'TSuccess

1080

A functional approach to error handling (Railway oriented programming)

| Failure of 'TFailure
// convert a single value into a two-track result
let succeed x =
Success x
// convert a single value into a two-track result
let fail x =
Failure x
// apply either a success function or failure function
let either successFunc failureFunc twoTrackInput =
match twoTrackInput with
| Success s -> successFunc s
| Failure f -> failureFunc f
// convert a switch function into a two-track function
let bind f =
either f fail
// pipe a two-track value into a switch function
let (>>=) x f =
bind f x
// compose two switches into another switch
let (>=>) s1 s2 =
s1 >> bind s2
// convert a one-track function into a switch
let switch f =
f >> succeed
// convert a one-track function into a two-track function
let map f =
either (f >> succeed) fail
// convert a dead-end function into a one-track function
let tee f x =
f x; x
// convert a one-track function into a switch with exception handling
let tryCatch f exnHandler x =
try
f x |> succeed
with
| ex -> exnHandler ex |> fail
// convert two one-track functions into a two-track function
let doubleMap successFunc failureFunc =
either (successFunc >> succeed) (failureFunc >> fail)
// add two switches in parallel
let plus addSuccess addFailure switch1 switch2 x =

1081

A functional approach to error handling (Railway oriented programming)

match (switch1 x),(switch2 x) with

| Success s1,Success s2 -> Success (addSuccess s1 s2)
| Failure f1,Success _ -> Failure f1
| Success _ ,Failure f2 -> Failure f2
| Failure f1,Failure f2 -> Failure (addFailure f1 f2)

Types vs. shapes

So far, we have focused entirely on the shape of the track, not the cargo on the trains.
This is a magical railway, where the goods being carried can change as they go along each
length of track.
For example, a cargo of pineapples will magically transform into apples when it goes through
the tunnel called function1 .

And a cargo of apples will transform into bananas when it goes through the tunnel called
function2 .

This magical railway has an important rule, namely that you can only connect tracks which
carry the same type of cargo. In this case we can connect function1 to function2
because the cargo coming out of function1 (apples) is the same as the cargo going into
function2 (also apples).

Of course, it is not always true that the tracks carry the same cargo, and a mismatch in the
kind of cargo will cause an error.
But you'll notice that in this discussion so far, we haven't mentioned the cargo once! Instead,
we have spent all our time talking about one-track vs. two track functions.

1082

A functional approach to error handling (Railway oriented programming)

Of course, it goes without saying that the cargo must match up. But I hope you can see that
it is the shape of the track that is really the important thing, not the cargo that is carried.

Generic types are powerful

Why have we not worried about the type of cargo? Because all the "adapter" and "combiner"
functions are completely generic! The bind and map and switch and plus functions do
not care about the type of the cargo, only the shape of the track.
Having extremely generic functions is a benefit in two ways. The first way is obvious: the
more generic a function is, the more reusable it is. The implementation of bind will work
with any types (as long as the shape is right).
But there is another, more subtle aspect of generic functions that is worth pointing out.
Because we generally know nothing about the types involved, we are very constrained in
what we can and can't do. As a result, we can't introduce bugs!
To see what I mean, let's look at the signature for map :
val map : ('a -> 'b) -> (Result<'a,'c> -> Result<'b,'c>)

It takes a function parameter 'a -> 'b and a value Result<'a,'c> and returns a value
Result<'b,'c> .

We don't know anything about the types 'a , 'b , and 'c . The only things we know are
that:
The same type 'a shows up in both the function parameter and the Success case of
the first Result .
The same type 'b shows up in both the function parameter and the Success case of
the second Result .
The same type 'c shows up in the Failure cases of both the first and second
Result s, but doesn't show up in the function parameter at all.

What can we deduce from this?

The return value has a type 'b in it. But where does it come from? We don't know what
type 'b is, so we don't know how to make one. But the function parameter knows how to
make one! Give it an 'a and it will make a 'b for us.
But where can we get an 'a from? We don't know what type 'a is either, so again we
don't know how to make one. But the first result parameter has an 'a we can use, so you
can see that we are forced to get the Success value from the Result<'a,'c> parameter and

1083

A functional approach to error handling (Railway oriented programming)

pass it to the function parameter. And then the Success case of the Result<'b,'c> return
value must be constructed from the result of the function.
Finally, the same logic applies to 'c . We are forced to get the Failure value from the
Result<'a,'c> input parameter and use it to construct the Failure case of the
Result<'a,'c> return value.

In other words, there is basically only one way to implement the map function! The type
signature is so generic that we have no choice.
On the other hand, imagine that the map function had been very specific about the types it
needed, like this:
val map : (int -> int) -> (Result<int,int> -> Result<int,int>)

In this case, we can come up a huge number of different implementations. To list a few:
We could have swapped the success and failure tracks.
We could have added a random number to the success track.
We could have ignored the function parameter altogether, and returned zero on both the
success and failure tracks.
All of these implementations are "buggy" in the sense that they don't do what we expect. But
they are all only possible because we know in advance that the type is int , and therefore
we can manipulate the values in ways we are not supposed to. The less we know about the
types, the less likely we are to make a mistake.

The failure type

In most of our functions, the transformation only applies to the success track. The failure
track is left alone ( map ), or merged with an incoming failure ( bind ).
This implies that the failure track must be same type all the way through. In this post we
have just used string , but in the next post we'll change the failure type to be something
more useful.

Summary and guidelines

At the beginning of this series, I promised to give you a simple recipe that you could follow.
But you might be feeling a bit overwhelmed now. Instead of making things simpler, I seem to
have made things more complicated. I have shown you lots of different ways of doing the
same thing! Bind vs. compose. Map vs. switch. Which approach should you use? Which way

1084

A functional approach to error handling (Railway oriented programming)

is best?
Of course, there is never one "right way" for all scenarios, but nevertheless, as promised,
here are some guidelines that can be used as the basis of a reliable and repeatable recipe.
Guidelines
Use double-track railway as your underlying model for dataflow situations.
Create a function for each step in the use case. The function for each step can in turn
be built from smaller functions (e.g. the validation functions).
Use standard composition ( >> ) to connect the functions.
If you need to insert a switch into the flow, use bind .
If you need to insert a single-track function into the flow, use map .
If you need to insert other types of functions into the flow, create an appropriate adapter
block and use it.
These guidelines may result in code that is not particularly concise or elegant, but on the
other hand, you will be using a consistent model, and it should be understandable to other
people when it needs to be maintained.
So with these guidelines, here are the main bits of the implementation so far. Note especially
the use of >> everywhere in the final usecase function.

1085

A functional approach to error handling (Railway oriented programming)

open RailwayCombinatorModule
let (&&&) v1 v2 =
let addSuccess r1 r2 = r1 // return first
let addFailure s1 s2 = s1 + "; " + s2 // concat
plus addSuccess addFailure v1 v2
let combinedValidation =
validate1
&&& validate2
&&& validate3
let canonicalizeEmail input =
{ input with email = input.email.Trim().ToLower() }
let updateDatabase input =
() // dummy dead-end function for now
// new function to handle exceptions
let updateDatebaseStep =
tryCatch (tee updateDatabase) (fun ex -> ex.Message)
let usecase =
combinedValidation
>> map canonicalizeEmail
>> bind updateDatebaseStep
>> log

One final suggestion. If you are working with a team of non-experts, unfamiliar operator
symbols will put people off. So here some extra guidelines with respect to operators:
Don't use any "strange" operators other than >> and |> .
In particular, that means you should not use operators like >>= or >=> unless
everyone is aware of them.
An exception can be made if you define the operator at the top of the module or function
where it is used. For example, the &&& operator could be defined at the top of the
validation module and then used later in that same module.

Further reading
If you like this "railway oriented" approach, you can also see it applied to FizzBuzz.
I also have some slides and video that show how take this approach further. (At some
point I will turn these into a proper blog post)
I presented on this topic at NDC Oslo 2014 (click image to view video)

1086

A functional approach to error handling (Railway oriented programming)

And here are the slides I used:

Railway Oriented Programming from my slideshare page

1087

Railway oriented programming: Carbonated edition

Railway oriented programming:

Carbonated edition
As a follow up to the Railway Oriented Programming post, I thought I'd apply the same
technique to the FizzBuzz problem, and compare it with other implementations.
A large part of this post was directly stolen from inspired by Dave Fayram's post on
FizzBuzz, with some additional ideas from raganwald.

FizzBuzz: The imperative version

As a reminder, here are the requirements for the FizzBuzz problem:
Write a program that prints the numbers from 1 to 100.
* For multiples of three print "Fizz" instead of the number.
* For multiples of five print "Buzz".
* For numbers which are multiples of both three and five print "FizzBuzz".

And here is a basic F# solution:

I have defined a function fizzBuzz that, given an integer i , uses match with when
clauses to do the various tests, and then prints the appropriate value.

1088

Railway oriented programming: Carbonated edition

Simple and straightforward, and fine for a quick hack, but there are a number of problems
with this implementation.
First, note that we had to have a special case for "fifteen". We couldn't just reuse the code
from the "three" and "five" cases. And this means that if we want to add another case, such
as "seven", we also need to add special cases for all the combinations as well (that is "21",
"35" and "105"). And of course, adding more numbers would lead to a combinatorial
explosion of cases.
Second, the order of matching is important. If the "fifteen" case had come last in the list of
patterns, the code would have run correctly, but not actually met the requirements. And
again, if we need to add new cases, we must always remember to put the largest ones first
to ensure correctness. This is just the kind of thing that causes subtle bugs.
Let's try another implementation, where we reuse the code for the "three" and "five" case,
and eliminate the need for a "fifteen" case altogether:
module FizzBuzz_IfPrime =
let fizzBuzz i =
let mutable printed = false
if i % 3 = 0 then
printed <- true
printf "Fizz"
if i % 5 = 0 then
printed <- true
printf "Buzz"
if not printed then
printf "%i" i
printf "; "
// do the fizzbuzz
[1..100] |> List.iter fizzBuzz

In this implementation, the printed value for "15" will be correct, because both the "3" and "5"
cases will be used. And also we don't have to worry about order -- as much, anyway.
But -- these branches are no longer independent, so we have to track whether any branch
has been used at all, so that we can handle the default case. And that has led to the mutable
variable. Mutables are a code smell in F#, so this implementation is not ideal.
However, this version does have the advantage that it can be easily refactored to support
multiple factors, not just 3 and 5.

1089

Railway oriented programming: Carbonated edition

Below is a version that does just this. We pass in a list of "rules" to fizzBuzz . Each rule
consists of a factor and a corresponding label to print out. The fizzBuzz function then just
iterates through these rules, processing each in turn.
module FizzBuzz_UsingFactorRules =
let fizzBuzz rules i =
let mutable printed = false
for factor,label in rules do
if i % factor = 0 then
printed <- true
printf "%s" label
if not printed then
printf "%i" i
printf "; "
// do the fizzbuzz
let rules = [ (3,"Fizz"); (5,"Buzz") ]
[1..100] |> List.iter (fizzBuzz rules)

If we want additional numbers to be processed, we just add them to the list of rules, like this:
module FizzBuzz_UsingFactorRules =
// existing code as above
let rules2 = [ (3,"Fizz"); (5,"Buzz"); (7,"Baz") ]
[1..105] |> List.iter (fizzBuzz rules2)

To sum up, we have created a very imperative implementation that would be almost the
same in C#. It's flexible, but the mutable variable is a bit of a code smell. Is there another
way?

FizzBuzz: The pipeline version

In this next version, we'll look at using the "pipeline" model, where data is fed through a
series of functions to arrive at a final result.
In this design, I envision a pipeline of functions, one to handle the "three" case, one to
handle the "five" case, and so on. And at the end, the appropriate label is spat out, ready to
be printed.

1090

Railway oriented programming: Carbonated edition

Here is some pseudocode to demonstrate the concept:

data |> handleThreeCase |> handleFiveCase |> handleAllOtherCases |> printResult

As an additional requirement, we want the pipeline to have no side effects. This means that
the intermediate functions must not print anything. They must instead pass any generated
labels down the pipe to the end, and only at that point print the results.

Designing the pipeline

As a first step, we need to define what data will be fed down the pipe.
Let's start with the first function, called handleThreeCase in the pseudocode above. What is
its input, and what is its output?
Obviously, the input is the integer being processed. But the output could be the string "Fizz"
if we're lucky. Or the original integer if we're not.
So now let's think about the input to the second function, handleFiveCase . It needs the
integer as well. But in the case of "15" it also needs the string "Fizz" as well, so it can
append "Buzz" to it.
Finally, the handleAllOtherCases function converts the int to a string, but only if "Fizz" or
"Buzz" have not been generated yet.
It's quite obvious then, that the data structure needs to contain both the integer being
processed and the "label so far".
The next question is: how do we know if an upstream function has created a label? The
handleAllOtherCases needs to know this in order to determine whether it needs to do

anything.
One way would be to use an empty string (or, horrors, a null string), but let's be good and
use a string option .
So, here's the final data type that we will be using:
type Data = {i:int; label:string option}

Pipeline version 1
With this data structure, we can define how handleThreeCase and handleFiveCase will work.
First, test the input int i to see if it is divisible by the factor.

1091

Railway oriented programming: Carbonated edition

If it is divisible, look at the label -- if it is None , then replace it with Some "Fizz" or
Some "Buzz" .

If the label already has a value, then append "Buzz" (or whatever) to it.
If the input is not divisible by the factor, just pass on the data unchanged.
Given this design, here's the implementation. It's a generic function that I will call carbonate
(after raganwald) because it works with both "Fizz" and "Buzz":
let carbonate factor label data =
let {i=i; label=labelSoFar} = data
if i % factor = 0 then
// pass on a new data record
let newLabel =
match labelSoFar with
| Some s -> s + label
| None -> label
{data with label=Some newLabel}
else
// pass on the unchanged data
data

The design for the handleAllOtherCases function is slightly different:

Look at the label -- if it is not None , then a previous function has created a label, so do
nothing.
But if the label is None , replace it with the string representation of the integer.
Here's the code -- I will call it labelOrDefault :
let labelOrDefault data =
let {i=i; label=labelSoFar} = data
match labelSoFar with
| Some s -> s
| None -> sprintf "%i" i

Now that we have the components, we can assemble the pipeline:

let fizzBuzz i =
{i=i; label=None}
|> carbonate 3 "Fizz"
|> carbonate 5 "Buzz"
|> labelOrDefault // convert to string
|> printf "%s; " // print

Note that we have to create an initial record using {i=i; label=None} for passing into the
first function ( carbonate 3 "Fizz" ).

1092

Railway oriented programming: Carbonated edition

Finally, here is all the code put together:

module FizzBuzz_Pipeline_WithRecord =
type Data = {i:int; label:string option}
let carbonate factor label data =
let {i=i; label=labelSoFar} = data
if i % factor = 0 then
// pass on a new data record
let newLabel =
match labelSoFar with
| Some s -> s + label
| None -> label
{data with label=Some newLabel}
else
// pass on the unchanged data
data
let labelOrDefault data =
let {i=i; label=labelSoFar} = data
match labelSoFar with
| Some s -> s
| None -> sprintf "%i" i
let fizzBuzz i =
{i=i; label=None}
|> carbonate 3 "Fizz"
|> carbonate 5 "Buzz"
|> labelOrDefault // convert to string
|> printf "%s; " // print
[1..100] |> List.iter fizzBuzz

Pipeline version 2
Creating a new record type can be useful as a form of documentation, but in a case like this,
it would probably be more idiomatic to just use a tuple rather than creating a special data
structure.
So here's a modified implementation that uses tuples.

1093

Railway oriented programming: Carbonated edition

module FizzBuzz_Pipeline_WithTuple =
// type Data = int * string option
let carbonate factor label data =
let (i,labelSoFar) = data
if i % factor = 0 then
// pass on a new data record
let newLabel =
labelSoFar
|> Option.map (fun s -> s + label)
|> defaultArg <| label
(i,Some newLabel)
else
// pass on the unchanged data
data
let labelOrDefault data =
let (i,labelSoFar) = data
labelSoFar
|> defaultArg <| sprintf "%i" i
let fizzBuzz i =
(i,None) // use tuple instead of record
|> carbonate 3 "Fizz"
|> carbonate 5 "Buzz"
|> labelOrDefault // convert to string
|> printf "%s; " // print
[1..100] |> List.iter fizzBuzz

As an exercise, try to find all the code that had to be changed.

Eliminating explicit tests for Some and None

In the tuple code above, I have also replaced the explicit Option matching code match ..
Some .. None with some built-in Option functions, map and defaultArg .

Here are the changes in carbonate :

1094

Railway oriented programming: Carbonated edition

and in labelOrDefault :
// before
match labelSoFar with
| Some s -> s
| None -> sprintf "%i" i
// after
labelSoFar
|> defaultArg <| sprintf "%i" i

You might be wondering about the strange looking |> defaultArg <| idiom.
I am using it because the option is the first parameter to defaultArg , not the second, so a
normal partial application won't work. But "bi-directional" piping does work, hence the
strange looking code.
Here's what I mean:
// OK - normal usage
defaultArg myOption defaultValue
// ERROR: piping doesn't work
myOption |> defaultArg defaultValue
// OK - bi-directional piping does work
myOption |> defaultArg <| defaultValue

Pipeline version 3
Our carbonate function is generic for any factor, so we can easily extend the code to
support "rules" just as in the earlier imperative version.

1095

Railway oriented programming: Carbonated edition

But one issue seems to be that we have hard-coded the "3" and "5" cases into the pipeline,
like this:
|> carbonate 3 "Fizz"
|> carbonate 5 "Buzz"

How can we dynamically add new functions into the pipeline?

The answer is quite simple. We dynamically create a function for each rule, and then
combine all these functions into one using composition.
Here's a snippet to demonstrate:
let allRules =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.reduce (>>)

Each rule is mapped into a function. And then the list of functions is combined into one
single function using >> .
Putting it all together, we have this final implementation:

1096

Railway oriented programming: Carbonated edition

module FizzBuzz_Pipeline_WithRules =
let carbonate factor label data =
let (i,labelSoFar) = data
if i % factor = 0 then
// pass on a new data record
let newLabel =
labelSoFar
|> Option.map (fun s -> s + label)
|> defaultArg <| label
(i,Some newLabel)
else
// pass on the unchanged data
data
let labelOrDefault data =
let (i,labelSoFar) = data
labelSoFar
|> defaultArg <| sprintf "%i" i
let fizzBuzz rules i =
// create a single function from all the rules
let allRules =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.reduce (>>)
(i,None)
|> allRules
|> labelOrDefault // convert to string
|> printf "%s; " // print
// test
let rules = [ (3,"Fizz"); (5,"Buzz"); (7,"Baz") ]
[1..105] |> List.iter (fizzBuzz rules)

Comparing this "pipeline" version with the previous imperative version, the design is much
more functional. There are no mutables and there are no side-effects anywhere (except in
the final printf statement).
There is a subtle bug in the use of List.reduce , however. Can you see what it is?** For a
discussion of the problem and the fix, see the postscript at the bottom of this page.
** Hint: try an empty list of rules.

FizzBuzz: The railway oriented version

1097

Railway oriented programming: Carbonated edition

The pipeline version is a perfectly adequate functional implementation of FizzBuzz, but for
fun, let's see if we can use the "two-track" design described in the railway oriented
programming post.
As a quick reminder, in "railway oriented programming" (a.k.a the "Either" monad), we define
a union type with two cases: "Success" and "Failure", each representing a different "track".
We then connect a set of "two-track" functions together to make the railway.
Most of the functions we actually use are what I called "switch" or "points" functions, with a
one track input, but a two-track output, one for the Success case, and one for the Failure
case. These switch functions are converted into two-track functions using a glue function
called "bind".
Here is a module containing definitions of the functions we will need.
module RailwayCombinatorModule =
let (|Success|Failure|) =
function
| Choice1Of2 s -> Success s
| Choice2Of2 f -> Failure f
/// convert a single value into a two-track result
let succeed x = Choice1Of2 x
/// convert a single value into a two-track result
let fail x = Choice2Of2 x
// appy either a success function or failure function
let either successFunc failureFunc twoTrackInput =
match twoTrackInput with
| Success s -> successFunc s
| Failure f -> failureFunc f
// convert a switch function into a two-track function
let bind f =
either f fail

I am using the Choice type here, which is built into the F# core library. But I have created
some helpers to make it look like the Success/Failure type: an active pattern and two
constructors.
Now, how will we adapt FizzBuzz to this?
Let's start by doing the obvious thing: defining "carbonation" as success, and an unmatched
integer as a failure.
In other words, the Success track contains the labels, and the Failure track contains the ints.

1098

Railway oriented programming: Carbonated edition

Our carbonate "switch" function will therefore look like this:

let carbonate factor label i =
if i % factor = 0 then
succeed label
else
fail i

This implementation is similar to the one used in the pipeline design discussed above, but it
is cleaner because the input is just an int, not a record or a tuple.
Next, we need to connect the components together. The logic will be:
if the int is already carbonated, ignore it
if the int is not carbonated, connect it to the input of the next switch function
Here is the implementation:
let connect f =
function
| Success x -> succeed x
| Failure i -> f i

Another way of writing this is to use the either function we defined in the library module:
let connect' f =
either succeed f

Make sure you understand that both of these implementations do exactly the same thing!
Next, we can create our "two-track" pipeline, like this:
let fizzBuzz =
carbonate 15 "FizzBuzz" // need the 15-FizzBuzz rule because of short-circuit
>> connect (carbonate 3 "Fizz")
>> connect (carbonate 5 "Buzz")
>> either (printf "%s; ") (printf "%i; ")

This is superficially similar to the "one-track" pipeline, but actually uses a different technique.
The switches are connected together through composition ( >> ) rather than piping ( |> ).
As a result, the fizzBuzz function does not have an int parameter -- we are defining a
function by combining other functions. There is no data anywhere.
A few other things have changed as well:

1099

Railway oriented programming: Carbonated edition

We have had to reintroduce the explicit test for the "15" case. This is because we have
only two tracks (success or failure). There is no "half-finished track" that allows the "5"
case to add to the output of the "3" case.
The labelOrDefault function from the previous example has been replaced with
either . In the Success case, a string is printed. In the Failure case, an int is printed.

Here is the complete implementation:

module FizzBuzz_RailwayOriented_CarbonationIsSuccess =
open RailwayCombinatorModule
// carbonate a value
let carbonate factor label i =
if i % factor = 0 then
succeed label
else
fail i
let connect f =
function
| Success x -> succeed x
| Failure i -> f i
let connect' f =
either succeed f
let fizzBuzz =
carbonate 15 "FizzBuzz" // need the 15-FizzBuzz rule because of short-cir
cuit
>> connect (carbonate 3 "Fizz")
>> connect (carbonate 5 "Buzz")
>> either (printf "%s; ") (printf "%i; ")
// test
[1..100] |> List.iter fizzBuzz

Carbonation as failure?
We defined carbonation as "success"in the example above -- it seems the natural thing to
do, surely. But if you recall, in the railway oriented programming model, "success" means
that data should be passed on to the next function, while "failure" means to bypass all the
intermediate functions and go straight to the end.
For FizzBuzz, the "bypass all the intermediate functions" track is the track with the
carbonated labels, while the "pass on to next function" track is the one with integers.

1100

Railway oriented programming: Carbonated edition

So we should really reverse the tracks: "Failure" now means carbonation, while "Success"
means no carbonation.
By doing this, we also get to reuse the pre-defined bind function, rather than having to
write our own connect function.
Here's the code with the tracks switched around:
module FizzBuzz_RailwayOriented_CarbonationIsFailure =
open RailwayCombinatorModule
// carbonate a value
let carbonate factor label i =
if i % factor = 0 then
fail label
else
succeed i
let fizzBuzz =
carbonate 15 "FizzBuzz"
>> bind (carbonate 3 "Fizz")
>> bind (carbonate 5 "Buzz")
>> either (printf "%i; ") (printf "%s; ")
// test
[1..100] |> List.iter fizzBuzz

What are the two tracks anyway?

The fact that we can swap the tracks so easily implies that that maybe there is a weakness
in the design. Are we trying to use a design that doesn't fit?
Why does one track have to be "Success" and another track "Failure" anyway? It doesn't
seem to make much difference.
So, why don't we keep the two-track idea, but get rid of the "Success" and "Failure" labels.
Instead, we can call one track "Carbonated" and the other "Uncarbonated".
To make this work, we can define an active pattern and constructor methods, just as we did
for "Success/Failure".

1101

Railway oriented programming: Carbonated edition

If you are doing domain driven design, it is good practice to write code that uses the
appropriate Ubiquitous Language rather than language that is not applicable.
In this case, if FizzBuzz was our domain, then our functions could now use the domainfriendly terminology of carbonated and uncarbonated rather than "success" or "failure".
let carbonate factor label i =
if i % factor = 0 then
carbonated label
else
uncarbonated i
let connect f =
function
| Uncarbonated i -> f i
| Carbonated x -> carbonated x

Note that, as before, the connect function can be rewritten using either (or we can just
use the predefined bind as before):
let connect' f =
either f carbonated

Here's all the code in one module:

1102

Railway oriented programming: Carbonated edition

module FizzBuzz_RailwayOriented_UsingCustomChoice =
open RailwayCombinatorModule
let (|Uncarbonated|Carbonated|) =
function
| Choice1Of2 u -> Uncarbonated u
| Choice2Of2 c -> Carbonated c
/// convert a single value into a two-track result
let uncarbonated x = Choice1Of2 x
let carbonated x = Choice2Of2 x
// carbonate a value
let carbonate factor label i =
if i % factor = 0 then
carbonated label
else
uncarbonated i
let connect f =
function
| Uncarbonated i -> f i
| Carbonated x -> carbonated x
let connect' f =
either f carbonated
let fizzBuzz =
carbonate 15 "FizzBuzz"
>> connect (carbonate 3 "Fizz")
>> connect (carbonate 5 "Buzz")
>> either (printf "%i; ") (printf "%s; ")
// test
[1..100] |> List.iter fizzBuzz

Adding rules
There are some problems with the version we have so far:
The "15" test is ugly. Can we get rid of it and reuse the "3" and "5" cases?
The "3" and "5" cases are hard-coded. Can we make this more dynamic?
As it happens, we can kill two birds with one stone and address both of these issues at
once.

1103

Railway oriented programming: Carbonated edition

Instead of combining all the "switch" functions in series, we can "add" them together in
parallel. In the railway oriented programming post, we used this technique for combining
validation functions. For FizzBuzz we are going to use it for doing all the factors at once.
The trick is to define a "append" or "concat" function for combining two functions. Once we
can add two functions this way, we can continue and add as many as we like.
So given that we have two carbonation functions, how do we concat them?
Well, here are the possible cases:
If they both have carbonated outputs, we concatenate the labels into a new carbonated
label.
If one has a carbonated output and the other doesn't, then we use the carbonated one.
If neither has a carbonated output, then we use either uncarbonated one (they will be
the same).
Here's the code:
// concat two carbonation functions
let (<+>) switch1 switch2 x =
match (switch1 x),(switch2 x) with
| Carbonated s1,Carbonated s2 -> carbonated (s1 + s2)
| Uncarbonated f1,Carbonated s2 -> carbonated s2
| Carbonated s1,Uncarbonated f2 -> carbonated s1
| Uncarbonated f1,Uncarbonated f2 -> uncarbonated f1

As an aside, notice that this code is almost like math, with uncarbonated playing the role of
"zero", like this:
something + something = combined somethings
zero + something = something
something + zero = something
zero + zero = zero

This is not a coincidence! We will see this kind of thing pop up over and over in functional
code. I'll be talking about this in a future post.
Anyway, with this "concat" function in place, we can rewrite the main fizzBuzz like this:
let fizzBuzz =
let carbonateAll =
carbonate 3 "Fizz" <+> carbonate 5 "Buzz"
carbonateAll
>> either (printf "%i; ") (printf "%s; ")

1104

Railway oriented programming: Carbonated edition

The two carbonate functions are added and then passed to either as before.
Here is the complete code:
module FizzBuzz_RailwayOriented_UsingAppend =
open RailwayCombinatorModule
let (|Uncarbonated|Carbonated|) =
function
| Choice1Of2 u -> Uncarbonated u
| Choice2Of2 c -> Carbonated c
/// convert a single value into a two-track result
let uncarbonated x = Choice1Of2 x
let carbonated x = Choice2Of2 x
// concat two carbonation functions
let (<+>) switch1 switch2 x =
match (switch1 x),(switch2 x) with
| Carbonated s1,Carbonated s2 -> carbonated (s1 + s2)
| Uncarbonated f1,Carbonated s2 -> carbonated s2
| Carbonated s1,Uncarbonated f2 -> carbonated s1
| Uncarbonated f1,Uncarbonated f2 -> uncarbonated f1
// carbonate a value
let carbonate factor label i =
if i % factor = 0 then
carbonated label
else
uncarbonated i
let fizzBuzz =
let carbonateAll =
carbonate 3 "Fizz" <+> carbonate 5 "Buzz"
carbonateAll
>> either (printf "%i; ") (printf "%s; ")
// test
[1..100] |> List.iter fizzBuzz

With this addition logic available, we can easily refactor the code to use rules. Just as with
the earlier "pipeline" implementation, we can use reduce to add all the rules together at
once.
Here's the version with rules:

1105

Railway oriented programming: Carbonated edition

module FizzBuzz_RailwayOriented_UsingAddition =
// code as above
let fizzBuzzPrimes rules =
let carbonateAll =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.reduce (<+>)
carbonateAll
>> either (printf "%i; ") (printf "%s; ")
// test
let rules = [ (3,"Fizz"); (5,"Buzz"); (7,"Baz") ]
[1..105] |> List.iter (fizzBuzzPrimes rules)

Summary
In this post, we've seen three different implementations:
An imperative version that used mutable values and mixed side-effects with logic.
A "pipeline" version that passed a data structure through a series of functions.
A "railway oriented" version that had two separate tracks, and used "addition" to
combine functions in parallel.
In my opinion, the imperative version is the worst design. Even though it was easy to hack
out quickly, it has a number of problems that make it brittle and error-prone.
Of the two functional versions, I think the "railway oriented" version is cleaner, for this
problem at least.
By using the Choice type rather than a tuple or special record, we made the code more
elegant thoughout. You can see the difference if you compare the pipeline version of
carbonate with the railway oriented version of carbonate .

In other situations, of course, the railway oriented approach might not work, and the pipeline
approach might be better. I hope this post has given some useful insight into both.
If you're a FizzBuzz fan, check out the Functional Reactive Programming page, which has
yet another variant of the problem.

Postscript: Be careful when using List.reduce

1106

Railway oriented programming: Carbonated edition

Be careful with List.reduce -- it will fail with empty lists. So if you have an empty rule set,
the code will throw a System.ArgumentException .
In the pipeline case, you can see this by adding the following snippet to the module:
module FizzBuzz_Pipeline_WithRules =
// code as before
// bug
let emptyRules = []
[1..105] |> List.iter (fizzBuzz emptyRules)

The fix is to replace List.reduce with List.fold . List.fold requires an additional

parameter: the initial (or "zero") value. In this case, we can use the identity function, id , as
the initial value.
Here is the fixed version of the code:
let allRules =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.fold (>>) id

Similarly, in the railway oriented example, we had:

let allRules =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.reduce (<+>)

which should be corrected to:

let allRules =
rules
|> List.map (fun (factor,label) -> carbonate factor label)
|> List.fold (<+>) zero

where zero is the "default" function to use if the list is empty.

As an exercise, define the zero function for this case. (Hint: we have actually already defined
it under another name).

1107

Railway oriented programming: Carbonated edition

1108

The "Understanding monoids" Series

In this series, we'll look at a very common "pattern" known as a monoid.

Monoids are not really a design pattern; more an approach to working with many different
types of values in a common way. In fact, once you understand monoids, you will start
seeing them everywhere!
Monoids without tears. A mostly mathless discussion of a common functional pattern.
Monoids in practice. Monoids without tears - Part 2.
Working with non-monoids. Monoids without tears - Part 3.

1109

Monoids without tears

If you are coming from an OO background, one of the more challenging aspects of learning
functional programming is the lack of obvious design patterns. There are plenty of idioms
such as partial application, and error handling techniques, but no apparent patterns in the
GoF sense.
In this post, we'll look at a very common "pattern" known as a monoid. Monoids are not
really a design pattern; more an approach to working with many different types of values in a
common way. In fact, once you understand monoids, you will start seeing them everywhere!
Unfortunately the term "monoid" itself is a bit off-putting. It originally comes from
mathematics but the concept as applied to programming is easy to grasp without any math
at all, as I hope to demonstrate. In fact, if we were to name the concept today in a
programming context, we might call it something like ICombinable instead, which is not
nearly as scary.
Finally, you might be wondering if a "monoid" has any connection with a "monad". Yes, there
is a mathematical connection between them, but in programming terms, they are very
different things, despite having similar names.

Uh-oh... some equations

On this site, I generally don't use any math, but in this case I'm going to break my selfimposed rule and show you some equations.
Ready? Here's the first one:
1 + 2 = 3

Could you handle that? How about another one?

1 + (2 + 3) = (1 + 2) + 3

And finally one more...

1 + 0 = 1 and 0 + 1 = 1

1110

Monoids without tears

Ok! We're done! If you can understand these equations, then you have all the math you
need to understand monoids.

Thinking like a mathematician

"A mathematician, like a painter or poet, is a maker of patterns. If his patterns are more
permanent than theirs, it is because they are made with ideas" -- G H Hardy
Most people imagine that mathematicians work with numbers, doing complicated arithmetic
and calculus.
This is a misconception. For example, if you look at typical high-level math discussions, you
will see lots of strange words, and lots of letter and symbols, but not a lot of arithmetic.
One of the things that mathematicians do do though, is try to find patterns in things. "What
do these things have in common?" and "How can we generalize these concepts?" are
typical mathematical questions.
So let's look at these three equations through a mathematician's eyes.

Generalizing the first equation

A mathematician would look at 1 + 2 = 3 and think something like:
We've got a bunch of things (integers in this case)
We've got some way of combining two of them (addition in this case)
And the result is another one of these things (that is, we get another integer)
And then a mathematician might try to see if this pattern could be generalized to other kinds
of things and operations.
Let's start by staying with integers as the "things". What other ways are there of combining
integers? And do they fit the pattern?
Let's try multiplication, does that fit this pattern?
The answer is yes, multiplication does fit this pattern because multiplying any two integers
results in another integer.
What about division? Does that fit the pattern? The answer is no, because in most cases,
dividing two integers results in a fraction, which is not an integer (I'm ignoring integer
division).
What about the max function? Does that fit the pattern? It combines two integers and
returns one of them, so the answer is yes.

1111

Monoids without tears

What about the equals function? It combines two integers but returns a boolean, not an
integer, so the answer is no.
Enough of integers! What other kinds of things can we think of?
Floats are similar to integers, but unlike integers, using division with floats does result in
another float, so the division operation fits the pattern.
How about booleans? They can be combined using operators such as AND, OR and so on.
Does aBool AND aBool result in another bool? Yes! And OR too fits the pattern.
Strings next. How can they be combined? One way is string concatenation, which returns
another string, which is what we want. But something like the equality operation doesn't fit,
because it returns a boolean.
Finally, let's consider lists. As for strings, the obvious way to combine them is with list
concatenation, which returns another list and fits the pattern.
We can continue on like this for all sorts of objects and combining operations, but you should
see how it works now.
You might ask: why is it so important that the operation return another thing of the same
type? The answer is that you can chain together multiple objects using the operation.
For example, because 1 + 2 is another integer, you can add 3 to it. And then because 1 +
2 + 3 is an integer as well, you can keep going and add say, 4, to the result. In other words,

it is only because integer addition fits the pattern that you can write a sequence of additions
like this: 1 + 2 + 3 + 4 . You couldn't write 1 = 2 = 3 = 4 in the same way, because
integer equality doesn't fit the pattern.
And of course, the chain of combined items can be as long as we like. In other words, this
pattern allows us to extend a pairwise operation into an operation that works on lists.
Mathematicians call the requirement that "the result is another one of these things" the
closure requirement.

Generalizing the second equation

Ok, what about the next equation, 1 + (2 + 3) = (1 + 2) + 3 ? Why is that important?
Well, if you think about the first pattern, it says we can build up a chain of operations such as
1 + 2 + 3 . But we have only got a pairwise operation. So what order should we do the

combining in? Should we combine 1 and 2 first, then combine the result with 3? Or should
we combine 2 and 3 first and then combine 1 with that result? Does it make a difference?

1112

Monoids without tears

That's where this second equation is useful. It says that, for addition, the order of
combination doesn't matter. You get the same result either way.
So for a chain of four items like this: 1 + 2 + 3 + 4 , we could start working from the left
hand side: ((1+2) + 3) + 4 or from the right hand side: 1 + (2 + (3+4)) or even do it in
two parts and then combine them like this: (1+2) + (3+4) .
Let's see if this pattern applies to the examples we've already looked at.
Again, let's start with other ways of combining integers.
We'll start with multiplication again. Does 1 * (2 * 3) give the same result as (1 * 2) *
3 ? Yes. Just as with addition, the order doesn't matter.

Now let's try subtraction. Does 1 - (2 - 3) give the same result as (1 - 2) - 3 ? No. For
subtraction, the order does matter.
What about division? Does 12 / (2 / 3) give the same result as (12 / 2) / 3 ? No. For
division also, the order matters.
But the max function does work. max( max(12,2), 3) gives the same result as max(12,
max(2,3) .

What about strings and lists? Does concatenation meet the requirement? What do you
think?
Here's a question... Can we come up with an operation for strings that is order dependent?
Well, how about a function like "subtractChars" which removes all characters in the right
string from the left string. So subtractChars("abc","ab") is just "c" . subtractChars is
indeed order dependent, as you can see with a simple example: subtractChars("abc",
subtractChars("abc","abc")) is not the same string as
subtractChars(subtractChars("abc","abc"),"abc") .

Mathematicians call the requirement that "the order doesn't matter" the associativity
requirement.
Important Note: When I say the "order of combining", I am talking about the order in which
you do the pairwise combining steps -- combining one pair, and then combining the result
with the next item.
But it is critical that the overall sequence of the items be left unchanged. This is because for
certain operations, if you change the sequencing of the items, then you get a completely
different result! 1 - 2 does not mean the same as 2 - 1 and 2 / 3 does not mean the
same as 3 / 2 .

1113

Monoids without tears

Of course, in many common cases, the sequence order doesn't matter. After all, 1+2 is the
same as 2+1 . In this case, the operation is said to be commutative.

The third equation

Now let's look at the third equation, 1 + 0 = 1 .
A mathematician would say something like: that's interesting -- there is a special kind of
thing ("zero") that, when you combine it with something, just gives you back the original
something, as if nothing had happened.
So once more, let's revisit our examples and see if we can extend this "zero" concept to
other operations and other things.
Again, let's start with multiplication. Is there some value, such that when you multiply a
number with it, you get back the original number?
Yes, of course! The number one. So for multiplication, the number 1 is the "zero".
What about max ? Is there a "zero" for that? For 32 bit ints, yes. Combining
System.Int32.MinValue with any other 32 bit integer using max will return the other integer.

That fits the definition of "zero" perfectly.

What about booleans combined using AND? Is there a zero for that? Yes. It is the value
True . Why? Because True AND False is False , and True AND True is True . In both

cases the other value is returned untouched.

What about booleans combined using OR? Is there a zero for that as well? I'll let you decide.
Moving on, what about string concatenation? Is there a "zero" for this? Yes, indeed -- it is
just the empty string.
"" + "hello" = "hello"
"hello" + "" = "hello"

Finally, for list concatenation, the "zero" is just the empty list.
[] @ [1;2;3] = [1;2;3]
[1;2;3] @ [] = [1;2;3]

You can see that the "zero" value depends very much on the operation, not just on the set of
things. The zero for integer addition is different from the "zero" for integer multiplication,
which is different again from the from "zero" for Max .
Mathematicians call the "zero" the identity element.

1114

Monoids without tears

The equations revisited

So now let's revisit the equations with our new generalizations in mind.
Before, we had:
1 + 2 = 3
1 + (2 + 3) = (1 + 2) + 3
1 + 0 = 1 and 0 + 1 = 1

But now we have something much more abstract, a set of generalized requirements that can
apply to all sorts of things:
You start with a bunch of things, and some way of combining them two at a time.
Rule 1 (Closure): The result of combining two things is always another one of the
things.
Rule 2 (Associativity): When combining more than two things, which pairwise
combination you do first doesn't matter.
Rule 3 (Identity element): There is a special thing called "zero" such that when you
combine any thing with "zero" you get the original thing back.
With these rules in place, we can come back to the definition of a monoid. A "monoid" is just
a system that obeys all three rules. Simple!
As I said at the beginning, don't let the mathematical background put you off. If programmers
had named this pattern, it probably would been called something like "the combinable
pattern" rather than "monoid". But that's life. The terminology is already well-established, so
we have to use it.
Note there are two parts to the definition of a monoid -- the things plus the associated
operation. A monoid is not just "a bunch of things", but "a bunch of things" and "some way of
combining them". So, for example, "the integers" is not a monoid, but "the integers under
addition" is a monoid.

Semigroups
In certain cases, you have a system that only follows the first two rules, and there is no
candidate for a "zero" value.
For example, if your domain consists only of strictly positive numbers, then under addition
they are closed and associative, but there is no positive number that can be "zero".
Another example might be the intersection of finite lists. It is closed and associative, but
there is no (finite) list that when intersected with any other finite list, leaves it untouched.

1115

Monoids without tears

This kind of system still quite useful, and is called a "semigroup" by mathematicians, rather
than a monoid. Luckily, there is a trick that can convert any semigroup into a monoid (which
I'll describe later).

A table of classifications
Let's put all our examples into a table, so you can see them all together.
Things

Operation

Closed?

Associative?

Identity?

Classification

Int32

Addition

Yes

Monoid

Int32

Multiplication

Yes

Monoid

Int32

Subtraction

Yes

Other

Int32

Max

Yes

Int32.MinValue

Monoid

Int32

Equality

Other

Int32

Less than

Other

Float

Multiplication

Yes

No (See note
1)

Other

Float

Division

Yes
(See
note 2)

Other

Positive
Numbers

Addition

Yes

No identity

Semigroup

Positive
Numbers

Multiplication

Yes

Monoid

Boolean

AND

Yes

true

Monoid

Boolean

Yes

false

Monoid

String

Concatenation

Yes

Empty string ""

Monoid

String

Equality

String

"subtractChars"

Yes

Empty string ""

Other

List

Concatenation

Yes

Empty list []

Monoid

List

Intersection

Yes

No identity

Semigroup

Other

There are many other kinds of things you can add to this list; polynomials, matrices,
probability distributions, and so on. This post won't discuss them, but once you get the idea
of monoids, you will see that the concept can be applied to all sorts of things.

1116

Monoids without tears

[Note 1] As Doug points out in the comments, floats are not associative. Replace 'float' with
'real number' to get associativity.
[Note 2] Mathematical real numbers are not closed under division, because you cannot
divide by zero and get another real number. However, with IEEE floating point numbers you
can divide by zero and get a valid value. So floats are indeed closed under division! Here's a
demonstration:
let x = 1.0/0.0 // infinity
let y = x * 2.0 // two times infinity
let z = 2.0 / x // two divided by infinity

What use are monoids to a programmer?

So far, we have described some abstract concepts, but what good are they for real-world
programming problems?

The benefit of closure

As we've seen, the closure rule has the benefit that you can convert pairwise operations into
operations that work on lists or sequences.
In other words, if we can define a pairwise operation, we can extend it to list operations "for
free".
The function that does this is typically called "reduce". Here are some examples:
Explicit

Using reduce

1 + 2 + 3 + 4

[ 1; 2; 3; 4 ] |> List.reduce (+)

1 * 2 * 3 * 4

[ 1; 2; 3; 4 ] |> List.reduce (*)

"a" + "b" + "c" + "d"

[ "a"; "b"; "c"; "d" ] |> List.reduce (+)

[1] @ [2] @ [3] @ [4]

[ [1]; [2]; [3]; [4] ] |> List.reduce (@)

You can see that reduce can be thought of as inserting the specified operation between
each element of the list.
Note that in the last example, the input to reduce is a list of lists, and the output is a single
list. Make sure you understand why this is.

The benefit of associativity

1117

Monoids without tears

If the pairwise combinations can be done in any order, that opens up some interesting
implementation techniques, such as:
Divide and conquer algorithms
Parallelization
Incrementalism
These are deep topics, but let's have a quick look!
Divide and conquer algorithms
Consider the task of summing the first 8 integers; how could we implement this?
One way would be a crude step-by-step sum, as follows:
let sumUpTo2 = 1 + 2
let sumUpTo3 = sumUpTo2 + 3
let sumUpTo4 = sumUpTo3 + 4
// etc
let result = sumUpTo7 + 8

But because the sums can be done in any order, we could also implement the requirement
by splitting the sum into two halves, like this
let sum1To4 = 1 + 2 + 3 + 4
let sum5To8 = 5 + 6 + 7 + 8
let result = sum1To4 + sum5To8

and then we can recursively split the sums into sub-sums in the same way until we get down
to the basic pairwise operation:
let sum1To2 = 1 + 2
let sum3To4 = 3 + 4
let sum1To4 = sum1To2 + sum3To4

This "divide and conquer" approach may seem like overkill for something like a simple sum,
but we'll see in a future post that, in conjunction with a map , it is the basis for some well
known aggregation algorithms.
Parallelization
Once we have a divide and conquer strategy, it can be easily converted into a parallel
algorithm.
For example, to sum the first 8 integers on a four-core CPU, we might do something like this:

1118

Monoids without tears

Core 1

Core 2

Core 3

Core 4

Step
1

sum12 = 1 + 2

sum34 = 3 + 4

sum56 = 5 +
6

sum78 = 7 +
8

Step
2

sum1234 = sum12 +
sum34

sum5678 = sum56 +
sum78

(idle)

Step
3

sum1234 + sum5678

(idle)

There are still seven calculations that need to be done, but because we are doing it parallel,
we can do them all in three steps.
Again, this might seem like a trivial example, but big data systems such as Hadoop are all
about aggregating large amounts of data, and if the aggregation operation is a monoid, then
you can, in theory, easily scale these aggregations by using multiple machines*.
*In practice, of course, the devil is in the details, and real-world systems don't work exactly
this way.
Incrementalism
Even if you do not need parallelism, a nice property of monoids is that they support
incremental calculations.
For example, let's say you have asked me to calculate the sum of one to five. Then of
course I give you back the answer fifteen.
But now you say that you have changed your mind, and you want the sum of one to six
instead. Do I have to add up all the numbers again, starting from scratch? No, I can use the
previous sum, and just add six to it incrementally. This is possible because integer addition
is a monoid.
That is, when faced with a sum like 1 + 2 + 3 + 4 + 5 + 6 , I can group the numbers any
way I like. In particular, I can make an incremental sum like this: (1 + 2 + 3 + 4 + 5) + 6 ,
which then reduces to 15 + 6 .
In this case, recalculating the entire sum from scratch might not be a big deal, but consider a
real-world example like web analytics, counting the number of visitors over the last 30 days,
say. A naive implementation might be to calculate the numbers by parsing the logs of the last
30 days data. A more efficient approach would be to recognize that the previous 29 days
have not changed, and to only process the incremental changes for one day. As a result, the
parsing effort is greatly reduced.
Similarly, if you had a word count of a 100 page book, and you added another page, you
shouldn't need to parse all 101 pages again. You just need to count the words on the last
page and add that to the previous total.*

1119

Monoids without tears

Technically, these are scary sounding monoid homomorphisms. I will explain what this is
in the next post.

The benefit of identity

Having an identity element is not always required. Having a closed, associative operation
(i.e. a semigroup) is sufficient to do many useful things.
But in some cases, it is not enough. For example, here are some cases that might crop up:
How can I use reduce on an empty list?
If I am designing a divide and conquer algorithm, what should I do if one of the "divide"
steps has nothing in it?
When using an incremental algorithm, what value should I start with when I have no
data?
In all cases we need a "zero" value. This allows us to say, for example, that the sum of an
empty list is 0 .
Regarding the first point above, if we are concerned that the list might be empty, then we
must replace reduce with fold , which allows an initial value to be passed in. (Of course,
fold can be used for more things than just monoid operations.)

Here are reduce and fold in action:

// ok
[1..10] |> List.reduce (+)
// error
[] |> List.reduce (+)
// ok with explicit zero
[1..10] |> List.fold (+) 0
// ok with explicit zero
[] |> List.fold (+) 0

Using a "zero" can result in counter-intuitive results sometimes. For example, what is the
product of an empty list of integers?
The answer is 1 , not 0 as you might expect! Here's the code to prove it:
[1..4] |> List.fold (*) 1 // result is 24
[] |> List.fold (*) 1 // result is 1

1120

Monoids without tears

Summary of the benefits

To sum up, a monoid is basically a way to describe an aggregation pattern -- we have a list
of things, we have some way of combining them, and we get a single aggregated object
back at the end.
Or in F# terms:
Monoid Aggregation : 'T list -> 'T

So when you are designing code, and you start using terms like "sum", "product",
"composition", or "concatenation", these are clues that you are dealing with a monoid.

Next steps
Now that we understand what a monoid is, let's see how they can be used in practice.
In the next post in this series, we'll look at how you might write real code that implements the
monoid "pattern".

1121

Monoids in practice

Monoids in practice
In the previous post, we looked at the definition of a monoid. In this post, we'll see how to
implement some monoids.
First, let's revisit the definition:
You start with a bunch of things, and some way of combining them two at a time.
Rule 1 (Closure): The result of combining two things is always another one of the
things.
Rule 2 (Associativity): When combining more than two things, which pairwise
combination you do first doesn't matter.
Rule 3 (Identity element): There is a special thing called "zero" such that when you
combine any thing with "zero" you get the original thing back.
For example, if strings are the things, and string concatenation is the operation, then we
have a monoid. Here's some code that demonstrates this:
let s1 = "hello"
let s2 = " world!"
// closure
let sum = s1 + s2 // sum is a string
// associativity
let s3 = "x"
let s4a = (s1+s2) + s3
let s4b = s1 + (s2+s3)
assert (s4a = s4b)
// an empty string is the identity
assert (s1 + "" = s1)
assert ("" + s1 = s1)

But now let's try to apply this to a more complicated object.

Say that we have have an OrderLine , a little structure that represents a line in a sales
order, say.
type OrderLine = {
ProductCode: string
Qty: int
Total: float
}

1122

Monoids in practice

And then perhaps we might want to find the total for an order, that is, we want to sum the
Total field for a list of lines.

The standard imperative approach would be to create a local total variable, and then loop
through the lines, summing as we go, like this:
let calculateOrderTotal lines =
let mutable total = 0.0
for line in lines do
total <- total + line.Total
total

Let's try it:

module OrdersUsingImperativeLoop =
type OrderLine = {
ProductCode: string
Qty: int
Total: float
}
let calculateOrderTotal lines =
let mutable total = 0.0
for line in lines do
total <- total + line.Total
total
let orderLines = [
{ProductCode="AAA"; Qty=2; Total=19.98}
{ProductCode="BBB"; Qty=1; Total=1.99}
{ProductCode="CCC"; Qty=3; Total=3.99}
]
orderLines
|> calculateOrderTotal
|> printfn "Total is %g"

But of course, being an experienced functional programmer, you would sneer at this, and
use fold in calculateOrderTotal instead, like this:

1123

Monoids in practice

module OrdersUsingFold =
type OrderLine = {
ProductCode: string
Qty: int
Total: float
}
let calculateOrderTotal lines =
let accumulateTotal total line =
total + line.Total
lines
|> List.fold accumulateTotal 0.0
let orderLines = [
{ProductCode="AAA"; Qty=2; Total=19.98}
{ProductCode="BBB"; Qty=1; Total=1.99}
{ProductCode="CCC"; Qty=3; Total=3.99}
]
orderLines
|> calculateOrderTotal
|> printfn "Total is %g"

So far, so good. Now let's look at a solution using a monoid approach.

For a monoid, we need to define some sort of addition or combination operation. How about
something like this?
let addLine orderLine1 orderLine2 =
orderLine1.Total + orderLine2.Total

But this is no good, because we forgot a key aspect of monoids. The addition must return a
value of the same type!
If we look at the signature for the addLine function...
addLine : OrderLine -> OrderLine -> float

...we can see that the return type is float not OrderLine .
What we need to do is return a whole other OrderLine . Here's a correct implementation:

1124

Monoids in practice

let addLine orderLine1 orderLine2 =

{
ProductCode = "TOTAL"
Qty = orderLine1.Qty + orderLine2.Qty
Total = orderLine1.Total + orderLine2.Total
}

Now the signature is correct: addLine : OrderLine -> OrderLine -> OrderLine .
Note that because we have to return the entire structure we have to specify something for
the ProductCode and Qty as well, not just the total. The Qty is easy, we can just do a
sum. For the ProductCode , I decided to use the string "TOTAL", because we don't have a
real product code we can use.
Let's give this a little test:
// utility method to print an OrderLine
let printLine {ProductCode=p; Qty=q;Total=t} =
printfn "%-10s %5i %6g" p q t
let orderLine1 = {ProductCode="AAA"; Qty=2; Total=19.98}
let orderLine2 = {ProductCode="BBB"; Qty=1; Total=1.99}
//add two lines to make a third
let orderLine3 = addLine orderLine1 orderLine2
orderLine3 |> printLine // and print it

We should get this result:

TOTAL 3 21.97

NOTE: For more on the printf formatting options used, see the post on printf here.
Now let's apply this to a list using reduce :
let orderLines = [
{ProductCode="AAA"; Qty=2; Total=19.98}
{ProductCode="BBB"; Qty=1; Total=1.99}
{ProductCode="CCC"; Qty=3; Total=3.99}
]
orderLines
|> List.reduce addLine
|> printLine

With the result:

1125

Monoids in practice

TOTAL 6 25.96

At first, this might seem like extra work, and just to add up a total. But note that we now have
more information than just the total; we also have the sum of the qtys as well.
For example, we can easily reuse the printLine function to make a simple receipt printing
function that includes the total, like this:
let printReceipt lines =
lines
|> List.iter printLine
printfn "-----------------------"
lines
|> List.reduce addLine
|> printLine
orderLines
|> printReceipt

Which gives an output like this:

AAA 2 19.98
BBB 1 1.99
CCC 3 3.99
----------------------TOTAL 6 25.96

More importantly, we can now use the incremental nature of monoids to keep a running
subtotal that we update every time a new line is added.
Here's an example:
let subtotal = orderLines |> List.reduce addLine
let newLine = {ProductCode="DDD"; Qty=1; Total=29.98}
let newSubtotal = subtotal |> addLine newLine
newSubtotal |> printLine

We could even define a custom operator such as ++ so that we can add lines together
naturally as it they were numbers:
let (++) a b = addLine a b // custom operator
let newSubtotal = subtotal ++ newLine

1126

Monoids in practice

You can see that using the monoid pattern opens up a whole new way of thinking. You can
apply this "add" approach to almost any kind of object.
For example, what would a product "plus" a product look like? Or a customer "plus" a
customer? Let your imagination run wild!

Are we there yet?

You might have noticed that we not quite done yet. There is a third requirement for a monoid
that we haven't discussed yet -- the zero or identity element.
In this case, the requirement means that we need some kind of OrderLine such that adding
it to another order line would leave the original untouched. Do we have such a thing?
Right now, no, because the addition operation always changes the product code to "TOTAL".
What we have right now is in fact a semigroup, not a monoid.
As you can see, a semigroup is perfectly useable. But a problem would arise if we had an
empty list of lines and we wanted to total them. What should the result be?
One workaround would be to change the addLine function to ignore empty product codes.
And then we could use an order line with an empty code as the zero element.
Here's what I mean:
let addLine orderLine1 orderLine2 =
match orderLine1.ProductCode, orderLine2.ProductCode with
// is one of them zero? If so, return the other one
| "", _ -> orderLine2
| _, "" -> orderLine1
// anything else is as before
| _ ->
{
ProductCode = "TOTAL"
Qty = orderLine1.Qty + orderLine2.Qty
Total = orderLine1.Total + orderLine2.Total
}
let zero = {ProductCode=""; Qty=0; Total=0.0}
let orderLine1 = {ProductCode="AAA"; Qty=2; Total=19.98}

We can then test that identity works as expected:

assert (orderLine1 = addLine orderLine1 zero)
assert (orderLine1 = addLine zero orderLine1)

1127

Monoids in practice

This does seem a bit hacky, so I wouldn't recommend this technique in general. There's
another way to get an identity that we'll be discussing later.

Introducing a special total type

In the example above, the OrderLine type was very simple and it was easy to overload the
fields for the total.
But what would happen if the OrderLine type was more complicated? For example, if it had
a Price field as well, like this:
type OrderLine = {
ProductCode: string
Qty: int
Price: float
Total: float
}

Now we have introduced a complication. What should we set the Price to when we
combine two lines? The average price? No price?
let addLine orderLine1 orderLine2 =
{
ProductCode = "TOTAL"
Qty = orderLine1.Qty + orderLine2.Qty
Price = 0 // or use average price?
Total = orderLine1.Total + orderLine2.Total
}

Neither seems very satisfactory.

The fact that we don't know what to do probably means that our design is wrong.
Really, we only need a subset of the data for the total, not all of it. How can we represent
this?
With a discriminated union of course! One case can be used for product lines, and the other
case can be used for totals only.
Here's what I mean:

1128

Monoids in practice

type ProductLine = {
ProductCode: string
Qty: int
Price: float
LineTotal: float
}
type TotalLine = {
Qty: int
OrderTotal: float
}
type OrderLine =
| Product of ProductLine
| Total of TotalLine

This design is much nicer. We now have a special structure just for totals and we don't have
to use contortions to make the excess data fit. We can even remove the dummy "TOTAL"
product code.
Note that I named the "total" field differently in each record. Having unique field names like
this means that you don't have to always specify the type explicitly.
Unfortunately, the addition logic is more complicated now, as we have to handle every
combination of cases:
let addLine orderLine1 orderLine2 =
let totalLine =
match orderLine1,orderLine2 with
| Product p1, Product p2 ->
{Qty = p1.Qty + p2.Qty;
OrderTotal = p1.LineTotal + p2.LineTotal}
| Product p, Total t ->
{Qty = p.Qty + t.Qty;
OrderTotal = p.LineTotal + t.OrderTotal}
| Total t, Product p ->
{Qty = p.Qty + t.Qty;
OrderTotal = p.LineTotal + t.OrderTotal}
| Total t1, Total t2 ->
{Qty = t1.Qty + t2.Qty;
OrderTotal = t1.OrderTotal + t2.OrderTotal}
Total totalLine // wrap totalLine to make OrderLine

Note that we cannot just return the TotalLine value. We have to wrap in the Total case to
make a proper OrderLine . If we didn't do that, then our addLine would have the signature
OrderLine -> OrderLine -> TotalLine , which is not correct. We have to have the signature
OrderLine -> OrderLine -> OrderLine -- nothing else will do!

1129

Monoids in practice

Now that we have two cases, we need to handle both of them in the printLine function:
let printLine = function
| Product {ProductCode=p; Qty=q; Price=pr; LineTotal=t} ->
printfn "%-10s %5i @%4g each %6g" p q pr t
| Total {Qty=q; OrderTotal=t} ->
printfn "%-10s %5i %6g" "TOTAL" q t

But once we have done this, we can now use addition just as before:
let orderLine1 = Product {ProductCode="AAA"; Qty=2; Price=9.99; LineTotal=19.98}
let orderLine2 = Product {ProductCode="BBB"; Qty=1; Price=1.99; LineTotal=1.99}
let orderLine3 = addLine orderLine1 orderLine2
orderLine1 |> printLine
orderLine2 |> printLine
orderLine3 |> printLine

Identity again
Again, we haven't dealt with the identity requirement. We could try using the same trick as
before, with a blank product code, but that only works with the Product case.
To get a proper identity, we really need to introduce a third case, EmptyOrder say, to the
union type:
type ProductLine = {
ProductCode: string
Qty: int
Price: float
LineTotal: float
}
type TotalLine = {
Qty: int
OrderTotal: float
}
type OrderLine =
| Product of ProductLine
| Total of TotalLine
| EmptyOrder

With this extra case available, we rewrite the addLine function to handle it:

1130

Monoids in practice

let addLine orderLine1 orderLine2 =

match orderLine1,orderLine2 with
// is one of them zero? If so, return the other one
| EmptyOrder, _ -> orderLine2
| _, EmptyOrder -> orderLine1
// otherwise as before
| Product p1, Product p2 ->
Total { Qty = p1.Qty + p2.Qty;
OrderTotal = p1.LineTotal + p2.LineTotal}
| Product p, Total t ->
Total {Qty = p.Qty + t.Qty;
OrderTotal = p.LineTotal + t.OrderTotal}
| Total t, Product p ->
Total {Qty = p.Qty + t.Qty;
OrderTotal = p.LineTotal + t.OrderTotal}
| Total t1, Total t2 ->
Total {Qty = t1.Qty + t2.Qty;
OrderTotal = t1.OrderTotal + t2.OrderTotal}

And now we can test it:

let zero = EmptyOrder
// test identity
let productLine = Product {ProductCode="AAA"; Qty=2; Price=9.99; LineTotal=19.98}
assert (productLine = addLine productLine zero)
assert (productLine = addLine zero productLine)
let totalLine = Total {Qty=2; OrderTotal=19.98}
assert (totalLine = addLine totalLine zero)
assert (totalLine = addLine zero totalLine)

Using the built in List.sum function

It turns out that the List.sum function knows about monoids! If you tell it what the addition
operation is, and what the zero is, then you can use List.sum directly rather than
List.fold .

The way you do this is by attaching two static members, + and Zero to your type, like this:
type OrderLine with
static member (+) (x,y) = addLine x y
static member Zero = EmptyOrder // a property

Once this has been done, you can use List.sum and it will work as expected.

1131

Monoids in practice

let lines1 = [productLine]

Note that for this to work you mustn't already have a method or case called Zero . If I had
used the name Zero instead of EmptyOrder for the third case it would not have worked.
Although this is a neat trick, in practice I don't think it is a good idea unless you are defining
a proper math-related type such as ComplexNumber or Vector . It's a bit too clever and nonobvious for my taste.
If you do want to use this trick, your Zero member cannot be an extension method -- it
must be defined with the type.
For example, in the code below, I'm trying to define the empty string as the "zero" for strings.
List.fold works because String.Zero is visible as an extension method in this code right

here, but List.sum fails because the extension method is not visible to it.
module StringMonoid =
// define extension method
type System.String with
static member Zero = ""
// OK.
["a";"b";"c"]
|> List.reduce (+)
|> printfn "Using reduce: %s"
// OK. String.Zero is visible as an extension method
["a";"b";"c"]
|> List.fold (+) System.String.Zero
|> printfn "Using fold: %s"
// Error. String.Zero is NOT visible to List.sum
["a";"b";"c"]
|> List.sum
|> printfn "Using sum: %s"

1132

Monoids in practice

Mapping to a different structure

Having two different cases in a union might be acceptable in the order line case, but in many
real world cases, that approach is too complicated or confusing.
Consider a customer record like this:
open System
type Customer = {
Name:string // and many more string fields!
LastActive:DateTime
TotalSpend:float }

How would we "add" two of these customers?

A helpful tip is to realize that aggregation really only works for numeric and similar types.
Strings can't really be aggregated easily.
So rather than trying to aggregate Customer , let's define a separate class CustomerStats
that contains all the aggregatable information:
// create a type to track customer statistics
type CustomerStats = {
// number of customers contributing to these stats
Count:int
// total number of days since last activity
TotalInactiveDays:int
// total amount of money spent
TotalSpend:float }

All the fields in CustomerStats are numeric, so it is obvious how we can add two stats
together:
let add stat1 stat2 = {
Count = stat1.Count + stat2.Count;
TotalInactiveDays = stat1.TotalInactiveDays + stat2.TotalInactiveDays
TotalSpend = stat1.TotalSpend + stat2.TotalSpend
}
// define an infix version as well
let (++) a b = add a b

1133

Monoids in practice

As always, the inputs and output of the add function must be the same type. We must have
CustomerStats -> CustomerStats -> CustomerStats , not Customer -> Customer ->
CustomerStats or any other variant.

Ok, so far so good.

Now let's say we have a collection of customers, and we want to get the aggregated stats for
them, how should we do this?
We can't add the customers directly, so what we need to do is first convert each customer to
a CustomerStats , and then add the stats up using the monoid operation.
Here's an example:
// convert a customer to a stat
let toStats cust =
let inactiveDays= DateTime.Now.Subtract(cust.LastActive).Days;
{Count=1; TotalInactiveDays=inactiveDays; TotalSpend=cust.TotalSpend}
// create a list of customers
let c1 = {Name="Alice"; LastActive=DateTime(2005,1,1); TotalSpend=100.0}
let c2 = {Name="Bob"; LastActive=DateTime(2010,2,2); TotalSpend=45.0}
let c3 = {Name="Charlie"; LastActive=DateTime(2011,3,3); TotalSpend=42.0}
let customers = [c1;c2;c3]
// aggregate the stats
customers
|> List.map toStats
|> List.reduce add
|> printfn "result = %A"

The first thing to note is that the toStats creates statistics for just one customer. We set the
count to just 1. It might seem a bit strange, but it does make sense, because if there was just
one customer in the list, that's what the aggregate stats would be.
The second thing to note is how the final aggregation is done. First we use map to convert
the source type to a type that is a monoid, and then we use reduce to aggregate all the
stats.
Hmmm.... map followed by reduce . Does that sound familiar to you?
Yes indeed, Google's famous MapReduce algorithm was inspired by this concept (although
the details are somewhat different).
Before we move on, here are some simple exercises for you to check your understanding.
What is the "zero" for CustomerStats ? Test your code by using List.fold on an empty
list.

1134

Monoids in practice

Write a simple OrderStats class and use it to aggregate the OrderLine type that we
introduced at the beginning of this post.

Monoid Homomorphisms
We've now got all the tools we need to understand something called a monoid
homomorphism.
I know what you're thinking... Ugh! Not just one, but two strange math words at once!
But I hope that the word "monoid" is not so intimidating now. And "homomorphism" is
another math word that is simpler than it sounds. It's just greek for "same shape" and it
describes a mapping or function that keeps the "shape" the same.
What does that mean in practice?
Well, we have seen that all monoids have a certain common structure. That is, even though
the underlying objects can be quite different (integers, strings, lists, CustomerStats , etc.) the
"monoidness" of them is the same. As George W. Bush once said, once you've seen one
monoid, you've seen them all.
So a monoid homomorphism is a transformation that preserves an essential "monoidness",
even if the "before" and "after" objects are quite different.
In this section, we'll look at a simple monoid homomorphism. It's the "hello world", the
"fibonacci series", of monoid homomorphisms -- word counting.

Documents as a monoid
Let's say we have a type which represents text blocks, something like this:
type Text = Text of string

And of course we can add two smaller text blocks to make a larger text block:
let addText (Text s1) (Text s2) =
Text (s1 + s2)

Here's an example of how adding works:

let t1 = Text "Hello"
let t2 = Text " World"
let t3 = addText t1 t2

1135

Monoids in practice

Since you are now a expert, you will quickly recognize this as a monoid, with the zero
obviously being Text "" .
Now let's say we are writing a book (such as this one) and we want a word count to show
how much we have written.
Here's a very crude implementation, plus a test:
let wordCount (Text s) =
s.Split(' ').Length
// test
Text "Hello world"
|> wordCount
|> printfn "The word count is %i"

So we are writing away, and now we have produced three pages of text. How do we
calculate the word count for the complete document?
Well, one way is to add the separate pages together to make a complete text block, and
then apply the wordCount function to that text block. Here's a diagram:

But everytime we finish a new page, we have to add all the text together and do the word
count all over again.
No doubt you can see that there is a better way of doing this. Instead of adding all the text
together and then counting, get the word count for each page separately, and then add
these counts up, like this:

The second approach relies on the fact that integers (the counts) are themselves a monoid,
and you can add them up to get the desired result.

1136

Monoids in practice

So the wordCount function has transformed an aggregation over "pages" into an

aggregation over "counts".
The big question now: is wordCount a monoid homomorphism?
Well, pages (text) and counts (integers) are both monoids, so it certainly transforms one
monoid into another.
But the more subtle condition is: does it preserve the "shape"? That is, does the adding of
the counts give the same answer as the adding of the pages?
In this case, the answer is yes. So wordCount is a monoid homomorphism!
You might think that this is obvious, and that all mappings like this must be monoid
homomorphisms, but we'll see an example later where this is not true.

The benefits of chunkability

The advantage of the monoid homomorphism approach is that it is "chunkable".
Each map and word count is independent of the others, so we can do them separately and
then add up the answers afterwards. For many algorithms, working on small chunks of data
is much more efficient than working on large chunks, so if we can, we should exploit this
whenever possible.
As a direct consequence of this chunkability, we get some of the benefits that we touched on
in the previous post.
First, it is incremental. That is, as we add text to the last page, we don't have to recalculate
the word counts for all the previous pages, which might save some time.
Second, it is parallelizable. The work for each chunk can be done independently, on different
cores or machines. Note that in practice, parallelism is much overrated. The chunkability into
small pieces has a much greater effect on performance than parallelism itself.

Comparing word count implementations

We're now ready to create some code that will demonstrate these two different techniques.
Let's start with the basic definitions from above, except that I will change the word count to
use regular expressions instead of split .

1137

Monoids in practice

module WordCountTest =
open System
type Text = Text of string
let addText (Text s1) (Text s2) =
Text (s1 + s2)
let wordCount (Text s) =
System.Text.RegularExpressions.Regex.Matches(s,@"\S+").Count

Next, we'll create a page with 1000 words in it, and a document with 1000 pages.
module WordCountTest =
// code as above
let page() =
List.replicate 1000 "hello "
|> List.reduce (+)
|> Text
let document() =
page() |> List.replicate 1000

We'll want to time the code to see if there is any difference between the implementations.
Here's a little helper function.
module WordCountTest =
// code as above
let time f msg =
let stopwatch = Diagnostics.Stopwatch()
stopwatch.Start()
f()
stopwatch.Stop()
printfn "Time taken for %s was %ims" msg stopwatch.ElapsedMilliseconds

Ok, let's implement the first approach. We'll add all the pages together using addText and
then do a word count on the entire million word document.

1138

Monoids in practice

module WordCountTest =
// code as above
let wordCountViaAddText() =
document()
|> List.reduce addText
|> wordCount
|> printfn "The word count is %i"
time wordCountViaAddText "reduce then count"

For the second approach, we'll do wordCount on each page first, and then add all the
results together (using reduce of course).
module WordCountTest =
// code as above
let wordCountViaMap() =
document()
|> List.map wordCount
|> List.reduce (+)
|> printfn "The word count is %i"
time wordCountViaMap "map then reduce"

Note that we have only changed two lines of code!

In wordCountViaAddText we had:
|> List.reduce addText
|> wordCount

And in wordCountViaMap we have basically swapped these lines. We now do wordCount first
and then reduce afterwards, like this:
|> List.map wordCount
|> List.reduce (+)

Finally, let's see what difference parallelism makes. We'll use the built-in
Array.Parallel.map instead of List.map , which means we'll need to convert the list into an

array first.

1139

Monoids in practice

module WordCountTest =
// code as above
let wordCountViaParallelAddCounts() =
document()
|> List.toArray
|> Array.Parallel.map wordCount
|> Array.reduce (+)
|> printfn "The word count is %i"
time wordCountViaParallelAddCounts "parallel map then reduce"

I hope that you are following along with the implementations, and that you understand what
is going on.

Analyzing the results

Here are the results for the different implementations running on my 4 core machine:
Time taken for reduce then count was 7955ms
Time taken for map then reduce was 698ms
Time taken for parallel map then reduce was 603ms

We must recognize that these are crude results, not a proper performance profile. But even
so, it is very obvious that the map/reduce version is about 10 times faster that the
ViaAddText version.

This is the key to why monoid homomorphisms are important -- they enable a "divide and
conquer" strategy that is both powerful and easy to implement.
Yes, you could argue that the algorithms used are very inefficient. String concat is a terrible
way to accumulate large text blocks, and there are much better ways of doing word counts.
But even with these caveats, the fundamental point is still valid: by swapping two lines of
code, we got a huge performance increase.
And with a little bit of hashing and caching, we would also get the benefits of incremental
aggregation -- only recalculating the minimum needed as pages change.
Note that the parallel map didn't make that much difference in this case, even though it did
use all four cores. Yes, we did add some minor expense with toArray but even in the best
case, you might only get a small speed up on a multicore machine. To reiterate, what really
made the most difference was the divide and conquer strategy inherent in the map/reduce
approach.

1140

Monoids in practice

A non-monoid homomorphism
I mentioned earlier that not all mappings are necessarily monoid homomorphisms. In this
section, we'll look at an example of one that isn't.
For this example, rather than using counting words, we're going to return the most frequent
word in a text block.
Here's the basic code.
module FrequentWordTest =
open System
open System.Text.RegularExpressions
type Text = Text of string
let addText (Text s1) (Text s2) =
Text (s1 + s2)
let mostFrequentWord (Text s) =
Regex.Matches(s,@"\S+")
|> Seq.cast<Match>
|> Seq.map (fun m -> m.ToString())
|> Seq.groupBy id
|> Seq.map (fun (k,v) -> k,Seq.length v)
|> Seq.sortBy (fun (_,v) -> -v)
|> Seq.head
|> fst

The mostFrequentWord function is bit more complicated than the previous wordCount
function, so I'll take you through it step by step.
First, we use a regex to match all non-whitespace. The result of this is a MatchCollection
not a list of Match , so we have to explicitly cast it into a sequence (an IEnumerable<Match>
in C# terms).
Next we convert each Match into the matched word, using ToString() . Then we group by
the word itself, which gives us a list of pairs, where each pair is a (word,list of words) . We
then turn those pairs into (word,list count) and then sort descending (using the negated
word count).
Finally we take the first pair, and return the first part of the pair. This is the most frequent
word.

1141

Monoids in practice

Ok, let's continue, and create some pages and a document as before. This time we're not
interested in performance, so we only need a few pages. But we do want to create different
pages. We'll create one containing nothing but "hello world", another containing nothing but
"goodbye world", and a third containing "foobar". (Not a very interesting book IMHO!)
module FrequentWordTest =
// code as above
let page1() =
List.replicate 1000 "hello world "
|> List.reduce (+)
|> Text
let page2() =
List.replicate 1000 "goodbye world "
|> List.reduce (+)
|> Text
let page3() =
List.replicate 1000 "foobar "
|> List.reduce (+)
|> Text
let document() =
[page1(); page2(); page3()]

It is obvious that, with respect to the entire document, "world" is the most frequent word
overall.
So let's compare the two approaches as before. The first approach will combine all the
pages and then apply mostFrequentWord , like this.

The second approach will do mostFrequentWord separately on each page and then combine
the results, like this:

1142

Monoids in practice

Here's the code:

Can you see what happened? The first approach was correct. But the second approach
gave a completely wrong answer!
Using add first, the most frequent word is world
Using map reduce, the most frequent word is hellogoodbyefoobar

The second approach just concatenated the most frequent words from each page. The
result is a new string that was not on any of the pages. A complete fail!
What went wrong?
Well, strings are a monoid under concatenation, so the mapping transformed a monoid
(Text) to another monoid (string).
But the mapping did not preserve the "shape". The most frequent word in a big chunk of text
cannot be derived from the most frequent words in smaller chunks of text. In other words, it
is not a proper monoid homomorphism.

Definition of a monoid homomorphism

Let's look at these two different examples again to understand what the distinction is
between them.
In the word count example, we got the same final result whether we added the blocks and
then did the word count, or whether we did the word counts and then added them together.
Here's a diagram:

1143

Monoids in practice

But for the most frequent word example, we did not get the same answer from the two
different approaches.

In other words, for wordCount , we had

wordCount(page1) + wordCount(page2) EQUALS wordCount(page1 + page)

But for mostFrequentWord , we had:

mostFrequentWord(page1) + mostFrequentWord(page2) NOT EQUAL TO mostFrequentWord(page1
+ page)

So this brings us to a slightly more precise definition of a monoid homomorphism:

Given a function that maps from one monoid to another (like 'wordCount' or 'mostFreque
ntWord')
Then to be a monoid homomorphism, the function must meet the requirement that:
function(chunk1) + function(chunk2) MUST EQUAL function(chunk1 + chunk2)

Alas, then, mostFrequentWord is not a monoid homomorphism.

That means that if we want to calculate the mostFrequentWord on a large number of text
files, we are sadly forced to add all the text together first, and we can't benefit from a divide
and conquer strategy.
... or can we? Is there a way to turn mostFrequentWord into a proper monoid
homomorphism? Stay tuned!

1144

Monoids in practice

Next steps
So far, we have only dealt with things that are proper monoids. But what if the thing you want
to work with is not a monoid? What then?
In the next post in this series, I'll give you some tips on converting almost anything into a
monoid.
We'll also fix up the mostFrequentWord example so that it is a proper monoid
homomorphism, and we'll revisit the thorny problem of zeroes, with an elegant approach for
creating them.
See you then!

Further reading
If you are interested in using monoids for data aggregation, there are lots of good
discussions in the following links:
Twitter's Algebird library
Most probabilistic data structures are monoids.
Gaussian distributions form a monoid.
Google's MapReduce Programming Model (PDF).
Monoidify! Monoids as a Design Principle for Efficient MapReduce Algorithms (PDF).
LinkedIn's Hourglass libary for Hadoop
From Stack Exchange: What use are groups, monoids, and rings in database
computations?
If you want to get a bit more technical, here is a detailed study of monoids and semigroups,
using graphics diagrams as the domain:
Monoids: Theme and Variations (PDF).

1145

Working with non-monoids

In the previous posts in this series, we only dealt with things that were proper monoids.
But what if the thing you want to work with is not a monoid? What then? Well, in this post, I'll
give you some tips on converting almost anything into a monoid.
In the process, we'll be introduced to a number of important and common functional design
idioms, such as preferring lists rather than singletons, and using the option type at every
opportunity.

Getting closure
If you recall, for a proper monoid, we need three things to be true: closure, associativity, and
identity. Each requirement can present a challenge, so we'll discuss each in turn.
We'll start with closure.
In some cases you might want to add values together, but the type of the combined value is
not the same as the type of the original values. How can you handle this?
One way is to just to map from the original type to a new type that is closed. We saw this
approach used with the Customer and CustomerStats example in the previous post. In
many cases, this is the easiest approach, because you don't have to mess with the design of
the original types.
On the other hand, sometimes you really don't want to use map , but instead want to design
your type from the beginning so that it meets the closure requirement.
Either way, whether you are designing a new type or redesigning an existing type, you can
use similar techniques to get closure.

Combining closed types to make new compound types

Obviously, we've seen that numeric types are closed under some basic math operations like
addition and multiplication. We've also seen that some non-numeric types, like strings and
lists, are closed under concatenation.
With this in mind, it should be obvious that any combination of these types will be closed too.
We just have to define the "add" function to do the appropriate "add" on the component
types.

1146

Working with non-monoids

Here's an example:
type MyType = {count:int; items:int list}
let addMyType t1 t2 =
{count = t1.count + t2.count;
items = t1.items @ t2.items}

The addMyType function uses integer addition on the int field, and list concatenation on
the list field. As a result the MyType is closed using the function addMyType -- in fact, not
only is it closed, it is a monoid too. So in this case, we're done!
This is exactly the approach we took with CustomerStats in the previous post.
So here's my first tip:
DESIGN TIP: To easily create a monoidal type, make sure that each field of the
type is also a monoid.
Question to think about: when you do this, what is the "zero" of the new compound type?

Dealing with non-numeric types

The approach above works when creating compound types. But what about non-numeric
types, which have no obvious numeric equivalent?
Here's a very simple case. Say that you have some chars that you want to add together, like
this:
'a' + 'b' -> what?

But, a char plus a char is not another char. If anything, it is a string.

'a' + 'b' -> "ab" // Closure fail!

But that is very unhelpful, as it does not meet the closure requirement.
One way to fix this is to force the chars into strings, which does work:
"a" + "b" -> "ab"

But that is a specific solution for chars -- is there a more generic solution that will work for
other types?

1147

Working with non-monoids

Well, think for a minute what the relationship of a string to a char is. A string can be
thought of as a list or array of chars.
In other words, we could have used lists of chars instead, like this:
['a'] @ ['b'] -> ['a'; 'b'] // Lists FTW!

This meets the closure requirement as well.

What's more, this is in fact a general solution to any problem like this, because anything can
be put into a list, and lists (with concatenation) are always monoids.
So here's my next tip:
DESIGN TIP: To enable closure for a non-numeric type, replace single items with
lists.
In some cases, you might need to convert to a list when setting up the monoid and then
convert to another type when you are done.
For example, in the Char case, you would do all your manipulation on lists of chars and
then only convert to a string at the end.
So let's have a go at creating a "monoidal char" module.
module MonoidalChar =
open System
/// "monoidal char"
type MChar = MChar of Char list
/// convert a char into a "monoidal char"
let toMChar ch = MChar [ch]
/// add two monoidal chars
let addChar (MChar l1) (MChar l2) =
MChar (l1 @ l2)
// infix version
let (++) = addChar
/// convert to a string
let toString (MChar cs) =
new System.String(List.toArray cs)

You can see that MChar is a wrapper around a list of chars, rather than a single char.
Now let's test it:

1148

Working with non-monoids

open MonoidalChar
// add two chars and convert to string
let a = 'a' |> toMChar
let b = 'b' |> toMChar
let c = a ++ b
c |> toString |> printfn "a + b = %s"
// result: "a + b = ab"

If we want to get fancy we can use map/reduce to work on a set of chars, like this:
[' '..'z'] // get a lot of chars
|> List.filter System.Char.IsPunctuation
|> List.map toMChar
|> List.reduce addChar
|> toString
|> printfn "punctuation chars are %s"
// result: "punctuation chars are !"#%&'()*,-./:;?@[\]_"

Monoids for errors

The MonoidalChar example is trivial, and could perhaps be implemented in other ways, but
in general this is an extremely useful technique.
For example, here is a simple module for doing some validation. There are two options,
Success and Failure , and the Failure case also has a error string associated with it.

module Validation =
type ValidationResult =
| Success
| Failure of string
let validateBadWord badWord (name:string) =
if name.Contains(badWord) then
Failure ("string contains a bad word: " + badWord)
else
Success
let validateLength maxLength name =
if String.length name > maxLength then
Failure "string is too long"
else
Success

In practice, we might perform multiple validations on a string, and we would like to return all
the results at once, added together somehow.
1149

Working with non-monoids

This calls out for being a monoid! If we can add two results pairwise, then we can extend the
operation to add as many results as we like!
So then the question is, how do we combine two validation results?
let result1 = Failure "string is null or empty"
let result2 = Failure "string is too long"
result1 + result2 = ????

A naive approach would be to concatenate the strings, but that wouldn't work if we were
using format strings, or resource ids with localization, etc.
No, a better way is to convert the Failure case to use a list of strings instead of a single
string. That will make combining results simple.
Here's the same code as above, with the Failure case redefined to use a list:
module MonoidalValidation =
type ValidationResult =
| Success
| Failure of string list
// helper to convert a single string into the failure case
let fail str =
Failure [str]
let validateBadWord badWord (name:string) =
if name.Contains(badWord) then
fail ("string contains a bad word: " + badWord)
else
Success
let validateLength maxLength name =
if String.length name > maxLength then
fail "string is too long"
else
Success

You can see that the individual validations call fail with a single string, but behind the
scenes it is being stored as a list of strings, which can, in turn, be concatenated together.
With this in place, we can now create the add function.
The logic will be:
If both results are Success , then the combined result is Success

1150

Working with non-monoids

If one result is Failure , then the combined result is that failure.

If both results are Failure , then the combined result is a Failure with both error lists
concatenated.
Here's the code:
module MonoidalValidation =
// as above
/// add two results
let add r1 r2 =
match r1,r2 with
| Success, Success -> Success
| Failure f1, Success -> Failure f1
| Success, Failure f2 -> Failure f2
| Failure f1, Failure f2 -> Failure (f1 @ f2)

Here are some tests to check the logic:

open MonoidalValidation
let test1 =
let result1 = Success
let result2 = Success
add result1 result2
|> printfn "Result is %A"
// "Result is Success"
let test2 =
let result1 = Success
let result2 = fail "string is too long"
add result1 result2
|> printfn "Result is %A"
// "Result is Failure ["string is too long"]"
let test3 =
let result1 = fail "string is null or empty"
let result2 = fail "string is too long"
add result1 result2
|> printfn "Result is %A"
// Result is Failure
// [ "string is null or empty";
// "string is too long"]

And here's a more realistic example, where we have a list of validation functions that we
want to apply:

1151

Working with non-monoids

let test4 =
let validationResults str =
[
validateLength 10
validateBadWord "monad"
validateBadWord "cobol"
]
|> List.map (fun validate -> validate str)
"cobol has native support for monads"
|> validationResults
|> List.reduce add
|> printfn "Result is %A"

The output is a Failure with three error messages.

Result is Failure
["string is too long"; "string contains a bad word: monad";
"string contains a bad word: cobol"]

One more thing is needed to finish up this monoid. We need a "zero" as well. What should it
be?
By definition, it is something that when combined with another result, leaves the other result
alone.
I hope you can see that by this definition, "zero" is just Success .
module MonoidalValidation =
// as above
// identity
let zero = Success

As you know, we would need to use zero if the list to reduce over is empty. So here's an
example where we don't apply any validation functions at all, giving us an empty list of
ValidationResult .

1152

Working with non-monoids

let test5 =
let validationResults str =
[]
|> List.map (fun validate -> validate str)
"cobol has native support for monads"
|> validationResults
|> List.fold add zero
|> printfn "Result is %A"
// Result is Success

Note that we needed to change reduce to fold as well, otherwise we would get a runtime
error.

Lists for performance

Here's one more example of the benefit of using lists. Compared with other methods of
combination, list concatenation is relatively cheap, both in computation and in memory use,
because the objects being pointed to don't have to change or be reallocated.
For example, in the previous post, we defined a Text block that wrapped a string, and used
string concatenation to add their contents.
type Text = Text of string
let addText (Text s1) (Text s2) =
Text (s1 + s2)

But for large strings this continual concatenation can be expensive.

Consider a different implementation, where the Text block contains a list of strings instead.
type Text = Text of string list
let addText (Text s1) (Text s2) =
Text (s1 @ s2)

Almost no change in implementation, but performance will probably be greatly improved.

You can do all your manipulation on lists of strings and you need only convert to a normal
string at the very end of the processing sequence.

1153

Working with non-monoids

And if lists aren't performant enough for you, you can easily extend this approach to use
classic data structures like trees, heaps, etc. or mutable types like ResizeArray. (See the
appendix on performance at the bottom of this post for some more discussion on this)

Jargon alert
The concept of using a list of objects as a monoid is common in mathematics, where it is
called a "free monoid". In computer science, it also called a "Kleene star" such as A* . And
if you don't allow empty lists, then you have no zero element. This variant is called a "free
semigroup" or "Kleene plus" such as A+ .
This "star" and "plus" notation will surely be familiar to you if you have ever used regular
expressions.*
* You probably weren't aware that there was a connection between regular expressions and
monoids! There's some even deeper relationships too.

Associativity
Now that we have dealt with closure, let's take on associativity.
We saw a couple of non-associative operations in the very first post, including subtraction
and division.
We can see that 5 - (3 - 2) is not equal to (5 - 3) - 2 . This shows that subtraction is
not associative, and also 12 / (3 / 2) is not equal to (12 / 3) / 2 , which shows that
division is not associative.
There's no single correct answer in these cases, because you might genuinely care about
different answers depending on whether you work from left to right or right to left.
In fact, the F# standard libraries have two versions of fold and reduce to cater for each
preference. The normal fold and reduce work left to right, like this:
//same as (12 - 3) - 2
[12;3;2] |> List.reduce (-) // => 7
//same as ((12 - 3) - 2) - 1
[12;3;2;1] |> List.reduce (-) // => 6

But there is also foldBack and reduceBack that work from right to left, like this:

1154

Working with non-monoids

//same as 12 - (3 - 2)
[12;3;2] |> List.reduceBack (-) // => 11
//same as 12 - (3 - (2 - 1))
[12;3;2;1] |> List.reduceBack (-) // => 10

In a sense, then, the associativity requirement is just a way of saying that you should get the
same answer no matter whether you use fold or foldBack .

Moving the operation into the element

But assuming that you do want a consistent monoidal approach, the trick in many cases is to
move the operation into a property of each element. Make the operation a noun, rather
than a verb.
For example 3 - 2 can be thought of as 3 + (-2) . Rather than "subtraction" as a verb,
we have "negative 2" as a noun.
In that case, the above example becomes 5 + (-3) + (-2) . And since we are now using
addition as the operator, we do have associativity, and 5 + (-3 + -2) is indeed the same as
(5 + -3) + -2 .

A similar approach works with division. 12 / 3 / 2 can be converted into 12 * (1/3) *

(1/2) , and now we are back to multiplication as the operator, which is associative.

This approach of converting the operator into a property of the element can be generalized
nicely.
So here's a tip:
DESIGN TIP: To get associativity for an operation, try to move the operation into
the object.
We can revisit an earlier example to understand how this works. If you recall, in the first post
we tried to come up with a non-associative operation for strings, and settled on
subtractChars .

Here's a simple implementation of subtractChars

let subtractChars (s1:string) (s2:string) =
let isIncluded (ch:char) = s2.IndexOf(ch) = -1
let chars = s1.ToCharArray() |> Array.filter isIncluded
System.String(chars)
// infix version
let (--) = subtractChars

1155

Working with non-monoids

With this implementation we can do some interactive tests:

"abcdef" -- "abd" // "cef"
"abcdef" -- "" // "abcdef"

And we can see for ourselves that the associativity requirement is violated:
("abc" -- "abc") -- "abc" // ""
"abc" -- ("abc" -- "abc") // "abc"

How can we make this associative?

The trick is move the "subtract-ness" from the operator into the object, just as we did with
the numbers earlier.
What I mean is that we replace the plain strings with a "subtract" or "chars to remove" data
structure that captures what we want to remove, like so:
let removalAction = (subtract "abd") // a data structure

And then we "apply" the data structure to the string:

let removalAction = (subtract "abd")
removalAction |> applyTo "abcdef" // "Result is cef"

Once we use this approach, we can rework the non-associative example above to look
something like this:
let removalAction = (subtract "abc") + (subtract "abc") + (subtract "abc")
removalAction |> applyTo "abc" // "Result is "

Yes, it is not exactly the same as the original code, but you might find that this is actually a
better fit in many situations.
The implementation is below. We define a CharsToRemove to contain a set of chars, and the
other function implementations fall out from that in a straightforward way.

1156

Working with non-monoids

/// store a list of chars to remove

type CharsToRemove = CharsToRemove of Set<char>
/// construct a new CharsToRemove
let subtract (s:string) =
s.ToCharArray() |> Set.ofArray |> CharsToRemove
/// apply a CharsToRemove to a string
let applyTo (s:string) (CharsToRemove chs) =
let isIncluded ch = Set.exists ((=) ch) chs |> not
let chars = s.ToCharArray() |> Array.filter isIncluded
System.String(chars)
// combine two CharsToRemove to get a new one
let (++) (CharsToRemove c1) (CharsToRemove c2) =
CharsToRemove (Set.union c1 c2)

Let's test!
let test1 =
let removalAction = (subtract "abd")
removalAction |> applyTo "abcdef" |> printfn "Result is %s"
// "Result is cef"
let test2 =
let removalAction = (subtract "abc") ++ (subtract "abc") ++ (subtract "abc")
removalAction |> applyTo "abcdef" |> printfn "Result is %s"
// "Result is "

The way to think about this approach is that, in a sense, we are modelling actions rather
than data. We have a list of CharsToRemove actions, then we combine them into a single
"big" CharsToRemove action, and then we execute that single action at the end, after we have
finished the intermediate manipulations.
We'll see another example of this shortly, but you might be thinking at this point: "this sounds
a bit like functions, doesn't it?" To which I will say "yes, it does!"
In fact rather than creating this CharsToRemove data structure, we could have just partially
applied the original subtractChars function, as shown below:
(Note that we reverse the parameters to make partial application easier)

1157

Working with non-monoids

// reverse for partial application

let subtract str charsToSubtract =
subtractChars charsToSubtract str
let removalAction = subtract "abd"
"abcdef" |> removalAction |> printfn "Result is %s"
// "Result is cef"

And now we don't even need a special applyTo function.

But when we have more than one of these subtraction functions, what do we do? Each of
these partially applied functions has signature string -> string , so how can we "add" them
together?
(subtract "abc") + (subtract "abc") + (subtract "abc") = ?

The answer is function composition, of course!

let removalAction2 = (subtract "abc") >> (subtract "abc") >> (subtract "abc")
removalAction2 "abcdef" |> printfn "Result is %s"
// "Result is def"

This is the functional equivalent of creating the CharsToRemove data structure.

The "data structure as action" and function approach are not exactly the same -- the
CharsToRemove approach may be more efficient, for example, because it uses a set, and is

only applied to strings at the end -- but they both achieve the same goal. Which one is better
depends on the particular problem you're working on.
I'll have more to say on functions and monoids in the next post.

Identity
Now to the last requirement for a monoid: identity.
As we have seen, identity is not always needed, but it is nice to have if you might be dealing
with empty lists.
For numeric values, finding an identity for an operation is generally easy, whether it be 0
(addition), 1 (multiplication) or Int32.MinValue (max).

1158

Working with non-monoids

And this carries over to structures that contain only numeric values as well -- just set all
values to their appropriate identity. The CustomerStats type from the previous post
demonstrates that nicely.
But what if you have objects that are not numeric? How can you create a "zero" or identity
element if there is no natural candidate?
The answer is: you just make one up.
Seriously!
We have already seen an example of this in the previous post, when we added an
EmptyOrder case to the OrderLine type:

type OrderLine =
| Product of ProductLine
| Total of TotalLine
| EmptyOrder

Let's look at this more closely. We performed two steps:

First, we created a new case and added it to the list of alternatives for an OrderLine
(as shown above).
Second, we adjusted the addLine function to take it into account (as shown below).
let addLine orderLine1 orderLine2 =
match orderLine1,orderLine2 with
// is one of them zero? If so, return the other one
| EmptyOrder, _ -> orderLine2
| _, EmptyOrder -> orderLine1
// logic for other cases ...

That's all there is to it.

The new, augmented type consists of the old order line cases, plus the new EmptyOrder
case, and so it can reuse much of the behavior of the old cases.
In particular, can you see that the new augmented type follows all the monoid rules?
A pair of values of the new type can be added to get another value of the new type
(closure)
If the combination order didn't matter for the old type, then it still doesn't matter for the
new type (associativity)
And finally... this extra case now gives us an identity for the new type.

Turning PositiveNumber into a monoid

1159

Working with non-monoids

We could do the same thing with the other semigroups we've seen.
For example, we noted earlier that strictly positive numbers (under addition) didn't have an
identity; they are only a semigroup. If we wanted to create a zero using the "augmentation
with extra case" technique (rather than just using 0 !) we would first define a special Zero
case (not an integer), and then create an addPositive function that can handle it, like this:
type PositiveNumberOrIdentity =
| Positive of int
| Zero
let addPositive i1 i2 =
match i1,i2 with
| Zero, _ -> i2
| _, Zero -> i1
| Positive p1, Positive p2 -> Positive (p1 + p2)

Admittedly, PositiveNumberOrIdentity is a contrived example, but you can see how this
same approach would work for any situation where you have "normal" values and a special,
separate, zero value.

A generic solution
There are a few drawbacks to this:
We have to deal with two cases now: the normal case and the zero case.
We have to create custom types and custom addition functions
Unfortunately, there's nothing you can do about the first issue. If you have a system with no
natural zero, and you create an artificial one, then you will indeed always have to deal with
two cases.
But there is something you can do about the second issue! Rather than create a new custom
type over and over, perhaps can we create a generic type that has two cases: one for all
normal values and one for the artificial zero, like this:
type NormalOrIdentity<'T> =
| Normal of 'T
| Zero

Does this type look familiar? It's just the Option type in disguise!
In other words, any time we need an identity which is outside the normal set of values, we
can use Option.None to represent it. And then Option.Some is used for all the other
"normal" values.

1160

Working with non-monoids

Another benefit of using Option is that we can also write a completely generic "add"
function as well. Here's a first attempt:
let optionAdd o1 o2 =
match o1, o2 with
| None, _ -> o2
| _, None -> o1
| Some s1, Some s2 -> Some (s1 + s2)

The logic is straightforward. If either option is None , the other option is returned. If both are
Some , then they are unwrapped, added together, and then wrapped in a Some again.

But the + in the last line makes assumptions about the types that we are adding. Better to
pass in the addition function explicitly, like this:
let optionAdd f o1 o2 =
match o1, o2 with
| None, _ -> o2
| _, None -> o1
| Some s1, Some s2 -> Some (f s1 s2)

In practice, this would used with partial application to bake in the addition function.
So now we have another important tip:
DESIGN TIP: To get identity for an operation, create a special case in a
discriminated union, or, even simpler, just use Option.

PositiveNumber revisited
So here is the Positive Number example again, now using the Option type.
type PositiveNumberOrIdentity = int option
let addPositive = optionAdd (+)

Much simpler!
Notice that we pass in the "real" addition function as a parameter to optionAdd so that it is
baked in. In other situations, you would do the same with the relevant aggregation function
that is associated with the semigroup.
As a result of this partial application, addPositive has the signature: int option -> int
option -> int option , which is exactly what we would expect from a monoid addition

function.

1161

Working with non-monoids

In other words, optionAdd turns any function 'a -> 'a -> 'a into the same function, but
"lifted" to the option type, that is, having a signature 'a option -> 'a option -> 'a option .
So, let's test it! Some test code might look like this:
// create some values
let p1 = Some 1
let p2 = Some 2
let zero = None
// test addition
addPositive p1 p2
addPositive p1 zero
addPositive zero p2
addPositive zero zero

You can see that unfortunately we do have to wrap the normal values in Some in order to
get the None as identity.
That sounds tedious but in practice, it is easy enough. The code below shows how we might
handle the two distinct cases when summing a list. First how to sum a non-empty list, and
then how to sum an empty list.
[1..10]
|> List.map Some
|> List.fold addPositive zero
[]
|> List.map Some
|> List.fold addPositive zero

ValidationResult revisited
While we're at it, let's also revisit the ValidationResult type that we described earlier when
talking about using lists to get closure. Here it is again:
type ValidationResult =
| Success
| Failure of string list

Now that we've got some insight into the positive integer case, let's look at this type from a
different angle as well.

1162

Working with non-monoids

The type has two cases. One case holds data that we care about, and the other case holds
no data. But the data we really care about are the error messages, not the success. As Leo
Tolstoy nearly said "All validation successes are alike; each validation failure is a failure in its
own way."
So, rather than thinking of it as a "Result", let's think of the type as storing failures, and
rewrite it like this instead, with the failure case first:
type ValidationFailure =
| Failure of string list
| Success

Does this type appear familar now?

Yes! It's the option type again! Can we never get away from the darn thing?
Using the option type, we can simplify the design of the ValidationFailure type to just this:
type ValidationFailure = string list option

The helper to convert a string into the failure case is now just Some with a list:
let fail str =
Some [str]

And the "add" function can reuse optionAdd , but this time with list concatenation as the
underlying operation:
let addFailure f1 f2 = optionAdd (@) f1 f2

Finally, the "zero" that was the Success case in the original design now simply becomes
None in the new design.

Here's all the code, plus tests

module MonoidalValidationOption =
type ValidationFailure = string list option
// helper to convert a string into the failure case
let fail str =
Some [str]
let validateBadWord badWord (name:string) =
if name.Contains(badWord) then

1163

Working with non-monoids

fail ("string contains a bad word: " + badWord)

else
None
let validateLength maxLength name =
if String.length name > maxLength then
fail "string is too long"
else
None
let optionAdd f o1 o2 =
match o1, o2 with
| None, _ -> o2
| _, None -> o1
| Some s1, Some s2 -> Some (f s1 s2)
/// add two results using optionAdd
let addFailure f1 f2 = optionAdd (@) f1 f2
// define the Zero
let Success = None
module MonoidalValidationOptionTest =
open MonoidalValidationOption
let test1 =
let result1 = Success
let result2 = Success
addFailure result1 result2
|> printfn "Result is %A"
// Result is <null>
let test2 =
let result1 = Success
let result2 = fail "string is too long"
addFailure result1 result2
|> printfn "Result is %A"
// Result is Some ["string is too long"]
let test3 =
let result1 = fail "string is null or empty"
let result2 = fail "string is too long"
addFailure result1 result2
|> printfn "Result is %A"
// Result is Some ["string is null or empty"; "string is too long"]
let test4 =
let validationResults str =
[
validateLength 10
validateBadWord "monad"
validateBadWord "cobol"

1164

Working with non-monoids

]
|> List.map (fun validate -> validate str)
"cobol has native support for monads"
|> validationResults
|> List.reduce addFailure
|> printfn "Result is %A"
// Result is Some
// ["string is too long"; "string contains a bad word: monad";
// "string contains a bad word: cobol"]
let test5 =
let validationResults str =
[]
|> List.map (fun validate -> validate str)
"cobol has native support for monads"
|> validationResults
|> List.fold addFailure Success
|> printfn "Result is %A"
// Result is <null>

Summary of the design tips

Let's pause for a second and see what we have covered so far.
Here are all the design tips together:
To easily create a monoidal type, make sure that each field of the type is also a monoid.
To enable closure for a non-numeric type, replace single items with lists (or a similar
data structure).
To get associativity for an operation, try to move the operation into the object.
To get identity for an operation, create a special case in a discriminated union, or, even
simpler, just use Option.
In the next two sections, we'll apply these tips to two of the non-monoids that we have seen
in previous posts: "average" and "most frequent word".

A case study: Average

So now we have the toolkit that will enable us to deal with the thorny case of averages.
Here's a simple implementation of a pairwise average function:

1165

Working with non-monoids

let avg i1 i2 =
float (i1 + i2) / 2.0
// test
avg 4 5 |> printfn "Average is %g"
// Average is 4.5

As we mentioned briefly in the first post, avg fails on all three monoid requirements!
First, it is not closed. Two ints that are combined together using avg do not result in
another int.
Second, even if it was closed, avg is not associative, as we can see by defining a similar
float function avgf :
let avgf i1 i2 =
(i1 + i2) / 2.0
// test
avgf (avgf 1.0 3.0) 5.0 |> printfn "Average from left is %g"
avgf 1.0 (avgf 3.0 5.0) |> printfn "Average from right is %g"
// Average from left is 3.5
// Average from right is 2.5

Finally, there is no identity.

What number, when averaged with any other number, returns the original value? Answer:
none!

Applying the design tips

So let's apply the design tips to see if they help us come up with a solution.
To easily create a monoidal type, make sure that each field of the type is also a monoid.
Well, "average" is a mathematical operation, so we could expect that a monoidal equivalent
would also be based on numbers.
To enable closure for a non-numeric type, replace single items with lists.
This looks at first glance like it won't be relevant, so we'll skip this for now.
To get associativity for an operation, try to move the operation into the object.
Here's the crux! How do we convert "average" from a verb (an operation) to a noun (a data
structure)?

1166

Working with non-monoids

The answer is that we create a structure that is not actually an average, but a "delayed
average" -- everything you need to make an average on demand.
That is, we need a data structure with two components: a total, and a count. With these two
numbers we can calculate an average as needed.
// store all the info needed for an average
type Avg = {total:int; count:int}
// add two Avgs together
let addAvg avg1 avg2 =
{total = avg1.total + avg2.total;
count = avg1.count + avg2.count}

The good thing about this, is that structure stores ints , not floats , so we don't need to
worry about loss of precision or associativity of floats.
The last tip is:
To get identity for an operation, create a special case in a discriminated union, or, even
simpler, just use Option.
In this case, the tip is not needed, as we can easily create a zero by setting the two
components to be zero:
let zero = {total=0; count=0}

We could also have used None for the zero, but it seems like overkill in this case. If the list
is empty, the Avg result is valid, even though we can't do the division.
Once we have had this insight into the data structure, the rest of the implementation follows
easily. Here is all the code, plus some tests:

1167

Working with non-monoids

module Average =
// store all the info needed for an average
type Avg = {total:int; count:int}
// add two Avgs together
let addAvg avg1 avg2 =
{total = avg1.total + avg2.total;
count = avg1.count + avg2.count}
// inline version of add
let (++) = addAvg
// construct an average from a single number
let avg n = {total=n; count=1}
// calculate the average from the data.
// return 0 for empty lists
let calcAvg avg =
if avg.count = 0
then 0.0
else float avg.total / float avg.count
// alternative - return None for empty lists
let calcAvg2 avg =
if avg.count = 0
then None
else Some (float avg.total / float avg.count)
// the identity
let zero = {total=0; count=0}
// test
addAvg (avg 4) (avg 5)
|> calcAvg
|> printfn "Average is %g"
// Average is 4.5
(avg 4) ++ (avg 5) ++ (avg 6)
|> calcAvg
|> printfn "Average is %g"
// Average is 5
// test
[1..10]
|> List.map avg
|> List.reduce addAvg
|> calcAvg
|> printfn "Average is %g"
// Average is 5.5

1168

Working with non-monoids

In the code above, you can see that I created a calcAvg function that uses the Avg
structure to calculate a (floating point) average. One nice thing about this approach is that
we can delay having to make a decision about what to do with a zero divisor. We can just
return 0 , or alternatively None , or we can just postpone the calculation indefinitely, and
only generate the average at the last possible moment, on demand!
And of course, this implementation of "average" has the ability to do incremental averages.
We get this for free because it is a monoid.
That is, if I have already calculated the average of a million numbers, and I want to add one
more, I don't have to recalculate everything, I can just add the new number to the totals so
far.

A slight diversion on metrics

If you have ever been responsible for managing any servers or services, you will be aware
of the importance of logging and monitoring metrics, such as CPU, I/O, etc.
One of the questions you often face then is how to design your metrics. Do you want
kilobytes per second, or just total kilobytes since the server started. Visitors per hour, or total
visitors?
If you look at some guidelines when creating metrics you will see the frequent
recommendation to only track metrics that are counters, not rates.
The advantage of counters is that (a) missing data doesn't affect the big picture, and (b) they
can be aggregated in many ways after the fact -- by minute, by hour, as a ratio with
something else, and so on.
Now that you have worked through this series, you can see that the recommendation can
really be rephrased as metrics should be monoids.
The work we did in the code above to transform "average" into two components, "total" and
"count", is exactly what you want to do to make a good metric.
Averages and other rates are not monoids, but "total" and "count" are, and then "average"
can be calculated from them at your leisure.

Case study: Turning "most frequent word" into

a monoid homomorphism

1169

Working with non-monoids

In the last post, we implemented a "most frequent word" function, but found that it wasn't a
monoid homomorphism. That is,
mostFrequentWord(text1) + mostFrequentWord(text2)

did not give the same result as:

mostFrequentWord( text1 + text2 )

Again, we can use the design tips to fix this up so that it works.
The insight here is again to delay the calculation until the last minute, just as we did in the
"average" example.
Rather than calculating the most frequent word upfront then, we create a data structure that
stores all the information that we need to calculate the most frequent word later.
module FrequentWordMonoid =
open System
open System.Text.RegularExpressions
type Text = Text of string
let addText (Text s1) (Text s2) =
Text (s1 + s2)
// return a word frequency map
let wordFreq (Text s) =
Regex.Matches(s,@"\S+")
|> Seq.cast<Match>
|> Seq.map (fun m -> m.ToString())
|> Seq.groupBy id
|> Seq.map (fun (k,v) -> k,Seq.length v)
|> Map.ofSeq

In the code above we have a new function wordFreq , that returns a Map<string,int> rather
just a single word. That is, we are now working with dictionaries, where each slot has a word
and its associated frequency.
Here is a demonstration of how it works:

1170

Working with non-monoids

module FrequentWordMonoid =
// code from above
let page1() =
List.replicate 1000 "hello world "
|> List.reduce (+)
|> Text
let page2() =
List.replicate 1000 "goodbye world "
|> List.reduce (+)
|> Text
let page3() =
List.replicate 1000 "foobar "
|> List.reduce (+)
|> Text
let document() =
[page1(); page2(); page3()]
// show some word frequency maps
page1() |> wordFreq |> printfn "The frequency map for page1 is %A"
page2() |> wordFreq |> printfn "The frequency map for page2 is %A"
//The frequency map for page1 is map [("hello", 1000); ("world", 1000)]
//The frequency map for page2 is map [("goodbye", 1000); ("world", 1000)]
document()
|> List.reduce addText
|> wordFreq
|> printfn "The frequency map for the document is %A"
//The frequency map for the document is map [
// ("foobar", 1000); ("goodbye", 1000);
// ("hello", 1000); ("world", 2000)]

With this map structure in place, we can create a function addMap to add two maps. It
simply merges the frequency counts of the words from both maps.

1171

Working with non-monoids

module FrequentWordMonoid =
// code from above
// define addition for the maps
let addMap map1 map2 =
let increment mapSoFar word count =
match mapSoFar |> Map.tryFind word with
| Some count' -> mapSoFar |> Map.add word (count + count')
| None -> mapSoFar |> Map.add word count
map2 |> Map.fold increment map1

And when we have combined all the maps together, we can then calculate the most frequent
word by looping through the map and finding the word with the largest frequency.
module FrequentWordMonoid =
// code from above
// as the last step,
// get the most frequent word in a map
let mostFrequentWord map =
let max (candidateWord,maxCountSoFar) word count =
if count > maxCountSoFar
then (word,count)
else (candidateWord,maxCountSoFar)
map |> Map.fold max ("None",0)

So, here are the two scenarios revisited using the new approach.
The first scenario combines all the pages into a single text, then applies wordFreq to get a
frequency map, and applies mostFrequentWord to get the most frequent word.
The second scenario applies wordFreq to each page separately to get a map for each page.
These maps are then combined with addMap to get a single global map. Then
mostFrequentWord is applied as the last step, as before.

1172

Working with non-monoids

module FrequentWordMonoid =
// code from above
document()
|> List.reduce addText
|> wordFreq
// get the most frequent word from the big map
|> mostFrequentWord
|> printfn "Using add first, the most frequent word and count is %A"
//Using add first, the most frequent word and count is ("world", 2000)
document()
|> List.map wordFreq
|> List.reduce addMap
// get the most frequent word from the merged smaller maps
|> mostFrequentWord
|> printfn "Using map reduce, the most frequent and count is %A"
//Using map reduce, the most frequent and count is ("world", 2000)

If you run this code, you will see that you now get the same answer.
This means that wordFreq is indeed a monoid homomorphism, and is suitable for running in
parallel, or incrementally.

Next time
We've seen a lot of code in this post, but it has all been focused on data structures.
However, there is nothing in the definition of a monoid that says that the things to be
combined have to be data structures -- they could be anything at all.
In the next post we'll look at monoids applied to other objects, such as types, functions, and
more.

Appendix: On Performance
In the examples above, I have made frequent use of @ to "add" two lists in the same way
that + adds two numbers. I did this to highlight the analogies with other monoidal
operations such as numeric addition and string concatenation.

1173

Working with non-monoids

I hope that it is clear that the code samples above are meant to be teaching examples, not
necessarily good models for the kind of real-world, battle-hardened, and all-too-ugly code
you need in a production environment.
A couple of people have pointed out that using List append ( @ ) should be avoided in
general. This is because the entire first list needs to be copied, which is not very efficient.
By far the best way to add something to a list is to add it to the front using the so-called
"cons" mechanism, which in F# is just :: . F# lists are implemented as linked lists, so
adding to the front is very cheap.
The problem with using this approach is that it is not symmetrical -- it doesn't add two lists
together, just a list and an element. This means that it cannot be used as the "add" operation
in a monoid.
If you don't need the benefits of a monoid, such as divide and conquer, then that is a
perfectly valid design decision. No need to sacrifice performance for a pattern that you are
not going to benefit from.
The other alternative to using @ is to not use lists in the first place!

Alternatives to lists
In the ValidationResult design, I used a list to hold the error results so that we could get
easy accumulation of the results. But I only chose the list type because it is really the
default collection type in F#. I could have equally well have chosen sequences, or arrays, or
sets. Almost any other collection type would have done the job just as well.
But not all types will have the same performance. For example, combining two sequences is
a lazy operation. You don't have to copy all the data; you just enumerate one sequence, then
the other. So that might be faster perhaps?
Rather than guessing, I wrote a little test script to measure performance at various list sizes,
for various collection types.
I have chosen a very simple model: we have a list of objects, each of which is a collection
containing one item. We then reduce this list of collections into a single giant collection using
the appropriate monoid operation. Finally, we iterate over the giant collection once.
This is very similar to the ValidationResult design, where we would combine all the results
into a single list of results, and then (presumably) iterate over them to show the errors.
It is also similar to the "most frequent word" design, above, where we combine all the
individual frequency maps into a single frequency map, and then iterate over it to find the
most frequent word. In that case, of course, we were using map rather than list , but the

1174

Working with non-monoids

set of steps is the same.

A performance experiment
Ok, here's the code:

1175

Working with non-monoids

module Performance =
let printHeader() =
printfn "Label,ListSize,ReduceAndIterMs"
// time the reduce and iter steps for a given list size and print the results
let time label reduce iter listSize =
System.GC.Collect() //clean up before starting
let stopwatch = System.Diagnostics.Stopwatch()
stopwatch.Start()
reduce() |> iter
stopwatch.Stop()
printfn "%s,%iK,%i" label (listSize/1000) stopwatch.ElapsedMilliseconds
let testListPerformance listSize =
let lists = List.init listSize (fun i -> [i.ToString()])
let reduce() = lists |> List.reduce (@)
let iter = List.iter ignore
time "List.@" reduce iter listSize
let testSeqPerformance_Append listSize =
let seqs = List.init listSize (fun i -> seq {yield i.ToString()})
let reduce() = seqs |> List.reduce Seq.append
let iter = Seq.iter ignore
time "Seq.append" reduce iter listSize
let testSeqPerformance_Yield listSize =
let seqs = List.init listSize (fun i -> seq {yield i.ToString()})
let reduce() = seqs |> List.reduce (fun x y -> seq {yield! x; yield! y})
let iter = Seq.iter ignore
time "seq(yield!)" reduce iter listSize
let testArrayPerformance listSize =
let arrays = List.init listSize (fun i -> [| i.ToString() |])
let reduce() = arrays |> List.reduce Array.append
let iter = Array.iter ignore
time "Array.append" reduce iter listSize
let testResizeArrayPerformance listSize =
let resizeArrays = List.init listSize (fun i -> new ResizeArray<string>( [i.To
String()] ) )
let append (x:ResizeArray<_>) y = x.AddRange(y); x
let reduce() = resizeArrays |> List.reduce append
let iter = Seq.iter ignore
time "ResizeArray.append" reduce iter listSize

Let's go through the code quickly:

The time function times the reduce and iteration steps. It deliberately does not test
how long it takes to create the collection. I do perform a GC before starting, but in
reality, the memory pressure that a particular type or algorithm causes is an important
1176

Working with non-monoids

part of the decision to use it (or not). Understanding how GC works is an important part
of getting performant code.
The testListPerformance function sets up the list of collections (lists in this case) and
also the reduce and iter functions. It then runs the timer on reduce and iter .
The other functions do the same thing, but with sequences, arrays, and ResizeArrays
(standard .NET Lists). Out of curiosity, I thought I'd test two ways of merging
sequences, one using the standard library function Seq.append and the other using two
yield! s in a row.

The testResizeArrayPerformance uses ResizeArrays and adds the right list to the left
one. The left one mutates and grows larger as needed, using a growth strategy that
keeps inserts efficient.
Now let's write code to check the performance on various sized lists. I chose to start with a
count of 2000 and move by increments of 4000 up to 50000.
open Performance
printHeader()
[2000..4000..50000]
|> List.iter testArrayPerformance
[2000..4000..50000]
|> List.iter testResizeArrayPerformance
[2000..4000..50000]
|> List.iter testListPerformance
[2000..4000..50000]
|> List.iter testSeqPerformance_Append
[2000..4000..50000]
|> List.iter testSeqPerformance_Yield

I won't list all the detailed output -- you can run the code for yourself -- but here is a chart of
the results.

1177

Working with non-monoids

There are a few things to note:

The two sequence-based examples crashed with stack overflows. The yield! was
about 30% faster than Seq.append , but also ran out of stack faster.
List.append didn't run out of stack, but got much slower as the lists got larger.
Array.append was fast, and increases more slowly with the size of the list
ResizeArray was fastest of all, and didn't break a sweat even with large lists.
For the three collection types that didn't crash, I also timed them for a list of 100K items. The
results were:
List = 150,730 ms
Array = 26,062 ms
ResizeArray = 33 ms
A clear winner there, then.

Analyzing the results

What conclusion can we draw from this little experiment?

1178

Working with non-monoids

First, you might have all sorts of questions, such as: Were you running in debug or release
mode? Did you have optimization turned on? What about using parallelism to increase
performance? And no doubt, there will be comments saying "why did you use technique X,
technique Y is so much better".
But here's the conclusion I would like to make:
You cannot draw any conclusion from these results!
Every situation is different and requires a different approach:
If you are working with small data sets you might not care about performance anyway.
In this case I would stick with lists -- I'd rather not sacrifice pattern matching and
immutability unless I have to.
The performance bottleneck might not be in the list addition code. There is no point
working on optimizing the list addition if you are actually spending all your time on disk
I/O or network delays. A real-world version of the word frequency example might
actually spend most of its time doing reading from disk, or parsing, rather than adding
lists.
If you working at the scale of Google, Twitter, or Facebook, you really need to go and
hire some algorithm experts.
The only principles that we can take away from any discussion on optimization and
performance are:
A problem must be dealt with in its own context. The size of the data being
processed, the type of hardware, the amount of memory, and so on. All these will make
a difference to your performance. What works for me may not work for you, which is
why...
You should always measure, not guess. Don't make assumptions about where your
code is spending its time -- learn to use a profiler! There are some good examples of
using a profiler here and here.
Be wary of micro-optimizations. Even if your profiler shows that your sorting routine
spends all its time in comparing strings, that doesn't necessarily mean that you need to
improve your string comparison function. You might be better off improving your
algorithm so that you don't need to do so many comparisons in the first place.
Premature optimization and all that.

1179

The "Understanding Parser Combinators" Series

In this series, we'll look at how so-called "applicative parsers" work. In order to understand
something, there's nothing like building it for yourself, and so we'll create a basic parser
library from scratch, then some useful "parser combinators", and then finish off by building a
complete JSON parser.
Understanding Parser Combinators. Building a parser combinator library from scratch.
Building a useful set of parser combinators. 15 or so combinators that can be combined
to parse almost anything.
Improving the parser library. Adding more informative errors.
Writing a JSON parser from scratch. In 250 lines of code.

1180

Understanding Parser Combinators

UPDATE: Slides and video from my talk on this topic
In this series, we'll look at how so-called "applicative parsers" work. In order to understand
something, there's nothing like building it for yourself, and so we'll create a basic parser
library from scratch, and then some useful "parser combinators", and then finish off by
building a complete JSON parser.
Now terms like "applicative parsers" and "parser combinators" can make this approach
seem complicated, but rather than attempting to explain these concepts up front, we'll just
dive in and start coding.
We'll build up to the complex stuff incrementally via a series of implementations, where each
implementation is only slightly different from the previous one. By using this approach, I
hope that at each stage the design and concepts will be easy to understand, and so by the
end of this series, parser combinators will have become completely demystified.
There will be four posts in this series:
In this, the first post, we'll look at the basic concepts of parser combinators and build the
core of the library.
In the second post, we'll build up a useful library of combinators.
In the third post, we'll work on providing helpful error messages.
In the last post, we'll build a JSON parser using this parser library.
Obviously, the focus here will not be on performance or efficiency, but I hope that it will give
you the understanding that will then enable you to use libraries like FParsec effectively. And
by the way, a big thank you to Stephan Tolksdorf, who created FParsec. You should make it
your first port of call for all your .NET parsing needs!

Implementation 1. Parsing a hard-coded

character
For our first implementation, let's create something that just parses a single, hard-coded,
character, in this case, the letter "A". You can't get much simpler than that!
Here is how it works:
The input to a parser is a stream of characters. We could use something complicated,

1181

Understanding Parser Combinators

but for now we'll just use a string .

If the stream is empty, then return a pair consisting of false and an empty string.
If the first character in the stream is an A , then return a pair consisting of true and
the remaining stream of characters.
If the first character in the stream is not an A , then return false and the (unchanged)
original stream of characters.
Here's the code:
let A_Parser str =
if String.IsNullOrEmpty(str) then
(false,"")
else if str.[0] = 'A' then
let remaining = str.[1..]
(true,remaining)
else
(false,str)

The signature of A_Parser is:

val A_Parser :
string -> (bool * string)

which tells us that the input is a string, and the output is a pair consisting of the boolean
result and another string (the remaining input), like this:

Let's test it now -- first with good input:

let inputABC = "ABC"
A_Parser inputABC

The result is:

(true, "BC")

As you can see, the A has been consumed and the remaining input is just "BC" .
And now with bad input:

1182

Understanding Parser Combinators

let inputZBC = "ZBC"

A_Parser inputZBC

which gives the result:

(false, "ZBC")

And in this case, the first character was not consumed and the remaining input is still
"ZBC" .

So, there's an incredibly simple parser for you. If you understand that, then everything that
follows will be easy!

Implementation 2. Parsing a specified

character
Let's refactor so that we can pass in the character we want to match, rather than having it be
hard coded.
And this time, rather than returning true or false, we'll return a message indicating what
happened.
We'll call the function pchar for "parse char". Here's the code:
let pchar (charToMatch,str) =
if String.IsNullOrEmpty(str) then
let msg = "No more input"
(msg,"")
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
let msg = sprintf "Found %c" charToMatch
(msg,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
(msg,str)

This code is just like the previous example, except that the unexpected character is now
shown in the error message.
The signature of pchar is:

1183

Understanding Parser Combinators

val pchar :
(char * string) -> (string * string)

which tells us that the input is a pair of (string,character to match) and the output is a pair
consisting of the (string) result and another string (the remaining input).
Let's test it now -- first with good input:
let inputABC = "ABC"
pchar('A',inputABC)

The result is:

("Found A", "BC")

As before, the A has been consumed and the remaining input is just "BC" .
And now with bad input:
let inputZBC = "ZBC"
pchar('A',inputZBC)

which gives the result:

("Expecting 'A'. Got 'Z'", "ZBC")

And again, as before, the first character was not consumed and the remaining input is still
"ZBC" .

If we pass in Z , then the parser does succeed:

pchar('Z',inputZBC) // ("Found Z", "BC")

Implementation 3. Returning a Success/Failure

We want to be able to tell the difference between a successful match and a failure, and
returning a stringly-typed message is not very helpful, so let's define a special "choice" type
to indicate the difference. I'll call it Result :

1184

Understanding Parser Combinators

type Result<'a> =
| Success of 'a
| Failure of string

The Success case is generic and can contain any value. The Failure case contains an
error message.
Note: for more on using this Success/Failure approach, see my talk on functional error
handling.
We can now rewrite the parser to return one of the Result cases, like this:
let pchar (charToMatch,str) =
if String.IsNullOrEmpty(str) then
Failure "No more input"
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
Success (charToMatch,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
Failure msg

The signature of pchar is now:

val pchar :
(char * string) -> Result<char * string>

which tells us that the the output is now a Result (which in the Success case, contains the
matched char and the remaining input string).
Let's test it again -- first with good input:
let inputABC = "ABC"
pchar('A',inputABC)

The result is:

Success ('A', "BC")

As before, the A has been consumed and the remaining input is just "BC" . We also get
the actual matched char ( A in this case).

1185

Understanding Parser Combinators

And now with bad input:

let inputZBC = "ZBC"
pchar('A',inputZBC)

which gives the result:

Failure "Expecting 'A'. Got 'Z'"

And in this case, the Failure case is returned with the appropriate error message.
This is a diagram of the function's inputs and outputs now:

Implementation 4. Switching to a curried

implementation
In the previous implementation, the input to the function has been a tuple -- a pair. This
requires you to pass both inputs at once.
In functional languages like F#, it's more idiomatic to use a curried version, like this:
let pchar charToMatch str =
if String.IsNullOrEmpty(str) then
Failure "No more input"
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
Success (charToMatch,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
Failure msg

Can you see the difference? The only difference is in the first line, and even then it is subtle.
Here's the uncurried (tuple) version:

1186

Understanding Parser Combinators

let pchar (charToMatch,str) =

...

And here's the curried version:

let pchar charToMatch str =
...

The difference is much more obvious when you look at the type signatures. Here's the
signature for the uncurried (tuple) version:
val pchar :
(char * string) -> Result<char * string>

And here's the signature for the curried version:

val pchar :
char -> string -> Result<char * string>

Here is the curried version of pchar represented as a diagram:

What is currying?
If you are unclear on how currying works, I have a post about it here, but basically it means
that a multi-parameter function can be written as a series of one-parameter functions.
In other words, this two-parameter function:
let add x y =
x + y

can be written as a one-parameter function that returns a lambda, like this:

let add x =
fun y -> x + y // return a lambda

or as a function that returns an inner function, like this:

1187

Understanding Parser Combinators

let add x =
let innerFn y = x + y
innerFn // return innerFn

Rewriting with an inner function

We can take advantage of currying and rewrite the parser as a one-parameter function
(where the parameter is charToMatch ) that returns a inner function.
Here's the new implementation, with the inner function cleverly named innerFn :
let pchar charToMatch =
// define a nested inner function
let innerFn str =
if String.IsNullOrEmpty(str) then
Failure "No more input"
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
Success (charToMatch,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
Failure msg
// return the inner function
innerFn

The type signature for this implementation looks like this:

val pchar :
char -> string -> Result<char * string>

It's exactly the same as the previous version!

That is, both of the above implementations are identical in practice:
// two-parameter implementation
let pchar charToMatch str =
...
// one-parameter implementation with inner function
let pchar charToMatch =
let innerFn str =
...
// return the inner function
innerFn

1188

Understanding Parser Combinators

The benefits of the curried implementation

What's nice about the curried implementation is that we can partially apply the character we
want to parse, like this:
let parseA = pchar 'A'

and then later on supply the second "input stream" parameter:

let inputABC = "ABC"
parseA inputABC // Success ('A', "BC")
let inputZBC = "ZBC"
parseA inputZBC // Failure "Expecting 'A'. Got 'Z'"

At this point, let's stop and review what is going on:

The pchar function has two inputs
We can provide one input (the char to match) and this results in a function being
returned.
We can then provide the second input (the stream of characters) to this parsing
function, and this creates the final Result value.
Here's a diagram of pchar again, but this time with the emphasis on partial application:

It's very important that you understand this logic before moving on, because the rest of the
post will build on this basic design.

Implementation 5. Encapsulating the parsing

function in a type
If we look at parseA (from the example above) we can see that it has a function type:
val parseA : string -> Result<char * string>

1189

Understanding Parser Combinators

That type is a bit complicated to use, so let's encapsulate it in a "wrapper" type called
Parser , like this:

type Parser<'T> = Parser of (string -> Result<'T * string>)

By encapsulating it, we'll go from this design:

to this design:

The change to the implementation is very simple. We just need to change the way the inner
function is returned.
That is, from this:
let pchar charToMatch =
let innerFn str =
...
// return the inner function
innerFn

to this:
let pchar charToMatch =
let innerFn str =
...
// return the "wrapped" inner function
Parser innerFn

Testing the wrapped function

Ok, now let's test again:
let parseA = pchar 'A'
let inputABC = "ABC"
parseA inputABC // compiler error

1190

Understanding Parser Combinators

But now we get a compiler error:

error FS0003: This value is not a function and cannot be applied

And of course that is because the function is wrapped in the Parser data structure! It's not
longer directly accessible.
So now we need a helper function that can extract the inner function and run it against the
input stream. Let's call it run !
Here's the implementation of run :
let run parser input =
// unwrap parser to get inner function
let (Parser innerFn) = parser
// call inner function with input
innerFn input

And now we can run the parseA parser against various inputs again:
let inputABC = "ABC"
run parseA inputABC // Success ('A', "BC")
let inputZBC = "ZBC"
run parseA inputZBC // Failure "Expecting 'A'. Got 'Z'"

That's it! We've got a basic Parser type! I hope that this all makes sense so far.

Combining two parsers in sequence: the "and

then" combinator
That last implementation is good enough for basic parsing logic. We'll revisit it later, but now
let's move up a level and develop some ways of combining parsers together -- the "parser
combinators" mentioned at the beginning.
We'll start with combining two parsers in sequence. For example, say that we want a parser
that matches "A" and then "B". We could try writing something like this:

1191

Understanding Parser Combinators

let parseA = pchar 'A'

let parseB = pchar 'B'
let parseAThenB = parseA >> parseB

but that gives us a compiler error, as the output of parseA does not match the input of
parseB , and so they cannot be composed like that.

If you are familiar with functional programming patterns, the need to chain a sequence of
wrapped types together like this happens frequently, and the solution is a bind function.
However, in this case, I won't implement bind but will instead go straight to an andThen
implemention.
The implementation logic will be as follows:
Run the first parser.
If there is a failure, return.
Otherwise, run the second parser with the remaining input.
If there is a failure, return.
If both parsers succeed, return a pair (tuple) that contains both parsed values.
Here's the code for andThen :

1192

Understanding Parser Combinators

let andThen parser1 parser2 =

let innerFn input =
// run parser1 with the input
let result1 = run parser1 input
// test the result for Failure/Success
match result1 with
| Failure err ->
// return error from parser1
Failure err
| Success (value1,remaining1) ->
// run parser2 with the remaining input
let result2 = run parser2 remaining1
// test the result for Failure/Success
match result2 with
| Failure err ->
// return error from parser2
Failure err
| Success (value2,remaining2) ->
// combine both values as a pair
let newValue = (value1,value2)
// return remaining input after parser2
Success (newValue,remaining2)
// return the inner function
Parser innerFn

The implementation follows the logic described above.

We'll also define an infix version of andThen so that we can use it like regular >>
composition:
let ( .>>. ) = andThen

Note: the parentheses are needed to define a custom operator, but are not needed in the
infix usage.
If we look at the signature of andThen :
val andThen :
parser1:Parser<'a> -> parser2:Parser<'b> -> Parser<'a * 'b>

we can see that it works for any two parsers, and they can be of different types ( 'a and
'b ).

1193

Understanding Parser Combinators

Testing andThen
Let's test it and see if it works!
First, create the compound parser:
let parseA = pchar 'A'
let parseB = pchar 'B'
let parseAThenB = parseA .>>. parseB

If you look at the types, you can see that all three values have type Parser :
val parseA : Parser<char>
val parseB : Parser<char>
val parseAThenB : Parser<char * char>

parseAThenB is of type Parser<char * char> meaning that the parsed value is a pair of

chars.
Now since the combined parser parseAThenB is just another Parser , we can use run with
it as before.
run parseAThenB "ABC" // Success (('A', 'B'), "C")
run parseAThenB "ZBC" // Failure "Expecting 'A'. Got 'Z'"
run parseAThenB "AZC" // Failure "Expecting 'B'. Got 'Z'"

You can see that in the success case, the pair ('A', 'B') was returned, and also that
failure happens when either letter is missing from the input.

Choosing between two parsers: the "or else"

combinator
Let's look at another important way of combining parsers -- the "or else" combinator.
For example, say that we want a parser that matches "A" or "B". How could we combine
them?
The implementation logic would be:
Run the first parser.

1194

Understanding Parser Combinators

On success, return the parsed value, along with the remaining input.
Otherwise, on failure, run the second parser with the original input...
...and in this case, return the result (success or failure) from the second parser.
Here's the code for orElse :
let orElse parser1 parser2 =
let innerFn input =
// run parser1 with the input
let result1 = run parser1 input
// test the result for Failure/Success
match result1 with
| Success result ->
// if success, return the original result
result1
| Failure err ->
// if failed, run parser2 with the input
let result2 = run parser2 input
// return parser2's result
result2
// return the inner function
Parser innerFn

And we'll define an infix version of orElse as well:

let ( <|> ) = orElse

If we look at the signature of orElse :

val orElse :
parser1:Parser<'a> -> parser2:Parser<'a> -> Parser<'a>

we can see that it works for any two parsers, but they must both be the same type 'a .

Testing orElse
Time to test it. First, create the combined parser:
let parseA = pchar 'A'
let parseB = pchar 'B'
let parseAOrElseB = parseA <|> parseB

1195

Understanding Parser Combinators

If you look at the types, you can see that all three values have type Parser<char> :
val parseA : Parser<char>
val parseB : Parser<char>
val parseAOrElseB : Parser<char>

Now if we run parseAOrElseB we can see that it successfully handles an "A" or a "B" as first
character.
run parseAOrElseB "AZZ" // Success ('A', "ZZ")
run parseAOrElseB "BZZ" // Success ('B', "ZZ")
run parseAOrElseB "CZZ" // Failure "Expecting 'B'. Got 'C'"

Combining andThen and orElse

With these two basic combinators, we can build more complex ones, such as "A and then (B
or C)".
Here's how to build up aAndThenBorC from simpler parsers:
let parseA = pchar 'A'
let parseB = pchar 'B'
let parseC = pchar 'C'
let bOrElseC = parseB <|> parseC
let aAndThenBorC = parseA .>>. bOrElseC

And here it is in action:

run aAndThenBorC "ABZ" // Success (('A', 'B'), "Z")
run aAndThenBorC "ACZ" // Success (('A', 'C'), "Z")
run aAndThenBorC "QBZ" // Failure "Expecting 'A'. Got 'Q'"
run aAndThenBorC "AQZ" // Failure "Expecting 'C'. Got 'Q'"

Note that the last example gives a misleading error. It says "Expecting 'C'" when it really
should say "Expecting 'B' or 'C'". We won't attempt to fix this right now, but in a later post
we'll implement better error messages.

Choosing from a list of parsers: "choice" and

"anyOf"
1196

Understanding Parser Combinators

This is where where the power of combinators starts kicking in, because with orElse in our
toolbox, we can use it to build even more combinators.
For example, let's say that we want choose from a list of parsers, rather than just two.
Well, that's easy. If we have a pairwise way of combining things, we can extend that to
combining an entire list using reduce (for more on working with reduce , see this post on
monoids ).
/// Choose any of a list of parsers
let choice listOfParsers =
List.reduce ( <|> ) listOfParsers

Note that this will fail if the input list is empty, but we will ignore that for now.
The signature of choice is:
val choice :
Parser<'a> list -> Parser<'a>

which shows us that, as expected, the input is a list of parsers, and the output is a single
parser.
With choice available, we can create an anyOf parser that matches any character in a list,
using the following logic:
The input is a list of characters
Each char in the list is transformed into a parser for that char using pchar
Finally, all the parsers are combined using choice
Here's the code:
/// Choose any of a list of characters
let anyOf listOfChars =
listOfChars
|> List.map pchar // convert into parsers
|> choice

Let's test it by creating a parser for any lowercase character and any digit character:
let parseLowercase =
anyOf ['a'..'z']
let parseDigit =
anyOf ['0'..'9']

1197

Understanding Parser Combinators

If we test them, they work as expected:

run parseLowercase "aBC" // Success ('a', "BC")
run parseLowercase "ABC" // Failure "Expecting 'z'. Got 'A'"
run parseDigit "1ABC" // Success ("1", "ABC")
run parseDigit "9ABC" // Success ("9", "ABC")
run parseDigit "|ABC" // Failure "Expecting '9'. Got '|'"

Again, the error messages are misleading. Any lowercase letter can be expected, not just 'z',
and any digit can be expected, not just '9'. As I said earlier, we'll work on the error messages
in a later post.

Review
Let's stop for now, and review what we have done:
We have created a type Parser that is a wrapper for a parsing function.
The parsing function takes an input (e.g. string) and attempts to match the input using
the criteria baked into the function.
If the match succeeds, the parsing function returns a Success with the matched item
and the remaining input.
If the match fails, the parsing function returns a Failure with reason for the failure.
And finally, we saw some "combinators" -- ways in which Parser s could be combined
to make a new Parser : andThen and orElse and choice .

Listing of the parser library so far

Here's the complete listing for the parsing library so far -- it's about 90 lines of code.
The source code displayed below is also available at this gist.
open System
/// Type that represents Success/Failure in parsing
type Result<'a> =
| Success of 'a
| Failure of string
/// Type that wraps a parsing function
type Parser<'T> = Parser of (string -> Result<'T * string>)
/// Parse a single character
let pchar charToMatch =

1198

Understanding Parser Combinators

// define a nested inner function

let innerFn str =
if String.IsNullOrEmpty(str) then
Failure "No more input"
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
Success (charToMatch,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
Failure msg
// return the "wrapped" inner function
Parser innerFn
/// Run a parser with some input
let run parser input =
// unwrap parser to get inner function
let (Parser innerFn) = parser
// call inner function with input
innerFn input
/// Combine two parsers as "A andThen B"
let andThen parser1 parser2 =
let innerFn input =
// run parser1 with the input
let result1 = run parser1 input
// test the result for Failure/Success
match result1 with
| Failure err ->
// return error from parser1
Failure err
| Success (value1,remaining1) ->
// run parser2 with the remaining input
let result2 = run parser2 remaining1
// test the result for Failure/Success
match result2 with
| Failure err ->
// return error from parser2
Failure err
| Success (value2,remaining2) ->
// combine both values as a pair
let newValue = (value1,value2)
// return remaining input after parser2
Success (newValue,remaining2)
// return the inner function
Parser innerFn

1199

Understanding Parser Combinators

/// Infix version of andThen

let ( .>>. ) = andThen
/// Combine two parsers as "A orElse B"
let orElse parser1 parser2 =
let innerFn input =
// run parser1 with the input
let result1 = run parser1 input
// test the result for Failure/Success
match result1 with
| Success result ->
// if success, return the original result
result1
| Failure err ->
// if failed, run parser2 with the input
let result2 = run parser2 input
// return parser2's result
result2
// return the inner function
Parser innerFn
/// Infix version of orElse
let ( <|> ) = orElse
/// Choose any of a list of parsers
let choice listOfParsers =
List.reduce ( <|> ) listOfParsers
/// Choose any of a list of characters
let anyOf listOfChars =
listOfChars
|> List.map pchar // convert into parsers
|> choice

Summary
In this post, we have created the foundations of a parsing library, and few simple
combinators.
In the next post, we'll build on this to create a library with many more combinators.
The source code for this post is available at this gist.

Further information
1200

Understanding Parser Combinators

If you are interesting in using this technique in production, be sure to investigate the
FParsec library for F#, which is optimized for real-world usage.
For more information about parser combinators in general, search the internet for
"Parsec", the Haskell library that influenced FParsec (and this post).
For some examples of using FParsec, try one of these posts:
Implementing a phrase search query for FogCreek's Kiln
A LOGO Parser
A Small Basic Parser
A C# Parser and building a C# compiler in F#
Write Yourself a Scheme in 48 Hours in F#
Parsing GLSL, the shading language of OpenGL

1201

Building a useful set of parser combinators

Building a useful set of parser

combinators
UPDATE: Slides and video from my talk on this topic
In this series, we are looking at how applicative parsers and parser combinators work.
In the first post, we created the foundations of a parsing library.
In this post, we'll build out the library with many other useful combinators. The
combinator names will be copied from those used by FParsec, so that you can easily
migrate to it.

1. map -- transforming the contents of a parser

When parsing, we often we want to match a particular string, such as a reserved word like
"if" or "where". A string is just a sequence of characters, so surely we could use the same
technique that we used to define anyOf in the first post, but using andThen instead of
orElse ?

Here's a (failed) attempt to create a pstring parser using that approach:

let pstring str =
str
|> Seq.map pchar // convert into parsers
|> Seq.reduce andThen

This doesn't work, because the output of andThen is different from the input (a tuple, not a
char) and so the reduce approach fails.
In order to solve this, we'll need to use a different technique.
To get started, let's try just matching a string of a specific length. Say, for example, that we
want to match a three digits in a row. Well, we can do that using andThen :
let parseDigit =
anyOf ['0'..'9']
let parseThreeDigits =
parseDigit .>>. parseDigit .>>. parseDigit

1202

Building a useful set of parser combinators

If we run it like this:

run parseThreeDigits "123A"

then we get the result:

Success ((('1', '2'), '3'), "A")

It does work, but the result contains a tuple inside a tuple (('1', '2'), '3') which is fugly
and hard to use. It would be so much more convenient to just have a simple string ( "123" ).
But in order to turn ('1', '2'), '3') into "123" , we'll need a function that can reach
inside of the parser and transform the result using an arbitrary passed in function.
Of course, what we need is the functional programmer's best friend, map .
To understand map and similar functions, I like to think of there being two worlds: a "Normal
World", where regular things live, and "Parser World", where Parser s live.
You can think of Parser World as a sort of "mirror" of Normal World because it obeys the
following rules:
Every type in Normal World (say char ) has a corresponding type in Parser World
( Parser<char> ).

And:
Every value in Normal World (say "ABC" ) has a corresponding value in Parser World
(that is, some Parser<string> that returns "ABC" ).
And:
Every function in Normal World (say char -> string ) has a corresponding function in
Parser World ( Parser<char> -> Parser<string> ).

1203

Building a useful set of parser combinators

Using this metaphor then, map transforms (or "lifts") a function in Normal World into a
function in Parser World.

And by the way, if you like this metaphor, I have a whole series of posts that develop it
further.
So that's what map does; how do we implement it?
The logic is:
Inside the innerFn , run the parser to get the result.
If the result was a success, apply the specified function to the success value to get a
new, transformed value, and...
...return the new, mapped, value instead of the original value.
Here's the code (I've named the map function mapP to avoid confusion with other map
functions):

1204

Building a useful set of parser combinators

let mapP f parser =

let innerFn input =
// run parser with the input
let result = run parser input
// test the result for Failure/Success
match result with
| Success (value,remaining) ->
// if success, return the value transformed by f
let newValue = f value
Success (newValue, remaining)
| Failure err ->
// if failed, return the error
Failure err
// return the inner function
Parser innerFn

If we look at the signature of mapP :

val mapP :
f:('a -> 'b) -> Parser<'a> -> Parser<'b>

we can see that it has exactly the signature we want, transforming a function 'a -> 'b into
a function Parser<'a> -> Parser<'b> .
It's common to define an infix version of map as well:
let ( <!> ) = mapP

And in the context of parsing, we'll often want to put the mapping function after the parser,
with the parameters flipped. This makes using map with the pipeline idiom much more
convenient:
let ( |>> ) x f = mapP f x

Parsing three digits with mapP

With mapP available, we can revisit parseThreeDigits and turn the tuple into a string.
Here's the code:

1205

Building a useful set of parser combinators

let parseDigit = anyOf ['0'..'9']

let parseThreeDigitsAsStr =
// create a parser that returns a tuple
let tupleParser =
parseDigit .>>. parseDigit .>>. parseDigit
// create a function that turns the tuple into a string
let transformTuple ((c1, c2), c3) =
String [| c1; c2; c3 |]
// use "map" to combine them
mapP transformTuple tupleParser

Or, if you prefer a more compact implementation:

let parseThreeDigitsAsStr =
(parseDigit .>>. parseDigit .>>. parseDigit)
|>> fun ((c1, c2), c3) -> String [| c1; c2; c3 |]

And if we test it, we get a string in the result now, rather than a tuple:
run parseThreeDigitsAsStr "123A" // Success ("123", "A")

We can go further, and map the string into an int:

let parseThreeDigitsAsInt =
mapP int parseThreeDigitsAsStr

If we test this, we get an int in the Success branch.

run parseThreeDigitsAsInt "123A" // Success (123, "A")

Let's check the type of parseThreeDigitsAsInt :

val parseThreeDigitsAsInt : Parser<int>

It's a Parser<int> now, not a Parser<char> or Parser<string> . The fact that a Parser can
contain any type, not just a char or string, is a key feature that will be very valuable when we
need to build more complex parsers.

1206

Building a useful set of parser combinators

2. apply and return -- lifting functions to the

world of Parsers
To achieve our goal of creating a parser that matches a list of characters, we need two more
helper functions which I will call returnP and applyP .
returnP simply transforms a normal value into a value in Parser World
applyP transforms a Parser containing a function ( Parser< 'a->'b > ) into a function in

Parser World ( Parser<'a> -> Parser<'b > )

Here's a diagram of returnP :

And here is the implementation of returnP :

let returnP x =
let innerFn input =
// ignore the input and return x
Success (x,input )
// return the inner function
Parser innerFn

The signature of returnP is just as we want:

val returnP :
'a -> Parser<'a>

Now here's a diagram of applyP :

1207

Building a useful set of parser combinators

And here is the implementation of applyP , which uses .>>. and map :
let applyP fP xP =
// create a Parser containing a pair (f,x)
(fP .>>. xP)
// map the pair by applying f to x
|> mapP (fun (f,x) -> f x)

The infix version of applyP is written as <*> :

let ( <*> ) = applyP

Again, the signature of applyP is just as we want:

val applyP :
Parser<('a -> 'b)> -> Parser<'a> -> Parser<'b>

Why do we need these two functions? Well, map will lift functions in Normal World into
functions in Parser World, but only for one-parameter functions.
What's great about returnP and applyP is that, together, they can lift any function in
Normal World into a function in Parser World, no matter how many parameters it has.
For example, we now can define a lift2 function that will lift a two parameter function into
Parser World like this:
// lift a two parameter function to Parser World
let lift2 f xP yP =
returnP f <*> xP <*> yP

The signature of lift2 is:

val lift2 :
f:('a -> 'b -> 'c) -> Parser<'a> -> Parser<'b> -> Parser<'c>

Here's a diagram of lift2 :

1208

Building a useful set of parser combinators

If you want to know more about how this works, check out my "man page" post on lift2 or
my explanation that involves the "Monadster".
Let's see some examples of using lift2 in practice. First, lifting integer addition to addition
of Parsers:
let addP =
lift2 (+)

The signature is:

val addP :
Parser<int> -> Parser<int> -> Parser<int>

which shows that addP does indeed take two Parser<int> parameters and returns another
Parser<int> .

And here's the startsWith function being lifted to Parser World:

let startsWith (str:string) prefix =
str.StartsWith(prefix)
let startsWithP =
lift2 startsWith

Again, the signature of startsWithP is parallel to the signature of startsWith , but lifted to
the world of Parsers.
val startsWith :
str:string -> prefix:string -> bool
val startsWithP :
Parser<string> -> Parser<string> -> Parser<bool>

1209

Building a useful set of parser combinators

3. sequence -- transforming a list of Parsers

into a single Parser
We now have the tools we need to implement our sequencing combinator! The logic will be:
Start with the list "cons" operator. This is the two-parameter function that prepends a
"head" element onto a "tail" of elements to make a new list.
Lift cons into the world of Parsers using lift2 .
We now have a a function that prepends a head Parser to a tail list of Parser s to
make a new list of Parser s, where:
The head Parser is the first element in the list of parsers that has been passed in.
The tail is generated by calling the same function recursively with the next parser in
the list.
When the input list is empty, just return a Parser containing an empty list.
Here's the implementation:
let rec sequence parserList =
// define the "cons" function, which is a two parameter function
let cons head tail = head::tail
// lift it to Parser World
let consP = lift2 cons
// process the list of parsers recursively
match parserList with
| [] ->
returnP []
| head::tail ->
consP head (sequence tail)

The signature of sequence is:

val sequence :
Parser<'a> list -> Parser<'a list>

which shows that the input is a list of Parser s and the output is a Parser containing a list
of elements.
Let's test it by creating a list of three parsers, and then combining them into one:

1210

Building a useful set of parser combinators

let parsers = [ pchar 'A'; pchar 'B'; pchar 'C' ]

let combined = sequence parsers
run combined "ABCD"
// Success (['A'; 'B'; 'C'], "D")

As you can see, when we run it we get back a list of characters, one for each parser in the
original list.

Implementing the pstring parser

At last, we can implement the parser that matches a string, which we'll call pstring .
The logic is:
Convert the string into a list of characters.
Convert each character into a Parser<char> .
Use sequence to convert the list of Parser<char> into a single Parser<char list> .
And finally, use map to convert the Parser<char list> into a Parser<string> .
Here's the code:
/// Helper to create a string from a list of chars
let charListToStr charList =
String(List.toArray charList)
// match a specific string
let pstring str =
str
// convert to list of char
|> List.ofSeq
// map each char to a pchar
|> List.map pchar
// convert to Parser<char list>
|> sequence
// convert Parser<char list> to Parser<string>
|> mapP charListToStr

Let's test it:

let parseABC = pstring "ABC"
run parseABC "ABCDE" // Success ("ABC", "DE")
run parseABC "A|CDE" // Failure "Expecting 'B'. Got '|'"
run parseABC "AB|DE" // Failure "Expecting 'C'. Got '|'"

1211

Building a useful set of parser combinators

It works as expected. Phew!

4. many and many1 -- matching a parser

multiple times
Another common need is to match a particular parser as many times as you can. For
example:
When matching an integer, you want to match as many digit characters as you can.
When matching a run of whitespace, you want to match as many whitespace characters
as you can.
There are slightly different requirements for these two cases.
When matching whitespace, it is often optional, so we want a "zero or more" matcher,
which we'll call many .
On the other hand, when matching digits for an integer, you want to match at least one
digit, so we want a "one or more" matcher, which we'll call many1 .
Before creating these, we'll define a helper function which matches a parser zero or more
times. The logic is:
Run the parser.
If the parser returns Failure (and this is key) just return an empty list. That is, this
function can never fail!
If the parser succeeds:
Call the function recursively to get the remaining values (which could also be an
empty list).
Then combine the first value and the remaining values.
Here's the code:

1212

Building a useful set of parser combinators

let rec parseZeroOrMore parser input =

// run parser with the input
let firstResult = run parser input
// test the result for Failure/Success
match firstResult with
| Failure err ->
// if parse fails, return empty list
([],input)
| Success (firstValue,inputAfterFirstParse) ->
// if parse succeeds, call recursively
// to get the subsequent values
let (subsequentValues,remainingInput) =
parseZeroOrMore parser inputAfterFirstParse
let values = firstValue::subsequentValues
(values,remainingInput)

With this helper function, we can easily define many now -- it's just a wrapper over
parseZeroOrMore :

/// match zero or more occurences of the specified parser

let many parser =
let rec innerFn input =
// parse the input -- wrap in Success as it always succeeds
Success (parseZeroOrMore parser input)
Parser innerFn

The signature of many shows that the output is indeed a list of values wrapped in a
Parser :

val many :
Parser<'a> -> Parser<'a list>

Now let's test many :

let manyA = many (pchar 'A')
// test some success cases
run manyA "ABCD" // Success (['A'], "BCD")
run manyA "AACD" // Success (['A'; 'A'], "CD")
run manyA "AAAD" // Success (['A'; 'A'; 'A'], "D")
// test a case with no matches
run manyA "|BCD" // Success ([], "|BCD")

1213

Building a useful set of parser combinators

Note that in the last case, even when there is nothing to match, the function succeeds.
There's nothing about many that restricts its use to single characters. For example, we can
use it to match repetitive string sequences too:
let manyAB = many (pstring "AB")
run manyAB "ABCD" // Success (["AB"], "CD")
run manyAB "ABABCD" // Success (["AB"; "AB"], "CD")
run manyAB "ZCD" // Success ([], "ZCD")
run manyAB "AZCD" // Success ([], "AZCD")

Finally, let's implement the original example of matching whitespace:

let whitespaceChar = anyOf [' '; '\t'; '\n']
let whitespace = many whitespaceChar
run whitespace "ABC" // Success ([], "ABC")
run whitespace " ABC" // Success ([' '], "ABC")
run whitespace "\tABC" // Success (['\t'], "ABC")

Defining many1
We can also define the "one or more" combinator many1 , using the following logic:
Run the parser.
If it fails, return the failure.
If it succeeds:
Call the helper function parseZeroOrMore to get the remaining values.
Then combine the first value and the remaining values.

1214

Building a useful set of parser combinators

/// match one or more occurences of the specified parser

let many1 parser =
let rec innerFn input =
// run parser with the input
let firstResult = run parser input
// test the result for Failure/Success
match firstResult with
| Failure err ->
Failure err // failed
| Success (firstValue,inputAfterFirstParse) ->
// if first found, look for zeroOrMore now
let (subsequentValues,remainingInput) =
parseZeroOrMore parser inputAfterFirstParse
let values = firstValue::subsequentValues
Success (values,remainingInput)
Parser innerFn

Again, the signature of many1 shows that the output is indeed a list of values wrapped in a
Parser :

val many1 :
Parser<'a> -> Parser<'a list>

Now let's test many1 :

// define parser for one digit
let digit = anyOf ['0'..'9']
// define parser for one or more digits
let digits = many1 digit
run digits "1ABC" // Success (['1'], "ABC")
run digits "12BC" // Success (['1'; '2'], "BC")
run digits "123C" // Success (['1'; '2'; '3'], "C")
run digits "1234" // Success (['1'; '2'; '3'; '4'], "")
run digits "ABC" // Failure "Expecting '9'. Got 'A'"

As we saw in an earlier example, the last case gives a misleading error. It says "Expecting
'9'" when it really should say "Expecting a digit". In the next post we'll fix this.

Parsing an integer
Using many1 , we can create a parser for an integer. The implementation logic is:
Create a parser for a digit.

1215

Building a useful set of parser combinators

Use many1 to get a list of digits.

Using map , transform the result (a list of digits) into a string and then into an int.
Here's the code:
let pint =
// helper
let resultToInt digitList =
// ignore int overflow for now
String(List.toArray digitList) |> int
// define parser for one digit
let digit = anyOf ['0'..'9']
// define parser for one or more digits
let digits = many1 digit
// map the digits to an int
digits
|> mapP resultToInt

And let's test it:

run pint "1ABC" // Success (1, "ABC")
run pint "12BC" // Success (12, "BC")
run pint "123C" // Success (123, "C")
run pint "1234" // Success (1234, "")
run pint "ABC" // Failure "Expecting '9'. Got 'A'"

5. opt -- matching a parser zero or one time

Sometimes we only want to match a parser zero or one time. For example, the pint parser
above does not handle negative values. To correct this, we need to be able to handle an
optional minus sign.
We can define an opt combinator easily:
Change the result of a specified parser to an option by mapping the result to Some .
Create another parser that always returns None .
Use <|> to choose the second ("None") parser if the first fails.
Here's the code:

1216

Building a useful set of parser combinators

let opt p =
let some = p |>> Some
let none = returnP None
some <|> none

Here's an example of it in use -- we match a digit followed by an optional semicolon:

let digit = anyOf ['0'..'9']
let digitThenSemicolon = digit .>>. opt (pchar ';')
run digitThenSemicolon "1;" // Success (('1', Some ';'), "")
run digitThenSemicolon "1" // Success (('1', None), "")

And here is pint rewritten to handle an optional minus sign:

let pint =
// helper
let resultToInt (sign,charList) =
let i = String(List.toArray charList) |> int
match sign with
| Some ch -> -i // negate the int
| None -> i
// define parser for one digit
let digit = anyOf ['0'..'9']
// define parser for one or more digits
let digits = many1 digit
// parse and convert
opt (pchar '-') .>>. digits
|>> resultToInt

Note that the resultToInt helper function now needs to handle the sign option as well as
the list of digits.
And here it is in action:
run pint "123C" // Success (123, "C")
run pint "-123C" // Success (-123, "C")

6. Throwing results away

1217

Building a useful set of parser combinators

We often want to match something in the input, but we don't care about the parsed value
itself. For example:
For a quoted string, we need to parse the quotes, but we don't need the quotes
themselves.
For a statement ending in a semicolon, we need to ensure the semicolon is there, but
we don't need the semicolon itself.
For whitespace separators, we need to ensure the whitespace is there, but we don't
need the actual whitespace data.
To handle these requirements, we will define some new combinators that throw away the
results of a parser:
p1 >>. p2 will apply p1 and p2 in sequence, just like .>>. , but throw away the

result of p1 and keep the result of p2 .

p1 .>> p2 will apply p1 and p2 in sequence, just like .>>. , but keep the result of
p1 and throw away the result of p2 .

These are easy to define -- just map over the result of .>>. , which is a tuple, and keep only
one element of the pair.
/// Keep only the result of the left side parser
let (.>>) p1 p2 =
// create a pair
p1 .>>. p2
// then only keep the first value
|> mapP (fun (a,b) -> a)
/// Keep only the result of the right side parser
let (>>.) p1 p2 =
// create a pair
p1 .>>. p2
// then only keep the second value
|> mapP (fun (a,b) -> b)

These combinators allow us to simplify the digitThenSemicolon example shown earlier:

let digit = anyOf ['0'..'9']
// use .>> below
let digitThenSemicolon = digit .>> opt (pchar ';')
run digitThenSemicolon "1;" // Success ('1', "")
run digitThenSemicolon "1" // Success ('1', "")

You can see that the result now is the same, whether or not the semicolon was present.

1218

Building a useful set of parser combinators

How about an example with whitespace?

The following code creates a parser that looks for "AB" followed by one or more whitespace
chars, followed by "CD".
let whitespaceChar = anyOf [' '; '\t'; '\n']
let whitespace = many1 whitespaceChar
let ab = pstring "AB"
let cd = pstring "CD"
let ab_cd = (ab .>> whitespace) .>>. cd
run ab_cd "AB \t\nCD" // Success (("AB", "CD"), "")

The result contains "AB" and "CD" only. The whitespace between them has been discarded.

Introducing between
A particularly common requirement is to look for a parser between delimiters such as quotes
or brackets.
Creating a combinator for this is trivial:
/// Keep only the result of the middle parser
let between p1 p2 p3 =
p1 >>. p2 .>> p3

And here it is in use, to parse a quoted integer:

let pdoublequote = pchar '"'
let quotedInteger = between pdoublequote pint pdoublequote
run quotedInteger "\"1234\"" // Success (1234, "")
run quotedInteger "1234" // Failure "Expecting '"'. Got '1'"

7. Parsing lists with separators

Another common requirement is parsing lists, seperated by something like commas or
whitespace.
To implement a "one or more" list, we need to:
First combine the separator and parser into one combined parser, but using >>. to
throw away the separator value.

1219

Building a useful set of parser combinators

Next, look for a list of the separator/parser combo using many .

Then prefix that with the first parser and combine the results.
Here's the code:
/// Parses one or more occurrences of p separated by sep
let sepBy1 p sep =
let sepThenP = sep >>. p
p .>>. many sepThenP
|>> fun (p,pList) -> p::pList

For the "zero or more" version, we can choose the empty list as an alternate if sepBy1 does
not find any matches:
/// Parses zero or more occurrences of p separated by sep
let sepBy p sep =
sepBy1 p sep <|> returnP []

Here's some tests for sepBy1 and sepBy , with results shown in the comments:
let comma = pchar ','
let digit = anyOf ['0'..'9']
let zeroOrMoreDigitList = sepBy digit comma
let oneOrMoreDigitList = sepBy1 digit comma
run oneOrMoreDigitList "1;" // Success (['1'], ";")
run oneOrMoreDigitList "1,2;" // Success (['1'; '2'], ";")
run oneOrMoreDigitList "1,2,3;" // Success (['1'; '2'; '3'], ";")
run oneOrMoreDigitList "Z;" // Failure "Expecting '9'. Got 'Z'"
run zeroOrMoreDigitList "1;" // Success (['1'], ";")
run zeroOrMoreDigitList "1,2;" // Success (['1'; '2'], ";")
run zeroOrMoreDigitList "1,2,3;" // Success (['1'; '2'; '3'], ";")
run zeroOrMoreDigitList "Z;" // Success ([], "Z;")

What about bind ?

One combinator that we haven't implemented so far is bind (or >>= ).
If you know anything about functional programming, or have seen my talk on FP patterns,
you'll know that bind is a powerful tool that can be used to implement many functions.
Up to this point, I thought that it would be better to show implementations for combinators
such as map and .>>. that were explicit and thus, hopefully, easier to understand.

1220

Building a useful set of parser combinators

But now that we have have some experience, let's implement bind and see what we can
do with it.
Here's the implementation of bindP (as I'll call it)
/// "bindP" takes a parser-producing function f, and a parser p
/// and passes the output of p into f, to create a new parser
let bindP f p =
let innerFn input =
let result1 = run p input
match result1 with
| Failure err ->
// return error from parser1
Failure err
| Success (value1,remainingInput) ->
// apply f to get a new parser
let p2 = f value1
// run parser with remaining input
run p2 remainingInput
Parser innerFn

The signature of bindP is:

val bindP :
f:('a -> Parser<'b>) -> Parser<'a> -> Parser<'b>

which conforms to a standard bind signature. The input f is a "diagonal" function ( 'a ->
Parser<'b> ) and the output is a "horizontal" function ( Parser<'a> -> Parser<'b> ). See this

post for more details on how bind works.

The infix version of bind is >>= . Note that the parameters are flipped: f is now the
second parameter which makes it more convenient for F#'s pipeline idiom.
let ( >>= ) p f = bindP f p

Reimplementing other combinators with bindP and

returnP
The combination of bindP and returnP can be used to re-implement many of the other
combinators. Here are some examples:

1221

Building a useful set of parser combinators

let mapP f =
bindP (f >> returnP)
let andThen p1 p2 =
p1 >>= (fun p1Result ->
p2 >>= (fun p2Result ->
returnP (p1Result,p2Result) ))
let applyP fP xP =
fP >>= (fun f ->
xP >>= (fun x ->
returnP (f x) ))
// (assuming "many" is defined)
let many1 p =
p >>= (fun head ->
many p >>= (fun tail ->
returnP (head::tail) ))

Note that the combinators that check the Failure path can not be implemented using
bind . These include orElse and many .

Review
We could keep building combinators for ever, but I think we have everything we need to
build a JSON parser now, so let's stop and review what we have done.
In the previous post we created these combinators:
.>>. ( andThen ) applies the two parsers in sequence and returns the results in a tuple.
<|> ( orElse ) applies the first parser, and if that fails, the second parsers.
choice extends orElse to choose from a list of parsers.

And in this post we created the following additional combinators:

bindP chains the result of a parser to another parser-producing function.
mapP transforms the result of a parser.
returnP lifts an normal value into the world of parsers.
applyP allows us to lift multi-parameter functions into functions that work on Parsers.
lift2 uses applyP to lift two-parameter functions into Parser World.
sequence converts a list of Parsers into a Parser containing a list.
many matches zero or more occurences of the specified parser.
many1 matches one or more occurences of the specified parser.
opt matches an optional occurrence of the specified parser.

1222

Building a useful set of parser combinators

.>> keeps only the result of the left side parser.

>>. keeps only the result of the right side parser.
between keeps only the result of the middle parser.
sepBy parses zero or more occurrences of a parser with a separator.
sepBy1 parses one or more occurrences of a parser with a separator.

I hope you can see why the concept of "combinators" is so powerful; given just a few basic
functions, we have built up a library of useful functions quickly and concisely.

Listing of the parser library so far

Here's the complete listing for the parsing library so far -- it's about 200 lines of code now!
The source code displayed below is also available at this gist.
open System
/// Type that represents Success/Failure in parsing
type Result<'a> =
| Success of 'a
| Failure of string
/// Type that wraps a parsing function
type Parser<'T> = Parser of (string -> Result<'T * string>)
/// Parse a single character
let pchar charToMatch =
// define a nested inner function
let innerFn str =
if String.IsNullOrEmpty(str) then
Failure "No more input"
else
let first = str.[0]
if first = charToMatch then
let remaining = str.[1..]
Success (charToMatch,remaining)
else
let msg = sprintf "Expecting '%c'. Got '%c'" charToMatch first
Failure msg
// return the "wrapped" inner function
Parser innerFn
/// Run a parser with some input
let run parser input =
// unwrap parser to get inner function
let (Parser innerFn) = parser
// call inner function with input
innerFn input

1223

Building a useful set of parser combinators

/// "bindP" takes a parser-producing function f, and a parser p

/// and passes the output of p into f, to create a new parser
let bindP f p =
let innerFn input =
let result1 = run p input
match result1 with
| Failure err ->
// return error from parser1
Failure err
| Success (value1,remainingInput) ->
// apply f to get a new parser
let p2 = f value1
// run parser with remaining input
run p2 remainingInput
Parser innerFn
/// Infix version of bindP
let ( >>= ) p f = bindP f p
/// Lift a value to a Parser
let returnP x =
let innerFn input =
// ignore the input and return x
Success (x,input)
// return the inner function
Parser innerFn
/// apply a function to the value inside a parser
let mapP f =
bindP (f >> returnP)
/// infix version of mapP
let ( <!> ) = mapP
/// "piping" version of mapP
let ( |>> ) x f = mapP f x
/// apply a wrapped function to a wrapped value
let applyP fP xP =
fP >>= (fun f ->
xP >>= (fun x ->
returnP (f x) ))
/// infix version of apply
let ( <*> ) = applyP
/// lift a two parameter function to Parser World
let lift2 f xP yP =
returnP f <*> xP <*> yP
/// Combine two parsers as "A andThen B"
let andThen p1 p2 =

1224

Building a useful set of parser combinators

p1 >>= (fun p1Result ->

p2 >>= (fun p2Result ->
returnP (p1Result,p2Result) ))
/// Infix version of andThen
let ( .>>. ) = andThen
/// Combine two parsers as "A orElse B"
let orElse p1 p2 =
let innerFn input =
// run parser1 with the input
let result1 = run p1 input
// test the result for Failure/Success
match result1 with
| Success result ->
// if success, return the original result
result1
| Failure err ->
// if failed, run parser2 with the input
let result2 = run p2 input
// return parser2's result
result2
// return the inner function
Parser innerFn
/// Infix version of orElse
let ( <|> ) = orElse
/// Choose any of a list of parsers
let choice listOfParsers =
List.reduce ( <|> ) listOfParsers
/// Choose any of a list of characters
let anyOf listOfChars =
listOfChars
|> List.map pchar // convert into parsers
|> choice
/// Convert a list of Parsers into a Parser of a list
let rec sequence parserList =
// define the "cons" function, which is a two parameter function
let cons head tail = head::tail
// lift it to Parser World
let consP = lift2 cons
// process the list of parsers recursively
match parserList with
| [] ->

1225

Building a useful set of parser combinators

returnP []
| head::tail ->
consP head (sequence tail)
/// (helper) match zero or more occurences of the specified parser
let rec parseZeroOrMore parser input =
// run parser with the input
let firstResult = run parser input
// test the result for Failure/Success
match firstResult with
| Failure err ->
// if parse fails, return empty list
([],input)
| Success (firstValue,inputAfterFirstParse) ->
// if parse succeeds, call recursively
// to get the subsequent values
let (subsequentValues,remainingInput) =
parseZeroOrMore parser inputAfterFirstParse
let values = firstValue::subsequentValues
(values,remainingInput)
/// matches zero or more occurences of the specified parser
let many parser =
let rec innerFn input =
// parse the input -- wrap in Success as it always succeeds
Success (parseZeroOrMore parser input)
Parser innerFn
/// matches one or more occurences of the specified parser
let many1 p =
p >>= (fun head ->
many p >>= (fun tail ->
returnP (head::tail) ))
/// Parses an optional occurrence of p and returns an option value.
let opt p =
let some = p |>> Some
let none = returnP None
some <|> none
/// Keep only the result of the left side parser
let (.>>) p1 p2 =
// create a pair
p1 .>>. p2
// then only keep the first value
|> mapP (fun (a,b) -> a)
/// Keep only the result of the right side parser
let (>>.) p1 p2 =
// create a pair
p1 .>>. p2
// then only keep the second value

1226

Building a useful set of parser combinators

|> mapP (fun (a,b) -> b)

/// Keep only the result of the middle parser
let between p1 p2 p3 =
p1 >>. p2 .>> p3
/// Parses one or more occurrences of p separated by sep
let sepBy1 p sep =
let sepThenP = sep >>. p
p .>>. many sepThenP
|>> fun (p,pList) -> p::pList
/// Parses zero or more occurrences of p separated by sep
let sepBy p sep =
sepBy1 p sep <|> returnP []

Summary
In this post, we have built on the basic parsing code from last time to create a library of a 15
or so combinators that can be combined to parse almost anything.
Soon, we'll use them to build a JSON parser, but before that, let's pause and take time to
clean up the error messages. That will be the topic of the next post.
The source code for this post is available at this gist.

1227

Improving the parser library

UPDATE: Slides and video from my talk on this topic
In this series, we are looking at how applicative parsers and parser combinators work.
In the first post, we created the foundations of a parsing library.
In the second post, we built out the library with many other useful combinators.
In this post, we'll rework the library to provide more helpful error messages.

1. Labelling a Parser
In some of the failing code examples from earlier posts, we got confusing errors:
let parseDigit = anyOf ['0'..'9']
run parseDigit "|ABC" // Failure "Expecting '9'. Got '|'"

parseDigit is defined as a choice of digit characters, so when the last choice ( '9' ) fails,

that is the error message we receive.

But that message is quite confusing. What we really want is to receive is an error that
mentions "digit", something like: Failure "Expecting digit. Got '|'" .
That is, what we need is a way of labeling parsers with a word like "digit" and then showing
that label when a failure occurs.
As a reminder, this is how the Parser type was defined in earlier posts:
type Parser<'a> = Parser of (string -> Result<'a * string>)

In order to add a label, we need to change it into a record structure:

type ParserLabel = string
/// A Parser structure has a parsing function & label
type Parser<'a> = {
parseFn : (string -> Result<'a * string>)
label: ParserLabel
}

1228

Improving the parser library

The record contains two fields: the parsing function ( parseFn ) and the label .
One problem is that the label is in the parser itself, but not in the Result , which means that
clients will not know how to display the label along with the error.
So let's add it to the Failure case of Result as well, in addition to the error message:
// Aliases
type ParserLabel = string
type ParserError = string
type Result<'a> =
| Success of 'a
| Failure of ParserLabel * ParserError

And while we are at it, let's define a helper function to display the result of a parse:
let printResult result =
match result with
| Success (value,input) ->
printfn "%A" value
| Failure (label,error) ->
printfn "Error parsing %s\n%s" label error

Updating the code

With this change to the definition of Parser and Result , we have to change some of the
basic functions, such as bindP :
/// "bindP" takes a parser-producing function f, and a parser p
/// and passes the output of p into f, to create a new parser
let bindP f p =
let label = "unknown" // <====== "label" is new!
let innerFn input =
...
match result1 with
| Failure (label,err) -> // <====== "label" is new!
...
| Success (value1,remainingInput) ->
...
{parseFn=innerFn; label=label} // <====== "parseFn" and "label" are new!

We have to make similar changes to returnP , orElse , and many . For the complete code,
see the gist linked to below.

Updating the label

1229

Improving the parser library

When we use a combinator to build a new compound parser, we will often want to assign a
new label to it. In order to do this, we replace the original parseFn with another one that
returns the new label.
Here's the code:
/// Update the label in the parser
let setLabel parser newLabel =
// change the inner function to use the new label
let newInnerFn input =
let result = parser.parseFn input
match result with
| Success s ->
// if Success, do nothing
Success s
| Failure (oldLabel,err) ->
// if Failure, return new label
Failure (newLabel,err) // <====== use newLabel here
// return the Parser
{parseFn=newInnerFn; label=newLabel} // <====== use newLabel here

And let's create an infix version of this called <?> :

/// infix version of setLabel
let ( <?> ) = setLabel

Let's test our new toy!

let parseDigit_WithLabel =
anyOf ['0'..'9']
<?> "digit"
run parseDigit_WithLabel "|ABC"
|> printResult

And the output is:

Error parsing digit
Unexpected '|'

The error message is now Error parsing digit rather than Expecting '9' . Much better!

Setting default labels:

1230

Improving the parser library

We can also set the default labels for certain combinators such as andThen and orElse
based on the inputs:
/// Combine two parsers as "A andThen B"
let andThen p1 p2 =
let label = sprintf "%s andThen %s" (getLabel p1) (getLabel p2)
p1 >>= (fun p1Result ->
p2 >>= (fun p2Result ->
returnP (p1Result,p2Result) ))
<?> label // <====== provide a custom label
// combine two parsers as "A orElse B"
let orElse parser1 parser2 =
// construct a new label
let label = // <====== provide a custom label
sprintf "%s orElse %s" (getLabel parser1) (getLabel parser2)

let innerFn input =

... etc ...
/// choose any of a list of characters
let anyOf listOfChars =
let label = sprintf "any of %A" listOfChars
listOfChars
|> List.map pchar
|> choice
<?> label // <====== provide a custom label

2. Replacing "pchar" with "satisfy"

One thing that has bothered me about all the implementations so far is pchar , the basic
primitive that all the other functions have built on.
I don't like that it is so tightly coupled to the input model. What happens if we want to parse
bytes from a binary format, or other kinds of input. All the combinators other than pchar are
loosely coupled. If we could decouple pchar as well, we would be set up for parsing any
stream of tokens, and that would make me happy!
At this point, I'll repeat one of my favorite FP slogans: "parameterize all the things!" In the
case of pchar , we'll remove the charToMatch parameter and replace it with a function -- a
predicate. We'll call the new function satisfy :

1231

Improving the parser library

/// Match an input token if the predicate is satisfied

let satisfy predicate label =
let innerFn input =
if String.IsNullOrEmpty(input) then
Failure (label,"No more input")
else
let first = input.[0]
if predicate first then // <====== use predicate here
let remainingInput = input.[1..]
Success (first,remainingInput)
else
let err = sprintf "Unexpected '%c'" first
Failure (label,err)
// return the parser
{parseFn=innerFn;label=label}

Other than the parameters, the only thing that has changed from the pchar implementation
is this one line:
let satisfy predicate label =
...
if predicate first then
...

With satisfy available, we can rewrite pchar :

/// parse a char
let pchar charToMatch =
let predicate ch = (ch = charToMatch)
let label = sprintf "%c" charToMatch
satisfy predicate label

Note that we are setting the label to be the charToMatch . This refactoring would not have
been as convenient before, because we didn't have the concept of "labels" yet, and so
pchar would not have been able to return a useful error message.

The satisfy function also lets us write more efficient versions of other parsers. For
example, parsing a digit looked like this originally:
/// parse a digit
let digitChar =
anyOf ['0'..'9']

But now we can rewrite it using a predicate directly, making it a lot more efficient:

1232

Improving the parser library

/// parse a digit

let digitChar =
let predicate = Char.IsDigit
let label = "digit"
satisfy predicate label

Similarly, we can create a more efficient whitespace parser too:

/// parse a whitespace char
let whitespaceChar =
let predicate = Char.IsWhiteSpace
let label = "whitespace"
satisfy predicate label

3. Adding position and context to error

messages
Another way to improve the error messages is to show the line and column that the error
occurred on.
Obviously, for simple one-liners, keeping track of the error location is not a problem, but
when you are parsing a 100 line JSON file, it will be very helpful.
In order to track the line and column we are going to have to abandon the simple string
input and replace it with something more complex, so let's start with that.

Defining a input that tracks position

First, we will need a Position type to store the line and column, with helper functions to
increment one column and one line:

1233

Improving the parser library

type Position = {
line : int
column : int
}
/// define an initial position
let initialPos = {line=0; column=0}
/// increment the column number
let incrCol pos =
{pos with column=pos.column + 1}
/// increment the line number and set the column to 0
let incrLine pos =
{line=pos.line + 1; column=0}

Next, we'll need to combine the input string with a position into a single "input state" type.
Since we are line oriented, we can make our lives easier and store the input string as a array
of lines rather than as one giant string:
/// Define the current input state
type InputState = {
lines : string[]
position : Position
}

We will also need a way to convert a string into a initial InputState :

/// Create a new InputState from a string
let fromStr str =
if String.IsNullOrEmpty(str) then
{lines=[||]; position=initialPos}
else
let separators = [| "\r\n"; "\n" |]
let lines = str.Split(separators, StringSplitOptions.None)
{lines=lines; position=initialPos}

Finally, and most importantly, we need a way to read the next character from the input -- let's
call it nextChar .
We know what the input for nextChar will be (an InputState ) but what should the output
look like?
If the input is at the end, we need a way to indicate that there is no next character, so in
that case return None .
Therefore in the case when a character is available, we will return Some .

1234

Improving the parser library

In addition, the input state will have changed because the column (or line) will have
been incremented as well.
So, putting this together, the input for nextChar is an InputState and the output is a pair
char option * InputState .

The logic for returning the next char will be as follows then:
If we are at the last character of the input, return EOF ( None ) and don't change the
state.
If the current column is not at the end of a line, return the character at that position and
change the state by incrementing the column position.
If the current column is at the end of a line, return a newline character and change the
state by incrementing the line position.
Here's the code:

1235

Improving the parser library

// return the current line

let currentLine inputState =
let linePos = inputState.position.line
if linePos < inputState.lines.Length then
inputState.lines.[linePos]
else
"end of file"
/// Get the next character from the input, if any
/// else return None. Also return the updated InputState
/// Signature: InputState -> InputState * char option
let nextChar input =
let linePos = input.position.line
let colPos = input.position.column
// three cases
// 1) if line >= maxLine ->
// return EOF
// 2) if col less than line length ->
// return char at colPos, increment colPos
// 3) if col at line length ->
// return NewLine, increment linePos
if linePos >= input.lines.Length then
input, None
else
let currentLine = currentLine input
if colPos < currentLine.Length then
let char = currentLine.[colPos]
let newPos = incrCol input.position
let newState = {input with position=newPos}
newState, Some char
else
// end of line, so return LF and move to next line
let char = '\n'
let newPos = incrLine input.position
let newState = {input with position=newPos}
newState, Some char

Unlike the earlier string implementation, the underlying array of lines is never altered or
copied -- only the position is changed. This means that making a new state each time the
position changes should be reasonably efficient, because the text is shared everywhere.
Let's quickly test that the implementation works. We'll create a helper function readAllChars
and then see what it returns for different inputs:

1236

Improving the parser library

let rec readAllChars input =

[
let remainingInput,charOpt = nextChar input
match charOpt with
| None ->
// end of input
()
| Some ch ->
// return first character
yield ch
// return the remaining characters
yield! readAllChars remainingInput
]

Here it is with some example inputs:

fromStr "" |> readAllChars // []
fromStr "a" |> readAllChars // ['a'; '\n']
fromStr "ab" |> readAllChars // ['a'; 'b'; '\n']
fromStr "a\nb" |> readAllChars // ['a'; '\n'; 'b'; '\n']

Note that the implementation returns a newline at the end of the input, even if the input
doesn't have one. I think that this is a feature, not a bug!

Changing the parser to use the input

We now need to change the Parser type again.
To start with, the Failure case needs to return some kind of data that indicates the
position, so we can show it in an error message.
We could just use the InputState as is, but let's be good and define a new type specially
for this use, called ParserPosition :
/// Stores information about the parser position for error messages
type ParserPosition = {
currentLine : string
line : int
column : int
}

We'll need some way to convert a InputState into a ParserPosition :

1237

Improving the parser library

let parserPositionFromInputState (inputState:Input) = {

currentLine = TextInput.currentLine inputState
line = inputState.position.line
column = inputState.position.column
}

And finally, we can update the Result type to include ParserPosition :

// Result type
type Result<'a> =
| Success of 'a
| Failure of ParserLabel * ParserError * ParserPosition

In addition, the Parser type needs to change from string to InputState :

type Input = TextInput.InputState // type alias
/// A Parser structure has a parsing function & label
type Parser<'a> = {
parseFn : (Input -> Result<'a * Input>)
label: ParserLabel
}

With all this extra information available, the printResult function can be enhanced to print
the text of the current line, along with a caret where the error is:
let printResult result =
match result with
| Success (value,input) ->
printfn "%A" value
| Failure (label,error,parserPos) ->
let errorLine = parserPos.currentLine
let colPos = parserPos.column
let linePos = parserPos.line
let failureCaret = sprintf "%*s^%s" colPos "" error
printfn "Line:%i Col:%i Error parsing %s\n%s\n%s" linePos colPos label errorLi
ne failureCaret

Let's test printResult with a dummy error value:

let exampleError =
Failure ("identifier", "unexpected |",
{currentLine = "123 ab|cd"; line=1; column=6})
printResult exampleError

1238

Improving the parser library

The output is shown below:

Line:1 Col:6 Error parsing identifier
123 ab|cd
^unexpected |

Much nicer than before!

Fixing up the run function

The run function now needs to take an InputState not a string. But we also want the
convenience of running against string input, so let's create two run functions, one that
takes an InputState and one that takes a string :
/// Run the parser on a InputState
let runOnInput parser input =
// call inner function with input
parser.parseFn input
/// Run the parser on a string
let run parser inputStr =
// call inner function with input
runOnInput parser (TextInput.fromStr inputStr)

Fixing up the combinators

We now have three items in the Failure case rather than two. This breaks some code but
is easy to fix. I'm tempted to create a special ParserError type so that it never happens
again, but for now, I'll just fix up the errors.
Here's a new version of satisfy :

1239

Improving the parser library

/// Match an input token if the predicate is satisfied

let satisfy predicate label =
let innerFn input =
let remainingInput,charOpt = TextInput.nextChar input
match charOpt with
| None ->
let err = "No more input"
let pos = parserPositionFromInputState input
//Failure (label,err) // <====== old version
Failure (label,err,pos) // <====== new version
| Some first ->
if predicate first then
Success (first,remainingInput)
else
let err = sprintf "Unexpected '%c'" first
let pos = parserPositionFromInputState input
//Failure (label,err) // <====== old version
Failure (label,err,pos) // <====== new version
// return the parser
{parseFn=innerFn;label=label}

Note that the failure case code is now Failure (label,err,pos) where the parser position is
built from the input state.
And here is bindP :
/// "bindP" takes a parser-producing function f, and a parser p
/// and passes the output of p into f, to create a new parser
let bindP f p =
let label = "unknown"
let innerFn input =
let result1 = runOnInput p input
match result1 with
| Failure (label,err,pos) -> // <====== new with pos
// return error from parser1
Failure (label,err,pos)
| Success (value1,remainingInput) ->
// apply f to get a new parser
let p2 = f value1
// run parser with remaining input
runOnInput p2 remainingInput
{parseFn=innerFn; label=label}

We can fix up the other functions in the same way.

Testing the positional errors

Let's test with a real parser now:

1240

Improving the parser library

let parseAB =
pchar 'A' .>>. pchar 'B'
<?> "AB"
run parseAB "A|C"
|> printResult

And the output is:

// Line:0 Col:1 Error parsing AB
// A|C
// ^Unexpected '|'

Excellent! I think we can stop now.

4. Adding some standard parsers to the library

In the previous posts, we've built parsers for strings and ints in passing, but now let's add
them to the core library, so that clients don't have to reinvent the wheel.
These parsers are based on those in the the FParsec library.
Let's start with some string-related parsers. I will present them without comment -- I hope
that the code is self-explanatory by now.

1241

Improving the parser library

/// parse a char

let pchar charToMatch =
// label is just the character
let label = sprintf "%c" charToMatch
let predicate ch = (ch = charToMatch)
satisfy predicate label
/// Choose any of a list of characters
let anyOf listOfChars =
let label = sprintf "anyOf %A" listOfChars
listOfChars
|> List.map pchar // convert into parsers
|> choice
<?> label
/// Convert a list of chars to a string
let charListToStr charList =
String(List.toArray charList)
/// Parses a sequence of zero or more chars with the char parser cp.
/// It returns the parsed chars as a string.
let manyChars cp =
many cp
|>> charListToStr
/// Parses a sequence of one or more chars with the char parser cp.
/// It returns the parsed chars as a string.
let manyChars1 cp =
many1 cp
|>> charListToStr
/// parse a specific string
let pstring str =
// label is just the string
let label = str
str
// convert to list of char
|> List.ofSeq
// map each char to a pchar
|> List.map pchar
// convert to Parser<char list>
|> sequence
// convert Parser<char list> to Parser<string>
|> mapP charListToStr
<?> label

Let's test pstring , for example:

1242

Improving the parser library

run (pstring "AB") "ABC"

Whitespace parsers
Whitespace is important in parsing, even if we do end up mostly throwing it away!
/// parse a whitespace char
let whitespaceChar =
let predicate = Char.IsWhiteSpace
let label = "whitespace"
satisfy predicate label
/// parse zero or more whitespace char
let spaces = many whitespaceChar
/// parse one or more whitespace char
let spaces1 = many1 whitespaceChar

And here's some whitespace tests:

run spaces " ABC"
|> printResult
// [' ']
run spaces "A"
|> printResult
// []
run spaces1 " ABC"
|> printResult
// [' ']
run spaces1 "A"
|> printResult
// Line:0 Col:0 Error parsing many1 whitespace
// A
// ^Unexpected 'A'

1243

Improving the parser library

Numeric parsers
Finally, we need a parser for ints and floats.
/// parse a digit
let digitChar =
let predicate = Char.IsDigit
let label = "digit"
satisfy predicate label
// parse an integer
let pint =
let label = "integer"
// helper
let resultToInt (sign,digits) =
let i = digits |> int // ignore int overflow for now
match sign with
| Some ch -> -i // negate the int
| None -> i
// define parser for one or more digits
let digits = manyChars1 digitChar
// an "int" is optional sign + one or more digits
opt (pchar '-') .>>. digits
|> mapP resultToInt
<?> label
// parse a float
let pfloat =
let label = "float"
// helper
let resultToFloat (((sign,digits1),point),digits2) =
let fl = sprintf "%s.%s" digits1 digits2 |> float
match sign with
| Some ch -> -fl // negate the float
| None -> fl
// define parser for one or more digits
let digits = manyChars1 digitChar
// a float is sign, digits, point, digits (ignore exponents for now)
opt (pchar '-') .>>. digits .>>. pchar '.' .>>. digits
|> mapP resultToFloat
<?> label

And some tests:

1244

Improving the parser library

run pint "-123Z"

|> printResult
// -123
run pint "-Z123"
|> printResult
// Line:0 Col:1 Error parsing integer
// -Z123
// ^Unexpected 'Z'
run pfloat "-123.45Z"
|> printResult
// -123.45
run pfloat "-123Z45"
|> printResult
// Line:0 Col:4 Error parsing float
// -123Z45
// ^Unexpected 'Z'

5. Backtracking
One more topic that we should discuss is "backtracking".
Let's say that you have two parsers: one to match the string A-1 and and another to match
the string A-2 . If the input is A-2 then the first parser will fail at the third character and the
second parser will be attempted.
Now the second parser must start at the beginning of the original sequence of characters,
not at the third character. That is, we need to undo the current position in the input stream
and go back to the first position.
If we were using a mutable input stream then this might be a tricky problem, but thankfully
we are using immutable data, and so "undoing" the position just means using the original
input value. And of course, this is exactly what combinators such as orElse ( <|> ) do.
In other words, we get backtracking "for free" when we use immutable input state. Yay!
Sometimes however, we don't want to backtrack. For example, let's say we have these
parsers:
let forExpression = the "for" keyword, then an identifier, then the "in" keyword, etc.
let ifExpression = the "if" keyword, then an identifier, then the "then" keyword, etc.
and we then create a combined expression parser that chooses between them:
let expression = forExpression <|> ifExpression

1245

Improving the parser library

Now, if the input stream is for &&& in something then the forExpression parser will error
when it hits the sequence &&& , because it is expecting a valid identifier. At this point we
don't want to backtrack and try the ifExpression -- we want to show an error such as
"identifier expected after 'for'".
The rule then is that: if input has been consumed successfully (in this case, the for
keyword was matched successfully) then do not backtrack.
We're not going to implement this rule in our simple library, but a proper library like FParsec
does implement this and also has support for bypassing it when needed.

Listing of the final parser library

The parsing library is up to 500 lines of code now, so I won't show it here. You can see it at
this gist.

Summary
In this post, we added better error handling and some more parsers.
Now we have everything we need to build a JSON parser! That will be the topic of the next
post.
The source code for this post is available at this gist.

1246

Writing a JSON parser from scratch

UPDATE: Slides and video from my talk on this topic
In this series, we are looking at how applicative parsers and parser combinators work.
In the first post, we created the foundations of a parsing library.
In the second post, we built out the library with many other useful combinators.
In the third post, we improved the error messages.
In this last post, we'll use the library we've written to build a JSON parser.

First, before we do anything else, we need to load the parser library script that we developed
over the last few posts, and then open the ParserLibrary namespace:
#load "ParserLibrary.fsx"
open System
open ParserLibrary

You can download ParserLibrary.fsx from here.

1. Building a model to represent the JSON spec

The JSON spec is available at json.org. I'll paraphase it here:
A value can be a string or a number or a bool or null or an object or an
array .

These structures can be nested.

A string is a sequence of zero or more Unicode characters, wrapped in double
quotes, using backslash escapes.
A number is very much like a C or Java number, except that the octal and hexadecimal
formats are not used.
A boolean is the literal true or false
A null is the literal null
An object is an unordered set of name/value pairs.
An object begins with { (left brace) and ends with } (right brace).
Each name is followed by : (colon) and the name/value pairs are separated by
, (comma).

1247

Writing a JSON parser from scratch

An array is an ordered collection of values.

An array begins with [ (left bracket) and ends with ] (right bracket).
Values are separated by , (comma).
Whitespace can be inserted between any pair of tokens.
In F#, this definition can be modelled naturally as:
type JValue =
| JString of string
| JNumber of float
| JBool of bool
| JNull
| JObject of Map<string, Json>
| JArray of Json list

So the goal of our JSON parser is:

Given a string, we want to output a JValue value.

2. Getting started with Null and Bool

Let's start with the simplest tasks -- parsing the literal values for null and the booleans.

Parsing Null
Parsing the null literal is trivial. The logic will be:
Match the string "null".
Map the result to the JNull case.
Here's the code:
let jNull =
pstring "null"
|>> (fun _ -> JNull) // map to JNull
<?> "null" // give it a label

Note that we don't actually care about the value returned by the parser because we know in
advance that it is going to be "null"!
This is a common situation, so let's write a little utility function, >>% to make this look nicer:

1248

Writing a JSON parser from scratch

// applies the parser p, ignores the result, and returns x.

let (>>%) p x =
p |>> (fun _ -> x)

Now we can rewrite jNull as follows:

let jNull =
pstring "null"
>>% JNull // using new utility combinator
<?> "null"

Let's test:
run jNull "null"
// Success: JNull
run jNull "nulp" |> printResult
// Line:0 Col:3 Error parsing null
// nulp
// ^Unexpected 'p'

That looks good. Let's try another one!

Parsing Bool
The bool parser will be similar to null:
Create a parser to match "true".
Create a parser to match "false".
And then choose between them using <|> .
Here's the code:
let jBool =
let jtrue =
pstring "true"
>>% JBool true // map to JBool
let jfalse =
pstring "false"
>>% JBool false // map to JBool
// choose between true and false
jtrue <|> jfalse
<?> "bool" // give it a label

1249

Writing a JSON parser from scratch

And here are some tests:

run jBool "true"
// Success: JBool true
run jBool "false"
// Success: JBool false
run jBool "truX" |> printResult
// Line:0 Col:0 Error parsing bool
// truX
// ^Unexpected 't'

Note that the error is misleading due to the backtracking issue discussed in the previous
post. Since "true" failed, it is trying to parse "false" now, and "t" is an unexpected character.

3. Parsing String
Now for something more complicated -- strings.
The spec for string parsing is available as a "railway diagram" like this:

All diagrams sourced from json.org.

To build a parser from a diagram like this, we work from the bottom up, building small
"primitive" parsers which we then combine into larger ones.

1250

Writing a JSON parser from scratch

Let's start with "any unicode character other than quote and backslash". We have a simple
condition to test, so we can just use the satisfy function:
let jUnescapedChar =
let label = "char"
satisfy (fun ch -> ch <> '\\' && ch <> '\"') label

We can test it immediately:

run jUnescapedChar "a" // Success 'a'
run jUnescapedChar "\\" |> printResult
// Line:0 Col:0 Error parsing char
// \
// ^Unexpected '\'

Ok, good.

Escaped characters
Now what about the next case, the escaped characters?
In this case we have a list of strings to match ( "\"" , "\n" , etc) and for each of these, a
character to use as the result.
The logic will be:
First define a list of pairs in the form (stringToMatch, resultChar) .
For each of these, build a parser using pstring stringToMatch >>% resultChar) .
Finally, combine all these parsers together using the choice function.
Here's the code:

1251

Writing a JSON parser from scratch

/// Parse an escaped char

let jEscapedChar =
[
// (stringToMatch, resultChar)
("\\\"",'\"') // quote
("\\\\",'\\') // reverse solidus
("\\/",'/') // solidus
("\\b",'\b') // backspace
("\\f",'\f') // formfeed
("\\n",'\n') // newline
("\\r",'\r') // cr
("\\t",'\t') // tab
]
// convert each pair into a parser
|> List.map (fun (toMatch,result) ->
pstring toMatch >>% result)
// and combine them into one
|> choice
<?> "escaped char" // set label

And again, let's test it immediately:

run jEscapedChar "\\\\" // Success '\'
run jEscapedChar "\\t" // Success '\009'
run jEscapedChar "a" |> printResult
// Line:0 Col:0 Error parsing escaped char
// a
// ^Unexpected 'a'

It works nicely!

Unicode characters
The final case is the parsing of unicode characters with hex digits.
The logic will be:
First define the primitives for backslash , u and hexdigit .
Combine them together, using four hexdigit s.
The output of the parser will be a nested, ugly tuple, so we need a helper function to
convert the digits to an int, and then a char.
Here's the code:

1252

Writing a JSON parser from scratch

/// Parse a unicode char

let jUnicodeChar =
// set up the "primitive" parsers
let backslash = pchar '\\'
let uChar = pchar 'u'
let hexdigit = anyOf (['0'..'9'] @ ['A'..'F'] @ ['a'..'f'])
// convert the parser output (nested tuples)
// to a char
let convertToChar (((h1,h2),h3),h4) =
let str = sprintf "%c%c%c%c" h1 h2 h3 h4
Int32.Parse(str,Globalization.NumberStyles.HexNumber) |> char
// set up the main parser
backslash >>. uChar >>. hexdigit .>>. hexdigit .>>. hexdigit .>>. hexdigit
|>> convertToChar

And let's test with a smiley face -- \u263A .

run jUnicodeChar "\\u263A"

The complete String parser

Putting it all together now:
Define a primitive for quote
Define a jchar as a choice between jUnescapedChar , jEscapedChar , and
jUnicodeChar .

The whole parser is then zero or many jchar between two quotes.
let quotedString =
let quote = pchar '\"' <?> "quote"
let jchar = jUnescapedChar <|> jEscapedChar <|> jUnicodeChar
// set up the main parser
quote >>. manyChars jchar .>> quote

One more thing, which is to wrap the quoted string in a JString case and give it a label:

1253

Writing a JSON parser from scratch

/// Parse a JString

let jString =
// wrap the string in a JString
quotedString
|>> JString // convert to JString
<?> "quoted string" // add label

Let's test the complete jString function:

run jString "\"\"" // Success ""
run jString "\"a\"" // Success "a"
run jString "\"ab\"" // Success "ab"
run jString "\"ab\\tde\"" // Success "ab\tde"
run jString "\"ab\\u263Ade\"" // Success "ab?de"

4. Parsing Number
The "railway diagram" for Number parsing is:

Again, we'll work bottom up. Let's start with the most primitive components, the single chars
and digits:

1254

Writing a JSON parser from scratch

let optSign = opt (pchar '-')

let zero = pstring "0"
let digitOneNine =
satisfy (fun ch -> Char.IsDigit ch && ch <> '0') "1-9"
let digit =
satisfy (fun ch -> Char.IsDigit ch ) "digit"
let point = pchar '.'
let e = pchar 'e' <|> pchar 'E'
let optPlusMinus = opt (pchar '-' <|> pchar '+')

Now let's build the "integer" part of the number. This is either:
The digit zero, or,
A nonZeroInt , which is a digitOneNine followed by zero or more normal digits.
let nonZeroInt =
digitOneNine .>>. manyChars digit
|>> fun (first,rest) -> string first + rest
let intPart = zero <|> nonZeroInt

Note that, for the nonZeroInt parser, we have to combine the output of digitOneNine (a
char) with manyChars digit (a string) so a simple map function is needed.
The optional fractional part is a decimal point followed by one or more digits:
let fractionPart = point >>. manyChars1 digit

And the exponent part is an e followed by an optional sign, followed by one or more digits:
let exponentPart = e >>. optPlusMinus .>>. manyChars1 digit

With these components, we can assemble the whole number:

optSign .>>. intPart .>>. opt fractionPart .>>. opt exponentPart
|>> convertToJNumber
<?> "number" // add label

1255

Writing a JSON parser from scratch

We haven't defined convertToJNumber yet though. This function will take the four-tuple
output by the parser and convert it into a float.
Now rather than writing custom float logic, we're going to be lazy and let the .NET framework
to the conversion for us! That is, each of the components will be turned into a string,
concatenated, and the whole string parsed into a float.
The problem is that some of the components (like the sign and exponent) are optional. Let's
write a helper that converts an option to a string using a passed in function, but if the option
is None return the empty string.
I'm going to call it |>? but it doesn't really matter because it is only used locally within the
jNumber parser.

// utility function to convert an optional value to a string, or "" if missing

let ( |>? ) opt f =
match opt with
| None -> ""
| Some x -> f x

Now we can create convertToJNumber :

The sign is converted to a string.
The fractional part is converted to a string, prefixed with a decimal point.
The exponent part is converted to a string, with the sign of the exponent also being
converted to a string.

1256

Writing a JSON parser from scratch

let convertToJNumber (((optSign,intPart),fractionPart),expPart) =

// convert to strings and let .NET parse them! - crude but ok for now.
let signStr =
optSign
|>? string // e.g. "-"
let fractionPartStr =
fractionPart
|>? (fun digits -> "." + digits ) // e.g. ".456"
let expPartStr =
expPart
|>? fun (optSign, digits) ->
let sign = optSign |>? string
"e" + sign + digits // e.g. "e-12"
// add the parts together and convert to a float, then wrap in a JNumber
(signStr + intPart + fractionPartStr + expPartStr)
|> float
|> JNumber

It's pretty crude, and converting things to strings can be slow, so feel free to write a better
version.
With that, we have everything we need for the complete jNumber function:
/// Parse a JNumber
let jNumber =
// set up the "primitive" parsers
let optSign = opt (pchar '-')
let zero = pstring "0"
let digitOneNine =
satisfy (fun ch -> Char.IsDigit ch && ch <> '0') "1-9"
let digit =
satisfy (fun ch -> Char.IsDigit ch ) "digit"
let point = pchar '.'
let e = pchar 'e' <|> pchar 'E'
let optPlusMinus = opt (pchar '-' <|> pchar '+')
let nonZeroInt =
digitOneNine .>>. manyChars digit
|>> fun (first,rest) -> string first + rest

1257

Writing a JSON parser from scratch

let intPart = zero <|> nonZeroInt

let fractionPart = point >>. manyChars1 digit
let exponentPart = e >>. optPlusMinus .>>. manyChars1 digit
// utility function to convert an optional value to a string, or "" if missing
let ( |>? ) opt f =
match opt with
| None -> ""
| Some x -> f x
let convertToJNumber (((optSign,intPart),fractionPart),expPart) =
// convert to strings and let .NET parse them! - crude but ok for now.
let signStr =
optSign
|>? string // e.g. "-"
let fractionPartStr =
fractionPart
|>? (fun digits -> "." + digits ) // e.g. ".456"
let expPartStr =
expPart
|>? fun (optSign, digits) ->
let sign = optSign |>? string
"e" + sign + digits // e.g. "e-12"
// add the parts together and convert to a float, then wrap in a JNumber
(signStr + intPart + fractionPartStr + expPartStr)
|> float
|> JNumber
// set up the main parser
optSign .>>. intPart .>>. opt fractionPart .>>. opt exponentPart
|>> convertToJNumber
<?> "number" // add label

It's a bit long-winded, but each component follows the spec, so I think it is still quite
readable.
Let's start testing it:
run jNumber "123" // JNumber 123.0
run jNumber "-123" // JNumber -123.0
run jNumber "123.4" // JNumber 123.4

And what about some failing cases?

1258

Writing a JSON parser from scratch

run jNumber "-123." // JNumber -123.0 -- should fail!

run jNumber "00.1" // JNumber 0 -- should fail!

Hmm. Something went wrong! These cases should fail, surely?

Well, no. What's happening in the -123. case is that the parser is consuming everything up
the to decimal point and then stopping, leaving the decimal point to be matched by the next
parser! So, not an error.
Similarly, in the 00.1 case, the parser is consuming only the first 0 then stopping, leaving
the rest of the input ( 0.4 ) to be matched by the next parser. Again, not an error.
To fix this properly is out of scope, so let's just add some whitespace to the parser to force it
to terminate.
let jNumber_ = jNumber .>> spaces1

Now let's test again:

run jNumber_ "123" // JNumber 123.0
run jNumber_ "-123" // JNumber -123.0
run jNumber_ "-123." |> printResult
// Line:0 Col:4 Error parsing number andThen many1 whitespace
// -123.
// ^Unexpected '.'

and we find the error is being detected properly now.

Let's test the fractional part:
run jNumber_ "123.4" // JNumber 123.4
run jNumber_ "00.4" |> printResult
// Line:0 Col:1 Error parsing number andThen many1 whitespace
// 00.4
// ^Unexpected '0'

and the exponent part now:

1259

Writing a JSON parser from scratch

// exponent only
run jNumber_ "123e4" // JNumber 1230000.0
// fraction and exponent
run jNumber_ "123.4e5" // JNumber 12340000.0
run jNumber_ "123.4e-5" // JNumber 0.001234

It's all looking good so far. Onwards and upwards!

5. Parsing Array
Next up is the Array case. Again, we can use the railway diagram to guide the
implementation:

We will start with the primitives again. Note that we are adding optional whitespace after
each token:
let jArray =
let left = pchar '[' .>> spaces
let right = pchar ']' .>> spaces
let comma = pchar ',' .>> spaces
let value = jValue .>> spaces

And then we create a list of values separated by a comma, with the whole list between the
left and right brackets.
let jArray =
...
// set up the list parser
let values = sepBy1 value comma
// set up the main parser
between left values right
|>> JArray
<?> "array"

Hold on -- what is this jValue ?

1260

Writing a JSON parser from scratch

let jArray =
...
let value = jValue .>> spaces // <=== what is "jValue"?
...

Well, the spec says that an Array can contain a list of values, so we'll assume that we have
a jValue parser that can parse them.
But to parse a JValue , we need to parse a Array first!
We have hit a common problem in parsing -- mutually recursive definitions. We need a
JValue parser to build an Array , but we need an Array parser to build a JValue .

How can we deal with this?

Forward references
The trick is to create a forward reference, a dummy JValue parser that we can use right
now to define the Array parser, and then later on, we will fix up the forward reference with
the "real" JValue parser.
This is one time where mutable references come in handy!
We will need a helper function to assist us with this, and the logic will be as follows:
Define a dummy parser that will be replaced later.
Define a real parser that forwards the input stream to the dummy parser.
Return both the real parser and a reference to the dummy parser.
Now when the client fixes up the reference, the real parser will forward the input to the new
parser that has replaced the dummy parser.
Here's the code:

1261

Writing a JSON parser from scratch

let createParserForwardedToRef<'a>() =
let dummyParser=
let innerFn input : Result<'a * Input> = failwith "unfixed forwarded parser"
{parseFn=innerFn; label="unknown"}
// ref to placeholder Parser
let parserRef = ref dummyParser
// wrapper Parser
let innerFn input =
// forward input to the placeholder
runOnInput !parserRef input
let wrapperParser = {parseFn=innerFn; label="unknown"}
wrapperParser, parserRef

With this in place, we can create a placeholder for a parser of type JValue :
let jValue,jValueRef = createParserForwardedToRef<JValue>()

Finishing up the Array parser

Going back to the Array parser, we can now compile it successfully, using the jValue
placeholder:
let jArray =
// set up the "primitive" parsers
let left = pchar '[' .>> spaces
let right = pchar ']' .>> spaces
let comma = pchar ',' .>> spaces
let value = jValue .>> spaces
// set up the list parser
let values = sepBy1 value comma
// set up the main parser
between left values right
|>> JArray
<?> "array"

If we try to test it now, we get an exception because we haven't fixed up the reference:

1262

Writing a JSON parser from scratch

run jArray "[ 1, 2 ]"

// System.Exception: unfixed forwarded parser

So for now, let's fix up the reference to use one of the parsers that we have already created,
such as jNumber :
jValueRef := jNumber

Now we can successfully test the jArray function, as long as we are careful to only use
numbers in our array!
run jArray "[ 1, 2 ]"
// Success (JArray [JNumber 1.0; JNumber 2.0],
run jArray "[ 1, 2, ]" |> printResult
// Line:0 Col:6 Error parsing array
// [ 1, 2, ]
// ^Unexpected ','

6. Parsing Object
The parser for Object is very similar to the one for Array .
First, the railway diagram:

Using this, we can create the parser directly, so I'll present it here without comment:

1263

Writing a JSON parser from scratch

let jObject =
// set up the "primitive" parsers
let left = pchar '{' .>> spaces
let right = pchar '}' .>> spaces
let colon = pchar ':' .>> spaces
let comma = pchar ',' .>> spaces
let key = quotedString .>> spaces
let value = jValue .>> spaces
// set up the list parser
let keyValue = (key .>> colon) .>>. value
let keyValues = sepBy1 keyValue comma
// set up the main parser
between left keyValues right
|>> Map.ofList // convert the list of keyValues into a Map
|>> JObject // wrap in JObject
<?> "object" // add label

A bit of testing to make sure it works (but remember, only numbers are supported as values
for now).
run jObject """{ "a":1, "b" : 2 }"""
// JObject (map [("a", JNumber 1.0); ("b", JNumber 2.0)]),
run jObject """{ "a":1, "b" : 2, }""" |> printResult
// Line:0 Col:18 Error parsing object
// { "a":1, "b" : 2, }
// ^Unexpected ','

7. Putting it all together

Finally, we can combine all six of the parsers using the choice combinator, and we can
assign this to the JValue parser reference that we created earlier:
jValueRef := choice
[
jNull
jBool
jNumber
jString
jArray
jObject
]

1264

Writing a JSON parser from scratch

And now we are ready to rock and roll!

Testing the complete parser: example 1

Here's an example of a JSON string that we can attempt to parse:
let example1 = """{
"name" : "Scott",
"isMale" : true,
"bday" : {"year":2001, "month":12, "day":25 },
"favouriteColors" : ["blue", "green"]
}"""
run jValue example1

And here is the result:

JObject
(map
[("bday", JObject(map
[("day", JNumber 25.0);
("month", JNumber 12.0);
("year", JNumber 2001.0)]));
("favouriteColors", JArray [JString "blue"; JString "green"]);
("isMale", JBool true);
("name", JString "Scott")
])

Testing the complete parser: example 2

Here's one from the example page on json.org:

1265

Writing a JSON parser from scratch

let example2= """{"widget": {

"debug": "on",
"window": {
"title": "Sample Konfabulator Widget",
"name": "main_window",
"width": 500,
"height": 500
},
"image": {
"src": "Images/Sun.png",
"name": "sun1",
"hOffset": 250,
"vOffset": 250,
"alignment": "center"
},
"text": {
"data": "Click Here",
"size": 36,
"style": "bold",
"name": "text1",
"hOffset": 250,
"vOffset": 100,
"alignment": "center",
"onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
}
}} """
run jValue example2

And here is the result:

1266

Writing a JSON parser from scratch

JObject(map
[("widget",JObject(map
[("debug", JString "on");
("image",JObject(map
[("alignment", JString "center");
("hOffset", JNumber 250.0); ("name", JString "sun1");
("src", JString "Images/Sun.png");
("vOffset", JNumber 250.0)]));
("text",JObject(map
[("alignment", JString "center");
("data", JString "Click Here");
("hOffset", JNumber 250.0);
("name", JString "text1");
("onMouseUp", JString "sun1.opacity = (sun1.opacity / 100) * 90;")
;
("size", JNumber 36.0);
("style", JString "bold");
("vOffset", JNumber 100.0)]));
("window",JObject(map
[("height", JNumber 500.0);
("name", JString "main_window");
("title", JString "Sample Konfabulator Widget");
("width", JNumber 500.0)]))]))]),

Complete listing of the JSON parser

Here's the complete listing for the JSON parser -- it's about 250 lines of useful code.
The source code displayed below is also available at this gist.
#load "ParserLibrary.fsx"
open System
open ParserLibrary
(*
// -------------------------------JSON spec from http://www.json.org/
// -------------------------------The JSON spec is available at [json.org](http://www.json.org/). I'll paraphase it here
:
* A `value` can be a `string` or a `number` or a `bool` or `null` or an `object` or an
`array`.
* These structures can be nested.
* A `string` is a sequence of zero or more Unicode characters, wrapped in double quote
s, using backslash escapes.
* A `number` is very much like a C or Java number, except that the octal and hexadecim

1267

Writing a JSON parser from scratch

al formats are not used.

* A `boolean` is the literal `true` or `false`
* A `null` is the literal `null`
* An `object` is an unordered set of name/value pairs.
* An object begins with { (left brace) and ends with } (right brace).
* Each name is followed by : (colon) and the name/value pairs are separated by , (co
mma).
* An `array` is an ordered collection of values.
* An array begins with [ (left bracket) and ends with ] (right bracket).
* Values are separated by , (comma).
* Whitespace can be inserted between any pair of tokens.
*)
type JValue =
| JString of string
| JNumber of float
| JBool of bool
| JNull
| JObject of Map<string, JValue>
| JArray of JValue list

// ======================================
// Forward reference
// ======================================
/// Create a forward reference
let createParserForwardedToRef<'a>() =
let dummyParser=
let innerFn input : Result<'a * Input> = failwith "unfixed forwarded parser"
{parseFn=innerFn; label="unknown"}
// ref to placeholder Parser
let parserRef = ref dummyParser
// wrapper Parser
let innerFn input =
// forward input to the placeholder
runOnInput !parserRef input
let wrapperParser = {parseFn=innerFn; label="unknown"}
wrapperParser, parserRef
let jValue,jValueRef = createParserForwardedToRef<JValue>()
// ======================================
// Utility function
// ======================================
// applies the parser p, ignores the result, and returns x.
let (>>%) p x =

1268

Writing a JSON parser from scratch

p |>> (fun _ -> x)

// ======================================
// Parsing a JNull
// ======================================
let jNull =
pstring "null"
>>% JNull // map to JNull
<?> "null" // give it a label
// ======================================
// Parsing a JBool
// ======================================
let jBool =
let jtrue =
pstring "true"
>>% JBool true // map to JBool
let jfalse =
pstring "false"
>>% JBool false // map to JBool
// choose between true and false
jtrue <|> jfalse
<?> "bool" // give it a label

// ======================================
// Parsing a JString
// ======================================
/// Parse an unescaped char
let jUnescapedChar =
satisfy (fun ch -> ch <> '\\' && ch <> '\"') "char"
/// Parse an escaped char
let jEscapedChar =
[
// (stringToMatch, resultChar)
("\\\"",'\"') // quote
("\\\\",'\\') // reverse solidus
("\\/",'/') // solidus
("\\b",'\b') // backspace
("\\f",'\f') // formfeed
("\\n",'\n') // newline
("\\r",'\r') // cr
("\\t",'\t') // tab
]
// convert each pair into a parser
|> List.map (fun (toMatch,result) ->
pstring toMatch >>% result)
// and combine them into one

1269

Writing a JSON parser from scratch

|> choice
/// Parse a unicode char
let jUnicodeChar =
// set up the "primitive" parsers
let backslash = pchar '\\'
let uChar = pchar 'u'
let hexdigit = anyOf (['0'..'9'] @ ['A'..'F'] @ ['a'..'f'])
// convert the parser output (nested tuples)
// to a char
let convertToChar (((h1,h2),h3),h4) =
let str = sprintf "%c%c%c%c" h1 h2 h3 h4
Int32.Parse(str,Globalization.NumberStyles.HexNumber) |> char
// set up the main parser
backslash >>. uChar >>. hexdigit .>>. hexdigit .>>. hexdigit .>>. hexdigit
|>> convertToChar

/// Parse a quoted string

let quotedString =
let quote = pchar '\"' <?> "quote"
let jchar = jUnescapedChar <|> jEscapedChar <|> jUnicodeChar
// set up the main parser
quote >>. manyChars jchar .>> quote
/// Parse a JString
let jString =
// wrap the string in a JString
quotedString
|>> JString // convert to JString
<?> "quoted string" // add label
// ======================================
// Parsing a JNumber
// ======================================
/// Parse a JNumber
let jNumber =
// set up the "primitive" parsers
let optSign = opt (pchar '-')
let zero = pstring "0"
let digitOneNine =
satisfy (fun ch -> Char.IsDigit ch && ch <> '0') "1-9"
let digit =
satisfy (fun ch -> Char.IsDigit ch ) "digit"

1270

Writing a JSON parser from scratch

let point = pchar '.'

let e = pchar 'e' <|> pchar 'E'
let optPlusMinus = opt (pchar '-' <|> pchar '+')
let nonZeroInt =
digitOneNine .>>. manyChars digit
|>> fun (first,rest) -> string first + rest
let intPart = zero <|> nonZeroInt
let fractionPart = point >>. manyChars1 digit
let exponentPart = e >>. optPlusMinus .>>. manyChars1 digit
// utility function to convert an optional value to a string, or "" if missing
let ( |>? ) opt f =
match opt with
| None -> ""
| Some x -> f x
let convertToJNumber (((optSign,intPart),fractionPart),expPart) =
// convert to strings and let .NET parse them! - crude but ok for now.
let signStr =
optSign
|>? string // e.g. "-"
let fractionPartStr =
fractionPart
|>? (fun digits -> "." + digits ) // e.g. ".456"
let expPartStr =
expPart
|>? fun (optSign, digits) ->
let sign = optSign |>? string
"e" + sign + digits // e.g. "e-12"
// add the parts together and convert to a float, then wrap in a JNumber
(signStr + intPart + fractionPartStr + expPartStr)
|> float
|> JNumber
// set up the main parser
optSign .>>. intPart .>>. opt fractionPart .>>. opt exponentPart
|>> convertToJNumber
<?> "number" // add label
// ======================================
// Parsing a JArray
// ======================================

1271

Writing a JSON parser from scratch

let jArray =
// set up the "primitive" parsers
let left = pchar '[' .>> spaces
let right = pchar ']' .>> spaces
let comma = pchar ',' .>> spaces
let value = jValue .>> spaces
// set up the list parser
let values = sepBy1 value comma
// set up the main parser
between left values right
|>> JArray
<?> "array"
// ======================================
// Parsing a JObject
// ======================================

1272

Writing a JSON parser from scratch

jArray
jObject
]

Summary
In this post, we built a JSON parser using the parser library that we have developed over the
previous posts.
I hope that, by building both the parser library and a real-world parser from scratch, you
have gained a good appreciation for how parser combinators work, and how useful they are.
I'll repeat what I said in the first post: if you are interesting in using this technique in
production, be sure to investigate the FParsec library for F#, which is optimized for realworld usage.
And if you are using languages other than F#, there is almost certainly a parser combinator
library available to use.
For more information about parser combinators in general, search the internet for
"Parsec", the Haskell library that influenced FParsec.
For some more examples of using FParsec, try one of these posts:
Implementing a phrase search query for FogCreek's Kiln
A LOGO Parser
A Small Basic Parser
A C# Parser and building a C# compiler in F#
Write Yourself a Scheme in 48 Hours in F#
Parsing GLSL, the shading language of OpenGL
Thanks!
The source code for this post is available at this gist.

1273

The "Handling State" Series

In this series of posts, I'll look at how you can thread state through a series of pure functions
in a convenient way.
To start with, I'll tell the story of Dr Frankenfunctor and the Monadster, and how the Doctor
needed a way to create "recipes" that were activated when lightning struck.
The Doctor then devised ways to work with these recipes using functions such as map ,
bind and apply .

In the final post, we'll see how we can use a computation expression to make the coding
cleaner, and how these techniques can be generalized into the so-called "state monad".
Warning! These posts contains gruesome topics, strained analogies, discussion of monads
Dr Frankenfunctor and the Monadster. Or, how a 19th century scientist nearly invented
the state monad.
Completing the body of the Monadster. Dr Frankenfunctor and the Monadster, part 2.
Refactoring the Monadster. Dr Frankenfunctor and the Monadster, part 3.

1274

Dr Frankenfunctor and the Monadster

UPDATE: Slides and video from my talk on this topic
Warning! This post contains gruesome topics, strained analogies, discussion of monads
For generations, we have been captivated by the tragic story of Dr Frankenfunctor. The
fascination with vital forces, the early experiments with electricity and galvanism, and finally
the breakthough culminating in the bringing to life of a collection of dead body parts -- the
Monadster.
But then, as we all know, the creature escaped and the free Monadster rampaged through
computer science conferences, bringing fear to the hearts of even the most seasoned
programmers.

CAPTION: The terrible events at the 1990 ACM Conference on LISP and Functional
Programming.
I will not repeat the details here; the story is still too terrible to recall.
But in all the millions of words devoted to this tragedy, one topic has never been
satisfactorily addressed.
How was the creature assembled and brought to life?
We know that Dr Frankenfunctor built the creature from dead body parts, and then animated
them in a single instant, using a bolt of lightning to create the vital force.
But the various body parts had to be assembled into a whole, and the vital force had to be
transmitted through the assembly in the appropriate manner, and all this done in a split
second, in the moment that the lightning struck.

1275

Dr Frankenfunctor and the Monadster

I have devoted many years of research into this matter, and recently, at great expense, I
have managed to obtain Dr Frankenfunctor's personal laboratory notebooks.
So at last, I can present Dr Frankenfunctor's technique to the world. Use it as you will. I do
not make any judgements as to its morality, after all, it is not for mere developers to question
the real-world effects of what we build.

Background
To start with, you need to understand the fundamental process involved.
First, you must know that no whole body was available to Dr Frankenfunctor. Instead, the
creature was created from an assemblage of body parts -- arms, legs, brain, heart -- whose
provenances were murky and best left unspoken.
Dr Frankenfunctor started with a dead body part, and infused it with some amount of vital
force. The result was two things: a now live body part, and the remaining, diminished, vital
force, because of course some of the vital force was transferred to the live part.
Here is a diagram demonstrating the principle:

But this creates only one body part. How can we create more than one? This is the
challenge that faced Dr Frankenfunctor.
The first problem is that we only have a limited quantity of the vital force. This means that
when we need to animate a second body part, we have available only the remaining vital
force from a previous step.
How can we connect the two steps together so that the vital force from the first step is fed
into the input of the second step?

1276

Dr Frankenfunctor and the Monadster

Even if we have chained the steps correctly, we need to take the various live body parts and
combine them somehow. But we only have access to live body parts during the moment of
creation. How can we combine them in that split second?

It was Dr Frankenfunctor's genius that led to an elegant approach that solved both of these
problems, the approach that I will present to you now.

The common context

Before discussing the particulars of assembling the body parts, we should spend a moment
on common functionality that is required for the rest of the procedure.
First, we need a label type. Dr Frankenfunctor was very disciplined in labeling the source of
every part used.
type Label = string

1277

Dr Frankenfunctor and the Monadster

The vital force we will model with a simple record type:

type VitalForce = {units:int}

Since we will be using vital force frequently, we will create a function that extracts one unit
and returns a tuple of the unit and remaining force.
let getVitalForce vitalForce =
let oneUnit = {units = 1}
let remaining = {units = vitalForce.units-1} // decrement
oneUnit, remaining // return both

The Left Leg

With the common code out of the way, we can return to the substance.
Dr Frankenfunctor's notebooks record that the lower extremities were created first. There
was a left leg lying around in the laboratory, and that was the starting point.
type DeadLeftLeg = DeadLeftLeg of Label

From this leg, a live leg could be created with the same label and one unit of vital force.
type LiveLeftLeg = LiveLeftLeg of Label * VitalForce

The type signature for the creation function would thus look like this:
type MakeLiveLeftLeg =
DeadLeftLeg * VitalForce -> LiveLeftLeg * VitalForce

And the actual implementation like this:

let makeLiveLeftLeg (deadLeftLeg,vitalForce) =
// get the label from the dead leg using pattern matching
let (DeadLeftLeg label) = deadLeftLeg
// get one unit of vital force
let oneUnit, remainingVitalForce = getVitalForce vitalForce
// create a live leg from the label and vital force
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
// return the leg and the remaining vital force
liveLeftLeg, remainingVitalForce

1278

Dr Frankenfunctor and the Monadster

As you can see, this implementation matched the earlier diagram precisely.

At this point Dr Frankenfunctor had two important insights.

The first insight was that, thanks to currying, the function could be converted from a function
taking a tuple to a two parameter function, with each parameter passed in turn.

And the code now looked like this:

type MakeLiveLeftLeg =
DeadLeftLeg -> VitalForce -> LiveLeftLeg * VitalForce
let makeLiveLeftLeg deadLeftLeg vitalForce =
let (DeadLeftLeg label) = deadLeftLeg
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
liveLeftLeg, remainingVitalForce

The second insight was that this same code can be interpreted as a function that in turn
returns a "becomeAlive" function.
That is, we have the dead part on hand, but we won't have any vital force until the final
moment, so why not process the dead part right now and return a function that can be used
when the vital force becomes available.
In other words, we pass in a dead part, and we get back a function that creates a live part
when given some vital force.

1279

Dr Frankenfunctor and the Monadster

These "become alive" functions can then be treated as "steps in a recipe", assuming we can
find some way of combining them.
The code looks like this now:
type MakeLiveLeftLeg =
DeadLeftLeg -> (VitalForce -> LiveLeftLeg * VitalForce)
let makeLiveLeftLeg deadLeftLeg =
// create an inner intermediate function
let becomeAlive vitalForce =
let (DeadLeftLeg label) = deadLeftLeg
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
liveLeftLeg, remainingVitalForce
// return it
becomeAlive

It may not be obvious, but this is exactly the same code as the previous version, just written
slightly differently.
This curried function (with two parameters) can be interpreted as a normal two parameter
function, or it can be interpreted as a one parameter function that returns another one
parameter function.
If this is not clear, consider the much simpler example of a two parameter add function:
let add x y =
x + y

Because F# curries functions by default, that implementation is exactly the same as this one:
let add x =
fun y -> x + y

Which, if we define an intermediate function, is also exactly the same as this one:
let add x =
let addX y = x + y
addX // return the function

Creating the Monadster type

Looking ahead, we can see that we can use a similar approach for all the functions that
create live body parts.
1280

Dr Frankenfunctor and the Monadster

All those functions will return a function that has a signature like: VitalForce -> LiveBodyPart
* VitalForce .

To make our life easy, let's give that function signature a name, M , which stands for
"Monadster part generator", and give it a generic type parameter 'LiveBodyPart so that we
can use it with many different body parts.
type M<'LiveBodyPart> =
VitalForce -> 'LiveBodyPart * VitalForce

We can now explicitly annotate the return type of the makeLiveLeftLeg function with
:M<LiveLeftLeg> .

let makeLiveLeftLeg deadLeftLeg :M<LiveLeftLeg> =

let becomeAlive vitalForce =
let (DeadLeftLeg label) = deadLeftLeg
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
liveLeftLeg, remainingVitalForce
becomeAlive

The rest of the function is unchanged because the becomeAlive return value is already
compatible with M<LiveLeftLeg> .
But I don't like having to explicitly annotate all the time. How about we wrap the function in a
single case union -- call it "M" -- to give it its own distinct type? Like this:
type M<'LiveBodyPart> =
M of (VitalForce -> 'LiveBodyPart * VitalForce)

That way, we can distinguish between a "Monadster part generator" and an ordinary function
returning a tuple.
To use this new definition, we need to tweak the code to wrap the intermediate function in
the single case union M when we return it, like this:
let makeLiveLeftLegM deadLeftLeg =
let becomeAlive vitalForce =
let (DeadLeftLeg label) = deadLeftLeg
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
liveLeftLeg, remainingVitalForce
// changed!
M becomeAlive // wrap the function in a single case union

1281

Dr Frankenfunctor and the Monadster

For this last version, the type signature will be correctly inferred without having to specify it
explicitly: a function that takes a dead left leg and returns an "M" of a live leg:
val makeLiveLeftLegM : DeadLeftLeg -> M<LiveLeftLeg>

Note that I've renamed the function makeLiveLeftLegM to make it clear that it returns a M of
LiveLeftLeg .

The meaning of M
So what does this "M" type mean exactly? How can we make sense of it?
One helpful way is to think of a M<T> as a recipe for creating a T . You give me some vital
force and I'll give you back a T .
But how can an M<T> create a T out of nothing?
That's where functions like makeLiveLeftLegM are critically important. They take a parameter
and "bake" it into the result. As a result, you will see lots of "M-making" functions with similar
signatures, all looking something like this:

Or in code terms:
DeadPart -> M<LivePart>

The challenge now will be how to combine these in an elegant way.

Testing the left leg

Ok, let's test what we've got so far.
We'll start by creating a dead leg and use makeLiveLeftLegM on it to get an M<LiveLeftLeg> .
let deadLeftLeg = DeadLeftLeg "Boris"
let leftLegM = makeLiveLeftLegM deadLeftLeg

What is leftLegM ? It's a recipe for creating a live left leg, given some vital force.
What's useful is that we can create this recipe up front, before the lightning strikes.

1282

Dr Frankenfunctor and the Monadster

Now let's pretend that the storm has arrived, the lightning has struck, and 10 units of vital
force are now available:
let vf = {units = 10}

Now, inside the leftLegM is a function which we can apply to the vital force. But first we
need to get the function out of the wrapper using pattern matching.
let (M innerFn) = leftLegM

And then we can run the inner function to get the live left leg and the remaining vital force:
let liveLeftLeg, remainingAfterLeftLeg = innerFn vf

The results look like this:

val liveLeftLeg : LiveLeftLeg =
LiveLeftLeg ("Boris",{units = 1;})
val remainingAfterLeftLeg : VitalForce =
{units = 9;}

You can see that a LiveLeftLeg was created successfully and that the remaining vital force
is reduced to 9 units now.
This pattern matching is awkward, so let's create a helper function that both unwraps the
inner function and calls it, all in one go.
We'll call it runM and it looks like this:
let runM (M f) vitalForce = f vitalForce

So the test code above would now be simplified to this:

let liveLeftLeg, remainingAfterLeftLeg = runM leftLegM vf

So now, finally, we have a function that can create a live left leg.
It took a while to get it working, but we've also built some useful tools and concepts that we
can use moving forwards.

The Right Leg

1283

Dr Frankenfunctor and the Monadster

Now that we know what we are doing, we should be able to use the same techniques for the
other body parts now.
How about a right leg then?
Unfortunately, according to the notebook, Dr Frankenfunctor could not find a right leg in the
laboratory. The problem was solved with a hack... but we'll come to that later.

The Left Arm

Next, the arms were created, starting with the left arm.
But there was a problem. The laboratory only had a broken left arm lying around. The arm
had to be healed before it could be used in the final body.
Now Dr Frankenfunctor, being a doctor, did know how to heal a broken arm, but only a live
one. Trying to heal a dead broken arm would be impossible.
In code terms, we have this:
type DeadLeftBrokenArm = DeadLeftBrokenArm of Label
// A live version of the broken arm.
type LiveLeftBrokenArm = LiveLeftBrokenArm of Label * VitalForce
// A live version of a heathly arm, with no dead version available
type LiveLeftArm = LiveLeftArm of Label * VitalForce
// An operation that can turn a broken left arm into a heathly left arm
type HealBrokenArm = LiveLeftBrokenArm -> LiveLeftArm

The challenge was therefore this: how can we make a live left arm out the material we have
on hand?
First, we have to rule out creating a LiveLeftArm from a DeadLeftUnbrokenArm , as there isn't
any such thing. Nor can we convert a DeadLeftBrokenArm into a healthy LiveLeftArm
directly.

1284

Dr Frankenfunctor and the Monadster

But what we can do is turn the DeadLeftBrokenArm into a live broken arm and then heal the
live broken arm, yes?

No, I'm afraid that won't work. We can't create live parts directly, we can only create live
parts in the context of the M recipe.
What we need to do then is create a special version of healBrokenArm (call it
healBrokenArmM ) that converts a M<LiveBrokenArm> to a M<LiveArm> .

But how do we create such a function? And how can we reuse healBrokenArm as part of it?
Let's start with the most straightforward implementation.
First, since the function will return an M something, it will have the same form as the
makeLiveLeftLegM function that we saw earlier. We'll need to create an inner function that

has a vitalForce parameter, and then return it wrapped in an M .

But unlike the function that we saw earlier, this one has an M as parameter too (an
M<LiveBrokenArm> ). How can we extract the data we need from this input?

Simple, just run it with some vitalForce. And where are we going to get the vitalForce from?
From the parameter to the inner function!
So our finished version will look like this:

1285

Dr Frankenfunctor and the Monadster

// implementation of HealBrokenArm
let healBrokenArm (LiveLeftBrokenArm (label,vf)) = LiveLeftArm (label,vf)
/// convert a M<LiveLeftBrokenArm> into a M<LiveLeftArm>
let makeHealedLeftArm brokenArmM =
// create a new inner function that takes a vitalForce parameter
let healWhileAlive vitalForce =
// run the incoming brokenArmM with the vitalForce
// to get a broken arm
let brokenArm,remainingVitalForce = runM brokenArmM vitalForce
// heal the broken arm
let healedArm = healBrokenArm brokenArm
// return the healed arm and the remaining VitalForce
healedArm, remainingVitalForce
// wrap the inner function and return it
M healWhileAlive

If we evaluate this code, we get the signature:

val makeHealedLeftArm : M<LiveLeftBrokenArm> -> M<LiveLeftArm>

which is exactly what we want!

But not so fast -- we can do better.
We've hard-coded the healBrokenArm transformation in there. What happens if we want to
do some other transformation, and for some other body part? Can we make this function a
bit more generic?
Yes, it's easy. All we need to is pass in a function ("f" say) that transforms the body part, like
this:
let makeGenericTransform f brokenArmM =
// create a new inner function that takes a vitalForce parameter
let healWhileAlive vitalForce =
let brokenArm,remainingVitalForce = runM brokenArmM vitalForce
// heal the broken arm using passed in f
let healedArm = f brokenArm
healedArm, remainingVitalForce
M healWhileAlive

1286

Dr Frankenfunctor and the Monadster

What's amazing about this is that by parameterizing that one transformation with the f
parameter, the whole function becomes generic!
We haven't made any other changes, but the signature for makeGenericTransform no longer
refers to arms. It works with anything!
val makeGenericTransform : f:('a -> 'b) -> M<'a> -> M<'b>

Introducing mapM
Since it is so generic now, the names are confusing. Let's rename it. I'll call it mapM . It works
with any body part and any transformation.
Here's the implementation, with the internal names fixed up too.
let mapM f bodyPartM =
let transformWhileAlive vitalForce =
let bodyPart,remainingVitalForce = runM bodyPartM vitalForce
let updatedBodyPart = f bodyPart
updatedBodyPart, remainingVitalForce
M transformWhileAlive

In particular, it works with the healBrokenArm function, so to create a version of "heal" that
has been lifted to work with M s we can just write this:
let healBrokenArmM = mapM healBrokenArm

The importance of mapM

One way of thinking about mapM is that it is a "function converter". Given any "normal"
function, it converts it to a function where the input and output are M s.

1287

Dr Frankenfunctor and the Monadster

Functions similar to mapM crop up in many situations. For example, Option.map transforms
a "normal" function into a function whose inputs and outputs are options. Similarly,
List.map transforms a "normal" function into a function whose inputs and outputs are lists.

And there are many other examples.

// map works with options
let healBrokenArmO = Option.map healBrokenArm
// LiveLeftBrokenArm option -> LiveLeftArm option
// map works with lists
let healBrokenArmL = List.map healBrokenArm
// LiveLeftBrokenArm list -> LiveLeftArm list

What might be new to you is that the "wrapper" type M contains a function, not a simple
data structure like Option or List. That might make your head hurt!
In addition, the diagram above implies that M could wrap any normal type and mapM could
map any normal function.
Let's try it and see!

1288

Dr Frankenfunctor and the Monadster

let isEven x = (x%2 = 0) // int -> bool

// map it
let isEvenM = mapM isEven // M<int> -> M<bool>
let isEmpty x = (String.length x)=0 // string -> bool
// map it
let isEmptyM = mapM isEmpty // M<string> -> M<bool>

So, yes, it works!

Testing the left arm

Again, let's test what we've got so far.
We'll start by creating a dead broken arm and use makeLiveLeftBrokenArm on it to get an
M<BrokenLeftArm> .

let makeLiveLeftBrokenArm deadLeftBrokenArm =

let (DeadLeftBrokenArm label) = deadLeftBrokenArm
let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftBrokenArm = LiveLeftBrokenArm (label,oneUnit)
liveLeftBrokenArm, remainingVitalForce
M becomeAlive
/// create a dead Left Broken Arm
let deadLeftBrokenArm = DeadLeftBrokenArm "Victor"
/// create a M<BrokenLeftArm> from the dead one
let leftBrokenArmM = makeLiveLeftBrokenArm deadLeftBrokenArm

Now we can use mapM and healBrokenArm to convert the M<BrokenLeftArm> into a
M<LeftArm> :

let leftArmM = leftBrokenArmM |> mapM healBrokenArm

What we have now in leftArmM is a recipe for creating a unbroken and live left arm. All we
need to do is add some vital force.
As before, we can do all these things up front, before the lightning strikes.
Now when the storm arrives, and the lightning has struck, and vital force is available, we can
run leftArmM with the vital force...

1289

Dr Frankenfunctor and the Monadster

let vf = {units = 10}

let liveLeftArm, remainingAfterLeftArm = runM leftArmM vf

...and we get this result:

val liveLeftArm : LiveLeftArm =
LiveLeftArm ("Victor",{units = 1;})
val remainingAfterLeftArm :
VitalForce = {units = 9;}

A live left arm, just as we wanted.

The Right Arm

On to the right arm next.
Again, there was a problem. Dr Frankenfunctor's notebooks record that there was no whole
arm available. However there was a lower arm and an upper arm...
type DeadRightLowerArm = DeadRightLowerArm of Label
type DeadRightUpperArm = DeadRightUpperArm of Label

...which could be turned into corresponding live ones:

type LiveRightLowerArm = LiveRightLowerArm of Label * VitalForce
type LiveRightUpperArm = LiveRightUpperArm of Label * VitalForce

Dr Frankenfunctor decided to do surgery to join the two arm sections into a whole arm.
// define the whole arm
type LiveRightArm = {
lowerArm : LiveRightLowerArm
upperArm : LiveRightUpperArm
}
// surgery to combine the two arm parts
let armSurgery lowerArm upperArm =
{lowerArm=lowerArm; upperArm=upperArm}

As with the broken arm, the surgery could only be done with live parts. Doing that with dead
parts would be yucky and gross.

1290

Dr Frankenfunctor and the Monadster

But also, as with the broken arm, we don't have access to the live parts directly, only within
the context of an M wrapper.
In other words we need to convert our armSurgery function that works with normal live
parts, and convert it into a armSurgeryM function that works with M s.

We can use the same approach as we did before:

create a inner function that takes a vitalForce parameter
run the incoming parameters with the vitalForce to extract the data
from the inner function return the new data after surgery
wrap the inner function in an "M" and return it
Here's the code:
/// convert a M<LiveRightLowerArm> and M<LiveRightUpperArm> into a M<LiveRightArm>
let makeArmSurgeryM_v1 lowerArmM upperArmM =
// create a new inner function that takes a vitalForce parameter
let becomeAlive vitalForce =
// run the incoming lowerArmM with the vitalForce
// to get the lower arm
let liveLowerArm,remainingVitalForce = runM lowerArmM vitalForce
// run the incoming upperArmM with the remainingVitalForce
// to get the upper arm
let liveUpperArm,remainingVitalForce2 = runM upperArmM remainingVitalForce
// do the surgery to create a liveRightArm
let liveRightArm = armSurgery liveLowerArm liveUpperArm
// return the whole arm and the SECOND remaining VitalForce
liveRightArm, remainingVitalForce2
// wrap the inner function and return it
M becomeAlive

1291

Dr Frankenfunctor and the Monadster

One big difference from the broken arm example is that we have two parameters, of course.
When we run the second parameter (to get the liveUpperArm ), we must be sure to pass in
the remaining vital force after the first step, not the original one.
And then, when we return from the inner function, we must be sure to return
remainingVitalForce2 (the remainder after the second step) not any other one.

If we compile this code, we get:

M<LiveRightLowerArm> -> M<LiveRightUpperArm> -> M<LiveRightArm>

which is just the signature we are looking for.

Introducing map2M
But as before, why not make this more generic? We don't need to hard-code armSurgery -we can pass it as a parameter.
We'll call the more generic function map2M -- just like mapM but with two parameters.
Here's the implementation:
let map2M f m1 m2 =
let becomeAlive vitalForce =
let v1,remainingVitalForce = runM m1 vitalForce
let v2,remainingVitalForce2 = runM m2 remainingVitalForce
let v3 = f v1 v2
v3, remainingVitalForce2
M becomeAlive

And it has the signature:

f:('a -> 'b -> 'c) -> M<'a> -> M<'b> -> M<'c>

Just as with mapM we can interpret this function as a "function converter" that converts a
"normal" two parameter function into a function in the world of M .

1292

Dr Frankenfunctor and the Monadster

Testing the right arm

Again, let's test what we've got so far.
As always, we need some functions to convert the dead parts into live parts.
let makeLiveRightLowerArm (DeadRightLowerArm label) =
let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveRightLowerArm = LiveRightLowerArm (label,oneUnit)
liveRightLowerArm, remainingVitalForce
M becomeAlive
let makeLiveRightUpperArm (DeadRightUpperArm label) =
let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveRightUpperArm = LiveRightUpperArm (label,oneUnit)
liveRightUpperArm, remainingVitalForce
M becomeAlive

By the way, are you noticing that there is a lot of duplication in these functions? Me too! We
will attempt to fix that later.
Next, we'll create the parts:

1293

Dr Frankenfunctor and the Monadster

let deadRightLowerArm = DeadRightLowerArm "Tom"

let lowerRightArmM = makeLiveRightLowerArm deadRightLowerArm
let deadRightUpperArm = DeadRightUpperArm "Jerry"
let upperRightArmM = makeLiveRightUpperArm deadRightUpperArm

And then create the function to make a whole arm:

let armSurgeryM = map2M armSurgery
let rightArmM = armSurgeryM lowerRightArmM upperRightArmM

As always, we can do all these things up front, before the lightning strikes, building a recipe
(or computation if you like) that will do everything we need when the time comes.
When the vital force is available, we can run rightArmM with the vital force...
let vf = {units = 10}
let liveRightArm, remainingFromRightArm = runM rightArmM vf

...and we get this result:

val liveRightArm : LiveRightArm =
{lowerArm = LiveRightLowerArm ("Tom",{units = 1;});
upperArm = LiveRightUpperArm ("Jerry",{units = 1;});}
val remainingFromRightArm : VitalForce =
{units = 8;}

A live right arm, composed of two subcomponents, just as required.

Also note that the remaining vital force has gone down to eight. We have correctly used up
two units of vital force.

Summary
In this post, we saw how to create a M type that wrapped a "become alive" function that in
turn could only be activated when lightning struck.
We also saw how various M-values could be processed and combined using mapM (for the
broken arm) and map2M (for the arm in two parts).
The code samples used in this post are available on GitHub.

1294

Dr Frankenfunctor and the Monadster

Next time
This exciting tale has more shocks in store for you! Stay tuned for the next installment, when
I reveal how the head and body were created.

1295

Completing the body of the Monadster

UPDATE: Slides and video from my talk on this topic
Warning! This post contains gruesome topics, strained analogies, discussion of monads
Welcome to the gripping tale of Dr Frankenfunctor and the Monadster!
We saw in the previous installment how Dr Frankenfunctor created life out of dead body
parts using "Monadster part generators" (or "M"s for short), that would, on being supplied
with some vital force, return a live body part.
We also saw how the leg and arms of the creature were created, and how these M-values
could be processed and combined using mapM (for the broken arm) and map2M (for the arm
in two parts).
In this second installment, we'll look at the other techniques Dr Frankenfunctor used to
create the head, the heart, and the complete body.

The Head
First, the head.
Just like the right arm, the head is composed of two parts, a brain and a skull.
Dr Frankenfunctor started by defining the dead brain and skull:
type DeadBrain = DeadBrain of Label
type Skull = Skull of Label

Unlike the two-part right arm, only the brain needs to become alive. The skull can be used
as is and does not need to be transformed before being used in a live head.
type LiveBrain = LiveBrain of Label * VitalForce
type LiveHead = {
brain : LiveBrain
skull : Skull // not live
}

The live brain is combined with the skull to make a live head using a headSurgery function,
analogous to the armSurgery we had earlier.

1296

Completing the body of the Monadster

let headSurgery brain skull =

{brain=brain; skull=skull}

Now we are ready to create a live head -- but how should we do it?
It would be great if we could reuse map2M , but there's a catch -- for map2M to work, it needs
a skull wrapped in a M .

But the skull doesn't need to become alive or use vital force, so we will need to create a
special function that converts a Skull to a M<Skull> .
We can use the same approach as we did before:
create a inner function that takes a vitalForce parameter
in this case, we leave the vitalForce untouched
from the inner function return the original skull and the untouched vitalForce
wrap the inner function in an "M" and return it
Here's the code:
let wrapSkullInM skull =
let becomeAlive vitalForce =
skull, vitalForce
M becomeAlive

But the signature of wrapSkullInM is quite interesting.

val wrapSkullInM : 'a -> M<'a>

No mention of skulls anywhere!

Introducing returnM
We've created a completely generic function that will turn anything into an M . So let's
rename it. I'm going to call it returnM , but in other contexts it might be called pure or
unit .

1297

Completing the body of the Monadster

let returnM x =
let becomeAlive vitalForce =
x, vitalForce
M becomeAlive

Testing the head

Let's put this into action.
First, we need to define how to create a live brain.
let makeLiveBrain (DeadBrain label) =
let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveBrain = LiveBrain (label,oneUnit)
liveBrain, remainingVitalForce
M becomeAlive

Next we obtain a dead brain and skull:

let deadBrain = DeadBrain "Abby Normal"
let skull = Skull "Yorick"

By the way, how this particular dead brain was obtained is an interesting story that I don't
have time to go into right now.

Next we build the "M" versions from the dead parts:

let liveBrainM = makeLiveBrain deadBrain
let skullM = returnM skull

1298

Completing the body of the Monadster

And combine the parts using map2M :

let headSurgeryM = map2M headSurgery
let headM = headSurgeryM liveBrainM skullM

Once again, we can do all these things up front, before the lightning strikes.
When the vital force is available, we can run headM with the vital force...
let vf = {units = 10}
let liveHead, remainingFromHead = runM headM vf

...and we get this result:

val liveHead : LiveHead =
{brain = LiveBrain ("Abby normal",{units = 1;});
skull = Skull "Yorick";}
val remainingFromHead : VitalForce =
{units = 9;}

A live head, composed of two subcomponents, just as required.

Also note that the remaining vital force is just nine, as the skull did not use up any units.

The Beating Heart

There is one more component we need, and that is a heart.
First, we have a dead heart and a live heart defined in the usual way:
type DeadHeart = DeadHeart of Label
type LiveHeart = LiveHeart of Label * VitalForce

But the creature needs more than a live heart -- it needs a beating heart. A beating heart is
constructed from a live heart and some more vital force, like this:
type BeatingHeart = BeatingHeart of LiveHeart * VitalForce

The code that creates a live heart is very similar to the previous examples:

1299

Completing the body of the Monadster

let makeLiveHeart (DeadHeart label) =

let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveHeart = LiveHeart (label,oneUnit)
liveHeart, remainingVitalForce
M becomeAlive

The code that creates a beating heart is also very similar. It takes a live heart as a
parameter, uses up another unit of vital force, and returns the beating heart and the
remaining vital force.
let makeBeatingHeart liveHeart =
let becomeAlive vitalForce =
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let beatingHeart = BeatingHeart (liveHeart, oneUnit)
beatingHeart, remainingVitalForce
M becomeAlive

If we look at the signatures for these functions, we see that they are very similar; both of the
form Something -> M<SomethingElse> .
val makeLiveHeart : DeadHeart -> M<LiveHeart>
val makeBeatingHeart : LiveHeart -> M<BeatingHeart>

Chaining together M-returning functions

We start with a dead heart, and we need to get a beating heat

But we don't have the tools to do this directly.

We have a function that turns a DeadHeart into a M<LiveHeart> , and we have a function
that turns a LiveHeart into a M<BeatingHeart> .
But the output of the first is not compatible with the input of the second, so we can't glue
them together.

1300

Completing the body of the Monadster

What we want then, is a function that, given a M<LiveHeart> as input, can convert it to a
M<BeatingHeart> .

And furthermore, we want to build it from the makeBeatingHeart function we already have.

Here's a first attempt, using the same pattern we've used many times before:
let makeBeatingHeartFromLiveHeartM liveHeartM =
let becomeAlive vitalForce =
// extract the liveHeart from liveHeartM
let liveHeart, remainingVitalForce = runM liveHeartM vitalForce
// use the liveHeart to create a beatingHeartM
let beatingHeartM = makeBeatingHeart liveHeart
// what goes here?
// return a beatingHeart and remaining vital force
beatingHeart, remainingVitalForce
M becomeAlive

But what goes in the middle? How can we get a beating heart from a beatingHeartM ? The
answer is to run it with some vital force (which we happen to have on hand, because we are
in the middle of the becomeAlive function).
What vital force though? It should be the remaining vital force after getting the liveHeart .

1301

Completing the body of the Monadster

So the final version looks like this:

let makeBeatingHeartFromLiveHeartM liveHeartM =
let becomeAlive vitalForce =
// extract the liveHeart from liveHeartM
let liveHeart, remainingVitalForce = runM liveHeartM vitalForce
// use the liveHeart to create a beatingHeartM
let beatingHeartM = makeBeatingHeart liveHeart
// run beatingHeartM to get a beatingHeart
let beatingHeart, remainingVitalForce2 = runM beatingHeartM remainingVitalForc
e
// return a beatingHeart and remaining vital force
beatingHeart, remainingVitalForce2
// wrap the inner function and return it
M becomeAlive

Notice that we return remainingVitalForce2 at the end, the remainder after both steps are
run.
If we look at the signature for this function, it is:
M<LiveHeart> -> M<BeatingHeart>

which is just what we wanted!

Introducing bindM
Once again, we can make this function generic by passing in a function parameter rather
than hardcoding makeBeatingHeart .
I'll call it bindM . Here's the code:
let bindM f bodyPartM =
let becomeAlive vitalForce =
let bodyPart, remainingVitalForce = runM bodyPartM vitalForce
let newBodyPartM = f bodyPart
let newBodyPart, remainingVitalForce2 = runM newBodyPartM remainingVitalForce
newBodyPart, remainingVitalForce2
M becomeAlive

and the signature is:

1302

Completing the body of the Monadster

f:('a -> M<'b>) -> M<'a> -> M<'b>

In other words, given any function Something -> M<SomethingElse> , I can convert it to a
function M<Something> -> M<SomethingElse> that has an M as input and output.
By the way, functions with a signature like Something -> M<SomethingElse> are often called
monadic functions.
Anyway, once you understand what is going on in bindM , a slightly shorter version can be
implemented like this:
let bindM f bodyPartM =
let becomeAlive vitalForce =
let bodyPart, remainingVitalForce = runM bodyPartM vitalForce
runM (f bodyPart) remainingVitalForce
M becomeAlive

So finally, we have a way of creating a function, that given a DeadHeart , creates a

M<BeatingHeart> .

Here's the code:

// create a dead heart
let deadHeart = DeadHeart "Anne"
// create a live heart generator (M<LiveHeart>)
let liveHeartM = makeLiveHeart deadHeart
// create a beating heart generator (M<BeatingHeart>)
// from liveHeartM and the makeBeatingHeart function
let beatingHeartM = bindM makeBeatingHeart liveHeartM

There are a lot of intermediate values in there, and it can be made simpler by using piping,
like this:

1303

Completing the body of the Monadster

let beatingHeartM =
DeadHeart "Anne"
|> makeLiveHeart
|> bindM makeBeatingHeart

The importance of bind

One way of thinking about bindM is that it is another "function converter", just like mapM .
That is, given any "M-returning" function, it converts it to a function where the input and
output are both M s.

Just like map , bind appears in many other contexts.

For example, Option.bind transforms a option-generating function ( 'a -> 'b option ) into a
function whose inputs and outputs are options. Similarly, List.bind transforms a listgenerating function ( 'a -> 'b list ) into a function whose inputs and outputs are lists.
And I discuss yet another version of bind at length in my talk on functional error handling.
The reason that bind is so important is that "M-returning" functions crop up a lot, and they
cannot be chained together easily because the output of one step does not match the input
of the the next step.
By using bindM , we can convert each step into a function where the input and output are
both M s, and then they can be chained together.

1304

Completing the body of the Monadster

Testing the beating heart

As always, we construct the recipe ahead of time, in this case, to make a BeatingHeart .
let beatingHeartM =
DeadHeart "Anne"
|> makeLiveHeart
|> bindM makeBeatingHeart

When the vital force is available, we can run beatingHeartM with the vital force...
let vf = {units = 10}
let beatingHeart, remainingFromHeart = runM beatingHeartM vf

...and we get this result:

val beatingHeart : BeatingHeart =
BeatingHeart (LiveHeart ("Anne",{units = 1;}),{units = 1;})
val remainingFromHeart : VitalForce =
{units = 8;}

Note that the remaining vital force is eight units, as we used up two units doing two steps.

The Whole Body

Finally, we have all the parts we need to assemble a complete body.
Here is Dr Frankenfunctor's definition of a live body:

1305

Completing the body of the Monadster

type LiveBody = {
leftLeg: LiveLeftLeg
rightLeg : LiveLeftLeg
leftArm : LiveLeftArm
rightArm : LiveRightArm
head : LiveHead
heart : BeatingHeart
}

You can see that it uses all the subcomponents that we have already developed.

Two left feet

Because there was no right leg available, Dr Frankenfunctor decided to take a short cut and
use two left legs in the body, hoping that no one would notice.
The result of this was that the creature had two left feet, which is not always a handicap, and
indeed, the creature not only overcame this disadvantage but became a creditable dancer,
as can be seen in this rare footage:

Gene Wilder - Young Frankenstein (1974) - Puttin' on the Ritz

Playback isn't supported on this device.

0:00 / 2:04

Assembling the subcomponents

1306

Completing the body of the Monadster

The LiveBody type has six fields. How can we construct it from the various M<BodyPart> s
that we have?
One way would be to repeat the technique that we used with mapM and map2M . We could
create a map3M and map4M and so on.
For example, map3M could be defined like this:
let map3M f m1 m2 m3 =
let becomeAlive vitalForce =
let v1,remainingVitalForce = runM m1 vitalForce
let v2,remainingVitalForce2 = runM m2 remainingVitalForce
let v3,remainingVitalForce3 = runM m3 remainingVitalForce2
let v4 = f v1 v2 v3
v4, remainingVitalForce3
M becomeAlive

But that gets tedious quite quickly. Is there a better way?

Why, yes there is!
To understand it, remember that record types like LiveBody have to be built all-or-nothing,
but functions can be assembled step by step, thanks to the magic of currying and partial
application.
So if we have a six parameter function that creates a LiveBody , like this:
val createBody :
leftLeg:LiveLeftLeg ->
rightLeg:LiveLeftLeg ->
leftArm:LiveLeftArm ->
rightArm:LiveRightArm ->
head:LiveHead ->
beatingHeart:BeatingHeart ->
LiveBody

we can actually think of it as a one parameter function that returns a five parameter function,
like this:
val createBody :
leftLeg:LiveLeftLeg -> (five param function)

and then when we apply the function to the first parameter ("leftLeg") we get back that five
parameter function:

1307

Completing the body of the Monadster

(six param function) apply (first parameter) returns (five param function)

where the five parameter function has the signature:

rightLeg:LiveLeftLeg ->
leftArm:LiveLeftArm ->
rightArm:LiveRightArm ->
head:LiveHead ->
beatingHeart:BeatingHeart ->
LiveBody

This five parameter function can in turn be thought of as a one parameter function that
returns a four parameter function:
rightLeg:LiveLeftLeg -> (four parameter function)

Again, we can apply a first parameter ("rightLeg") and get back that four parameter function:
(five param function) apply (first parameter) returns (four param function)

where the four parameter function has the signature:

leftArm:LiveLeftArm ->
rightArm:LiveRightArm ->
head:LiveHead ->
beatingHeart:BeatingHeart ->
LiveBody

And so on and so on, until eventually we get a function with one parameter. The function will
have the signature BeatingHeart -> LiveBody .
When we apply the final parameter ("beatingHeart") then we get back our completed
LiveBody .

We can use this trick for M-things as well!

We start with the six parameter function wrapped in an M, and an M parameter.
Let's assume there is some way to "apply" the M-function to the M-parameter. We should get
back a five parameter function wrapped in an M .

1308

Completing the body of the Monadster

// normal version
(six param function) apply (first parameter) returns (five param function)
// M-world version
M<six param function> applyM M<first parameter> returns M<five param function>

And then doing that again, we can apply the next M-parameter
// normal version
(five param function) apply (first parameter) returns (four param function)
// M-world version
M<five param function> applyM M<first parameter> returns M<four param function>

and so on, applying the parameters one by one until we get the final result.

Introducing applyM
This applyM function will have two parameters then, a function wrapped in an M, and a
parameter wrapped in an M. The output will be the result of the function wrapped in an M.
Here's the implementation:
let applyM mf mx =
let becomeAlive vitalForce =
let f,remainingVitalForce = runM mf vitalForce
let x,remainingVitalForce2 = runM mx remainingVitalForce
let y = f x
y, remainingVitalForce2
M becomeAlive

As you can see it is quite similar to map2M , except that the "f" comes from unwrapping the
first parameter itself.
Let's try it out!
First we need our six parameter function:

1309

Completing the body of the Monadster

let createBody leftLeg rightLeg leftArm rightArm head beatingHeart =

{
leftLeg = leftLeg
rightLeg = rightLeg
leftArm = leftArm
rightArm = rightArm
head = head
heart = beatingHeart
}

And we're going to need to clone the left leg to use it for the right leg:
let rightLegM = leftLegM

Next, we need to wrap this createBody function in an M . How can we do that?

With the returnM function we defined earlier for skull, of course!
So putting it together, we have this code:
// move createBody to M-world -- a six parameter function wrapped in an M
let fSixParamM = returnM createBody
// apply first M-param to get a five parameter function wrapped in an M
let fFiveParamM = applyM fSixParamM leftLegM
// apply second M-param to get a four parameter function wrapped in an M
let fFourParamM = applyM fFiveParamM rightLegM
// etc
let fThreeParamM = applyM fFourParamM leftArmM
let fTwoParamM = applyM fThreeParamM rightArmM
let fOneParamM = applyM fTwoParamM headM
// after last application, the result is a M<LiveBody>
let bodyM = applyM fOneParamM beatingHeartM

It works! The result is a M<LiveBody> just as we want.

But that code sure is ugly! What can we do to make it look nicer?
One trick is to turn applyM into an infix operation, just like normal function application. The
operator used for this is commonly written <*> .
let (<*>) = applyM

With this in place, we can rewite the above code as:

1310

Completing the body of the Monadster

let bodyM =
returnM createBody
<*> leftLegM
<*> rightLegM
<*> leftArmM
<*> rightArmM
<*> headM
<*> beatingHeartM

which is much nicer!

Another trick is to notice that the returnM followed by applyM is the same as mapM . So if
we create an infix operator for mapM too...
let (<!>) = mapM

...we can get rid of the returnM as well and write the code like this:
let bodyM =
createBody
<!> leftLegM
<*> rightLegM
<*> leftArmM
<*> rightArmM
<*> headM
<*> beatingHeartM

What's nice about this is that it reads almost as if you were just calling the original function
(once you get used to the symbols!)

Testing the whole body

As always, we want to construct the recipe ahead of time. In this case, we have already
created the bodyM that will give us a complete LiveBody when the vital force arrives.
Now all we have to do is wait for lightning to strike and charge the machinery that generates
the vital force!

1311

Completing the body of the Monadster

Source: Misfit Robot Daydream

Here it comes -- the vital force is available! Quickly we run bodyM in the usual way...
let vf = {units = 10}
let liveBody, remainingFromBody = runM bodyM vf

...and we get this result:

val liveBody : LiveBody =
{leftLeg = LiveLeftLeg ("Boris",{units = 1;});
rightLeg = LiveLeftLeg ("Boris",{units = 1;});
leftArm = LiveLeftArm ("Victor",{units = 1;});
rightArm = {lowerArm = LiveRightLowerArm ("Tom",{units = 1;});
upperArm = LiveRightUpperArm ("Jerry",{units = 1;});};
head = {brain = LiveBrain ("Abby Normal",{units = 1;});
skull = Skull "Yorick";};
heart = BeatingHeart (LiveHeart ("Anne",{units = 1;}),{units = 1;});}
val remainingFromBody : VitalForce = {units = 2;}

It's alive! We have successfully reproduced Dr Frankenfunctor's work!

Note that the body contains all the correct subcomponents, and that the remaining vital force
has been correctly reduced to two units, as we used up eight units creating the body.

Summary
In this post, we extended our repertoire of manipulation techniques to include:

1312

Completing the body of the Monadster

returnM for the skull

bindM for the beating heart
applyM to assemble the whole body

The code samples used in this post are available on GitHub.

Next time
In the final installment, we'll refactor the code and review all the techniques used.

1313

Refactoring the Monadster

UPDATE: Slides and video from my talk on this topic
Warning! This post contains gruesome topics, strained analogies, discussion of monads
Welcome to the third installment in the gripping tale of Dr Frankenfunctor and the
Monadster!
We saw in the first installment how Dr Frankenfunctor created life out of dead body parts
using "Monadster part generators" (or "M"s for short), that would, on being supplied with
some vital force, return a live body part.
We also saw how the leg and arms of the creature were created, and how these M-values
could be processed and combined using mapM and map2M .
In the second installment we learned how the head, heart and body were built using other
powerful techniques such as returnM , bindM and applyM .
In this last installment, we'll review all the techniques used, refactor the code, and compare
Dr Frankenfunctor's techniques to the modern-day state monad.
Links for the complete series:
Part 1 - Dr Frankenfunctor and the Monadster
Part 2 - Completing the body
Part 3 - Review and refactoring (this post)

Review of the the techniques used

Before we refactor, let's review all the techniques we have used.

The M type
We couldn't create an actual live body part until the vital force was available, yet we wanted
to manipulate them, combine them, etc. before the lightning struck. We did this by creating a
type M that wrapped a "become alive" function for each part. We could then think of
M<BodyPart> as a recipe or instructions for creating a BodyPart when the time came.

The definition of M was:

1314

Refactoring the Monadster

type M<'a> = M of (VitalForce -> 'a * VitalForce)

mapM
Next, we wanted to transform the contents of an M without using any vital force. In our
specific case, we wanted to turn a broken arm recipe ( M<BrokenLeftArm> ) into a unbroken
arm recipe ( M<LeftArm> ). The solution was to implement a function mapM that took a
normal function 'a -> 'b and turned it into a M<'a> -> M<'b> function.
The signature of mapM was:
val mapM : f:('a -> 'b) -> M<'a> -> M<'b>

map2M
We also wanted to combine two M-recipes to make a new one. In that particular case, it was
combining an upper arm ( M<UpperRightArm> ) and lower arm ( M<LowerRightArm> ) into a
whole arm ( M<RightArm> ). The solution was map2M .
The signature of map2M was:
val map2M : f:('a -> 'b -> 'c) -> M<'a> -> M<'b> -> M<'c>

returnM
Another challenge was to lift a normal value into the world of M-recipes directly, without
using any vital force. In that particular case, it was turning a Skull into an M<Skull> so it
could be used with map2M to make a whole head.
The signature of returnM was:
val returnM : 'a -> M<'a>

Monadic functions
We created many functions that had a similar shape. They all take something as input and
return a M-recipe as output. In other words, they have this signature:
val monadicFunction : 'a -> M<'b>

1315

Refactoring the Monadster

Here are some examples of actual monadic functions that we used:

val makeLiveLeftLeg : DeadLeftLeg -> M<LiveLeftLeg>
val makeLiveRightLowerArm : DeadRightLowerArm -> M<LiveRightLowerArm>
val makeLiveHeart : DeadHeart -> M<LiveHeart>
val makeBeatingHeart : LiveHeart -> M<BeatingHeart>
// and also
val returnM : 'a -> M<'a>

bindM
The functions up to now did not require access to the vital force. But then we found that we
needed to chain two monadic functions together. In particular, we needed to chain the output
of makeLiveHeart (with signature DeadHeart -> M<LiveHeart> ) to the input of
makeBeatingHeart (with signature LiveHeart -> M<BeatingHeart> ). The solution was bindM ,

which transforms monadic functions of the form 'a -> M<'b> into functions in the M-world
( M<'a> -> M<'b> ) which could then be composed together.
The signature of bindM was:
val bindM : f:('a -> M<'b>) -> M<'a> -> M<'b>

applyM
Finally, we needed a way to combine a large number of M-parameters to make the live body.
Rather than having to create special versions of map ( map4M , map5M , map6M , etc), we
implemented a generic applyM function that could apply an M-function to an M-parameter.
From that, we could work with a function of any size, step by step, using partial application to
apply one M-parameter at a time.
The signature of applyM was:
val applyM : M<('a -> 'b)> -> M<'a> -> M<'b>

Defining the others functions in terms of bind and return

Note that of all these functions, only bindM needed access to the vital force.
In fact, as we'll see below, the functions mapM , map2M , and applyM can actually be defined
in terms of bindM and returnM !

1316

Refactoring the Monadster

Refactoring to a computation expression

A lot of the functions we have created have a very similar shape, resulting in a lot of
duplication. Here's one example:
let makeLiveLeftLegM deadLeftLeg =
let becomeAlive vitalForce =
let (DeadLeftLeg label) = deadLeftLeg
let oneUnit, remainingVitalForce = getVitalForce vitalForce
let liveLeftLeg = LiveLeftLeg (label,oneUnit)
liveLeftLeg, remainingVitalForce
M becomeAlive // wrap the function in a single case union

In particular, there is a lot of explicit handling of the vital force.

In most functional languages, there is a way to hide this so that the code looks much
cleaner.
In Haskell, developers use "do-notation", in Scala people use "for-yield" (the "for
comprehension"). And in F#, people use computation expressions.
To create a computation expression in F#, you just need two things to start with, a "bind" and
a "return", both of which we have.
Next, you define a class with specially named methods Bind and Return :
type MonsterBuilder()=
member this.Return(x) = returnM x
member this.Bind(xM,f) = bindM f xM

And finally, you create an instance of this class:

let monster = new MonsterBuilder()

When this is done, we have access to the special syntax monster {...} , just like
async{...} , seq{...} , etc.

The let! x = xM syntax requires that the right side is an M-type, say M<X> .
let! unwraps the M<X> into an X and binds it to the left side -- "x" in this case.

The return y syntax requires that return value is a "normal" type, Y say.
return wraps it into a M<Y> (using returnM ) and returns it as the overall value of the
monster expression.

So some example code would look like this:

1317

Refactoring the Monadster

monster {
let! x = xM // unwrap an M<X> into an X and bind to "x"
return y // wrap a Y and return an M<Y>
}

If you want more on computation expressions, I have an in-depth series of posts about them.

Redefining mapM and friends

With monster expressions available, let's rewrite mapM and the other functions.
mapM
mapM takes a function and a wrapped M-value and returns the function applied to the inner

value.
Here is an implementation using monster :
let mapM f xM =
monster {
let! x = xM // unwrap the M<X>
return f x // return M of (f x)
}

If we compile this implementation, we get the same signature as the previous

implementation:
val mapM : f:('a -> 'b) -> M<'a> -> M<'b>

map2M
map2M takes a function and two wrapped M-values and returns the function applied to both

the values.
It's also easy to write using monster expressions:
let map2M f xM yM =
monster {
let! x = xM // unwrap M<X>
let! y = yM // unwrap M<Y>
return f x y // return M of (f x y)
}

If we compile this implementation, we again get the same signature as the previous
implementation:
1318

Refactoring the Monadster

val map2M : f:('a -> 'b -> 'c) -> M<'a> -> M<'b> -> M<'c>

applyM
applyM takes a wrapped function and a wrapped value and returns the function applied to

the value.
Again, it's trivial to write using monster expressions:
let applyM fM xM =
monster {
let! f = fM // unwrap M<F>
let! x = xM // unwrap M<X>
return f x // return M of (f x)
}

And the signature is as expected

val applyM : M<('a -> 'b)> -> M<'a> -> M<'b>

Manipulating the vital force from within a

monster context
We'd like to use the monster expression to rewrite all our other functions too, but there is a
stumbling block.
Many of our functions have a body that looks like this:
// extract a unit of vital force from the context
let oneUnit, remainingVitalForce = getVitalForce vitalForce
// do something
// return value and remaining vital force
liveBodyPart, remainingVitalForce

In other words, we're getting some of the vital force and then putting a new vital force to be
used for the next step.
We are familiar with "getters" and "setters" in object-oriented programming, so let's see if we
can write similar ones that will work in the monster context.

1319

Refactoring the Monadster

Introducing getM
Let's start with the getter. How should we implement it?
Well, the vital force is only available in the context of becoming alive, so the function must
follow the familiar template:
let getM =
let doSomethingWhileLive vitalForce =
// what here ??
what to return??, vitalForce
M doSomethingWhileLive

Note that getting the vitalForce doesn't use any up, so the original amount can be
returned untouched.
But what should happen in the middle? And what should be returned as the first element of
the tuple?
The answer is simple: just return the vital force itself!
let getM =
let doSomethingWhileLive vitalForce =
// return the current vital force in the first element of the tuple
vitalForce, vitalForce
M doSomethingWhileLive

getM is a M<VitalForce> value, which means that we can unwrap it inside a monster

expression like this:

monster {
let! vitalForce = getM
// do something with vital force
}

Introducing putM
For the putter, the implementation is a function with a parameter for the new vital force.
let putM newVitalForce =
let doSomethingWhileLive vitalForce =
what here ??
M doSomethingWhileLive

1320

Refactoring the Monadster

Again, what should we do in the middle?

The most important thing is that the newVitalForce becomes the value that is passed on to
the next step. We must throw away the original vital force!
Which in turn means that newVitalForce must be used as the second part of the tuple that
is returned.
And what should be in the first part of the tuple that is returned? There is no sensible value
to return, so we'll just use unit .
Here's the final implementation:
let putM newVitalForce =
let doSomethingWhileLive vitalForce =
// return nothing in the first element of the tuple
// return the newVitalForce in the second element of the tuple
(), newVitalForce
M doSomethingWhileLive

With getM and putM in place, we can now create a function that
gets the current vital force from the context
extracts one unit from that
replaces the current vital force with the remaining vital force
returns the one unit of vital force to the caller
And here's the code:
let useUpOneUnitM =
monster {
let! vitalForce = getM
let oneUnit, remainingVitalForce = getVitalForce vitalForce
do! putM remainingVitalForce
return oneUnit
}

Using the monster expression to rewrite all the

other functions
With useUpOneUnitM , we can start to rewrite all the other functions.
For example, the original function makeLiveLeftLegM looked like this, with lots of explicit
handling of the vital force.

1321

Refactoring the Monadster

let makeLiveLeftLegM deadLeftLeg =

The new version, using a monster expression, has implicit handling of the vital force, and
consequently looks much cleaner.
let makeLiveLeftLegM deadLeftLeg =
monster {
let (DeadLeftLeg label) = deadLeftLeg
let! oneUnit = useUpOneUnitM
return LiveLeftLeg (label,oneUnit)
}

Similarly, we can rewrite all the arm surgery code like this:
let makeLiveRightLowerArm (DeadRightLowerArm label) =
monster {
let! oneUnit = useUpOneUnitM
return LiveRightLowerArm (label,oneUnit)
}
let makeLiveRightUpperArm (DeadRightUpperArm label) =
monster {
let! oneUnit = useUpOneUnitM
return LiveRightUpperArm (label,oneUnit)
}
// create the M-parts
let lowerRightArmM = DeadRightLowerArm "Tom" |> makeLiveRightLowerArm
let upperRightArmM = DeadRightUpperArm "Jerry" |> makeLiveRightUpperArm
// turn armSurgery into an M-function
let armSurgeryM = map2M armSurgery
// do surgery to combine the two M-parts into a new M-part
let rightArmM = armSurgeryM lowerRightArmM upperRightArmM

And so on. This new code is much cleaner.

In fact, we can make it cleaner yet by eliminating the intermediate values such as
armSurgery and armSurgeryM and putting everything in the one monster expression.

1322

Refactoring the Monadster

let rightArmM = monster {

let! lowerArm = DeadRightLowerArm "Tom" |> makeLiveRightLowerArm
let! upperArm = DeadRightUpperArm "Jerry" |> makeLiveRightUpperArm
return {lowerArm=lowerArm; upperArm=upperArm}
}

We can use this approach for the head as well. We don't need headSurgery or returnM any
more.
let headM = monster {
let! brain = makeLiveBrain deadBrain
return {brain=brain; skull=skull}
}

Finally, we can use a monster expression to create the whole body too:
// a function to create a M-body given all the M-parts
let createBodyM leftLegM rightLegM leftArmM rightArmM headM beatingHeartM =
monster {
let! leftLeg = leftLegM
let! rightLeg = rightLegM
let! leftArm = leftArmM
let! rightArm = rightArmM
let! head = headM
let! beatingHeart = beatingHeartM
// create the record
return {
leftLeg = leftLeg
rightLeg = rightLeg
leftArm = leftArm
rightArm = rightArm
head = head
heart = beatingHeart
}
}
// create the M-body
let bodyM = createBodyM leftLegM rightLegM leftArmM rightArmM headM beatingHeartM

NOTE: The complete code using monster expressions is available on GitHub.

Monster expressions vs applyM

We previously used an alternative way to create the body, using applyM .
For reference, here's that way using applyM :

1323

Refactoring the Monadster

let createBody leftLeg rightLeg leftArm rightArm head beatingHeart =

{
leftLeg = leftLeg
rightLeg = rightLeg
leftArm = leftArm
rightArm = rightArm
head = head
heart = beatingHeart
}
let bodyM =
createBody
<!> leftLegM
<*> rightLegM
<*> leftArmM
<*> rightArmM
<*> headM
<*> beatingHeartM

So what's the difference?

Aesthetically there is a bit of difference, but you could legitimately prefer either.
However, there is a much more important difference between the applyM approach and the
monster expression approach.

The applyM approach allows the parameters to be run independently or in parallel, while
the monster expression approach requires that the parameters are run in sequence, with
the output of one being fed into the input of the next.
That's not relevant for this scenario, but can be important for other situations such as
validation or async. For example, in a validation context, you may want to collect all the
validation errors at once, rather than only returning the first one that fails.

Relationship to the state monad

Dr Frankenfunctor was a pioneer in her time, blazing a new trail, but she did not generalize
her discoveries to other domains.
Nowadays, this pattern of threading some information through a series of functions is very
common, and we give it a standard name: "State Monad".
Now to be a true monad, there are various properties that must be satisfied (the so-called
monad laws), but I am not going to discuss them here, as this post is not intended to be a
monad tutorial.

1324

Refactoring the Monadster

Instead, I'll just focus on how the state monad might be defined and used in practice.
First off, to be truly reusable, we need to replace the VitalForce type with other types. So
our function-wrapping type (call it S ) must have two type parameters, one for the type of
the state, and one for the type of the value.
type S<'State,'Value> =
S of ('State -> 'Value * 'State)

With this defined, we can create the usual suspects: runS , returnS and bindS .
// encapsulate the function call that "runs" the state
let runS (S f) state = f state
// lift a value to the S-world
let returnS x =
let run state =
x, state
S run
// lift a monadic function to the S-world
let bindS f xS =
let run state =
let x, newState = runS xS state
runS (f x) newState
S run

Personlly, I'm glad that we got to understood how these worked in the M context before
making them completely generic. I don't know about you, but signatures like these
val runS : S<'a,'b> -> 'a -> 'b * 'a
val bindS : f:('a -> S<'b,'c>) -> S<'b,'a> -> S<'b,'c>

would be really hard to understand on their own, without any preparation.

Anyway, with those basics in place we can create a state expression.
type StateBuilder()=
member this.Return(x) = returnS x
member this.Bind(xS,f) = bindS f xS
let state = new StateBuilder()

getS and putS are defined in a similar way as getM and putM for monster .

1325

Refactoring the Monadster

let getS =
let run state =
// return the current state in the first element of the tuple
state, state
S run
// val getS : S<State>
let putS newState =
let run _ =
// return nothing in the first element of the tuple
// return the newState in the second element of the tuple
(), newState
S run
// val putS : 'State -> S<unit>

Property based testing of the state expression

Before moving on, how do we know that our state implementation is correct? What does it
even mean to be correct?
Well, rather than writing lots of example based tests, this is a great candidate for a propertybased testing approach.
The properties we might expect to be satisfied include:
The monad laws.
Only the last put counts. That is, putting X then putting Y should be the same as just
putting Y.
Get should return the last put. That is, putting X then doing a get should return the
same X.
and so on.
I won't go into this any more right now. I suggest watching the talk for a more in-depth
discussion.

Using the state expression instead of the monster

expression
We can now use state expressions exactly as we did monster expressions. Here's an
example:

1326

Refactoring the Monadster

// combine get and put to extract one unit

let useUpOneUnitS = state {
let! vitalForce = getS
let oneUnit, remainingVitalForce = getVitalForce vitalForce
do! putS remainingVitalForce
return oneUnit
}
type DeadLeftLeg = DeadLeftLeg of Label
type LiveLeftLeg = LiveLeftLeg of Label * VitalForce
// new version with implicit handling of vital force
let makeLiveLeftLeg (DeadLeftLeg label) = state {
let! oneUnit = useUpOneUnitS
return LiveLeftLeg (label,oneUnit)
}

Another example is how to build a BeatingHeart :

type DeadHeart = DeadHeart of Label
type LiveHeart = LiveHeart of Label * VitalForce
type BeatingHeart = BeatingHeart of LiveHeart * VitalForce
let makeLiveHeart (DeadHeart label) = state {
let! oneUnit = useUpOneUnitS
return LiveHeart (label,oneUnit)
}
let makeBeatingHeart liveHeart = state {
let! oneUnit = useUpOneUnitS
return BeatingHeart (liveHeart,oneUnit)
}
let beatingHeartS = state {
let! liveHeart = DeadHeart "Anne" |> makeLiveHeart
return! makeBeatingHeart liveHeart
}
let beatingHeart, remainingFromHeart = runS beatingHeartS vf

As you can see, the state expression automatically picked up that VitalForce was being
used as the state -- we didn't need to specify it explicitly.
So, if you have a state expression type available to you, you don't need to create your own
expressions like monster at all!
For a more detailed and complex example of the state monad in F#, check out the FSharpx
library.

1327

Refactoring the Monadster

NOTE: The complete code using state expressions is available on GitHub.

Other examples of using a state expression

The state computation expression, once defined, can be used for all sorts of things. For
example, we can use state to model a stack.
Let's start by defining a Stack type and associated functions:
// define the type to use as the state
type Stack<'a> = Stack of 'a list
// define pop outside of state expressions
let popStack (Stack contents) =
match contents with
| [] -> failwith "Stack underflow"
| head::tail ->
head, (Stack tail)
// define push outside of state expressions
let pushStack newTop (Stack contents) =
Stack (newTop::contents)
// define an empty stack
let emptyStack = Stack []
// get the value of the stack when run
// starting with the empty stack
let getValue stackM =
runS stackM emptyStack |> fst

Note that none of that code knows about or uses the state computation expression.
To make it work with state , we need to define a customized getter and putter for use in a
state context:

1328

Refactoring the Monadster

let pop() = state {

let! stack = getS
let top, remainingStack = popStack stack
do! putS remainingStack
return top
}
let push newTop = state {
let! stack = getS
let newStack = pushStack newTop stack
do! putS newStack
return ()
}

With these in place we can start coding our domain!

Stack-based Hello World

Here's a simple one. We push "world", then "hello", then pop the stack and combine the
results.
let helloWorldS = state {
do! push "world"
do! push "hello"
let! top1 = pop()
let! top2 = pop()
let combined = top1 + " " + top2
return combined
}
let helloWorld = getValue helloWorldS // "hello world"

Stack-based calculator
Here's a simple stack-based calculator:
let one = state {do! push 1}
let two = state {do! push 2}
let add = state {
let! top1 = pop()
let! top2 = pop()
do! push (top1 + top2)
}

And now we can combine these basic states to build more complex ones:

1329

Refactoring the Monadster

let three = state {

do! one
do! two
do! add
}
let five = state {
do! two
do! three
do! add
}

Remember that, just as with the vital force, all we have now is a recipe for building a stack.
We still need to run it to execute the recipe and get the result.
Let's add a helper to run all the operations and return the top of the stack:
let calculate stackOperations = state {
do! stackOperations
let! top = pop()
return top
}

Now we can evaluate the operations, like this:

let threeN = calculate three |> getValue // 3
let fiveN = calculate five |> getValue // 5

OK, OK, some monad stuff

People always want to know about monads, even though I do not want these posts to
degenerate into yet another monad tutorial.
So here's how they fit in with what we have worked with in these posts.
A functor (in a programming sense, anyway) is a data structure (such as Option, or List, or
State) which has a map function associated with it. And the map function has some
properties that it must satisfy (the "functor laws").
A applicative functor (in a programming sense) is a data structure (such as Option, or List,
or State) which has two functions associated with it: apply and pure (which is the same
as return ). And these functions have some properties that they must satisfy (the
"applicative functor laws").

1330

Refactoring the Monadster

Finally, a monad (in a programming sense) is a data structure (such as Option, or List, or
State) which has two functions associated with it: bind (often written as >>= ) and
return . And again, these functions have some properties that they must satisfy (the

"monad laws").
Of these three, the monad is most "powerful" in a sense, because the bind function allows
you to chain M-producing functions together, and as we have seen, map and apply can be
written in terms of bind and return .
So you can see that both our original M type and the more generic State type, in
conjunction with their supporting functions, are monads, (assuming that our bind and
return implementations satisfy the monad laws).

For a visual version of these definitions, there is a great post called Functors, Applicatives,
And Monads In Pictures.

Further reading
There are a lot of posts about state monads on the web, of course. Most of them are Haskell
oriented, but I hope that those explanations will make more sense after reading this series of
posts, so I'm only going to mention a few follow up links.
State monad in pictures
"A few monads more", from "Learn You A Haskell"
Much Ado About Monads. Discussion about state monad in F#.
And for another important use of "bind", you might find my talk on functional error handling
useful.
If you want to see F# implementations of other monads, look no further than the FSharpx
project.

Summary
Dr Frankenfunctor was a groundbreaking experimentalist, and I'm glad that I have been able
to share insights on her way of working.
We've seen how she discovered a primitive monad-like type, M<BodyPart> , and how mapM ,
map2M , returnM , bindM and applyM were all developed to solve specific problems.

We've also seen how the need to solve the same problems led to the modern-day state
monad and computation expression.

1331

Refactoring the Monadster

Anyway, I hope that this series of posts has been enlightening. My not-so-secret wish is that
monads and their associated combinators will no longer be so shocking to you now...

...and that you can use them wisely in your own projects. Good luck!
NOTE: The code samples used in this post are available on GitHub.

1332

The "Map and Bind and Apply, Oh my!" Series

In this series of posts, I'll attempt to describe some of the core functions for dealing with
generic data types (such as Option and List ). This is a follow-up post to my talk on
functional patterns.
Yes, I know that I promised not to do this kind of thing, but for this post I thought I'd take a
different approach from most people. Rather than talking about abstractions such as type
classes, I thought it might be useful to focus on the core functions themselves and how they
are used in practice.
In other words, a sort of "man page" for map , return , apply , and bind .
So, there is a section for each function, describing their name (and common aliases),
common operators, their type signature, and then a detailed description of why they are
needed and how they are used, along with some visuals (which I always find helpful).
Understanding map and apply. A toolset for working with elevated worlds.
Understanding bind. Or, how to compose world-crossing functions.
Using the core functions in practice. Working with independent and dependent data.
Understanding traverse and sequence. Mixing lists and elevated values.
Using map, apply, bind and sequence in practice. A real-world example that uses all the
techniques.
Reinventing the Reader monad. Or, designing your own elevated world.
Map and Bind and Apply, a summary. .

1333

Understanding map and apply

In this series of posts, I'll attempt to describe some of the core functions for dealing with
generic data types (such as Option and List ). This is a follow-up post to my talk on
functional patterns.
Yes, I know that I promised not to do this kind of thing, but for this post I thought I'd take a
different approach from most people. Rather than talking about abstractions such as type
classes, I thought it might be useful to focus on the core functions themselves and how they
are used in practice.
In other words, a sort of "man page" for map , return , apply , and bind .
So, there is a section for each function, describing their name (and common aliases),
common operators, their type signature, and then a detailed description of why they are
needed and how they are used, along with some visuals (which I always find helpful).
Haskellers and category-theorists may want to look away now! There will be no mathematics
and quite a lot of hand-waving. I am going to avoid jargon and Haskell-specific concepts
such as type classes and focus on the big picture as much as possible. The concepts here
should be applicable to any kind of functional programming in any language.
I know that some people might dislike this approach. That's fine. There is no shortage of
more academic explanations on the web. Start with this and this.
Finally, as with most of the posts on this site, I am writing this up for my own benefit as well,
as part of my own learning process. I don't claim to be an expert at all, so if I have made any
errors please let me know.

Background
To start with, let me provide the background and some terminology.
Imagine that there are two worlds that we could program in: a "normal" everyday world and a
world that I will call the "elevated world" (for reasons that I will explain shortly).
The elevated world is very similar to the normal world. In fact, every thing in the normal
world has a corresponding thing in the elevated world.
So, for example, we have the set of values called Int in the normal world, and in the
elevated world there is a parallel set of values called, say, E<Int> . Similarly, we have the
set of values called String in the normal world, and in the elevated world there is a parallel

1334

Understanding map and apply

set of values called E<String> .

Also, just as there are functions between Int s and String s in the normal world, so there
are functions between E<Int> s and E<String> s in the elevated world.

Note that I am deliberately using the term "world" rather than "type" to emphasis that the
relationships between values in the world are just as important as the underlying data type.

What exactly is an elevated world?

I can't define what an elevated world is exactly, because there are too many different kinds
of elevated worlds, and they don't have anything in common.
Some of them represent data structures ( Option<T> ), some of them represent workflows
( State<T> ), some of them represent signals ( Observable<T> ), or asychronous values
( Async<T> ), or other concepts.

1335

Understanding map and apply

But even though the various elevated worlds have nothing in common specifically, there are
commonalities in the way they can be worked with. We find that certain issues occur over
and over again in different elevated worlds, and we can use standard tools and patterns to
deal with these issues.
The rest of this series will attempt to document these tools and patterns.

Series contents
This series is developed as follows:
First, I'll discuss the tools we have for lifting normal things into the elevated world. This
includes functions such as map , return , apply and bind .
Next, I'll discuss how you can combine elevated values in different ways, based on
whether the values are independent or dependent.
Next, we'll look at some ways of mixing lists with other elevated values.
Finally, we'll look at two real-world examples that put all these techniques to use, and
we'll find ourselves accidentally inventing the Reader monad.
Here's a list of shortcuts to the various functions:
Part 1: Lifting to the elevated world
The map function
The return function
The apply function
The liftN family of functions
The zip function and ZipList world
Part 2: How to compose world-crossing functions
The bind function
List is not a monad. Option is not a monad.
Part 3: Using the core functions in practice
Independent and dependent data
Example: Validation using applicative style and monadic style
Lifting to a consistent world
Kleisli world
Part 4: Mixing lists and elevated values
Mixing lists and elevated values
The traverse / MapM function
The sequence function
"Sequence" as a recipe for ad-hoc implementations
Readability vs. performance

1336

Understanding map and apply

Dude, where's my filter ?

Part 5: A real-world example that uses all the techniques
Example: Downloading and processing a list of websites
Treating two worlds as one
Part 6: Designing your own elevated world
Designing your own elevated world
Filtering out failures
The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 1: Lifting to the elevated world

The first challenge is: how do we get from the normal world to the elevated world?
First, we will assume that for any particular elevated world:
Every type in the normal world has a corresponding type in the elevated world.
Every value in the normal world has a corresponding value in the elevated world.
Every function in the normal world has a corresponding function in the elevated world.
The concept of moving something from the normal world to the elevated world is called
"lifting" (which is why I used the term "elevated world" in the first place).
We'll call these corresponding things "lifted types" and "lifted values" and "lifted functions".
Now because each elevated world is different, there is no common implementation for lifting,
but we can give names to the various "lifting" patterns, such as map and return .
NOTE: There is no standard name for these lifted types. I have seen them called "wrapper
types" or "augmented types" or "monadic types". I'm not really happy with any of these
names, so I invented a new one! Also, I'm trying avoid any assumptions, so I don't want to
imply that the lifted types are somehow better or contain extra information. I hope that by
using the word "elevated" in this post, I can focus on the lifting process rather than on the
types themselves.
As for using the word "monadic", that would be inaccurate, as there is no requirement that
these types are part of a monad.

1337

Understanding map and apply

The map function

Common Names: map , fmap , lift , Select
Common Operators: <$> <!>
What it does: Lifts a function into the elevated world
Signature: (a->b) -> E<a> -> E . Alternatively with the parameters reversed: E<a> ->
(a->b) -> E

Description
"map" is the generic name for something that takes a function in the normal world and
transforms it into a corresponding function in the elevated world.

Each elevated world will have its own implementation of map.

Alternative interpretation
An alternative interpretation of map is that it is a two parameter function that takes an
elevated value ( E<a> ) and a normal function ( a->b ), and returns a new elevated value
( E ) generated by applying the function a->b to the internal elements of E<a> .

In languages where functions are curried by default, such as F#, both these interpretation
are the same. In other languages, you may need to curry/uncurry to switch between the two
uses.
Note that the two parameter version often has the signature E<a> -> (a->b) -> E , with
the elevated value first and the normal function second. From an abstract point of view,
there's no difference between them -- the map concept is the same -- but obviously, the
parameter order affects how you might use map functions in practice.

1338

Understanding map and apply

Implementation examples
Here are two examples of how map can be defined for options and lists in F#.
/// map for Options
let mapOption f opt =
match opt with
| None ->
None
| Some x ->
Some (f x)
// has type : ('a -> 'b) -> 'a option -> 'b option
/// map for Lists
let rec mapList f list =
match list with
| [] ->
[]
| head::tail ->
// new head + new tail
(f head) :: (mapList f tail)
// has type : ('a -> 'b) -> 'a list -> 'b list

These functions are built-in of course, so we don't need to define them, I've done it just to
show what they might look for some common types.

Usage examples
Here are some examples of how map can be used in F#:
// Define a function in the normal world
let add1 x = x + 1
// has type : int -> int
// A function lifted to the world of Options
let add1IfSomething = Option.map add1
// has type : int option -> int option
// A function lifted to the world of Lists
let add1ToEachElement = List.map add1
// has type : int list -> int list

With these mapped versions in place you can write code like this:
Some 2 |> add1IfSomething // Some 3
[1;2;3] |> add1ToEachElement // [2; 3; 4]

1339

Understanding map and apply

In many cases, we don't bother to create an intermediate function -- partial application is

used instead:
Some 2 |> Option.map add1 // Some 3
[1;2;3] |> List.map add1 // [2; 3; 4]

The properties of a correct map implementation

I said earlier that the elevated world is in some ways a mirror of the normal world. Every
function in the normal world has a corresponding function in the elevated world, and so on.
We want map to return this corresponding elevated function in a sensible way.
For example, map of add should not (wrongly) return the elevated version of multiply ,
and map of lowercase should not return the elevated version of uppercase ! But how can
we be sure that a particular implementation of map does indeed return the correct
corresponding function?
In my post on property based testing I showed how a correct implementation of a function
can be defined and tested using general properties rather than specific examples.
This is true for map as well. The implementation will vary with the specific elevated world,
but in all cases, there are certain properties that the implementation should satisfy to avoid
strange behavior.
First, if you take the id function in the normal world, and you lift it into the elevated world
with map , the new function must be the same as the id function in the elevated world.

Next, if you take two functions f and g in the normal world, and you compose them (into
h , say), and then lift the resulting function using map , the resulting function should be the

same as if you lifted f and g into the elevated world first, and then composed them there
afterwards.

1340

Understanding map and apply

These two properties are the so-called "Functor Laws", and a Functor (in the programming
sense) is defined as a generic data type -- E<T> in our case -- plus a map function that
obeys the functor laws.
NOTE: "Functor" is a confusing word. There is "functor" in the category theory sense, and
there is "functor" in the programming sense (as defined above). There are also things called
"functors" defined in libraries, such as the Functor type class in Haskell, and the Functor
trait in Scalaz, not to mention functors in SML and OCaml (and C++), which are different yet
again!
Conseqently, I prefer to talk about "mappable" worlds. In practical programming, you will
almost never run into a elevated world that does not support being mapped over somehow.

Variants of map
There are some variants of map that are common:
Const map. A const or "replace-by" map replaces all values with a constant rather than
the output of a function. In some cases, a specialized function like this can allow for a
more efficient implementation.
Maps that work with cross-world functions. The map function a->b lives entirely in
the normal world. But what if the function you want to map with does not return
something in the normal world, but a value in another, different, enhanced world? We'll
see how to address this challenge in a later post.

The return function

Common Names: return , pure , unit , yield , point
Common Operators: None
What it does: Lifts a single value into the elevated world

1341

Understanding map and apply

Signature: a -> E<a>

Description
"return" (also known as "unit" or "pure") simply creates a elevated value from a normal
value.

This function goes by many names, but I'm going to be consistent and call it return as that
is the common term for it in F#, and is the term used in computation expressions.
NOTE: I'm ignoring the difference between pure and return , because type classes are
not the focus of this post.

Implementation examples
Here are two examples of return implementations in F#:
// A value lifted to the world of Options
let returnOption x = Some x
// has type : 'a -> 'a option
// A value lifted to the world of Lists
let returnList x = [x]
// has type : 'a -> 'a list

Obviously, we don't need to define special functions like this for options and lists. Again, I've
just done it to show what return might look for some common types.

The apply function

Common Names: apply , ap
Common Operators: <*>

1342

Understanding map and apply

What it does: Unpacks a function wrapped inside a elevated value into a lifted function
E<a> -> E

Signature: E<(a->b)> -> E<a> -> E

Description
"apply" unpacks a function wrapped inside a elevated value ( E<(a->b)> ) into a lifted
function E<a> -> E

This might seem unimportant, but is actually very valuable, as it allows you to lift a multiparameter function in the normal world into a multi-parameter function in the elevated world,
as we'll see shortly.

Alternative interpretation
An alternative interpretation of apply is that it is a two parameter function that takes a
elevated value ( E<a> ) and a elevated function ( E<(a->b)> ), and returns a new elevated
value ( E ) generated by applying the function a->b to the internal elements of E<a> .
For example, if you have a one-parameter function ( E<(a->b)> ), you can apply it to a single
elevated parameter to get the output as another elevated value.

If you have a two-parameter function ( E<(a->b->c)> ), you can use apply twice in
succession with two elevated parameters to get the elevated output.

You can continue to use this technique to work with as many parameters as you wish.

Implementation examples
Here are some examples of defining apply for two different types in F#:

1343

Understanding map and apply

module Option =
// The apply function for Options
let apply fOpt xOpt =
match fOpt,xOpt with
| Some f, Some x -> Some (f x)
| _ -> None
module List =
// The apply function for lists
// [f;g] apply [x;y] becomes [f x; f y; g x; g y]
let apply (fList: ('a->'b) list) (xList: 'a list) =
[ for f in fList do
for x in xList do
yield f x ]

In this case, rather than have names like applyOption and applyList , I have given the
functions the same name but put them in a per-type module.
Note that in the List.apply implementation, each function in the first list is applied to each
value in the second list, resulting in a "cross-product" style result. That is, the list of functions
[f; g] applied to the list of values [x; y] becomes the four-element list [f x; f y; g x;
g y] . We'll see shortly that this is not the only way to do it.

Also, of course, I'm cheating by building this implementation on a for..in..do loop -functionality that already exists!
I did this for clarity in showing how apply works. It's easy enough to create a "from scratch"
recursive implementation, (though it is not so easy to create one that is properly tailrecursive!) but I want to focus on the concepts not on the implementation for now.

Infix version of apply

Using the apply function as it stands can be awkward, so it is common to create an infix
version, typically called <*> . With this in place you can write code like this:
let resultOption =
let (<*>) = Option.apply
(Some add) <*> (Some 2) <*> (Some 3)
// resultOption = Some 5
let resultList =
let (<*>) = List.apply
[add] <*> [1;2] <*> [10;20]
// resultList = [11; 21; 12; 22]

1344

Understanding map and apply

Apply vs. Map

The combination of apply and return is considered "more powerful" than map , because
if you have apply and return , you can construct map from them, but not vice versa.
Here's how it works: to construct a lifted function from a normal function, just use return on
the normal function and then apply . This gives you the same result as if you had simply
done map in the first place.

This trick also means that our infix notation can be simplified a little. The initial return then
apply can be replaced with map , and we so typically create an infix operator for map as

well, such as <!> in F#.

let resultOption2 =
let (<!>) = Option.map
let (<*>) = Option.apply
add <!> (Some 2) <*> (Some 3)
// resultOption2 = Some 5
let resultList2 =
let (<!>) = List.map
let (<*>) = List.apply
add <!> [1;2] <*> [10;20]
// resultList2 = [11; 21; 12; 22]

This makes the code look more like using the function normally. That is, instead of the
normal add x y , we can use the similar looking add <!> x <*> y , but now x and y can
be elevated values rather than normal values. Some people have even called this style
"overloaded whitespace"!
Here's one more for fun:

1345

Understanding map and apply

let batman =
let (<!>) = List.map
let (<*>) = List.apply
// string concatenation using +
(+) <!> ["bam"; "kapow"; "zap"] <*> ["!"; "!!"]
// result =
// ["bam!"; "bam!!"; "kapow!"; "kapow!!"; "zap!"; "zap!!"]

The properties of a correct apply/return implementation

As with map , a correct implementation of the apply / return pair should have some
properties that are true no matter what elevated world we are working with.
There are four so-called "Applicative Laws", and an Applicative Functor (in the
programming sense) is defined as a generic data type constructor -- E<T> -- plus a pair of
functions ( apply and return ) that obey the applicative laws.
Just as with the laws for map , these laws are quite sensible. I'll show two of them.
The first law says that if you take the id function in the normal world, and you lift it into the
elevated world with return , and then you do apply , the new function, which is of type
E<a> -> E<a> , should be the same as the id function in the elevated world.

The second law says that if you take a function f and a value x in the normal world, and
you apply f to x to get a result ( y , say), and then lift the result using return , the
resulting value should be the same as if you lifted f and x into the elevated world first,
and then applied them there afterwards.

1346

Understanding map and apply

The other two laws are not so easily diagrammed, so I won't document them here, but
together the laws ensure that any implementation is sensible.

The liftN family of functions

Common Names: lift2 , lift3 , lift4 and similar
Common Operators: None
What it does: Combines two (or three, or four) elevated values using a specified function
Signature:
lift2: (a->b->c) -> E<a> -> E -> E<c> ,
lift3: (a->b->c->d) -> E<a> -> E -> E<c> -> E<d> ,
etc.

Description
The apply and return functions can be used to define a series of helper functions liftN
( lift2 , lift3 , lift4 , etc) that take a normal function with N parameters (where
N=2,3,4, etc) and transform it to a corresponding elevated function.
Note that lift1 is just map , and so it is not usually defined as a separate function.
Here's what an implementation might look like:

1347

Understanding map and apply

module Option =
let (<*>) = apply
let (<!>) = Option.map
let lift2 f x y =
f <!> x <*> y
let lift3 f x y z =
f <!> x <*> y <*> z
let lift4 f x y z w =
f <!> x <*> y <*> z <*> w

Here's a visual representation of lift2 :

The lift series of functions can be used to make code a bit more readable because, by
using one of the pre-made lift functions, we can avoid the <*> syntax.
First, here's an example of lifting a two-parameter function:
// define a two-parameter function to test with
let addPair x y = x + y
// lift a two-param function
let addPairOpt = Option.lift2 addPair
// call as normal
addPairOpt (Some 1) (Some 2)
// result => Some 3

And here's an example of lifting a three-parameter function:

1348

Understanding map and apply

// define a three-parameter function to test with

let addTriple x y z = x + y + z
// lift a three-param function
let addTripleOpt = Option.lift3 addTriple
// call as normal
addTripleOpt (Some 1) (Some 2) (Some 3)
// result => Some 6

Interpreting "lift2" as a "combiner"

There is an alternative interpretation of apply as a "combiner" of elevated values, rather
than as function application.
For example, when using lift2 , the first parameter is a parameter specifying how to
combine the values.
Here's an example where the same values are combined in two different ways: first with
addition, then with multiplication.
Option.lift2 (+) (Some 2) (Some 3) // Some 5
Option.lift2 (*) (Some 2) (Some 3) // Some 6

Going further, can we eliminate the need for this first function parameter and have a generic
way of combining the values?
Why, yes we can! We can just use a tuple constructor to combine the values. When we do
this we are combining the values without making any decision about how they will be used
yet.
Here's what it looks like in a diagram:

and here's how you might implement it for options and lists:

1349

Understanding map and apply

// define a tuple creation function

let tuple x y = x,y
// create a generic combiner of options
// with the tuple constructor baked in
let combineOpt x y = Option.lift2 tuple x y
// create a generic combiner of lists
// with the tuple constructor baked in
let combineList x y = List.lift2 tuple x y

Let's see what happens when we use the combiners:

combineOpt (Some 1) (Some 2)
// Result => Some (1, 2)
combineList [1;2] [100;200]
// Result => [(1, 100); (1, 200); (2, 100); (2, 200)]

Now that we have an elevated tuple, we can work with the pair in any way we want, we just
need to use map to do the actual combining.
Want to add the values? Just use + in the map function:
combineOpt (Some 2) (Some 3)
|> Option.map (fun (x,y) -> x + y)
// Result => // Some 5
combineList [1;2] [100;200]
|> List.map (fun (x,y) -> x + y)
// Result => [101; 201; 102; 202]

Want to multiply the values? Just use * in the map function:

combineOpt (Some 2) (Some 3)
|> Option.map (fun (x,y) -> x * y)
// Result => Some 6
combineList [1;2] [100;200]
|> List.map (fun (x,y) -> x * y)
// Result => [100; 200; 200; 400]

And so on. Obviously, real-world uses would be somewhat more interesting.

Defining apply in terms of lift2

1350

Understanding map and apply

Interestingly, the lift2 function above can be actually used as an alternative basis for
defining apply .
That is, we can define apply in terms of the lift2 function by setting the combining
function to be just function application.
Here's a demonstration of how this works for Option :
module Option =
/// define lift2 from scratch
let lift2 f xOpt yOpt =
match xOpt,yOpt with
| Some x,Some y -> Some (f x y)
| _ -> None
/// define apply in terms of lift2
let apply fOpt xOpt =
lift2 (fun f x -> f x) fOpt xOpt

This alternative approach is worth knowing about because for some types it's easier to
define lift2 than apply .

Combining missing or bad data

Notice that in all the combiners we've looked at, if one of the elevated values is "missing" or
"bad" somehow, then the overall result is also bad.
For example, with combineList , if one of the parameters is an empty list, the result is also
an empty list, and with combineOpt , if one of the parameters is None , the result is also
None .

combineOpt (Some 2) None

|> Option.map (fun (x,y) -> x + y)
// Result => None
combineList [1;2] []
|> List.map (fun (x,y) -> x * y)
// Result => Empty list

It's possible to create an alternative kind of combiner that ignores missing or bad values, just
as adding "0" to a number is ignored. For more information, see my post on "Monoids
without tears".

One sided combiners <* and *>

1351

Understanding map and apply

In some cases you might have two elevated values, and want to discard the value from one
side or the other.
Here's an example for lists:
let ( <* ) x y =
List.lift2 (fun left right -> left) x y
let ( *> ) x y =
List.lift2 (fun left right -> right) x y

We can then combine a 2-element list and a 3-element list to get a 6-element list as
expected, but the contents come from only one side or the other.
[1;2] <* [3;4;5] // [1; 1; 1; 2; 2; 2]
[1;2] *> [3;4;5] // [3; 4; 5; 3; 4; 5]

We can turn this into a feature! We can replicate a value N times by crossing it with [1..n] .
let repeat n pattern =
[1..n] *> pattern
let replicate n x =
[1..n] *> [x]
repeat 3 ["a";"b"]
// ["a"; "b"; "a"; "b"; "a"; "b"]
replicate 5 "A"
// ["A"; "A"; "A"; "A"; "A"]

Of course, this is by no means an efficient way to replicate a value, but it does show that
starting with just the two functions apply and return , you can build up some quite
complex behavior.
On a more practical note though, why might this "throwing away data" be useful? Well in
many cases, we might not want the values, but we do want the effects.
For example, in a parser, you might see code like this:
let readQuotedString =
readQuoteChar *> readNonQuoteChars <* readQuoteChar

1352

Understanding map and apply

In this snippet, readQuoteChar means "match and read a quote character from the input
stream" and readNonQuoteChars means "read a series of non-quote characters from the
input stream".
When we are parsing a quoted string we want ensure the input stream that contains the
quote character is read, but we don't care about the quote characters themselves, just the
inner content.
Hence the use of *> to ignore the leading quote and <* to ignore the trailing quote.

The zip function and ZipList world

Common Names: zip , zipWith , map2
Common Operators: <*> (in the context of ZipList world)
What it does: Combines two lists (or other enumerables) using a specified function
Signature: E<(a->b->c)> -> E<a> -> E -> E<c> where E is a list or other enumerable
type, or E<a> -> E -> E<a,b> for the tuple-combined version.

Description
Some data types might have more than one valid implementation of apply . For example,
there is another possible implementation of apply for lists, commonly called ZipList or
some variant of that.
In this implementation, the corresponding elements in each list are processed at the same
time, and then both lists are shifted to get the next element. That is, the list of functions [f;
g] applied to the list of values [x; y] becomes the two-element list [f x; g y]

// alternate "zip" implementation

// [f;g] apply [x;y] becomes [f x; g y]
let rec zipList fList xList =
match fList,xList with
| [],_
| _,[] ->
// either side empty, then done
[]
| (f::fTail),(x::xTail) ->
// new head + new tail
(f x) :: (zipList fTail xTail)
// has type : ('a -> 'b) -> 'a list -> 'b list

1353

Understanding map and apply

WARNING: This implementation is just for demonstration. It's not tail-recursive, so don't use
it for large lists!
If the lists are of different lengths, some implementations throw an exception (as the F#
library functions List.map2 and List.zip do), while others silently ignore the extra data
(as the implementation above does).
Ok, let's see it in use:
let add10 x = x + 10
let add20 x = x + 20
let add30 x = x + 30
let result =
let (<*>) = zipList
[add10; add20; add30] <*> [1; 2; 3]
// result => [11; 22; 33]

Note that the result is [11; 22; 33] -- only three elements. If we had used the standard
List.apply , there would have been nine elements.

Interpreting "zip" as a "combiner"

We saw above that List.apply , or rather List.lift2 , could be intepreted as a combiner.
Similarly, so can zipList .
let add x y = x + y
let resultAdd =
let (<*>) = zipList
[add;add] <*> [1;2] <*> [10;20]
// resultAdd = [11; 22]
// [ (add 1 10); (add 2 20) ]

Note that we can't just have one add function in the first list -- we have to have one add
for every element in the second and third lists!
That could get annoying, so often, a "tupled" version of zip is used, whereby you don't
specify a combining function at all, and just get back a list of tuples instead, which you can
then process later using map . This is the same approach as was used in the combine
functions discussed above, but for zipList .

ZipList world

1354

Understanding map and apply

In standard List world, there is an apply and a return . But with our different version of
apply we can create a different version of List world called ZipList world.

ZipList world is quite different from the standard List world.

In ZipList world, the apply function is implemented as above. But more interestingly, ZipList
world has a completely different implementation of return compared with standard List
world. In the standard List world, return is just a list with a single element, but for ZipList
world, it has to be an infinitely repeated value!
In a non-lazy language like F#, we can't do this, but if we replace List with Seq (aka
IEnumerable ) then we can create an infinitely repeated value, as shown below:

module ZipSeq =
// define "return" for ZipSeqWorld
let retn x = Seq.initInfinite (fun _ -> x)
// define "apply" for ZipSeqWorld
// (where we can define apply in terms of "lift2", aka "map2")
let apply fSeq xSeq =
Seq.map2 (fun f x -> f x) fSeq xSeq
// has type : ('a -> 'b) seq -> 'a seq -> 'b seq
// define a sequence that is a combination of two others
let triangularNumbers =
let (<*>) = apply
let addAndDivideByTwo x y = (x + y) / 2
let numbers = Seq.initInfinite id
let squareNumbers = Seq.initInfinite (fun i -> i * i)
(retn addAndDivideByTwo) <*> numbers <*> squareNumbers
// evaulate first 10 elements
// and display result
triangularNumbers |> Seq.take 10 |> List.ofSeq |> printfn "%A"
// Result =>
// [0; 1; 3; 6; 10; 15; 21; 28; 36; 45]

This example demonstrates that an elevated world is not just a data type (like the List type)
but consists of the datatype and the functions that work with it. In this particular case, "List
world" and "ZipList world" share the same data type but have quite different environments.

What types support map and apply and

return ?

1355

Understanding map and apply

So far we have defined all these useful functions in an abstract way. But how easy is it to
find real types that have implementations of them, including all the various laws?
The answer is: very easy! In fact almost all types support these set of functions. You'd be
hard-pressed to find a useful type that didn't.
That means that map and apply and return are available (or can be easily implemented)
for standard types such as Option , List , Seq , Async , etc., and also any types you are
likely to define yourself.

Summary
In this post, I described three core functions for lifting simple "normal" values to elevated
worlds: map , return , and apply , plus some derived functions like liftN and zip .
In practice however, things are not that simple. We frequently have to work with functions
that cross between the worlds. Their input is in the normal world but their output is in the
elevated world.
In the next post we'll show how these world-crossing functions can be lifted to the elevated
world as well.

1356

Understanding bind

Understanding bind
This post is the second in a series. In the previous post, I described some of the core
functions for lifting a value from a normal world to an elevated world.
In this post, we'll look at "world-crossing" functions, and how they can be tamed with the
bind function.

Series contents
Here's a list of shortcuts to the various functions mentioned in this series:
Part 1: Lifting to the elevated world
The map function
The return function
The apply function
The liftN family of functions
The zip function and ZipList world
Part 2: How to compose world-crossing functions
The bind function
List is not a monad. Option is not a monad.
Part 3: Using the core functions in practice
Independent and dependent data
Example: Validation using applicative style and monadic style
Lifting to a consistent world
Kleisli world
Part 4: Mixing lists and elevated values
Mixing lists and elevated values
The traverse / MapM function
The sequence function
"Sequence" as a recipe for ad-hoc implementations
Readability vs. performance
Dude, where's my filter ?
Part 5: A real-world example that uses all the techniques
Example: Downloading and processing a list of websites
Treating two worlds as one
Part 6: Designing your own elevated world
Designing your own elevated world

1357

Understanding bind

Filtering out failures

The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 2: How to compose world-crossing

functions
The bind function
Common Names: bind , flatMap , andThen , collect , SelectMany
Common Operators: >>= (left to right), =<< (right to left )
What it does: Allows you to compose world-crossing ("monadic") functions
Signature: (a->E) -> E<a> -> E . Alternatively with the parameters reversed: E<a> ->
(a->E) -> E

Description
We frequently have to deal with functions that cross between the normal world and the
elevated world.
For example: a function that parses a string to an int might return an Option<int>
rather than a normal int , a function that reads lines from a file might return
IEnumerable<string> , a function that fetches a web page might return Async<string> , and

so on.
These kinds of "world-crossing" functions are recognizable by their signature a -> E ;
their input is in the normal world but their output is in the elevated world. Unfortunately, this
means that these kinds of functions cannot be linked together using standard composition.

What "bind" does is transform a world-crossing function (commonly known as a "monadic

function") into a lifted function E<a> -> E .

1358

Understanding bind

The benefit of doing this is that the resulting lifted functions live purely in the elevated world,
and so can be combined easily by composition.
For example, a function of type a -> E cannot be directly composed with a function of
type b -> E<c> , but after bind is used, the second function becomes of type E ->
E<c> , which can be composed.

In this way, bind lets us chain together any number of monadic functions.

Alternative interpretation
An alternative interpretation of bind is that it is a two parameter function that takes a
elevated value ( E<a> ) and a "monadic function" ( a -> E ), and returns a new elevated
value ( E ) generated by "unwrapping" the value inside the input, and running the function
a -> E against it. Of course, the "unwrapping" metaphor does not work for every

elevated world, but still it can often be useful to think of it this way.

Implementation examples
Here are some examples of defining bind for two different types in F#:

1359

Understanding bind

module Option =
// The bind function for Options
let bind f xOpt =
match xOpt with
| Some x -> f x
| _ -> None
// has type : ('a -> 'b option) -> 'a option -> 'b option
module List =
// The bind function for lists
let bindList (f: 'a->'b list) (xList: 'a list) =
[ for x in xList do
for y in f x do
yield y ]
// has type : ('a -> 'b list) -> 'a list -> 'b list

Notes:
Of course, in these two particular cases, the functions already exist in F#, called
Option.bind and List.collect .

For List.bind I'm cheating again and using for..in..do , but I think that this particular
implementation shows clearly how bind works with lists. There is a purer recursive
implementation, but I won't show that here.

Usage example
As explained at the beginning of this section, bind can be used to compose cross-world
functions. Let's see how this works in practice with a simple example.
First let's say we have a function that parses certain string s into int s. Here's a very
simple implementation:
let parseInt str =
match str with
| "-1" -> Some -1
| "0" -> Some 0
| "1" -> Some 1
| "2" -> Some 2
// etc
| _ -> None
// signature is string -> int option

1360

Understanding bind

Sometimes it returns a int, sometimes not. So the signature is string -> int option -- a
cross-world function.
And let's say we have another function that takes an int as input and returns a OrderQty
type:
type OrderQty = OrderQty of int
let toOrderQty qty =
if qty >= 1 then
Some (OrderQty qty)
else
// only positive numbers allowed
None
// signature is int -> OrderQty option

Again, it might not return an OrderQty if the input is not positive. The signature is therefore
int -> OrderQty option -- another cross-world function.

Now, how can we create a function that starts with an string and returns an OrderQty in one
step?
The output of parseInt cannot be fed directly into toOrderQty , so this is where bind
comes to the rescue!
Doing Option.bind toOrderQty lifts it to a int option -> OrderQty option function and so
the output of parseInt can be used as input, just as we need.
let parseOrderQty str =
parseInt str
|> Option.bind toOrderQty
// signature is string -> OrderQty option

The signature of our new parseOrderQty is string -> OrderQty option , yet another crossworld function. So if we want to do something with the OrderQty that is output we may well
have to use bind again on the next function in the chain.

Infix version of bind

As with apply , using the named bind function can be awkward, so it is common to create
an infix version, typically called >>= (for left to right data flow) or =<< (for right to left data
flow) .
With this in place you can write an alternative version of parseOrderQty like this:

1361

Understanding bind

let parseOrderQty_alt str =

str |> parseInt >>= toOrderQty

You can see that >>= performs the same kind of role as pipe ( |> ) does, except that it
works to pipe "elevated" values into cross-world functions.

Bind as a "programmable semicolon"

Bind can be used to chain any number of functions or expressions together, so you often
see code looking something like this:
expression1 >>=
expression2 >>=
expression3 >>=
expression4

This is not too different from how an imperative program might look if you replace the >>=
with a ; :
statement1;
statement2;
statement3;
statement4;

Because of this, bind is sometimes called a "programmable semicolon".

Language support for bind/return

Most functional programming languages have some kind of syntax support for bind that
lets you avoid having to write a series of continuations or use explicit binds.
In F# it is (one component) of computation expressions, so the following explicit chaining of
bind :

initialExpression >>= (fun x ->

expressionUsingX >>= (fun y ->
expressionUsingY >>= (fun z ->
x+y+z ))) // return

becomes implicit, using let! syntax:

1362

Understanding bind

elevated {
let! x = initialExpression
let! y = expressionUsingX x
let! z = expressionUsingY y
return x+y+z }

In Haskell, the equivalent is the "do notation":

do
x <- initialExpression
y <- expressionUsingX x
z <- expressionUsingY y
return x+y+z

And in Scala, the equivalent is the "for comprehension":

for {
x <- initialExpression
y <- expressionUsingX(x)
z <- expressionUsingY(y)
} yield {
x+y+z
}

It's important to emphasize that you do not have to use the special syntax when using
bind/return. You can always use bind or >>= in the same way as any other function.

Bind vs. Apply vs. Map

The combination of bind and return are considered even more powerful than apply and
return , because if you have bind and return , you can construct map and apply from

them, but not vice versa.

Here's how bind can be used to emulate map , for example:
First, you construct a world-crossing function from a normal function by applying
return to the output.

Next, convert this world-crossing function into a lifted function using bind . This gives
you the same result as if you had simply done map in the first place.

1363

Understanding bind

Similarly, bind can emulate apply . Here is how map and apply can be defined using
bind and return for Options in F#:

// map defined in terms of bind and return (Some)

let map f =
Option.bind (f >> Some)
// apply defined in terms of bind and return (Some)
let apply fOpt xOpt =
fOpt |> Option.bind (fun f ->
let map = Option.bind (f >> Some)
map xOpt)

At this point, people often ask "why should I use apply instead of bind when bind is
more powerful?"
The answer is that just because apply can be emulated by bind , doesn't mean it should
be. For example, it is possible to implement apply in a way that cannot be emulated by a
bind implementation.

In fact, using apply ("applicative style") or bind ("monadic style") can have a profound
effect on how your program works! We'll discuss these two approaches in more detail in part
3 of this post.

The properties of a correct bind/return implementation

As with map , and as with apply / return , a correct implementation of the bind / return
pair should have some properties that are true no matter what elevated world we are
working with.
There are three so-called "Monad Laws", and one way of defining a Monad (in the
programming sense) is to say that it consists of three things: a generic type constructor
E<T> plus a pair of functions ( bind and return ) that obey the monad laws. This is not the

only way to define a monad, and mathematicians typically use a slightly different definition,
but this one is most useful to programmers.

1364

Understanding bind

Just as with the the Functor and Applicative laws we saw earlier, these laws are quite
sensible.
First, note that return function is itself a cross-world function:

That means that we can use bind to lift it into a function in the elevated world. And what
does this lifted function do? Hopefully, nothing! It should just return its input.
So that is exactly the first monad law: it says that this lifted function must be the same as the
id function in the elevated world.

The second law is similar but with bind and return reversed. Say that we have a normal
value a and cross-world function f that turns an a into a E .

Let's lift both of them to the elevated world, using bind on f and return on a .

1365

Understanding bind

Now if we apply the elevated version of f to the elevated verson of a we get some value
E .

On the other hand if we apply the normal version of f to the normal verson of a we also
get some value E .

The second monad law says that these two elevated values ( E ) should be the same. In
other words, all this binding and returning should not distort the data.
The third monad law is about associativity.
In the normal world, function composition is associative. For example, we could pipe a value
into a function f and then take that result and pipe it into another function g . Alternatively,
we can compose f and g first into a single function and then pipe a into it.
let groupFromTheLeft = (a |> f) |> g
let groupFromTheRight = a |> (f >> g)

In the normal world, we expect both of these alternatives to give the same answer.

1366

Understanding bind

The third monad law says that, after using bind and return , the grouping doesn't matter
either. The two examples below correspond to the examples above:
let groupFromTheLeft = (a >>= f) >>= g
let groupFromTheRight = a >>= (fun x -> f x >>= g)

And again, we expect both of these to give the same answer.

List is not a monad. Option is not a monad.

If you look at the definition above, a monad has a type constructor (a.k.a "generic type") and
two functions and a set of properties that must be satisfied.
The List data type is therefore just one component of a monad, as is the Option data
type. List and Option , by themselves, are not monads.
It might be better to think of a monad as a transformation, so that the "List monad" is the
transformation that converts the normal world to the elevated "List world", and the "Option
monad" is the transformation that converts the normal world to the elevated "Option world".
I think this is where a lot of the confusion comes in. The word "List" can mean many different
things:
1. A concrete type or data structure such as List<int> .
2. A type constructor (generic type): List<T> .
3. A type constructor and some operations, such as a List class or module.
4. A type constructor and some operations and the operations satisfy the monad laws.
Only the last one is a monad! The other meanings are valid but contribute to the confusion.
Also the last two cases are hard to tell apart by looking at the code. Unfortunately, there
have been cases where implementations did not satisfy the monad laws. Just because it's a
"Monad" doesn't mean that it's a monad.
Personally, I try to avoid using the word "monad" on this site and focus on the bind function
instead, as part of a toolkit of functions for solving problems rather than an abstract concept.
So don't ask: Do I have a monad?
Do ask: Do I have useful bind and return functions? And are they implemented correctly?

1367

Understanding bind

Summary
We now have a set of four core functions: map , return , apply , and bind , and I hope
that you are clear on what each one does.
But there are some questions that have not been addressed yet, such as "why should I
choose apply instead of bind ?", or "how can I deal with multiple elevated worlds at the
same time?"
In the next post we'll address these questions and demonstrate how to use the toolset with a
series of practical examples.
UPDATE: Fixed error in monad laws pointed out by @joseanpg. Thanks!

1368

Using the core functions in practice

This post is the third in a series. In the previous two posts, I described some of the core
functions for dealing with generic data types: map , apply , bind , and so on.
In this post, I'll show how to use these functions in practice, and will explain the difference
between the so-called "applicative" and "monadic" styles.

1369

Using the core functions in practice

Filtering out failures

The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 3: Using the core functions in practice

Now that we have the basic tools for lifting normal values to elevated values and working
with cross-world functions, it's time to start working with them!
In this section, we'll look at some examples how these functions are actually used.

Independent vs. dependent data

I briefly mentioned earlier that there is a important difference between using apply and
bind . Let's go into this now.

When using apply , you can see that each parameter ( E<a> , E ) is completely
independent of the other. The value of E does not depend on what E<a> is.

On the other hand, when using bind , the value of E does depend on what E<a> is.

The distinction between working with independent values or dependent values leads to two
different styles:
The so-called "applicative" style uses functions such as apply , lift , and combine
where each elevated value is independent.
The so-called "monadic" style uses functions such as bind to chain together functions

1370

Using the core functions in practice

that are dependent on a previous value.

What does that mean in practice? Well, let's look at an example where you could choose
from both approaches.
Say that you have to download data from three websites and combine them. And say that
we have an action, say GetURL , that gets the data from a website on demand.
Now you have a choice:
Do you want to fetch all the URLs in parallel? If so, treat the GetURL s as
independent data and use the applicative style.
Do you want to fetch each URL one at a time, and skip the next in line if the
previous one fails? If so, treat the GetURL s as dependent data and use the monadic
style. This linear approach will be slower overall than the "applicative" version above,
but will also avoid unnecessary I/O.
Does the URL for the next site depend on what you download from the previous
site? In this case, you are forced to use "monadic" style, because each GetURL
depends on the output of the previous one.
As you can see, the choice between applicative style and monadic style is not clear cut; it
depends on what you want to do.
We'll look at a real implementation of this example in the final post of this series.
but...
It's important to say that just because you choose a style doesn't mean it will be
implemented as you expect. As we have seen, you can easily implement apply in terms of
bind , so even if you use <*> in your code, the implementation may be proceeding

monadically.
In the example above, the implementation does not have to run the downloads in parallel. It
could run them serially instead. By using applicative style, you're just saying that you don't
care about dependencies and so they could be downloaded in parallel.

Static vs. dynamic structure

If you use the applicative style, that means that you define all the actions up front -"statically", as it were.
In the downloading example, the applicative style requires that you specific in advance
which URLs will be visited. And because there is more knowledge up front it means that we
can potentially do things like parallelization or other optimizations.

1371

Using the core functions in practice

On the other hand, the monadic style means that only the initial action is known up front.
The remainder of the actions are determined dynamically, based on the output of previous
actions. This is more flexible, but also limits our ability to see the big picture in advance.

Order of evaluation vs. dependency

Sometimes dependency is confused with order of evaluation.
Certainly, if one value depends on another then the first value must be evaluated before the
second value. And in theory, if the values are completely independent (and have no side
effects), then they can be evaluated in any order.
However, even if the values are completely independent, there can still be an implicit order
in how they are evaluated.
For example, even if the list of GetURL s is done in parallel, it's likely that the urls will begin
to be fetched in the order in which they are listed, starting with the first one.
And in the List.apply implemented in the previous post, we saw that [f; g] apply [x; y]
resulted in [f x; f y; g x; g y] rather than [f x; g x; f y; g y] . That is, all the f
values are first, then all the g values.
In general, then, there is a convention that values are evaluated in a left to right order, even
if they are independent.

Example: Validation using applicative style and

monadic style
To see how both the applicative style and monadic style can be used, let's look at an
example using validation.
Say that we have a simple domain containing a CustomerId , an EmailAddress , and a
CustomerInfo which is a record containing both of these.

type CustomerId = CustomerId of int

type EmailAddress = EmailAddress of string
type CustomerInfo = {
id: CustomerId
email: EmailAddress
}

1372

Using the core functions in practice

And let's say that there is some validation around creating a CustomerId . For example, that
the inner int must be positive. And of course, there will be some validation around
creating a EmailAddress too. For example, that it must contain an "@" sign at least.
How would we do this?
First we create a type to represent the success/failure of validation.
type Result<'a> =
| Success of 'a
| Failure of string list

Note that I have defined the Failure case to contain a list of strings, not just one. This will
become important later.
With Result in hand, we can go ahead and define the two constructor/validation functions
as required:
let createCustomerId id =
if id > 0 then
Success (CustomerId id)
else
Failure ["CustomerId must be positive"]
// int -> Result<CustomerId>
let createEmailAddress str =
if System.String.IsNullOrEmpty(str) then
Failure ["Email must not be empty"]
elif str.Contains("@") then
Success (EmailAddress str)
else
Failure ["Email must contain @-sign"]
// string -> Result<EmailAddress>

Notice that createCustomerId has type int -> Result<CustomerId> , and

createEmailAddress has type string -> Result<EmailAddress> .

That means that both of these validation functions are world-crossing functions, going from
the normal world to the Result<_> world.

Defining the core functions for Result

Since we are dealing with world-crossing functions, we know that we will have to use
functions like apply and bind , so let's define them for our Result type.

1373

Using the core functions in practice

module Result =
let map f xResult =
match xResult with
| Success x ->
Success (f x)
| Failure errs ->
Failure errs
// Signature: ('a -> 'b) -> Result<'a> -> Result<'b>
// "return" is a keyword in F#, so abbreviate it
let retn x =
Success x
// Signature: 'a -> Result<'a>
let apply fResult xResult =
match fResult,xResult with
| Success f, Success x ->
Success (f x)
| Failure errs, Success x ->
Failure errs
| Success f, Failure errs ->
Failure errs
| Failure errs1, Failure errs2 ->
// concat both lists of errors
Failure (List.concat [errs1; errs2])
// Signature: Result<('a -> 'b)> -> Result<'a> -> Result<'b>
let bind f xResult =
match xResult with
| Success x ->
f x
| Failure errs ->
Failure errs
// Signature: ('a -> Result<'b>) -> Result<'a> -> Result<'b>

If we check the signatures, we can see that they are exactly as we want:
map has signature: ('a -> 'b) -> Result<'a> -> Result<'b>
retn has signature: 'a -> Result<'a>
apply has signature: Result<('a -> 'b)> -> Result<'a> -> Result<'b>
bind has signature: ('a -> Result<'b>) -> Result<'a> -> Result<'b>

I defined a retn function in the module to be consistent, but I don't bother to use it very
often. The concept of return is important, but in practice, I'll probably just use the Success
constructor directly. In languages with type classes, such as Haskell, return is used much
more.

1374

Using the core functions in practice

Also note that apply will concat the error messages from each side if both parameters are
failures. This allows us to collect all the failures without discarding any. This is the reason
why I made the Failure case have a list of strings, rather than a single string.
NOTE: I'm using string for the failure case to make the demonstration easier. In a more
sophisticated design I would list the possible failures explicitly. See my functional error
handling talk for more details.

Validation using applicative style

Now that we have have the domain and the toolset around Result , let's try using the
applicative style to create a CustomerInfo record.
The outputs of the validation are already elevated to Result , so we know we'll need to use
some sort of "lifting" approach to work with them.
First we'll create a function in the normal world that creates a CustomerInfo record given a
normal CustomerId and a normal EmailAddress :
let createCustomer customerId email =
{ id=customerId; email=email }
// CustomerId -> EmailAddress -> CustomerInfo

Note that the signature is CustomerId -> EmailAddress -> CustomerInfo .

Now we can use the lifting technique with <!> and <*> that was explained in the previous
post:
let (<!>) = Result.map
let (<*>) = Result.apply
// applicative version
let createCustomerResultA id email =
let idResult = createCustomerId id
let emailResult = createEmailAddress email
createCustomer <!> idResult <*> emailResult
// int -> string -> Result<CustomerInfo>

The signature of this shows that we start with a normal int and string and return a
Result<CustomerInfo>

1375

Using the core functions in practice

Let's try it out with some good and bad data:

let goodId = 1
let badId = 0
let goodEmail = "test@example.com"
let badEmail = "example.com"
let goodCustomerA =
createCustomerResultA goodId goodEmail
// Result<CustomerInfo> =
// Success {id = CustomerId 1; email = EmailAddress "test@example.com";}
let badCustomerA =
createCustomerResultA badId badEmail
// Result<CustomerInfo> =
// Failure ["CustomerId must be positive"; "Email must contain @-sign"]

The goodCustomerA is a Success and contains the right data, but the badCustomerA is a
Failure and contains two validation error messages. Excellent!

Validation using monadic style

Now let's do another implementation, but this time using monadic style. In this version the
logic will be:
try to convert an int into a CustomerId
if that is successful, try to convert a string into a EmailAddress
if that is successful, create a CustomerInfo from the customerId and email.
Here's the code:

1376

Using the core functions in practice

let (>>=) x f = Result.bind f x

// monadic version
let createCustomerResultM id email =
createCustomerId id >>= (fun customerId ->
createEmailAddress email >>= (fun emailAddress ->
let customer = createCustomer customerId emailAddress
Success customer
))
// int -> string -> Result<CustomerInfo>

The signature of the monadic-style createCustomerResultM is exactly the same as the

applicative-style createCustomerResultA but internally it is doing something different, which
will be reflected in the different results we get.

let goodCustomerM =
createCustomerResultM goodId goodEmail
// Result<CustomerInfo> =
// Success {id = CustomerId 1; email = EmailAddress "test@example.com";}
let badCustomerM =
createCustomerResultM badId badEmail
// Result<CustomerInfo> =
// Failure ["CustomerId must be positive"]

In the good customer case, the end result is the same, but in the bad customer case, only
one error is returned, the first one. The rest of the validation was short circuited after the
CustomerId creation failed.

Comparing the two styles

This example has demonstrated the difference between applicative and monadic style quite
well, I think.
The applicative example did all the validations up front, and then combined the results.
The benefit was that we didn't lose any of the validation errors. The downside was we
did work that we might not have needed to do.

1377

Using the core functions in practice

On the other hand, the monadic example did one validation at a time, chained together.
The benefit was that we short-circuited the rest of the chain as soon as an error
occurred and avoided extra work. The downside was that we only got the first error.

Mixing the two styles

Now there is nothing to say that we can't mix and match applicative and monadic styles.
For example, we might build a CustomerInfo using applicative style, so that we don't lose
any errors, but later on in the program, when a validation is followed by a database update,
we probably want to use monadic style, so that the database update is skipped if the
validation fails.

Using F# computation expressions

Finally, let's build a computation expression for these Result types.
To do this, we just define a class with members called Return and Bind , and then we
create an instance of that class, called result , say:
module Result =
type ResultBuilder() =
member this.Return x = retn x
member this.Bind(x,f) = bind f x
let result = new ResultBuilder()

We can then rewrite the createCustomerResultM function to look like this:

1378

Using the core functions in practice

let createCustomerResultCE id email = result {

let! customerId = createCustomerId id
let! emailAddress = createEmailAddress email
let customer = createCustomer customerId emailAddress
return customer }

This computation expression version looks almost like using an imperative language.
Note that F# computation expressions are always monadic, as is Haskell do-notation and
Scala for-comprehensions. That's not generally a problem, because if you need applicative
style it is very easy to write without any language support.

Lifting to a consistent world

In practice, we often have a mish-mash of different kinds of values and functions that we
need to combine together.
The trick for doing this is to convert all them to the same type, after which they can be
combined easily.

Making values consistent

Let's revisit the previous validation example, but let's change the record so that it has an
extra property, a name of type string:
type CustomerId = CustomerId of int
type EmailAddress = EmailAddress of string
type CustomerInfo = {
id: CustomerId
name: string // New!
email: EmailAddress
}

As before, we want to create a function in the normal world that we will later lift to the
Result world.

let createCustomer customerId name email =

{ id=customerId; name=name; email=email }
// CustomerId -> String -> EmailAddress -> CustomerInfo

Now we are ready to update the lifted createCustomer with the extra parameter:
1379

Using the core functions in practice

let (<!>) = Result.map

let (<*>) = Result.apply
let createCustomerResultA id name email =
let idResult = createCustomerId id
let emailResult = createEmailAddress email
createCustomer <!> idResult <*> name <*> emailResult
// ERROR ~~~~

But this won't compile! In the series of parameters idResult <*> name <*> emailResult one
of them is not like the others. The problem is that idResult and emailResult are both
Results, but name is still a string.
The fix is just to lift name into the world of results (say nameResult ) by using return , which
for Result is just Success . Here is the corrected version of the function that does work:
let createCustomerResultA id name email =
let idResult = createCustomerId id
let emailResult = createEmailAddress email
let nameResult = Success name // lift name to Result
createCustomer <!> idResult <*> nameResult <*> emailResult

Making functions consistent

The same trick can be used with functions too.
For example, let's say that we have a simple customer update workflow with four steps:
First, we validate the input. The output of this is the same kind of Result type we
created above. Note that this validation function could itself be the result of combining
other, smaller validation functions using apply .
Next, we canonicalize the data. For example: lowercasing emails, trimming whitespace,
etc. This step never raises an error.
Next, we fetch the existing record from the database. For example, getting a customer
for the CustomerId . This step could fail with an error too.
Finally, we update the database. This step is a "dead-end" function -- there is no output.
For error handling, I like to think of there being two tracks: a Success track and a Failure
track. In this model, an error-generating function is analogous to a railway switch (US) or
points (UK).

1380

Using the core functions in practice

The problem is that these functions cannot be glued together; they are all different shapes.
The solution is to convert all of them to the same shape, in this case the two-track model
with success and failure on different tracks. Let's call this Two-Track world!

Transforming functions using the toolset

Each original function, then, needs to be elevated to Two-Track world, and we know just the
tools that can do this!
The Canonicalize function is a single track function. We can turn it into a two-track function
using map .

The DbFetch function is a world-crossing function. We can turn it into a wholly two-track
function using bind .

The DbUpdate function is more complicated. We don't like dead-end functions, so first we
need to transform it to a function where the data keeps flowing. I'll call this function tee .
The output of tee has one track in and one track out, so we need to convert it to a twotrack function, again using map .

After all these transformations, we can reassemble the new versions of these functions. The
result looks like this:

And of course, these functions can now be composed together very easily, so that we end
up with a single function looking like this, with one input and a success/failure output:

1381

Using the core functions in practice

This combined function is yet another world-crossing function of the form a->Result , and
so it in turn can be used as a component part of a even bigger function.
For more examples of this "elevating everything to the same world" approach, see my posts
on functional error handling and threading state.

Kleisli world
There is an alternative world which can be used as a basic for consistency which I will call
"Kleisli" world, named after Professor Kleisli -- a mathematician, of course!
In Kleisli world everything is a cross-world function! Or, using the railway track analogy,
everything is a switch (or points).
In Kleisli world, the cross-world functions can be composed directly, using an operator called
>=> for left-to-right composition or <=< for right-to-left composition.

Using the same example as before, we can lift all our functions to Kleisli world.
The Validate and DbFetch functions are already in the right form so they don't need
to be changed.
The one-track Canonicalize function can be lifted to a switch just by lifting the output to
a two-track value. Let's call this toSwitch .

The tee-d DbUpdate function can be also lifted to a switch just by doing toSwitch after
the tee.

1382

Using the core functions in practice

Once all the functions have been lifted to Kleisli world, they can be composed with Kleisli
composition:

Kleisli world has some nice properties that Two-Track world doesn't but on the other hand, I
find it hard to get my head around it! So I generally stick to using Two-Track world as my
foundation for things like this.

Summary
In this post, we learned about "applicative" vs "monadic" style, and why the choice could
have an important effect on which actions are executed, and what results are returned.
We also saw how to lift different kinds values and functions to a a consistent world so that
the could be worked with easily.
In the next post we'll look at a common problem: working with lists of elevated values.

1383

Understanding traverse and sequence

This post is one in a series. In the first two posts, I described some of the core functions for
dealing with generic data types: map , bind , and so on. In the previous post, I discussed
"applicative" vs "monadic" style, and how to lift values and functions to be consistent with
each other.
In this post, we'll look at a common problem: working with lists of elevated values.

1384

Understanding traverse and sequence

Designing your own elevated world

Filtering out failures
The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 4: Mixing lists and elevated values

A common issue is how to deal with lists or other collections of elevated values.
Here are some examples:
Example 1: We have a parseInt with signature string -> int option , and we have a
list of strings. We want to parse all the strings at once. Now of course we can use map
to convert the list of strings to a list of options. But what we really want is not a "list of
options" but an "option of list", a list of parsed ints, wrapped in an option in case any fail.
Example 2: We have a readCustomerFromDb function with signature CustomerId ->
Result<Customer> , that will return Success if the record can be found and returned, and
Failure otherwise. And say we have a list of CustomerId s and we want to read all the

customers at once. Again, we can use map to convert the list of ids to a list of results.
But what we really want is not a list of Result<Customer> , but a Result containing a
Customer list , with the Failure case in case of errors.

Example 3: We have a fetchWebPage function with signature Uri -> Async<string> ,

that will return a task that will download the page contents on demand. And say we
have a list of Uris s and we want to fetch all the pages at once. Again, we can use
map to convert the list of Uri s to a list of Async s. But what we really want is not a list

of Async , but a Async containing a list of strings.

Mapping an Option generating function

Let's start by coming up with a solution for the first case and then seeing if we can
generalize it to the others.
The obvious approach would be:
First, use map to turn the list of string into a list of Option<int> .
Next, create a function that turns the list of Option<int> into an Option<int list> .
But this requires two passes through the list. Can we do it in one pass?

1385

Understanding traverse and sequence

Yes! If we think about how a list is built, there is a "cons" function ( :: in F#) that is used to
join the head to the tail. If we elevate this to the Option world, we can use Option.apply to
join a head Option to a tail Option using the lifted version of cons .
let (<*>) = Option.apply
let retn = Some
let rec mapOption f list =
let cons head tail = head :: tail
match list with
| [] ->
retn []
| head::tail ->
retn cons <*> (f head) <*> (mapOption f tail)

NOTE: I defined cons explicitly because :: is not a function and List.Cons takes a tuple
and is thus not usable in this context.
Here is the implementation as a diagram:

If you are confused as to how this works, please read the section on apply in the first post
in this series.
Note also that I am explicitly defining retn and using it in the implementation rather than
just using Some . You'll see why in the next section.
Now let's test it!
let parseInt str =
match (System.Int32.TryParse str) with
| true,i -> Some i
| false,_ -> None
// string -> int option
let good = ["1";"2";"3"] |> mapOption parseInt
// Some [1; 2; 3]
let bad = ["1";"x";"y"] |> mapOption parseInt
// None

1386

Understanding traverse and sequence

We start by defining parseInt of type string -> int option (piggybacking on the existing
.NET library).
We use mapOption to run it against a list of good values, and we get Some [1; 2; 3] , with
the list inside the option, just as we want.
And if we use a list where some of the values are bad, we get None for the entire result.

Mapping a Result generating function

Let's repeat this, but this time using the Result type from the earlier validation example.
Here's the mapResult function:
let (<*>) = Result.apply
let retn = Success
let rec mapResult f list =
let cons head tail = head :: tail
match list with
| [] ->
retn []
| head::tail ->
retn cons <*> (f head) <*> (mapResult f tail)

Again I am explicitly defining a retn rather than just using Success . And because of this,
the body of the code for mapResult and mapOption is exactly the same!
Now let's change parseInt to return a Result rather than an Option :
let parseInt str =
match (System.Int32.TryParse str) with
| true,i -> Success i
| false,_ -> Failure [str + " is not an int"]

And then we can rerun the tests again, but this time getting more informative errors in the
failure case:
let good = ["1";"2";"3"] |> mapResult parseInt
// Success [1; 2; 3]
let bad = ["1";"x";"y"] |> mapResult parseInt
// Failure ["x is not an int"; "y is not an int"]

Can we make a generic mapXXX function?

1387

Understanding traverse and sequence

The implementations of mapOption and mapResult have exactly the same code, the only
difference is the different retn and <*> functions (from Option and Result, respectively).
So the question naturally arises, rather than having mapResult , mapOption , and other
specific implementations for each elevated type, can we make a completely generic version
of mapXXX that works for all elevated types?
The obvious thing would be able to pass these two functions in as an extra parameter, like
this:
let rec mapE (retn,ap) f list =
let cons head tail = head :: tail
let (<*>) = ap
match list with
| [] ->
retn []
| head::tail ->
(retn cons) <*> (f head) <*> (mapE retn ap f tail)

There are some problems with this though. First, this code doesn't compile in F#! But even if
it did, we'd want to make sure that the same two parameters were passed around
everywhere.
We might attempt this by creating a record structure containing the two parameters, and
then create one instance for each type of elevated world:
type Applicative<'a,'b> = {
retn: 'a -> E<'a>
apply: E<'a->'b> -> E<'a> -> E<'b>
}
// functions for applying Option
let applOption = {retn = Option.Some; apply=Option.apply}
// functions for applying Result
let applResult = {retn = Result.Success; apply=Result.apply}

The instance of the Applicative record ( appl say) would be an extra parameter to our
generic mapE function, like this:

1388

Understanding traverse and sequence

let rec mapE appl f list =

let cons head tail = head :: tail
let (<*>) = appl.apply
let retn = appl.retn
match list with
| [] ->
retn []
| head::tail ->
(retn cons) <*> (f head) <*> (mapE retn ap f tail)

In use, we would pass in the specific applicative instance that we want, like this:
// build an Option specific version...
let mapOption = mapE applOption
// ...and use it
let good = ["1";"2";"3"] |> mapOption parseInt

Unfortunately, none of this works either, at least in F#. The Applicative type, as defined,
won't compile. This is because F# does not support "higher-kinded types". That is, we can't
parameterize the Applicative type with a generic type, only with concrete types.
In Haskell and languages that do support "higher-kinded types", the Applicative type that
we've defined is similar to a "type class". What's more, with type classes, we don't have to
pass around the functions explicitly -- the compiler will do that for us.
There is actually a clever (and hacky) way of getting the same effect in F# though using
static type constraints. I'm not going to discuss it here, but you can see it used in the
FSharpx library.
The alternative to all this abstraction is just creating a mapXXX function for each elevated
world that we want to work with: mapOption , mapResult , mapAsync and so on.
Personally I am OK with this cruder approach. There are not that many elevated worlds that
you work with regularly, and even though you lose out on abstraction, you gain on
explicitness, which is often useful when working in a team of mixed abilities.
So let's look at these mapXXX functions, also called traverse .

The traverse / mapM function

Common Names: mapM , traverse , for

1389

Understanding traverse and sequence

Common Operators: None

What it does: Transforms a world-crossing function into a world-crossing function that works
with collections
Signature: (a->E) -> a list -> E (or variants where list is replaced with other
collection types)

Description
We saw above that we can define a set of mapXXX functions, where XXX stands for an
applicative world -- a world that has apply and return . Each of these mapXXX functions
transforms a world-crossing function into a world-crossing function that works with
collections.

And as we noted above, if the language supports type classes, we can get away with a
single implementation, called mapM or traverse . I'm going to call the general concept
traverse from now on to make it clear that is different from map .

Map vs. Traverse

Understanding the difference between map and traverse can be hard, so let's see if we
can explain it in pictures.
First, let's introduce some visual notation using the analogy of an "elevated" world sitting
above a "normal" world.
Some of these elevated worlds (in fact almost all of them!) have apply and return
functions. We'll call these "Applicative worlds". Examples include Option , Result , Async ,
etc.
And some of these elevated worlds have a traverse function. We'll call these "Traversable
worlds", and we'll use List as a classic example.

1390

Understanding traverse and sequence

If a Traversable world is on top, that produces a type such as List<a> , and if an Applicative
world is on top, that produces a type such as Result<a> .

IMPORTANT: I will be using the syntax List<_> to represent "List world" for consistency
with Result<_> , etc. This is not meant to be the same as the .NET List class! In F#, this
would be implemented by the immutable list type.
But from now on we are going to be dealing with both kinds of elevated worlds in the same
"stack".
The Traversable world can be stacked on top of the Applicative world, which produces a
type such as List<Result<a>> , or alternatively, the Applicative world world can be stacked
on top of the Traversable world, which produces a type such as Result<List<a>> .

Now let's see what the different kinds of functions look like using this notation.
Let's start with a plain cross-world function such as a -> Result , where the target world
is an applicative world. In the diagram, the input is a normal world (on the left), and the
output (on the right) is an applicative world stacked on top of the normal world.

Now if we have a list of normal a values, and then we use map to transform each a
value using a function like a -> Result , the result will also be a list, but where the
contents are Result values instead of a values.

1391

Understanding traverse and sequence

When it comes to traverse the effect is quite different. If we use traverse to transform a
list of a values using that function, the output will be a Result , not a list. And the contents
of the Result will be a List .

In other words, with traverse , the List stays attached to the normal world, and the
Applicative world (such as Result ) is added at the top.
Ok, I know this all sounds very abstract, but it is actually a very useful technique. We'll see
an example of this is used in practice below.

Applicative vs. monadic versions of traverse

It turns out that traverse can be implemented in an applicative style or a monadic style, so
there are often two separate implementations to choose from. The applicative versions tend
to end in A and the monadic versions end in M , which is helpful!
Let's see how this works with our trusty Result type.
First, we'll implement traverseResult using both applicative and monadic approaches.

1392

Understanding traverse and sequence

module List =
/// Map a Result producing function over a list to get a new Result
/// using applicative style
/// ('a -> Result<'b>) -> 'a list -> Result<'b list>
let rec traverseResultA f list =
// define the applicative functions
let (<*>) = Result.apply
let retn = Result.Success
// define a "cons" function
let cons head tail = head :: tail
// loop through the list
match list with
| [] ->
// if empty, lift [] to a Result
retn []
| head::tail ->
// otherwise lift the head to a Result using f
// and cons it with the lifted version of the remaining list
retn cons <*> (f head) <*> (traverseResultA f tail)

/// Map a Result producing function over a list to get a new Result
/// using monadic style
/// ('a -> Result<'b>) -> 'a list -> Result<'b list>
let rec traverseResultM f list =
// define the monadic functions
let (>>=) x f = Result.bind f x
let retn = Result.Success
// define a "cons" function
let cons head tail = head :: tail
// loop through the list
match list with
| [] ->
// if empty, lift [] to a Result
retn []
| head::tail ->
// otherwise lift the head to a Result using f
// then lift the tail to a Result using traverse
// then cons the head and tail and return it
f head >>= (fun h ->
traverseResultM f tail >>= (fun t ->
retn (cons h t) ))

The applicative version is the same implementation that we used earlier.

1393

Understanding traverse and sequence

The monadic version applies the function f to the first element and then passes it to
bind . As always with monadic style, if the result is bad the rest of the list will be skipped.

On the other hand, if the result is good, the next element in the list is processed, and so on.
Then the results are cons'ed back together again.
NOTE: These implementations are for demonstration only! Neither of these implementations
are tail-recursive, and so they will fail on large lists!
Alright, let's test the two functions and see how they differ. First we need our parseInt
function:
/// parse an int and return a Result
/// string -> Result<int>
let parseInt str =
match (System.Int32.TryParse str) with
| true,i -> Result.Success i
| false,_ -> Result.Failure [str + " is not an int"]

Now if we pass in a list of good values (all parsable), the result for both implementations is
the same.
// pass in strings wrapped in a List
// (applicative version)
let goodA = ["1"; "2"; "3"] |> List.traverseResultA parseInt
// get back a Result containing a list of ints
// Success [1; 2; 3]
// pass in strings wrapped in a List
// (monadic version)
let goodM = ["1"; "2"; "3"] |> List.traverseResultM parseInt
// get back a Result containing a list of ints
// Success [1; 2; 3]

But if we pass in a list with some bad values, the results differ.
// pass in strings wrapped in a List
// (applicative version)
let badA = ["1"; "x"; "y"] |> List.traverseResultA parseInt
// get back a Result containing a list of ints
// Failure ["x is not an int"; "y is not an int"]
// pass in strings wrapped in a List
// (monadic version)
let badM = ["1"; "x"; "y"] |> List.traverseResultM parseInt
// get back a Result containing a list of ints
// Failure ["x is not an int"]

1394

Understanding traverse and sequence

The applicative version returns all the errors, while the monadic version returns only the first
error.

Implementing traverse using fold

I mentioned above that the "from-scratch" implementation was not tail recursive and would
fail for large lists. That could be fixed of course, at the price of making the code more
complicated.
On the other hand, if your collection type has a "right fold" function, as List does, then you
can use that to make the implementation simpler, faster, and safer too.
In fact, I always like to use fold and its ilk wherever possible so that I never have to worry
about getting tail-recursion right!
So, here are the re-implementations of traverseResult , using List.foldBack . I have kept
the code as similar as possible, but delegated the looping over the list to the fold function,
rather than creating a recursive function.

1395

Understanding traverse and sequence

/// Map a Result producing function over a list to get a new Result
/// using applicative style
/// ('a -> Result<'b>) -> 'a list -> Result<'b list>
let traverseResultA f list =
// define the applicative functions
let (<*>) = Result.apply
let retn = Result.Success
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
retn cons <*> (f head) <*> tail
List.foldBack folder list initState
/// Map a Result producing function over a list to get a new Result
/// using monadic style
/// ('a -> Result<'b>) -> 'a list -> Result<'b list>
let traverseResultM f list =
// define the monadic functions
let (>>=) x f = Result.bind f x
let retn = Result.Success
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
f head >>= (fun h ->
tail >>= (fun t ->
retn (cons h t) ))
List.foldBack folder list initState

Note that this approach will not work for all collection classes. Some types do not have a
right fold, so traverse must be implemented differently.

What about types other than lists?

All these examples have used the list type as the collection type. Can we implement
traverse for other types too?

1396

Understanding traverse and sequence

Yes. For example, an Option can be considered a one-element list, and we can use the
same trick.
For example, here's an implementation of traverseResultA for Option
module Option =
/// Map a Result producing function over an Option to get a new Result
/// ('a -> Result<'b>) -> 'a option -> Result<'b option>
let traverseResultA f opt =
// define the applicative functions
let (<*>) = Result.apply
let retn = Result.Success
// loop through the option
match opt with
| None ->
// if empty, lift None to an Result
retn None
| Some x ->
// lift value to an Result
(retn Some) <*> (f x)

Now we can wrap a string in an Option and use parseInt on it. Rather than getting a
Option of Result , we invert the stack and get a Result of Option .

// pass in an string wrapped in an Option

let good = Some "1" |> Option.traverseResultA parseInt
// get back a Result containing an Option
// Success (Some 1)

If we pass in an unparsable string, we get failure:

// pass in an string wrapped in an Option
let bad = Some "x" |> Option.traverseResultA parseInt
// get back a Result containing an Option
// Failure ["x is not an int"]

If we pass in None , we get Success containing None !

// pass in an string wrapped in an Option
let goodNone = None |> Option.traverseResultA parseInt
// get back a Result containing an Option
// Success (None)

1397

Understanding traverse and sequence

This last result might be surprising at first glance, but think of it this way, the parsing didn't
fail, so there was no Failure at all.

Traversables
Types that can implement a function like mapXXX or traverseXXX are called Traversable.
For example, collection types are Traversables as well as some others.
As we saw above, in a language with type classes a Traversable type can get away with just
one implementation of traverse , but in a language without type classes a Traversable type
will need one implementation per elevated type.
Also note that, unlike all the generic functions we have created before, the type being acted
on (inside the collection) must have appropriate apply and return functions in order for
traverse to be implemented. That is, the inner type must be an Applicative.

The properties of a correct traverse implementation

As always, a correct implementation of traverse should have some properties that are true
no matter what elevated world we are working with.
These are the "Traversable Laws", and a Traversable is defined as a generic data type
constructor -- E<T> -- plus a set of functions ( traverse or traverseXXX ) that obey these
laws.
The laws are similar to the previous ones. For example, the identity function should be
mapped correctly, composition should be preserved, etc.

The sequence function

Common Names: sequence
Common Operators: None
What it does: Transforms a list of elevated values into an elevated value containing a list
Signature: E<a> list -> E<a list> (or variants where list is replaced with other collection
types)

Description

1398

Understanding traverse and sequence

We saw above how you can use the traverse function as a substitute for map when you
have a function that generates an applicative type such as Result .
But what happens if you are just handed a List<Result> and you need to change it to a
Result<List> . That is, you need to swap the order of the worlds on the stack:

This is where sequence is useful -- that's exactly what it does! The sequence function
"swaps layers".
The order of swapping is fixed:
The Traversable world starts higher and is swapped down.
The Applicative world starts lower and is swapped up.
Note that if you aleady have an implementation of traverse , then sequence can be derived
from it easily. In fact, you can think of sequence as traverse with the id function baked
in.

Applicative vs Monadic versions of sequence

Just as with traverse , there can be applicative and monadic versions of sequence :
sequenceA for the applicative one.
sequenceM (or just sequence ) for the monadic one.

A simple example
Let's implement and test a sequence implementation for Result :

1399

Understanding traverse and sequence

module List =
/// Transform a "list<Result>" into a "Result<list>"
/// and collect the results using apply
/// Result<'a> list -> Result<'a list>
let sequenceResultA x = traverseResultA id x
/// Transform a "list<Result>" into a "Result<list>"
/// and collect the results using bind.
/// Result<'a> list -> Result<'a list>
let sequenceResultM x = traverseResultM id x

Ok, that was too easy! Now let's test it, starting with the applicative version:
let goodSequenceA =
["1"; "2"; "3"]
|> List.map parseInt
|> List.sequenceResultA
// Success [1; 2; 3]
let badSequenceA =
["1"; "x"; "y"]
|> List.map parseInt
|> List.sequenceResultA
// Failure ["x is not an int"; "y is not an int"]

and then the monadic version:

let goodSequenceM =
["1"; "2"; "3"]
|> List.map parseInt
|> List.sequenceResultM
// Success [1; 2; 3]
let badSequenceM =
["1"; "x"; "y"]
|> List.map parseInt
|> List.sequenceResultM
// Failure ["x is not an int"]

As before, we get back a Result<List> , and as before the monadic version stops on the
first error, while the applicative version accumulates all the errors.

1400

Understanding traverse and sequence

"Sequence" as a recipe for ad-hoc

implementations
We saw above that having type classes like Applicative means that you only need to
implement traverse and sequence once. In F# and other languages without high-kinded
types you have to create an implementation for each type that you want to traverse over.
Does that mean that the concepts of traverse and sequence are irrelevant or too abstract?
I don't think so.
Instead of thinking of them as library functions, I find that it is useful to think of them as
recipes -- a set of instructions that you can follow mechanically to solve a particular problem.
In many cases, the problem is unique to a context, and there is no need to create a library
function -- you can create a helper function as needed.
Let me demonstrate with an example. Say that you are given a list of options, where each
option contains a tuple, like this:
let tuples = [Some (1,2); Some (3,4); None; Some (7,8);]
// List<Option<Tuple<int>>>

This data is in the form List<Option<Tuple<int>>> . And now say, that for some reason, you
need to turn it into a tuple of two lists, where each list contains options, like this:
let desiredOutput = [Some 1; Some 3; None; Some 7],[Some 2; Some 4; None; Some 8]
// Tuple<List<Option<int>>>

The desired result is in the form Tuple<List<Option<int>>> .

So, how would you write a function to do this? Quick!
No doubt you could come up with one, but it might require a bit of thought and testing to be
sure you get it right.
On the other hand, if you recognize that this task is just transforming a stack of worlds to
another stack, you can create a function mechanically, almost without thinking.

1401

Understanding traverse and sequence

Designing the solution

To design the solution, we need to pay attention to which worlds move up and which worlds
move down.
The tuple world needs to end up at the top, so it will have to be swapped "up", which in
turn means that it will play the role of "applicative".
The option and list worlds need to be swapped "down", which in turn means that they
will both play the role of "traversable".
In order to do this transform then, I will need two helper functions:
optionSequenceTuple will move an option down and a tuple up.

listSequenceTuple will move a list down and a tuple up.

Do these helper functions need to be in a library? No. It's unlikely that I will need them again,
and even I need them occasionally, I'd prefer to write them scratch to avoid having to take a
dependency.
On the other hand, the List.sequenceResult function implemented earlier that converts a
List<Result<a>> to a Result<List<a>> is something I do use frequently, and so that one is

worth centralizing.

Implementing the solution

Once we know how the solution will look, we can start coding mechanically.
First, the tuple is playing the role of the applicative, so we need to define the apply and
return functions:

1402

Understanding traverse and sequence

let tupleReturn x = (x, x)

let tupleApply (f,g) (x,y) = (f x, g y)

Next, define listSequenceTuple using exactly the same right fold template as we did before,
with List as the traversable and tuple as the applicative:
let listSequenceTuple list =
// define the applicative functions
let (<*>) = tupleApply
let retn = tupleReturn
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail = retn cons <*> head <*> tail
List.foldBack folder list initState

There is no thinking going on here. I'm just following the template!

We can test it immediately:
[ (1,2); (3,4)] |> listSequenceTuple
// Result => ([1; 3], [2; 4])

And it gives a tuple with two lists, as expected.

Similarly, define optionSequenceTuple using the same right fold template again. This time
Option is the traversable and tuple is still the applicative:

let optionSequenceTuple opt =

// define the applicative functions
let (<*>) = tupleApply
let retn = tupleReturn
// right fold over the option
let initState = retn None
let folder x _ = (retn Some) <*> x
Option.foldBack folder opt initState

We can test it too:

1403

Understanding traverse and sequence

Some (1,2) |> optionSequenceTuple

// Result => (Some 1, Some 2)

And it gives a tuple with two options, as expected.

Finally, we can glue all the parts together. Again, no thinking required!
let convert input =
input
// from List<Option<Tuple<int>>> to List<Tuple<Option<int>>>
|> List.map optionSequenceTuple
// from List<Tuple<Option<int>>> to Tuple<List<Option<int>>>
|> listSequenceTuple

And if we use it, we get just what we wanted:

let output = convert tuples
// ( [Some 1; Some 3; None; Some 7], [Some 2; Some 4; None; Some 8] )
output = desiredOutput |> printfn "Is output correct? %b"
// Is output correct? true

Ok, this solution is more work than having one reusable function, but because it is
mechanical, it only takes a few minutes to code, and is still easier than trying to come up
with your own solution!
Want more? For an example of using sequence in a real-world problem, please read this
post.

Readability vs. performance

At the beginning of this post I noted our tendency as functional programmers to map first
and ask questions later.
In other words, given a Result producing function like parseInt , we would start by
collecting the results and only then figure out how to deal with them. Our code would look
something like this, then:

1404

Understanding traverse and sequence

["1"; "2"; "3"]

|> List.map parseInt
|> List.sequenceResultM

But of course, this does involve two passes over the list, and we saw how traverse could
combine the map and the sequence in one step, making only one pass over the list, like
this:
["1"; "2"; "3"]
|> List.traverseResultM parseInt

So if traverse is more compact and potentially faster, why ever use sequence ?
Well, sometimes you are given a certain structure and you have no choice, but in other
situations I might still prefer the two-step map-sequence approach just because it is easier to
understand. The mental model for "map" then "swap" seems easier to grasp for most people
than the one-step traverse.
In other words, I would always go for readability unless you can prove that performance is
impacted. Many people are still learning FP, and being overly cryptic is not helpful, in my
experience.

Dude, where's my filter ?

We've seen that functions like map and sequence work on lists to transform them into other
types, but what about filtering? How can I filter things using these methods?
The answer is -- you can't! map , and traverse and sequence are all "structure
preserving". If you start with a list of 10 things you still have a list of 10 things afterwards,
albeit somewhere else in the stack. Or if you start with a tree with three branches, you still
have a tree of three branches at the end.
In the tuple example above, the original list had four elements, and after the transformation,
the two new lists in the tuple also had four elements.
If you need to change the structure of a type, you need to use something like fold . Fold
allows you to build a new structure from an old one, which means that you can use it to
create a new list with some elements missing (i.e. a filter).
The various uses of fold are worthy of their own series, so I'm going to save a discussion of
filtering for another time.

1405

Understanding traverse and sequence

Summary
In this post, we learned about traverse and sequence as a way of working with lists of
elevated values.
In the next post we'll finish up by working through a practical example that uses all the
techniques that have been discussed.

1406

Using map, apply, bind and sequence in practice

Using map, apply, bind and sequence in

practice
This post is the fifth in a series. In the first two posts, I described some of the core functions
for dealing with generic data types: map , bind , and so on. In the third post, I discussed
"applicative" vs "monadic" style, and how to lift values and functions to be consistent with
each other. In the previous post, I introduced traverse and sequence as a way of working
with lists of elevated values.
In this post, we'll finish up by working through a practical example that uses all the
techniques that have been discussed so far.

1407

Using map, apply, bind and sequence in practice

Example: Downloading and processing a list of websites

Treating two worlds as one
Part 6: Designing your own elevated world
Designing your own elevated world
Filtering out failures
The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 5: A real-world example that uses all the

techniques
Example: Downloading and processing a list of
websites
The example will be a variant of the one mentioned at the beginning of the third post:
Given a list of websites, create an action that finds the site with the largest home page.
Let's break this down into steps:
First we'll need to transform the urls into a list of actions, where each action downloads the
page and gets the size of the content.
And then we need to find the largest content, but in order to do this we'll have to convert the
list of actions into a single action containing a list of sizes. And that's where traverse or
sequence will come in.

Let's get started!

The downloader
First we need to create a downloader. I would use the built-in System.Net.WebClient class,
but for some reason it doesn't allow override of the timeout. I'm going to want to have a
small timeout for the later tests on bad uris, so this is important.
One trick is to just subclass WebClient and intercept the method that builds a request. So
here it is:
1408

Using map, apply, bind and sequence in practice

// define a millisecond Unit of Measure

type [<Measure>] ms
/// Custom implementation of WebClient with settable timeout
type WebClientWithTimeout(timeout:int<ms>) =
inherit System.Net.WebClient()
override this.GetWebRequest(address) =
let result = base.GetWebRequest(address)
result.Timeout <- int timeout
result

Notice that I'm using units of measure for the timeout value. I find that units of measure are
invaluable to distiguish seconds from milliseconds. I once accidentally set a timeout to 2000
seconds rather than 2000 milliseconds and I don't want to make that mistake again!
The next bit of code defines our domain types. We want to be able to keep the url and the
size together as we process them. We could use a tuple, but I am a proponent of using
types to model your domain, if only for documentation.
// The content of a downloaded page
type UriContent =
UriContent of System.Uri * string
// The content size of a downloaded page
type UriContentSize =
UriContentSize of System.Uri * int

Yes, this might be overkill for a trivial example like this, but in a more serious project I think it
is very much worth doing.
Now for the code that does the downloading:

1409

Using map, apply, bind and sequence in practice

/// Get the contents of the page at the given Uri

/// Uri -> Async<Result<UriContent>>
let getUriContent (uri:System.Uri) =
async {
use client = new WebClientWithTimeout(1000<ms>) // 1 sec timeout
try
printfn " [%s] Started ..." uri.Host
let! html = client.AsyncDownloadString(uri)
printfn " [%s] ... finished" uri.Host
let uriContent = UriContent (uri, html)
return (Result.Success uriContent)
with
| ex ->
printfn " [%s] ... exception" uri.Host
let err = sprintf "[%s] %A" uri.Host ex.Message
return Result.Failure [err ]
}

Notes:
The .NET library will throw on various errors, so I am catching that and turning it into a
Failure .

The use client = section ensures that the client will be correctly disposed at the end
of the block.
The whole operation is wrapped in an async workflow, and the let! html =
client.AsyncDownloadString is where the download happens asynchronously.

I've added some printfn s for tracing, just for this example. In real code, I wouldn't do
this of course!
Before moving on, let's test this code interactively. First we need a helper to print the result:
let showContentResult result =
match result with
| Success (UriContent (uri, html)) ->
printfn "SUCCESS: [%s] First 100 chars: %s" uri.Host (html.Substring(0,100))
| Failure errs ->
printfn "FAILURE: %A" errs

And then we can try it out on a good site:

1410

Using map, apply, bind and sequence in practice

System.Uri ("http://google.com")
|> getUriContent
|> Async.RunSynchronously
|> showContentResult
// [google.com] Started ...
// [google.com] ... finished
// SUCCESS: [google.com] First 100 chars: <!doctype html><html itemscope="" itemtype="
http://schema.org/WebPage" lang="en-GB"><head><meta cont

and a bad one:

System.Uri ("http://example.bad")
|> getUriContent
|> Async.RunSynchronously
|> showContentResult
// [example.bad] Started ...
// [example.bad] ... exception
// FAILURE: ["[example.bad] "The remote name could not be resolved: 'example.bad'""]

Extending the Async type with map and apply and

bind
At this point, we know that we are going to be dealing with the world of Async , so before we
go any further, let's make sure that we have our four core functions available:

1411

Using map, apply, bind and sequence in practice

module Async =
let map f xAsync = async {
// get the contents of xAsync
let! x = xAsync
// apply the function and lift the result
return f x
}
let retn x = async {
// lift x to an Async
return x
}
let apply fAsync xAsync = async {
// start the two asyncs in parallel
let! fChild = Async.StartChild fAsync
let! xChild = Async.StartChild xAsync
// wait for the results
let! f = fChild
let! x = xChild
// apply the function to the results
return f x
}
let bind f xAsync = async {
// get the contents of xAsync
let! x = xAsync
// apply the function but don't lift the result
// as f will return an Async
return! f x
}

These implementations are straightforward:

I'm using the async workflow to work with Async values.
The let! syntax in map extracts the content from the Async (meaning run it and
await the result).
The return syntax in map , retn , and apply lifts the value to an Async using
return .

The apply function runs the two parameters in parallel using a fork/join pattern. If I had
instead written let! fChild = ... followed by a let! xChild = ... that would have
been monadic and sequential, which is not what I wanted.
The return! syntax in bind means that the value is already lifted and not to call
return on it.

1412

Using map, apply, bind and sequence in practice

Getting the size of the downloaded page

Getting back on track, we can continue from the downloading step and move on to the
process of converting the result to a UriContentSize :
/// Make a UriContentSize from a UriContent
/// UriContent -> Result<UriContentSize>
let makeContentSize (UriContent (uri, html)) =
if System.String.IsNullOrEmpty(html) then
Result.Failure ["empty page"]
else
let uriContentSize = UriContentSize (uri, html.Length)
Result.Success uriContentSize

If the input html is null or empty we'll treat this an error, otherwise we'll return a
UriContentSize .

Now we have two functions and we want to combine them into one "get UriContentSize
given a Uri" function. The problem is that the outputs and inputs don't match:
getUriContent is Uri -> Async<Result<UriContent>>
makeContentSize is UriContent -> Result<UriContentSize>

The answer is to transform makeContentSize from a function that takes a UriContent as

input into a function that takes a Async<Result<UriContent>> as input. How can we do that?
First, use Result.bind to convert it from an a -> Result function to a Result<a> ->
Result function. In this case, UriContent -> Result<UriContentSize> becomes
Result<UriContent> -> Result<UriContentSize> .

Next, use Async.map to convert it from an a -> b function to a Async<a> -> Async
function. In this case, Result<UriContent> -> Result<UriContentSize> becomes
Async<Result<UriContent>> -> Async<Result<UriContentSize>> .

1413

Using map, apply, bind and sequence in practice

And now that it has the right kind of input, so we can compose it with getUriContent :
/// Get the size of the contents of the page at the given Uri
/// Uri -> Async<Result<UriContentSize>>
let getUriContentSize uri =
getUriContent uri
|> Async.map (Result.bind makeContentSize)

That's some gnarly type signature, and it's only going to get worse! It's at times like these
that I really appreciate type inference.
Let's test again. First a helper to format the result:
let showContentSizeResult result =
match result with
| Success (UriContentSize (uri, len)) ->
printfn "SUCCESS: [%s] Content size is %i" uri.Host len
| Failure errs ->
printfn "FAILURE: %A" errs

And then we can try it out on a good site:

1414

Using map, apply, bind and sequence in practice

System.Uri ("http://google.com")
|> getUriContentSize
|> Async.RunSynchronously
|> showContentSizeResult
// [google.com] Started ...
// [google.com] ... finished
//SUCCESS: [google.com] Content size is 44293

and a bad one:

System.Uri ("http://example.bad")
|> getUriContentSize
|> Async.RunSynchronously
|> showContentSizeResult
// [example.bad] Started ...
// [example.bad] ... exception
//FAILURE: ["[example.bad] "The remote name could not be resolved: 'example.bad'""]

Getting the largest size from a list

The last step in the process is to find the largest page size.
That's easy. Once we have a list of UriContentSize , we can easily find the largest one using
List.maxBy :

/// Get the largest UriContentSize from a list

/// UriContentSize list -> UriContentSize
let maxContentSize list =
// extract the len field from a UriContentSize
let contentSize (UriContentSize (_, len)) = len
// use maxBy to find the largest
list |> List.maxBy contentSize

Putting it all together

We're ready to assemble all the pieces now, using the following algorithm:
Start with a list of urls
Turn the list of strings into a list of uris ( Uri list )
Turn the list of Uri s into a list of actions ( Async<Result<UriContentSize>> list )
Next we need to swap the top two parts of the stack. That is, transform a List<Async>

1415

Using map, apply, bind and sequence in practice

into a Async<List> .

Next we need to swap the bottom two parts of the stack -- transform a List<Result>
into a Result<List> . But the two bottom parts of the stack are wrapped in an Async so
we need to use Async.map to do this.

Finally we need to use List.maxBy on the bottom List to convert it into a single
value. That is, transform a List<UriContentSize> into a UriContentSize . But the
bottom of the stack is wrapped in a Result wrapped in an Async so we need to use
Async.map and Result.map to do this.

Here's the complete code:

1416

Using map, apply, bind and sequence in practice

/// Get the largest page size from a list of websites

let largestPageSizeA urls =
urls
// turn the list of strings into a list of Uris
// (In F# v4, we can call System.Uri directly!)
|> List.map (fun s -> System.Uri(s))
// turn the list of Uris into a "Async<Result<UriContentSize>> list"
|> List.map getUriContentSize
// turn the "Async<Result<UriContentSize>> list"
// into an "Async<Result<UriContentSize> list>"
|> List.sequenceAsyncA
// turn the "Async<Result<UriContentSize> list>"
// into a "Async<Result<UriContentSize list>>"
|> Async.map List.sequenceResultA
// find the largest in the inner list to get
// a "Async<Result<UriContentSize>>"
|> Async.map (Result.map maxContentSize)

This function has signature string list -> Async<Result<UriContentSize>> , which is just
what we wanted!
There are two sequence functions involved here: sequenceAsyncA and sequenceResultA .
The implementations are as you would expect from all the previous discussion, but I'll show
the code anyway:

1417

Using map, apply, bind and sequence in practice

module List =
/// Map a Async producing function over a list to get a new Async
/// using applicative style
/// ('a -> Async<'b>) -> 'a list -> Async<'b list>
let rec traverseAsyncA f list =
// define the applicative functions
let (<*>) = Async.apply
let retn = Async.retn
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
retn cons <*> (f head) <*> tail
List.foldBack folder list initState
/// Transform a "list<Async>" into a "Async<list>"
/// and collect the results using apply.
let sequenceAsyncA x = traverseAsyncA id x
/// Map a Result producing function over a list to get a new Result
/// using applicative style
/// ('a -> Result<'b>) -> 'a list -> Result<'b list>
let rec traverseResultA f list =
// define the applicative functions
let (<*>) = Result.apply
let retn = Result.Success
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
retn cons <*> (f head) <*> tail
List.foldBack folder list initState
/// Transform a "list<Result>" into a "Result<list>"
/// and collect the results using apply.
let sequenceResultA x = traverseResultA id x

Adding a timer

1418

Using map, apply, bind and sequence in practice

It will be interesting to see how long the download takes for different scenarios, so let's
create a little timer that runs a function a certain number of times and takes the average:
/// Do countN repetitions of the function f and print the time per run
let time countN label f =
let stopwatch = System.Diagnostics.Stopwatch()
// do a full GC at the start but not thereafter
// allow garbage to collect for each iteration
System.GC.Collect()
printfn "======================="
printfn "%s" label
printfn "======================="
let mutable totalMs = 0L
for iteration in [1..countN] do
stopwatch.Restart()
f()
stopwatch.Stop()
printfn "#%2i elapsed:%6ims " iteration stopwatch.ElapsedMilliseconds
totalMs <- totalMs + stopwatch.ElapsedMilliseconds
let avgTimePerRun = totalMs / int64 countN
printfn "%s: Average time per run:%6ims " label avgTimePerRun

Ready to download at last

Let's download some sites for real!
We'll define two lists of sites: a "good" one, where all the sites should be accessible, and a
"bad" one, containing invalid sites.
let goodSites = [
"http://google.com"
"http://bbc.co.uk"
"http://fsharp.org"
"http://microsoft.com"
]
let badSites = [
"http://example.com/nopage"
"http://bad.example.com"
"http://verybad.example.com"
"http://veryverybad.example.com"
]

1419

Using map, apply, bind and sequence in practice

Let's start by running largestPageSizeA 10 times with the good sites list:
let f() =
largestPageSizeA goodSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeA_Good" f

The output is something like this:

[google.com] Started ...
[bbc.co.uk] Started ...
[fsharp.org] Started ...
[microsoft.com] Started ...
[bbc.co.uk] ... finished
[fsharp.org] ... finished
[google.com] ... finished
[microsoft.com] ... finished
SUCCESS: [bbc.co.uk] Content size is 108983
largestPageSizeA_Good: Average time per run: 533ms

We can see immediately that the downloads are happening in parallel -- they have all started
before the first one has finished.
Now what about if some of the sites are bad?
let f() =
largestPageSizeA badSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeA_Bad" f

The output is something like this:

1420

Using map, apply, bind and sequence in practice

[example.com] Started ...

[bad.example.com] Started ...
[verybad.example.com] Started ...
[veryverybad.example.com] Started ...
[verybad.example.com] ... exception
[veryverybad.example.com] ... exception
[example.com] ... exception
[bad.example.com] ... exception
FAILURE: [
"[example.com] "The remote server returned an error: (404) Not Found."";
"[bad.example.com] "The remote name could not be resolved: 'bad.example.com'"";
"[verybad.example.com] "The remote name could not be resolved: 'verybad.example.com'"
";
"[veryverybad.example.com] "The remote name could not be resolved: 'veryverybad.examp
le.com'""]
largestPageSizeA_Bad: Average time per run: 2252ms

Again, all the downloads are happening in parallel, and all four failures are returned.

Optimizations
The largestPageSizeA has a series of maps and sequences in it which means that the list is
being iterated over three times and the async mapped over twice.
As I said earlier, I prefer clarity over micro-optimizations unless there is proof otherwise, and
so this does not bother me.
However, let's look at what you could do if you wanted to.
Here's the original version, with comments removed:
let largestPageSizeA urls =
urls
|> List.map (fun s -> System.Uri(s))
|> List.map getUriContentSize
|> List.sequenceAsyncA
|> Async.map List.sequenceResultA
|> Async.map (Result.map maxContentSize)

The first two List.map s could be combined:

1421

Using map, apply, bind and sequence in practice

let largestPageSizeA urls =

The map-sequence can be replaced with a traverse :

let largestPageSizeA urls =
urls
|> List.traverseAsyncA (fun s -> System.Uri(s) |> getUriContentSize)
|> Async.map List.sequenceResultA
|> Async.map (Result.map maxContentSize)

and finally the two Async.map s can be combined too:

let largestPageSizeA urls =
urls
|> List.traverseAsyncA (fun s -> System.Uri(s) |> getUriContentSize)
|> Async.map (List.sequenceResultA >> Result.map maxContentSize)

Personally, I think we've gone too far here. I prefer the original version to this one!
As an aside, one way to get the best of both worlds is to use a "streams" library that
automatically merges the maps for you. In F#, a good one is Nessos Streams. Here is a blog
post showing the difference between streams and the standard seq .

Downloading the monadic way

Let's reimplement the downloading logic using monadic style and see what difference it
makes.
First we need a monadic version of the downloader:
let largestPageSizeM urls =
urls
|> List.map (fun s -> System.Uri(s))
|> List.map getUriContentSize
|> List.sequenceAsyncM // <= "M" version
|> Async.map List.sequenceResultM // <= "M" version
|> Async.map (Result.map maxContentSize)

This one uses the monadic sequence functions (I won't show them -- the implementation is
as you expect).
1422

Using map, apply, bind and sequence in practice

Let's run largestPageSizeM 10 times with the good sites list and see if there is any
difference from the applicative version:
let f() =
largestPageSizeM goodSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeM_Good" f

The output is something like this:

[google.com] Started ...
[google.com] ... finished
[bbc.co.uk] Started ...
[bbc.co.uk] ... finished
[fsharp.org] Started ...
[fsharp.org] ... finished
[microsoft.com] Started ...
[microsoft.com] ... finished
SUCCESS: [bbc.co.uk] Content size is 108695
largestPageSizeM_Good: Average time per run: 955ms

There is a big difference now -- it is obvious that the downloads are happening in series -each one starts only when the previous one has finished.
As a result, the average time is 955ms per run, almost twice that of the applicative version.
Now what about if some of the sites are bad? What should we expect? Well, because it's
monadic, we should expect that after the first error, the remaining sites are skipped, right?
Let's see if that happens!
let f() =
largestPageSizeM badSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeM_Bad" f

The output is something like this:

1423

Using map, apply, bind and sequence in practice

[example.com] Started ...

[example.com] ... exception
[bad.example.com] Started ...
[bad.example.com] ... exception
[verybad.example.com] Started ...
[verybad.example.com] ... exception
[veryverybad.example.com] Started ...
[veryverybad.example.com] ... exception
FAILURE: ["[example.com] "The remote server returned an error: (404) Not Found.""]
largestPageSizeM_Bad: Average time per run: 2371ms

Well that was unexpected! All of the sites were visited in series, even though the first one
had an error. But in that case, why is only the first error returned, rather than all the the
errors?
Can you see what went wrong?

Explaining the problem

The reason why the implementation did not work as expected is that the chaining of the
Async s was independent of the chaining of the Result s.

If you step through this in a debugger you can see what is happening:
The first Async in the list was run, resulting in a failure.
Async.bind was used with the next Async in the list. But Async.bind has no concept

of error, so the next Async was run, producing another failure.

In this way, all the Async s were run, producing a list of failures.
This list of failures was then traversed using Result.bind . Of course, because of the
bind, only the first one was processed and the rest ignored.
The final result was that all the Async s were run but only the first failure was returned.

Treating two worlds as one

The fundamental problem is that we are treating the Async list and Result list as separate
things to be traversed over. But that means that a failed Result has no influence on
whether the next Async is run.
What we want to do, then, is tie them together so that a bad result does determine whether
the next Async is run.

1424

Using map, apply, bind and sequence in practice

And in order to do that, we need to treat the Async and the Result as a single type -- let's
imaginatively call it AsyncResult .
If they are a single type, then bind looks like this:

meaning that the previous value will determine the next value.
And also, the "swapping" becomes much simpler:

Defining the AsyncResult type

OK, let's define the AsyncResult type and it's associated map , return , apply and bind
functions.

1425

Using map, apply, bind and sequence in practice

/// type alias (optional)

type AsyncResult<'a> = Async<Result<'a>>
/// functions for AsyncResult
module AsyncResult =
module AsyncResult =
let map f =
f |> Result.map |> Async.map
let retn x =
x |> Result.retn |> Async.retn
let apply fAsyncResult xAsyncResult =
fAsyncResult |> Async.bind (fun fResult ->
xAsyncResult |> Async.map (fun xResult ->
Result.apply fResult xResult))
let bind f xAsyncResult = async {
let! xResult = xAsyncResult
match xResult with
| Success x -> return! f x
| Failure err -> return (Failure err)
}

Notes:
The type alias is optional. We can use Async<Result<'a>> directly in the code and it wil
work fine. The point is that conceptually AsyncResult is a separate type.
The bind implementation is new. The continuation function f is now crossing two
worlds, and has the signature 'a -> Async<Result<'b>> .
If the inner Result is successful, the continuation function f is evaluated with the
result. The return! syntax means that the return value is already lifted.
If the inner Result is a failure, we have to lift the failure to an Async.

Defining the traverse and sequence functions

With bind and return in place, we can create the appropriate traverse and sequence
functions for AsyncResult :

1426

Using map, apply, bind and sequence in practice

module List =
/// Map an AsyncResult producing function over a list to get a new AsyncResult
/// using monadic style
/// ('a -> AsyncResult<'b>) -> 'a list -> AsyncResult<'b list>
let rec traverseAsyncResultM f list =
// define the monadic functions
let (>>=) x f = AsyncResult.bind f x
let retn = AsyncResult.retn
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
f head >>= (fun h ->
tail >>= (fun t ->
retn (cons h t) ))
List.foldBack folder list initState
/// Transform a "list<AsyncResult>" into a "AsyncResult<list>"
/// and collect the results using bind.
let sequenceAsyncResultM x = traverseAsyncResultM id x

Defining and testing the downloading functions

Finally, the largestPageSize function is simpler now, with only one sequence needed.
let largestPageSizeM_AR urls =
urls
|> List.map (fun s -> System.Uri(s) |> getUriContentSize)
|> List.sequenceAsyncResultM
|> AsyncResult.map maxContentSize

Let's run largestPageSizeM_AR 10 times with the good sites list and see if there is any
difference from the applicative version:
let f() =
largestPageSizeM_AR goodSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeM_AR_Good" f

The output is something like this:

1427

Using map, apply, bind and sequence in practice

[google.com] Started ...

[google.com] ... finished
[bbc.co.uk] Started ...
[bbc.co.uk] ... finished
[fsharp.org] Started ...
[fsharp.org] ... finished
[microsoft.com] Started ...
[microsoft.com] ... finished
SUCCESS: [bbc.co.uk] Content size is 108510
largestPageSizeM_AR_Good: Average time per run: 1026ms

Again, the downloads are happening in series. And again, the time per run is almost twice
that of the applicative version.
And now the moment we've been waiting for! Will it skip the downloading after the first bad
site?
let f() =
largestPageSizeM_AR badSites
|> Async.RunSynchronously
|> showContentSizeResult
time 10 "largestPageSizeM_AR_Bad" f

The output is something like this:

[example.com] Started ...
[example.com] ... exception
FAILURE: ["[example.com] "The remote server returned an error: (404) Not Found.""]
largestPageSizeM_AR_Bad: Average time per run: 117ms

Success! The error from the first bad site prevented the rest of the downloads, and the short
run time is proof of that.

Summary
In this post, we worked through a small practical example. I hope that this example
demonstrated that map , apply , bind , traverse , and sequence are not just academic
abstractions but essential tools in your toolbelt.
In the next post we'll working through another practical example, but this time we will end up
creating our own elevated world. See you then!

1428

Using map, apply, bind and sequence in practice

1429

Reinventing the Reader monad

This post is the sixth in a series. In the first two posts, I described some of the core functions
for dealing with generic data types: map , bind , and so on. In the third post, I discussed
"applicative" vs "monadic" style, and how to lift values and functions to be consistent with
each other. In the fourth and previous posts, I introduced traverse and sequence as a way
of working with lists of elevated values, and we saw this used in a practical example:
downloading some URLs.
In this post, we'll finish up by working through another practical example, but this time we'll
create our own "elevated world" as a way to deal with awkward code. We'll see that this
approach is so common that it has a name -- the "Reader monad".

1430

Reinventing the Reader monad

Part 5: A real-world example that uses all the techniques

Example: Downloading and processing a list of websites
Treating two worlds as one
Part 6: Designing your own elevated world
Designing your own elevated world
Filtering out failures
The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Part 6: Designing your own elevated world

The scenario we'll be working with in this post is just this:
A customer comes to your site and wants to view information about the products they have
purchased.
In this example, we'll assume that you have a API for a key/value store (such as Redis or a
NoSql database), and all the information you need is stored there.
So the code we need will look something like this:
Open API connection
Get product ids purchased by customer id using the API
For each product id:
get the product info for that id using the API
Close API connection
Return the list of product infos

How hard can that be?

Well, it turns out to be surprisingly tricky! Luckily, we can find a way to make it easier using
the concepts in this series.

Defining the domain and a dummy ApiClient

First let's define the domain types:
There will be a CustomerId and ProductId of course.

1431

Reinventing the Reader monad

For the product information, we'll just define a simple ProductInfo with a ProductName
field.
Here are the types:
type CustId = CustId of string
type ProductId = ProductId of string
type ProductInfo = {ProductName: string; }

For testing our api, let's create an ApiClient class with some Get and Set methods,
backed by a static mutable dictionary. This is based on similar APIs such as the Redis client.
Notes:
The Get and Set both work with objects, so I've added a casting mechanism.
In case of errors such as a failed cast, or a missing key, I'm using the Result type that
we've been using throughout this series. Therefore, both Get and Set return
Result s rather than plain objects.

To make it more realistic, I've also added dummy methods for Open , Close and
Dispose .

All methods trace a log to the console.

1432

Reinventing the Reader monad

type ApiClient() =
// static storage
static let mutable data = Map.empty<string,obj>
/// Try casting a value
/// Return Success of the value or Failure on failure
member private this.TryCast<'a> key (value:obj) =
match value with
| :? 'a as a ->
Result.Success a
| _ ->
let typeName = typeof<'a>.Name
Result.Failure [sprintf "Can't cast value at %s to %s" key typeName]
/// Get a value
member this.Get<'a> (id:obj) =
let key = sprintf "%A" id
printfn "[API] Get %s" key
match Map.tryFind key data with
| Some o ->
this.TryCast<'a> key o
| None ->
Result.Failure [sprintf "Key %s not found" key]
/// Set a value
member this.Set (id:obj) (value:obj) =
let key = sprintf "%A" id
printfn "[API] Set %s" key
if key = "bad" then // for testing failure paths
Result.Failure [sprintf "Bad Key %s " key]
else
data <- Map.add key value data
Result.Success ()
member this.Open() =
printfn "[API] Opening"
member this.Close() =
printfn "[API] Closing"
interface System.IDisposable with
member this.Dispose() =
printfn "[API] Disposing"

Let's do some tests:

1433

Reinventing the Reader monad

And the results are:

[API] Get "K1"
[K1] Failure ["Key "K1" not found"]
[API] Set "K2"
[API] Get "K2"
[K2] Success "hello"
[API] Set "K3"
[API] Get "K3"
[K3] Failure ["Can't cast value at "K3" to Int32"]
[API] Disposing

A first implementation attempt

For our first attempt at implementing the scenario, let's start with the pseudo-code from
above:
let getPurchaseInfo (custId:CustId) : Result<ProductInfo list> =
// Open api connection
use api = new ApiClient()
api.Open()
// Get product ids purchased by customer id
let productIdsResult = api.Get<ProductId list> custId
let productInfosResult = ??
// Close api connection
api.Close()
// Return the list of product infos
productInfosResult

So far so good, but there is a bit of a problem already.

1434

Reinventing the Reader monad

The getPurchaseInfo function takes a CustId as input, but it can't just output a list of
ProductInfo s, because there might be a failure. That means that the return type needs to

be Result<ProductInfo list> .
Ok, how do we create our productInfosResult ?
Well that should be easy. If the productIdsResult is Success, then loop through each id and
get the info for each id. If the productIdsResult is Failure, then just return that failure.
let getPurchaseInfo (custId:CustId) : Result<ProductInfo list> =
// Open api connection
use api = new ApiClient()
api.Open()
// Get product ids purchased by customer id
let productIdsResult = api.Get<ProductId list> custId
let productInfosResult =
match productIdsResult with
| Success productIds ->
let productInfos = ResizeArray() // Same as .NET List<T>
for productId in productIds do
let productInfo = api.Get<ProductInfo> productId
productInfos.Add productInfo // mutation!
Success productInfos
| Failure err ->
Failure err
// Close api connection
api.Close()
// Return the list of product infos
productInfosResult

Hmmm. It's looking a bit ugly. And I'm having to use a mutable data structure
( productInfos ) to accumulate each product info and then wrap it in Success .
And there's a worse problem The productInfo that I'm getting from api.Get<ProductInfo>
is not a ProductInfo at all, but a Result<ProductInfo> , so productInfos is not the right
type at all!
Let's add code to test each ProductInfo result. If it's a success, then add it to the list of
product infos, and if it's a failure, then return the failure.

1435

Reinventing the Reader monad

let getPurchaseInfo (custId:CustId) : Result<ProductInfo list> =

// Open api connection
use api = new ApiClient()
api.Open()
// Get product ids purchased by customer id
let productIdsResult = api.Get<ProductId list> custId
let productInfosResult =
match productIdsResult with
| Success productIds ->
let productInfos = ResizeArray() // Same as .NET List<T>
let mutable anyFailures = false
for productId in productIds do
let productInfoResult = api.Get<ProductInfo> productId
match productInfoResult with
| Success productInfo ->
productInfos.Add productInfo
| Failure err ->
Failure err
Success productInfos
| Failure err ->
Failure err
// Close api connection
api.Close()
// Return the list of product infos
productInfosResult

Um, no. That won't work at all. The code above will not compile. We can't do an "early
return" in the loop when a failure happens.
So what do we have so far? Some really ugly code that won't even compile.
There has to be a better way.

A second implementation attempt

It would be great if we could hide all this unwrapping and testing of Result s. And there is -computation expressions to the rescue.
If we create a computation expression for Result we can write the code like this:

1436

Reinventing the Reader monad

/// CustId -> Result<ProductInfo list>

let getPurchaseInfo (custId:CustId) : Result<ProductInfo list> =
// Open api connection
use api = new ApiClient()
api.Open()
let productInfosResult = Result.result {
// Get product ids purchased by customer id
let! productIds = api.Get<ProductId list> custId
let productInfos = ResizeArray() // Same as .NET List<T>
for productId in productIds do
let! productInfo = api.Get<ProductInfo> productId
productInfos.Add productInfo
return productInfos |> List.ofSeq
}
// Close api connection
api.Close()
// Return the list of product infos
productInfosResult

In let productInfosResult = Result.result { .. } code we create a result computation

expression that simplifies all the unwrapping (with let! ) and wrapping (with return ).
And so this implementation has no explicit xxxResult values anywhere. However, it still has
to use a mutable collection class to do the accumulation, because the for productId in
productIds do is not actually a real for loop, and we can't replace it with List.map , say.

The result computation expression.

Which brings us onto the implementation of the result computation expression. In the
previous posts, ResultBuilder only had two methods, Return and Bind , but in order to
get the for..in..do functionality, we have to implement a lot of other methods too, and it
ends up being a bit more complicated.

1437

Reinventing the Reader monad

module Result =
let bind f xResult = ...
type ResultBuilder() =
member this.Return x = retn x
member this.ReturnFrom(m: Result<'T>) = m
member this.Bind(x,f) = bind f x
member this.Zero() = Failure []
member this.Combine (x,f) = bind f x
member this.Delay(f: unit -> _) = f
member this.Run(f) = f()
member this.TryFinally(m, compensation) =
try this.ReturnFrom(m)
finally compensation()
member this.Using(res:#System.IDisposable, body) =
this.TryFinally(body res, fun () ->
match res with
| null -> ()
| disp -> disp.Dispose())
member this.While(guard, f) =
if not (guard()) then
this.Zero()
else
this.Bind(f(), fun _ -> this.While(guard, f))
member this.For(sequence:seq<_>, body) =
this.Using(sequence.GetEnumerator(), fun enum ->
this.While(enum.MoveNext, this.Delay(fun () ->
body enum.Current)))
let result = new ResultBuilder()

I have a series about the internals of computation expressions, so I don't want to explain all
that code here. Instead, for the rest of the post we'll work on refactoring getPurchaseInfo ,
and by the end of it we'll see that we don't need the result computation expression at all.

Refactoring the function

The problem with the getPurchaseInfo function as it stands is that it mixes concerns: it both
creates the ApiClient and does some work with it.

1438

Reinventing the Reader monad

There a number of problems with this approach:

If we want to do different work with the API, we have to repeat the open/close part of
this code. And it's possible that one of the implementations might open the API but
forget to close it.
It's not testable with a mock API client.
We can solve both of these problems by separating the creation of an ApiClient from its
use by parameterizing the action, like this.
let executeApiAction apiAction =
// Open api connection
use api = new ApiClient()
api.Open()
// do something with it
let result = apiAction api
// Close api connection
api.Close()
// return result
result

The action function that is passed in would look like this, with a parameter for the ApiClient
as well as for the CustId :
/// CustId -> ApiClient -> Result<ProductInfo list>
let getPurchaseInfo (custId:CustId) (api:ApiClient) =
let productInfosResult = Result.result {
let! productIds = api.Get<ProductId list> custId
let productInfos = ResizeArray() // Same as .NET List<T>
for productId in productIds do
let! productInfo = api.Get<ProductInfo> productId
productInfos.Add productInfo
return productInfos |> List.ofSeq
}
// return result
productInfosResult

Note that getPurchaseInfo has two parameters, but executeApiAction expects a function
with only one.
No problem! Just use partial application to bake in the first parameter:

1439

Reinventing the Reader monad

let action = getPurchaseInfo (CustId "C1") // partially apply

executeApiAction action

That's why the ApiClient is the second parameter in the parameter list -- so that we can do
partial application.

More refactoring
We might need to get the product ids for some other purpose, and also the productInfo, so
let's refactor those out into separate functions too:
/// CustId -> ApiClient -> Result<ProductId list>
let getPurchaseIds (custId:CustId) (api:ApiClient) =
api.Get<ProductId list> custId
/// ProductId -> ApiClient -> Result<ProductInfo>
let getProductInfo (productId:ProductId) (api:ApiClient) =
api.Get<ProductInfo> productId
/// CustId -> ApiClient -> Result<ProductInfo list>
let getPurchaseInfo (custId:CustId) (api:ApiClient) =
let result = Result.result {
let! productIds = getPurchaseIds custId api
let productInfos = ResizeArray()
for productId in productIds do
let! productInfo = getProductInfo productId api
productInfos.Add productInfo
return productInfos |> List.ofSeq
}
// return result
result

Now, we have these nice core functions getPurchaseIds and getProductInfo , but I'm
annoyed that I have to write messy code to glue them together in getPurchaseInfo .
Ideally, what I'd like to do is pipe the output of getPurchaseIds into getProductInfo like this:
let getPurchaseInfo (custId:CustId) =
custId
|> getPurchaseIds
|> List.map getProductInfo

Or as a diagram:

1440

Reinventing the Reader monad

But I can't, and there are two reasons why:

First, getProductInfo has two parameters. Not just a ProductId but also the
ApiClient .

Second, even if ApiClient wasn't there, the input of getProductInfo is a simple

ProductId but the output of getPurchaseIds is a Result .

Wouldn't it be great if we could solve both of these problems!

Introducing our own elevated world

Let's address the first problem. How can we compose functions when the extra ApiClient
parameter keeps getting in the way?
This is what a typical API calling function looks like:

If we look at the type signature we see this, a function with two parameters:

But another way to interpret this function is as a function with one parameter that returns
another function. The returned function has an ApiClient parameter and returns the final
ouput.

You might think of it like this: I have an input right now, but I won't have an actual ApiClient
until later, so let me use the input to create a api-consuming function that can I glue together
in various ways right now, without needing a ApiClient at all.
Let's give this api-consuming function a name. Let's call it ApiAction .

1441

Reinventing the Reader monad

In fact, let's do more than that -- let's make it a type!

type ApiAction<'a> = (ApiClient -> 'a)

Unfortunately, as it stands, this is just a type alias for a function, not a separate type. We
need to wrap it in a single case union to make it a distinct type.
type ApiAction<'a> = ApiAction of (ApiClient -> 'a)

Rewriting to use ApiAction

Now that we have a real type to use, we can rewrite our core domain functions to use it.
First getPurchaseIds :
// CustId -> ApiAction<Result<ProductId list>>
let getPurchaseIds (custId:CustId) =
// create the api-consuming function
let action (api:ApiClient) =
api.Get<ProductId list> custId
// wrap it in the single case
ApiAction action

The signature is now CustId -> ApiAction<Result<ProductId list>> , which you can interpret
as meaning: "give me a CustId and I will give a you a ApiAction that, when given an api, will
make a list of ProductIds".
Similarly, getProductInfo can be rewritten to return an ApiAction :

1442

Reinventing the Reader monad

// ProductId -> ApiAction<Result<ProductInfo>>

let getProductInfo (productId:ProductId) =
// create the api-consuming function
let action (api:ApiClient) =
api.Get<ProductInfo> productId
// wrap it in the single case
ApiAction action

Notice those signatures:

CustId -> ApiAction<Result<ProductId list>>
ProductId -> ApiAction<Result<ProductInfo>>

This is starting to look awfully familiar. Didn't we see something just like this in the previous
post, with Async<Result<_>> ?

ApiAction as an elevated world

If we draw diagrams of the various types involved in these two functions, we can clearly see
that ApiAction is an elevated world, just like List and Result . And that means that we
should be able to use the same techniques as we have used before: map , bind ,
traverse , etc.

Here's getPurchaseIds as a stack diagram. The input is a CustId and the output is an
ApiAction<Result<List<ProductId>>> :

and with getProductInfo the input is a ProductId and the output is an

ApiAction<Result<ProductInfo>> :

1443

Reinventing the Reader monad

The combined function that we want, getPurchaseInfo , should look like this:

And now the problem in composing the two functions is very clear: the output of
getPurchaseIds can not be used as the input for getProductInfo :

But I think that you can see that we have some hope! There should be some way of
manipulating these layers so that they do match up, and then we can compose them easily.
So that's what we will work on next.

Introducting ApiActionResult
In the last post we merged Async and Result into the compound type AsyncResult . We
can do the same here, and create the type ApiActionResult .
When we make this change, our two functions become slightly simpler:

Enough diagrams -- let's write some code now.

First, we need to define map , apply , return and bind for ApiAction :

1444

Reinventing the Reader monad

module ApiAction =
/// Evaluate the action with a given api
/// ApiClient -> ApiAction<'a> -> 'a
let run api (ApiAction action) =
let resultOfAction = action api
resultOfAction
/// ('a -> 'b) -> ApiAction<'a> -> ApiAction<'b>
let map f action =
let newAction api =
let x = run api action
f x
ApiAction newAction
/// 'a -> ApiAction<'a>
let retn x =
let newAction api =
x
ApiAction newAction
/// ApiAction<('a -> 'b)> -> ApiAction<'a> -> ApiAction<'b>
let apply fAction xAction =
let newAction api =
let f = run api fAction
let x = run api xAction
f x
ApiAction newAction
/// ('a -> ApiAction<'b>) -> ApiAction<'a> -> ApiAction<'b>
let bind f xAction =
let newAction api =
let x = run api xAction
run api (f x)
ApiAction newAction
/// Create an ApiClient and run the action on it
/// ApiAction<'a> -> 'a
let execute action =
use api = new ApiClient()
api.Open()
let result = run api action
api.Close()
result

Note that all the functions use a helper function called run which unwraps an ApiAction to
get the function inside, and applies this to the api that is also passed in. The result is the
value wrapped in the ApiAction .
For example, if we had an ApiAction<int> then run api myAction would result in an int .

1445

Reinventing the Reader monad

And at the bottom, there is a execute function that creates an ApiClient , opens the
connection, runs the action, and then closes the connection.
And with the core functions for ApiAction defined, we can go ahead and define the
functions for the compound type ApiActionResult , just as we did for AsyncResult in the
previous post:
module ApiActionResult =
let map f =
ApiAction.map (Result.map f)
let retn x =
ApiAction.retn (Result.retn x)
let apply fActionResult xActionResult =
let newAction api =
let fResult = ApiAction.run api fActionResult
let xResult = ApiAction.run api xActionResult
Result.apply fResult xResult
ApiAction newAction
let bind f xActionResult =
let newAction api =
let xResult = ApiAction.run api xActionResult
// create a new action based on what xResult is
let yAction =
match xResult with
| Success x ->
// Success? Run the function
f x
| Failure err ->
// Failure? wrap the error in an ApiAction
(Failure err) |> ApiAction.retn
ApiAction.run api yAction
ApiAction newAction

Working out the transforms

Now that we have all the tools in place, we must decide on what transforms to use to change
the shape of getProductInfo so that the input matches up.
Should we choose map , or bind , or traverse ?
Let's play around with the stacks visually and see what happens for each kind of transform.
Before we get started, let's be explicit about what we are trying to achieve:

1446

Reinventing the Reader monad

We have two functions getPurchaseIds and getProductInfo that we want to combine

into a single function getPurchaseInfo .
We have to manipulate the left side (the input) of getProductInfo so that it matches the
output of getPurchaseIds .
We have to manipulate the right side (the output) of getProductInfo so that it matches
the output of our ideal getPurchaseInfo .

Map
As a reminder, map adds a new stack on both sides. So if we start with a generic worldcrossing function like this:

Then, after List.map say, we will have a new List stack on each site.

Here's our getProductInfo before transformation:

And here is what it would look like after using List.map

1447

Reinventing the Reader monad

This might seem promising -- we have a List of ProductId as input now, and if we can
stack a ApiActionResult on top we would match the output of getPurchaseId .
But the output is all wrong. We want the ApiActionResult to stay on the top. That is, we
don't want a List of ApiActionResult but a ApiActionResult of List .

Bind
Ok, what about bind ?
If you recall, bind turns a "diagonal" function into a horizontal function by adding a new
stack on the left sides. So for example, whatever the top elevated world is on the right, that
will be added to the left.

And here is what our getProductInfo would look like after using ApiActionResult.bind

This is no good to us. We need to have a List of ProductId as input.

Traverse
Finally, let's try traverse .

1448

Reinventing the Reader monad

traverse turns a diagonal function of values into diagonal function with lists wrapping the

values. That is, List is added as the top stack on the left hand side, and the second-fromtop stack on the right hand side.

if we try that out on getProductInfo we get something very promising.

The input is a list as needed. And the output is perfect. We wanted a

ApiAction<Result<List<ProductInfo>>> and we now have it.

So all we need to do now is add an ApiActionResult to the left side.

Well, we just saw this! It's bind . So if we do that as well, we are finished.

And here it is expressed as code:

let getPurchaseInfo =
let getProductInfo1 = traverse getProductInfo
let getProductInfo2 = ApiActionResult.bind getProductInfo1
getPurchaseIds >> getProductInfo2

Or to make it a bit less ugly:

1449

Reinventing the Reader monad

let getPurchaseInfo =
let getProductInfoLifted =
getProductInfo
|> traverse
|> ApiActionResult.bind
getPurchaseIds >> getProductInfoLifted

Let's compare that with the earlier version of getPurchaseInfo :

let getPurchaseInfo (custId:CustId) (api:ApiClient) =
let result = Result.result {
let! productIds = getPurchaseIds custId api
let productInfos = ResizeArray()
for productId in productIds do
let! productInfo = getProductInfo productId api
productInfos.Add productInfo
return productInfos |> List.ofSeq
}
// return result
result

Let's compare the two versions in a table:

Earlier verson

Latest function

Composite function is non-trivial and needs special

code to glue the two smaller functions together

Composite function is just

piping and composition

Uses the "result" computation expression

No special syntax needed

Has special code to loop through the results

Uses "traverse"

Uses a intermediate (and mutable) List object to

accumulate the list of product infos

No intermediate values
needed. Just a data pipeline.

Implementing traverse
The code above uses traverse , but we haven't implemented it yet. As I noted earlier, it can
be implemented mechanically, following a template.
Here it is:

1450

Reinventing the Reader monad

let traverse f list =

// define the applicative functions
let (<*>) = ApiActionResult.apply
let retn = ApiActionResult.retn
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
retn cons <*> f head <*> tail
List.foldBack folder list initState

Testing the implementation

Let's test it!
First we need a helper function to show results:
let showResult result =
match result with
| Success (productInfoList) ->
printfn "SUCCESS: %A" productInfoList
| Failure errs ->
printfn "FAILURE: %A" errs

Next, we need to load the API with some test data:

let setupTestData (api:ApiClient) =
//setup purchases
api.Set (CustId "C1") [ProductId "P1"; ProductId "P2"] |> ignore
api.Set (CustId "C2") [ProductId "PX"; ProductId "P2"] |> ignore
//setup product info
api.Set (ProductId "P1") {ProductName="P1-Name"} |> ignore
api.Set (ProductId "P2") {ProductName="P2-Name"} |> ignore
// P3 missing
// setupTestData is an api-consuming function
// so it can be put in an ApiAction
// and then that apiAction can be executed
let setupAction = ApiAction setupTestData
ApiAction.execute setupAction

Customer C1 has purchased two products: P1 and P2.

1451

Reinventing the Reader monad

Customer C2 has purchased two products: PX and P2.

Products P1 and P2 have some info.
Product PX does not have any info.
Let's see how this works out for different customer ids.
We'll start with Customer C1. For this customer we expect both product infos to be returned:
CustId "C1"
|> getPurchaseInfo
|> ApiAction.execute
|> showResult

And here are the results:

[API] Opening
[API] Get CustId "C1"
[API] Get ProductId "P1"
[API] Get ProductId "P2"
[API] Closing
[API] Disposing
SUCCESS: [{ProductName = "P1-Name";}; {ProductName = "P2-Name";}]

What happens if we use a missing customer, such as CX?

CustId "CX"
|> getPurchaseInfo
|> ApiAction.execute
|> showResult

As expected, we get a nice "key not found" failure, and the rest of the operations are skipped
as soon as the key is not found.
[API] Opening
[API] Get CustId "CX"
[API] Closing
[API] Disposing
FAILURE: ["Key CustId "CX" not found"]

What about if one of the purchased products has no info? For example, customer C2
purchased PX and P2, but there is no info for PX.

1452

Reinventing the Reader monad

CustId "C2"
|> getPurchaseInfo
|> ApiAction.execute
|> showResult

The overall result is a failure. Any bad product causes the whole operation to fail.
[API] Opening
[API] Get CustId "C2"
[API] Get ProductId "PX"
[API] Get ProductId "P2"
[API] Closing
[API] Disposing
FAILURE: ["Key ProductId "PX" not found"]

But note that the data for product P2 is fetched even though product PX failed. Why?
Because we are using the applicative version of traverse , so every element of the list is
fetched "in parallel".
If we wanted to only fetch P2 once we knew that PX existed, then we should be using
monadic style instead. We already seen how to write a monadic version of traverse , so I
leave that as an exercise for you!

Filtering out failures

In the implementation above, the getPurchaseInfo function failed if any product failed to be
found. Harsh!
A real application would probably be more forgiving. Probably what should happen is that the
failed products are logged, but all the successes are accumulated and returned.
How could we do this?
The answer is simple -- we just need to modify the traverse function to skip failures.
First, we need to create a new helper function for ApiActionResult . It will allow us to pass in
two functions, one for the success case and one for the error case:

1453

Reinventing the Reader monad

module ApiActionResult =
let map = ...
let retn = ...
let apply = ...
let bind = ...
let either onSuccess onFailure xActionResult =
let newAction api =
let xResult = ApiAction.run api xActionResult
let yAction =
match xResult with
| Result.Success x -> onSuccess x
| Result.Failure err -> onFailure err
ApiAction.run api yAction
ApiAction newAction

This helper function helps us match both cases inside a ApiAction without doing
complicated unwrapping. We will need this for our traverse that skips failures.
By the way, note that ApiActionResult.bind can be defined in terms of either :
let bind f =
either
// Success? Run the function
(fun x -> f x)
// Failure? wrap the error in an ApiAction
(fun err -> (Failure err) |> ApiAction.retn)

Now we can define our "traverse with logging of failures" function:

let traverseWithLog log f list =
// define the applicative functions
let (<*>) = ApiActionResult.apply
let retn = ApiActionResult.retn
// define a "cons" function
let cons head tail = head :: tail
// right fold over the list
let initState = retn []
let folder head tail =
(f head)
|> ApiActionResult.either
(fun h -> retn cons <*> retn h <*> tail)
(fun errs -> log errs; tail)
List.foldBack folder list initState

1454

Reinventing the Reader monad

The only difference between this and the previous implementation is this bit:
let folder head tail =
(f head)
|> ApiActionResult.either
(fun h -> retn cons <*> retn h <*> tail)
(fun errs -> log errs; tail)

This says that:

If the new first element ( f head ) is a success, lift the inner value ( retn h ) and cons it
with the tail to build a new list.
But if the new first element is a failure, then log the inner errors ( errs ) with the passed
in logging function ( log ) and just reuse the current tail. In this way, failed elements are
not added to the list, but neither do they cause the whole function to fail.
Let's create a new function getPurchasesInfoWithLog and try it with customer C2 and the
missing product PX:
let getPurchasesInfoWithLog =
let log errs = printfn "SKIPPED %A" errs
let getProductInfoLifted =
getProductInfo
|> traverseWithLog log
|> ApiActionResult.bind
getPurchaseIds >> getProductInfoLifted
CustId "C2"
|> getPurchasesInfoWithLog
|> ApiAction.execute
|> showResult

The result is a Success now, but only one ProductInfo , for P2, is returned. The log shows
that PX was skipped.
[API] Opening
[API] Get CustId "C2"
[API] Get ProductId "PX"
SKIPPED ["Key ProductId "PX" not found"]
[API] Get ProductId "P2"
[API] Closing
[API] Disposing
SUCCESS: [{ProductName = "P2-Name";}]

1455

Reinventing the Reader monad

The Reader monad

If you look closely at the ApiResult module, you will see that map , bind , and all the other
functions do not use any information about the api that is passed around. We could have
made it any type and those functions would still have worked.
So in the spirit of "parameterize all the things", why not make it a parameter?
That means that we could have defined ApiAction as follows:
type ApiAction<'anything,'a> = ApiAction of ('anything -> 'a)

But if it can be anything, why call it ApiAction any more? It could represent any set of
things that depend on an object (such as an api ) being passed in to them.
We are not the first people to discover this! This type is commonly called the Reader type
and is defined like this:
type Reader<'environment,'a> = Reader of ('environment -> 'a)

The extra type 'environment plays the same role that ApiClient did in our definition of
ApiAction . There is some environment that is passed around as an extra parameter to all

your functions, just as a api instance was.

In fact, we can actually define ApiAction in terms of Reader very easily:
type ApiAction<'a> = Reader<ApiClient,'a>

The set of functions for Reader are exactly the same as for ApiAction . I have just taken
the code and replaced ApiAction with Reader and api with environment !

1456

Reinventing the Reader monad

module Reader =
/// Evaluate the action with a given environment
/// 'env -> Reader<'env,'a> -> 'a
let run environment (Reader action) =
let resultOfAction = action environment
resultOfAction
/// ('a -> 'b) -> Reader<'env,'a> -> Reader<'env,'b>
let map f action =
let newAction environment =
let x = run environment action
f x
Reader newAction
/// 'a -> Reader<'env,'a>
let retn x =
let newAction environment =
x
Reader newAction
/// Reader<'env,('a -> 'b)> -> Reader<'env,'a> -> Reader<'env,'b>
let apply fAction xAction =
let newAction environment =
let f = run environment fAction
let x = run environment xAction
f x
Reader newAction
/// ('a -> Reader<'env,'b>) -> Reader<'env,'a> -> Reader<'env,'b>
let bind f xAction =
let newAction environment =
let x = run environment xAction
run environment (f x)
Reader newAction

The type signatures are a bit harder to read now though!

The Reader type, plus bind and return , plus the fact that bind and return implement
the monad laws, means that Reader is typically called "the Reader monad" .
I'm not going to delve into the Reader monad here, but I hope that you can see how it is
actually a useful thing and not some bizarre ivory tower concept.

The Reader monad vs. an explicit type

Now if you like, you could replace all the ApiAction code above with Reader code, and it
would work just the same. But should you?

1457

Reinventing the Reader monad

Personally, I think that while understanding the concept behind the Reader monad is
important and useful, I prefer the actual implementation of ApiAction as I defined it
originally, an explicit type rather than an alias for Reader<ApiClient,'a> .
Why? Well, F# doesn't have typeclasses, F# doesn't have partial application of type
constructors, F# doesn't have "newtype". Basically, F# isn't Haskell! I don't think that idioms
that work well in Haskell should be carried over to F# directly when the language does not
offer support for it.
If you understand the concepts, you can implement all the necessary transformations in a
few lines of code. Yes, it's a little extra work, but the upside is less abstraction and fewer
dependencies.
I would make an exception, perhaps, if your team were all Haskell experts, and the Reader
monad was familiar to everyone. But for teams of different abilities, I would err on being too
concrete rather than too abstract.

Summary
In this post, we worked through another practical example, created our own elevated world
which made things much easier, and in the process, accidentally re-invented the reader
monad.
If you liked this, you can see a similar practical example, this time for the State monad, in my
series on "Dr Frankenfunctor and the Monadster".
The next and final post has a quick summary of the series, and some further reading.

1458

Map and Bind and Apply, a summary

Series summary
Well, this series turned out to be longer than I originally planned. Thanks for making it to the
end!
I hope that this discussion has been helpful in understanding the various function
transformations like map and bind , and given you some useful techniques for dealing with
world-crossing functions -- maybe even demystified the m-word a bit!
If you want to start using these kinds of functions in your own code, I hope that you can see
how easy they are to write, but you should also consider using one of the excellent F# utility
libraries that contain these and much more:
ExtCore (source, NuGet). ExtCore provides extensions to the F# core library
(FSharp.Core) and aims to help you build industrial-strength F# applications. These
extensions include additional functions for modules such as Array, List, Set, and Map;
immutable IntSet, IntMap, LazyList, and Queue collections; a variety of computation
expressions (workflows); and "workflow collections" -- collections modules which have
been adapted to work seamlessly from within workflows.
FSharpx.Extras (home page). FSharpx.Extras is part of the FSharpx series of libraries.
It implements several standard monads (State, Reader, Writer, Either, Continuation,
Distribution), validation with applicative functors, general functions like flip, and some
asynchronous programming utilities, and functions to make C# - F# interop easier.
For example, the monadic traverse List.traverseResultM that I implemented in this post is
already available in ExtCore here.
And if you liked this series, I have posts explaining the State monad in my series on "Dr
Frankenfunctor and the Monadster" and the Either monad in my talk "Railway Oriented
Programming".
As I said at the very beginning, writing this up has been a learning process for me too. I am
not an expert, so if I have made any errors please do let me know.
Thanks!

Series contents
1459

Map and Bind and Apply, a summary

Here's a list of shortcuts to the various functions mentioned in this series:

Part 1: Lifting to the elevated world
The map function
The return function
The apply function
The liftN family of functions
The zip function and ZipList world
Part 2: How to compose world-crossing functions
The bind function
List is not a monad. Option is not a monad.
Part 3: Using the core functions in practice
Independent and dependent data
Example: Validation using applicative style and monadic style
Lifting to a consistent world
Kleisli world
Part 4: Mixing lists and elevated values
Mixing lists and elevated values
The traverse / MapM function
The sequence function
"Sequence" as a recipe for ad-hoc implementations
Readability vs. performance
Dude, where's my filter ?
Part 5: A real-world example that uses all the techniques
Example: Downloading and processing a list of websites
Treating two worlds as one
Part 6: Designing your own elevated world
Designing your own elevated world
Filtering out failures
The Reader monad
Part 7: Summary
List of operators mentioned
Further reading

Appendix: List of operators mentioned

Unlike OO languages, functional programming languages are known for their strange
operators, so I thought it would be helpful to document the ones that have been used in this
series, with links back to the relevant discussion.

1460

Map and Bind and Apply, a summary

Equivalent
function

Operator

Discussion

Left-to-right
composition

Not part of this series, but discussed here

Right-to-left
composition

As above

Left-to-right
piping

As above

Right-to-left
piping

As above

<!>

map

Discussed here

<$>

map

Haskell operator for map, but not a valid operator in

F#, so I'm using <!> in this series.

<*>

apply

Discussed here

One sided combiner. Discussed here

>>=

Left-to-right

Discussed here

bind
=<<

Right-to-left
bind

As above

>=>

Left-to-right
Kleisli
composition

Discussed here

<=<

Right-to-left
Kleisli
composition

As above

Appendix: Further reading

Alternative tutorials:
You Could Have Invented Monads! (And Maybe You Already Have).
Functors, Applicatives and Monads in pictures.
Kleisli composition ? la Up-Goer Five. I think this one is funny.
Eric Lippert's series on monads in C#.
For the academically minded:
Monads for Functional Programming (PDF), by Philip Wadler. One of the first monad

1461

Map and Bind and Apply, a summary

papers.
Applicative Programming with Effects (PDF), by Conor McBride and Ross Paterson.
The Essence of the Iterator Pattern (PDF), by Jeremy Gibbons and Bruno Oliveira.
F# examples:
F# ExtCore and FSharpx.Extras have lots of useful code.
FSharpx.Async has map , apply , liftN (called "Parallel"), bind , and other useful
extensions for Async .
Applicatives are very well suited for parsing, as explained in these posts:
Parsing with applicative functors in F#.
Dive into parser combinators: parsing search queries with F# and FParsec in Kiln.

1462

The "Recursive types and folds" Series

In this series, we'll look at recursive types and how to use them, and on the way, we'll look at
catamorphisms, tail recursion, the difference between left and right folds, and more.
Introduction to recursive types. Don't fear the catamorphism....
Catamorphism examples. Applying the rules to other domains.
Introducing Folds. Threading state through a recursive data structure.
Understanding Folds. Recursion vs. iteration.
Generic recursive types. Implementing a domain in three ways.
Trees in the real world. Examples using databases, JSON and error handling.

1463

Introduction to recursive types

In this series, we'll look at recursive types and how to use them, and on the way, we'll look at
catamorphisms, tail recursion, the difference between left and right folds, and more.

Series contents
Here's the contents of this series:
Part 1: Introduction to recursive types and catamorphisms
A simple recursive type
Parameterize all the things
Introducing catamorphisms
Benefits of catamorphisms
Rules for creating a catamorphism
Part 2: Catamorphism examples
Catamorphism example: File system domain
Catamorphism example: Product domain
Part 3: Introducing folds
A flaw in our catamorphism implementation
Introducing fold
Problems with fold
Using functions as accumulators
Introducing foldback
Rules for creating a fold
Part 4: Understanding folds
Iteration vs. recursion
Fold example: File system domain
Common questions about "fold"
Part 5: Generic recursive types
LinkedList: A generic recursive type
Making the Gift domain generic
Defining a generic Container type
A third way to implement the gift domain
Abstract or concrete? Comparing the three designs
Part 6: Trees in the real world
Defining a generic Tree type
The Tree type in the real world

1464

Introduction to recursive types

Mapping the Tree type

Example: Creating a directory listing
Example: A parallel grep
Example: Storing the file system in a database
Example: Serializing a Tree to JSON
Example: Deserializing a Tree from JSON
Example: Deserializing a Tree from JSON - with error handling

A basic recursive type

Let's start with a simple example -- how to model a gift.
As it happens, I'm a very lazy gift-giver. I always give people a book or chocolates. I
generally wrap them, and sometimes, if I'm feeling particularly extravagant, I put them in a
gift box and add a card too.
Let's see how I might model this in types:
type Book = {title: string; price: decimal}
type ChocolateType = Dark | Milk | SeventyPercent
type Chocolate = {chocType: ChocolateType ; price: decimal}
type WrappingPaperStyle =
| HappyBirthday
| HappyHolidays
| SolidColor
type Gift =
| Book of Book
| Chocolate of Chocolate
| Wrapped of Gift * WrappingPaperStyle
| Boxed of Gift
| WithACard of Gift * message:string

You can see that three of the cases are "containers" that refer to another Gift . The
Wrapped case has some paper and a inner gift, as does the Boxed case, as does the
WithACard case. The two other cases, Book and Chocolate , do not refer to a gift and can

be considered "leaf" nodes or terminals.

The presence of a reference to an inner Gift in those three cases makes Gift a
recursive type. Note that, unlike functions, the rec keyword is not needed for defining
recursive types.

1465

Introduction to recursive types

Let's create some example values:

// a Book
let wolfHall = {title="Wolf Hall"; price=20m}
// a Chocolate
let yummyChoc = {chocType=SeventyPercent; price=5m}
// A Gift
let birthdayPresent = WithACard (Wrapped (Book wolfHall, HappyBirthday), "Happy Birthd
ay")
// WithACard (
// Wrapped (
// Book {title = "Wolf Hall"; price = 20M},
// HappyBirthday),
// "Happy Birthday")
// A Gift
let christmasPresent = Wrapped (Boxed (Chocolate yummyChoc), HappyHolidays)
// Wrapped (
// Boxed (
// Chocolate {chocType = SeventyPercent; price = 5M}),
// HappyHolidays)

Before we start working with these values, a word of advice...

Guideline: Avoid infinitely recursive types

I suggest that, in F#, every recursive type should consist of a mix of recursive and nonrecursive cases. If there were no non-recursive elements, such as Book , all values of the
type would have to be infinitely recursive.
For example, in the ImpossibleGift type below, all the cases are recursive. To construct
any one of the cases you need an inner gift, and that needs to be constructed too, and so
on.
type ImpossibleGift =
| Boxed of ImpossibleGift
| WithACard of ImpossibleGift * message:string

It is possible to create such types if you allow laziness, mutation, or reflection. But in
general, in a non-lazy language like F#, it's a good idea to avoid such types.

Working with recursive types

End of public service announcement -- let's get coding!

1466

Introduction to recursive types

First, say that we want a description of the gift. The logic will be:
For the two non-recursive cases, return a string describing that case.
For the three recursive cases, return a string that describes the case, but also includes
the description of the inner gift. This means that description function is going to refer
to itself, and therefore it must be marked with the rec keyword.
Here's an example implementation:
let rec description gift =
match gift with
| Book book ->
sprintf "'%s'" book.title
| Chocolate choc ->
sprintf "%A chocolate" choc.chocType
| Wrapped (innerGift,style) ->
sprintf "%s wrapped in %A paper" (description innerGift) style
| Boxed innerGift ->
sprintf "%s in a box" (description innerGift)
| WithACard (innerGift,message) ->
sprintf "%s with a card saying '%s'" (description innerGift) message

Note the recursive calls like this one in the Boxed case:
| Boxed innerGift ->
sprintf "%s in a box" (description innerGift)
~~~~~~~~~~~ <= recursive call

If we try this with our example values, let's see what we get:
birthdayPresent |> description
// "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birthday'"
christmasPresent |> description
// "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

That looks pretty good to me. Things like HappyHolidays look a bit funny without spaces, but
it's good enough to demonstrate the idea.
What about creating another function? For example, what is the total cost of a gift?
For totalCost , the logic will be:
Books and chocolate capture the price in the case-specific data, so use that.
Wrapping adds 0.5 to the cost.
A box adds 1.0 to the cost.

1467

Introduction to recursive types

A card adds 2.0 to the cost.

let rec totalCost gift =
match gift with
| Book book ->
book.price
| Chocolate choc ->
choc.price
| Wrapped (innerGift,style) ->
(totalCost innerGift) + 0.5m
| Boxed innerGift ->
(totalCost innerGift) + 1.0m
| WithACard (innerGift,message) ->
(totalCost innerGift) + 2.0m

And here are the costs for the two examples:

birthdayPresent |> totalCost
// 22.5m
christmasPresent |> totalCost
// 6.5m

Sometimes, people ask what is inside the box or wrapping paper. A whatsInside function is
easy to implement -- just ignore the container cases and return something for the nonrecursive cases.
let rec whatsInside gift =
match gift with
| Book book ->
"A book"
| Chocolate choc ->
"Some chocolate"
| Wrapped (innerGift,style) ->
whatsInside innerGift
| Boxed innerGift ->
whatsInside innerGift
| WithACard (innerGift,message) ->
whatsInside innerGift

And the results:

birthdayPresent |> whatsInside
// "A book"
christmasPresent |> whatsInside
// "Some chocolate"

1468

Introduction to recursive types

So that's a good start -- three functions, all quite easy to write.

Parameterize all the things

However, these three functions have some duplicate code. In addition to the unique
application logic, each function is doing its own pattern matching and its own logic for
recursively visiting the inner gift.
How can we separate the navigation logic from the application logic?
Answer: Parameterize all the things!
As always, we can parameterize the application logic by passing in functions. In this case,
we will need five functions, one for each case.
Here is the new, parameterized version -- I'll explain why I have called it cataGift shortly.
let rec cataGift fBook fChocolate fWrapped fBox fCard gift =
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (innerGift,style) ->
let innerGiftResult = cataGift fBook fChocolate fWrapped fBox fCard innerGift
fWrapped (innerGiftResult,style)
| Boxed innerGift ->
let innerGiftResult = cataGift fBook fChocolate fWrapped fBox fCard innerGift
fBox innerGiftResult
| WithACard (innerGift,message) ->
let innerGiftResult = cataGift fBook fChocolate fWrapped fBox fCard innerGift
fCard (innerGiftResult,message)

You can see this function is created using a purely mechanical process:
Each function parameter ( fBook , fChocolate , etc) corresponds to a case.
For the two non-recursive cases, the function parameter is passed all the data
associated with that case.
For the three recursive cases, there are two steps:
First, the cataGift function is called recursively on the innerGift to get an
innerGiftResult

Then the appropriate handler is passed all the data associated with that case, but
with innerGiftResult replacing innerGift .
Let's rewrite total cost using the generic cataGift function.

1469

Introduction to recursive types

let totalCostUsingCata gift =

let fBook (book:Book) =
book.price
let fChocolate (choc:Chocolate) =
choc.price
let fWrapped (innerCost,style) =
innerCost + 0.5m
let fBox innerCost =
innerCost + 1.0m
let fCard (innerCost,message) =
innerCost + 2.0m
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

Notes:
The innerGiftResult is now the total cost of the inner gift, so I have renamed it to
innerCost .

The totalCostUsingCata function itself is not recursive, because it uses the cataGift
function, and so no longer needs the rec keyword.
And this function gives the same result as before:
birthdayPresent |> totalCostUsingCata
// 22.5m

We can rewrite the description function using cataGift in the same way, changing
innerGiftResult to innerText .

let descriptionUsingCata gift =

let fBook (book:Book) =
sprintf "'%s'" book.title
let fChocolate (choc:Chocolate) =
sprintf "%A chocolate" choc.chocType
let fWrapped (innerText,style) =
sprintf "%s wrapped in %A paper" innerText style
let fBox innerText =
sprintf "%s in a box" innerText
let fCard (innerText,message) =
sprintf "%s with a card saying '%s'" innerText message
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

And the results are the same as before:

1470

Introduction to recursive types

birthdayPresent |> descriptionUsingCata

// "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birthday'"
christmasPresent |> descriptionUsingCata
// "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

Introducing catamorphisms
The cataGift function we wrote above is called a "catamorphism", from the Greek
components "down + shape". In normal usage, a catamorphism is a function that "collapses"
a recursive type into a new value based on its structure. In fact, you can think of a
catamorphism as a sort of "visitor pattern".
A catamorphism is very powerful concept, because it is the most fundamental function that
you can define for a structure like this. Any other function can be defined in terms of it.
That is, if we want to create a function with signature Gift -> string or Gift -> int , we
can use a catamorphism to create it by specifying a function for each case in the Gift
structure.
We saw above how we could rewrite totalCost as totalCostUsingCata using the
catamorphism, and we'll see lots of other examples later.

Catamorphisms and folds

Catamorphisms are often called "folds", but there is more than one kind of fold, so I will tend
to use "catamorphism" to refer the concept and "fold" to refer to a specific kind of
implementation.
I will talk in detail about the various kinds of folds in a subsequent post, so for the rest of this
post I will just use "catamorphism".

Tidying up the implementation

The cataGift implementation above was deliberately long-winded so that you could see
each step. Once you understand what is going on though, you can simplify it somewhat.
First, the cataGift fBook fChocolate fWrapped fBox fCard crops up three times, once for
each recursive case. Let's assign it a name like recurse :

1471

Introduction to recursive types

let rec cataGift fBook fChocolate fWrapped fBox fCard gift =

let recurse = cataGift fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (innerGift,style) ->
let innerGiftResult = recurse innerGift
fWrapped (innerGiftResult,style)
| Boxed innerGift ->
let innerGiftResult = recurse innerGift
fBox innerGiftResult
| WithACard (innerGift,message) ->
let innerGiftResult = recurse innerGift
fCard (innerGiftResult,message)

The recurse function has the simple signature Gift -> 'a -- that is, it converts a Gift to
the return type we need, and so we can use it to work with the various innerGift values.
The other thing is to replace innerGift with just gift in the recursive cases -- this is
called "shadowing". The benefit is that the "outer" gift is no longer visible to the casehandling code, and so we can't accidentally recurse into it, which would cause an infinite
loop.
Generally I avoid shadowing, but this is one case where it actually is a good practice,
because it eliminates a particularly nasty kind of bug.
Here's the version after the clean up:
let rec cataGift fBook fChocolate fWrapped fBox fCard gift =
let recurse = cataGift fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (gift,style) ->
fWrapped (recurse gift,style)
| Boxed gift ->
fBox (recurse gift)
| WithACard (gift,message) ->
fCard (recurse gift,message)

One more thing. I'm going to explicitly annotate the return type and call it 'r . Later on in
this series we'll be dealing with other generic types such as 'a and 'b , so it will be helpful
to be consistent and always have a standard name for the return type.

1472

Introduction to recursive types

let rec cataGift fBook fChocolate fWrapped fBox fCard gift :'r =
// name the return type => ~~~~

Here's the final version:

let rec cataGift fBook fChocolate fWrapped fBox fCard gift :'r =
let recurse = cataGift fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (gift,style) ->
fWrapped (recurse gift,style)
| Boxed gift ->
fBox (recurse gift)
| WithACard (gift,message) ->
fCard (recurse gift,message)

It's much simpler than the original implementation, and also demonstrates the symmetry
between a case constructor like Wrapped (gift,style) and the corresponding handler
fWrapped (recurse gift,style) . Which leads us nicely to...

The relationship between the type constructors and the

handlers
Here is the signature for the cataGift function. You can see that each case handler
function ( fBook , fBox , etc.) has the same pattern: an input which contains all the data for
that case, and a common output type 'r .
val cataGift :
fBook:(Book -> 'r) ->
fChocolate:(Chocolate -> 'r) ->
fWrapped:('r * WrappingPaperStyle -> 'r) ->
fBox:('r -> 'r) ->
fCard:('r * string -> 'r) ->
// input value
gift:Gift ->
// return value
'r

Another way to think about this is that: everywhere that there is a Gift type in the
constructor, it has been replaced with an 'r .
For example:

1473

Introduction to recursive types

The Gift.Book constructor takes a Book and returns a Gift . The fBook handler
takes a Book and returns an 'r .
The Gift.Wrapped constructor takes a Gift * WrappingPaperStyle and returns a Gift .
The fWrapped handler takes an 'r * WrappingPaperStyle as input and returns an 'r .
Here is that relationship expressed through type signatures:
// The Gift.Book constructor
Book -> Gift
// The fBook handler
Book -> 'r
// The Gift.Wrapped constructor
Gift * WrappingPaperStyle -> Gift
// The fWrapped handler
'r * WrappingPaperStyle -> 'r
// The Gift.Boxed constructor
Gift -> Gift
// The fBox handler
'r -> 'r

and so on for all of the rest.

Benefits of catamorphisms
There is a lot of theory behind catamorphisms, but what are the benefits in practice?
Why bother to create a special function like cataGift ? Why not just leave the original
functions alone?
There are number of reasons, including:
Reuse. Later, we will be creating quite complex catamorphisms. It's nice if you only
have to get the logic right once.
Encapsulation. By exposing functions only, you hide the internal structure of the data
type.
Flexibility. Functions are more flexible than pattern matching -- they can be composed,
partially applied, etc.
Mapping. With a catamorphism in hand you can easily create functions that map the
various cases to new structures.

1474

Introduction to recursive types

It's true that most of these benefits apply to non-recursive types as well, but recursive types
tend to be more complex so the benefits of encapsulation, flexibility, etc. are correspondingly
stronger.
In the following sections, we'll look at the last three points in more detail.

Using function parameters to hide internal structure

The first benefit is that a catamorphism abstracts out the internal design. By using functions,
the client code is somewhat isolated from the internal structure. Again, you can think of the
Visitor pattern as analogous in the OO world.
For example, if all the clients used the catamorphism function rather than pattern matching, I
could safely rename cases, and even, with a bit of care, add or remove cases.
Here's an example. Let's say that I had a earlier design for Gift that didn't have the
WithACard case. I'll call it version 1:

type Gift =
| Book of Book
| Chocolate of Chocolate
| Wrapped of Gift * WrappingPaperStyle
| Boxed of Gift

And say that we built and published a catamorphism function for that structure:
let rec cataGift fBook fChocolate fWrapped fBox gift :'r =
let recurse = cataGift fBook fChocolate fWrapped fBox
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (gift,style) ->
fWrapped (recurse gift,style)
| Boxed gift ->
fBox (recurse gift)

Note that this has only four function parameters.

Now suppose that version 2 of Gift comes along, which adds the WithACard case:

1475

Introduction to recursive types

and now there are five cases.

Often, when we add a new case, we want to break all the clients and force them to deal with
the new case.
But sometimes, we don't. And so we can stay compatible with the original cataGift by
handling the extra case silently, like this:
/// Uses Gift_V2 but is still backwards compatible with the earlier "cataGift".
let rec cataGift fBook fChocolate fWrapped fBox gift :'r =
let recurse = cataGift fBook fChocolate fWrapped fBox
match gift with
| Book book ->
fBook book
| Chocolate choc ->
fChocolate choc
| Wrapped (gift,style) ->
fWrapped (recurse gift,style)
| Boxed gift ->
fBox (recurse gift)
// pass through the new case silently
| WithACard (gift,message) ->
recurse gift

This function still has only four function parameters -- there is no special behavior for the
WithACard case.

There are a number of alternative ways of being compatible, such as returning a default
value. The important point is that the clients are not aware of the change.
Aside: Using active patterns to hide data
While we're on the topic of hiding the structure of a type, I should mention that you can also
use active patterns to do this.
For example, we could create a active pattern for the first four cases, and ignore the
WithACard case.

1476

Introduction to recursive types

let rec (|Book|Chocolate|Wrapped|Boxed|) gift =

The clients can pattern match on the four cases without knowing that the new case even
exists:
let rec whatsInside gift =
match gift with
| Book book ->
"A book"
| Chocolate choc ->
"Some chocolate"
| Wrapped (gift,style) ->
whatsInside gift
| Boxed gift ->
whatsInside gift

Case-handling functions vs. pattern matching

Catamorphisms use function parameters, and as noted above, functions are more flexible
than pattern matching due to tools such composition, partial application, etc.
Here's an example where all the "container" cases are ignored, and only the "content" cases
are handled.
let handleContents fBook fChocolate gift =
let fWrapped (innerGiftResult,style) =
innerGiftResult
let fBox innerGiftResult =
innerGiftResult
let fCard (innerGiftResult,message) =
innerGiftResult
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

1477

Introduction to recursive types

And here it is in use, with the two remaining cases handled "inline" using piping:
birthdayPresent
|> handleContents
(fun book -> "The book you wanted for your birthday")
(fun choc -> "Your fave chocolate")
// Result => "The book you wanted for your birthday"
christmasPresent
|> handleContents
(fun book -> "The book you wanted for Christmas")
(fun choc -> "Don't eat too much over the holidays!")
// Result => "Don't eat too much over the holidays!"

Of course this could be done with pattern matching, but being able to work with the existing
cataGift function directly makes life easier.

Using catamorphisms for mapping

I said above that a catamorphism is a function that "collapses" a recursive type into a new
value. For example, in totalCost , the recursive gift structure was collapsed into a single
cost value.
But a "single value" can be more than just a primitive -- it can be a complicated structure too,
such as a another recursive structure.
In fact, catamorphisms are great for mapping one kind of structure onto another, especially if
they are very similar.
For example, let's say that I have a chocolate-loving room-mate who will stealthily remove
and devour any chocolate in a gift, leaving the wrapping untouched, but discarding the box
and gift card.
What's left at the end is a "gift minus chocolate" that we can model as follows:
type GiftMinusChocolate =
| Book of Book
| Apology of string
| Wrapped of GiftMinusChocolate * WrappingPaperStyle

We can easily map from a Gift to a GiftMinusChocolate , because the cases are almost
parallel.
A Book is passed through untouched.
Chocolate is eaten and replaced with an Apology .

The Wrapped case is passed through untouched.

1478

Introduction to recursive types

The Box and WithACard cases are ignored.

Here's the code:
let removeChocolate gift =
let fBook (book:Book) =
Book book
let fChocolate (choc:Chocolate) =
Apology "sorry I ate your chocolate"
let fWrapped (innerGiftResult,style) =
Wrapped (innerGiftResult,style)
let fBox innerGiftResult =
innerGiftResult
let fCard (innerGiftResult,message) =
innerGiftResult
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

And if we test...
birthdayPresent |> removeChocolate
// GiftMinusChocolate =
// Wrapped (Book {title = "Wolf Hall"; price = 20M}, HappyBirthday)
christmasPresent |> removeChocolate
// GiftMinusChocolate =
// Wrapped (Apology "sorry I ate your chocolate", HappyHolidays)

Deep copying
One more thing. Remember that each case-handling function takes the data associated with
that case? That means that we can just use the original case constructors as the functions!
To see what I mean, let's define a function called deepCopy that clones the original value.
Each case handler is just the corresponding case constructor:

1479

Introduction to recursive types

let deepCopy gift =

let fBook book =
Book book
let fChocolate (choc:Chocolate) =
Chocolate choc
let fWrapped (innerGiftResult,style) =
Wrapped (innerGiftResult,style)
let fBox innerGiftResult =
Boxed innerGiftResult
let fCard (innerGiftResult,message) =
WithACard (innerGiftResult,message)
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

We can simplify this further by removing the redundant parameters for each handler:
let deepCopy gift =
let fBook = Book
let fChocolate = Chocolate
let fWrapped = Wrapped
let fBox = Boxed
let fCard = WithACard
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

You can test that this works yourself:

christmasPresent |> deepCopy
// Result =>
// Wrapped (
// Boxed (Chocolate {chocType = SeventyPercent; price = 5M;}),
// HappyHolidays)

So that leads to another way of thinking about a catamorphism:

A catamorphism is a function for a recursive type such that when you pass in the type's
case constructors, you get a "clone" function.

Mapping and transforming in one pass

A slight variant on deepCopy allows us to recurse through an object and change bits of it as
we do so.
For example, let's say I don't like milk chocolate. Well, I can write a function that upgrades
the gift to better quality chocolate and leaves all the other cases alone.

1480

Introduction to recursive types

let upgradeChocolate gift =

let fBook = Book
let fChocolate (choc:Chocolate) =
Chocolate {choc with chocType = SeventyPercent}
let fWrapped = Wrapped
let fBox = Boxed
let fCard = WithACard
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

And here it is in use:

// create some chocolate I don't like
let cheapChoc = Boxed (Chocolate {chocType=Milk; price=5m})
// upgrade it!
cheapChoc |> upgradeChocolate
// Result =>
// Boxed (Chocolate {chocType = SeventyPercent; price = 5M})

If you are thinking that this is beginning to smell like a map , you'd be right. We'll look at
generic maps in the sixth post, as part of a discussion of generic recursive types.

Rules for creating catamorphisms

We saw above that creating a catamorphism is a mechanical process:
Create a function parameter to handle each case in the structure.
For non-recursive cases, pass the function parameter all the data associated with that
case.
For recursive cases, perform two steps:
First, call the catamorphism recursively on the nested value.
Then pass the handler all the data associated with that case, but with the result of
the catamorphism replacing the original nested value.
Let's now see if we can apply these rules to create catamorphisms in other domains.

Summary
We've seen in this post how to define a recursive type, and been introduced to
catamorphisms.

1481

Introduction to recursive types

In the next post we'll uses these rules to create catamorphisms for some other domains.
See you then!
The source code for this post is available at this gist.

1482

Catamorphism examples

Catamorphism examples
This post is the second in a series.
In the previous post, I introduced "catamorphisms", a way of creating functions for recursive
types, and listed some rules which can be used to implement them mechanically. In this
post, we'll use these rules to implement catamorphisms for some other domains.

1483

Catamorphism examples

Part 6: Trees in the real world

Defining a generic Tree type
The Tree type in the real world
Mapping the Tree type
Example: Creating a directory listing
Example: A parallel grep
Example: Storing the file system in a database
Example: Serializing a Tree to JSON
Example: Deserializing a Tree from JSON
Example: Deserializing a Tree from JSON - with error handling

Rules for creating catamorphisms

We saw in the previous post that creating a catamorphism is a mechanical process, and the
rules were:
Create a function parameter to handle each case in the structure.
For non-recursive cases, pass the function parameter all the data associated with that
case.
For recursive cases, perform two steps:
First, call the catamorphism recursively on the nested value.
Then pass the handler all the data associated with that case, but with the result of
the catamorphism replacing the original nested value.
Let's now see if we can apply these rules to create catamorphisms in other domains.

Catamorphism example: File system domain

Let's start with a very crude model of a file system:
Each file has a name and a size.
Each directory has a name and a size and a list of subitems.
Here's how I might model that:

1484

Catamorphism examples

type FileSystemItem =
| File of File
| Directory of Directory
and File = {name:string; fileSize:int}
and Directory = {name:string; dirSize:int; subitems:FileSystemItem list}

I admit it's a pretty bad model, but it's just good enough for this example!
Ok, here are some sample files and directories:
let readme = File {name="readme.txt"; fileSize=1}
let config = File {name="config.xml"; fileSize=2}
let build = File {name="build.bat"; fileSize=3}
let src = Directory {name="src"; dirSize=10; subitems=[readme; config; build]}
let bin = Directory {name="bin"; dirSize=10; subitems=[]}
let root = Directory {name="root"; dirSize=5; subitems=[src; bin]}

Time to create the catamorphism!

Let's start by looking at the signatures to figure out what we need.
The File constructor takes a File and returns a FileSystemItem . Using the guidelines
above, the handler for the File case needs to have the signature File -> 'r .
// case constructor
File : File -> FileSystemItem
// function parameter to handle File case
fFile : File -> 'r

That's simple enough. Let's put together an initial skeleton of cataFS , as I'll call it:
let rec cataFS fFile fDir item :'r =
let recurse = cataFS fFile fDir
match item with
| File file ->
fFile file
| Directory dir ->
// to do

The Directory case is more complicated. If we naively applied the guidelines above, the
handler for the Directory case would have the signature Directory -> 'r , but that would
be incorrect, because the Directory record itself contains a FileSystemItem that needs to
be replaced with an 'r too. How can we do this?

1485

Catamorphism examples

One way is to "explode" the Directory record into a tuple of (string,int,FileSystemItem

list) , and then replace the FileSystemItem with 'r in there too.

In other words, we have this sequence of transformations:

// case constructor (Directory as record)
Directory : Directory -> FileSystemItem
// case constructor (Directory unpacked as tuple)
Directory : (string, int, FileSystemItem list) -> FileSystemItem
// replace with 'r ===> ~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
// function parameter to handle Directory case
fDir : (string, int, 'r list) -> 'r

Another issue is that the data associated with the Directory case is a list of
FileSystemItem s. How can we convert that into a list of 'r s?

Well, the recurse helper turns a FileSystemItem into an 'r , so we can just use List.map
passing in recurse as the mapping function, and that will give us the list of 'r s we need!
Putting it all together, we get this implementation:
let rec cataFS fFile fDir item :'r =
let recurse = cataFS fFile fDir
match item with
| File file ->
fFile file
| Directory dir ->
let listOfRs = dir.subitems |> List.map recurse
fDir (dir.name,dir.dirSize,listOfRs)

and if we look at the type signature, we can see that it is just what we want:
val cataFS :
fFile : (File -> 'r) ->
fDir : (string * int * 'r list -> 'r) ->
// input value
FileSystemItem ->
// return value
'r

So we're done. It's a bit complicated to set up, but once built, we have a nice reusable
function that can be the basis for many others.

File system domain: totalSize example

1486

Catamorphism examples

Alrighty then, let's use it in practice.

To start with, we can easily define a totalSize function that returns the total size of an item
and all its subitems:
let totalSize fileSystemItem =
let fFile (file:File) =
file.fileSize
let fDir (name,size,subsizes) =
(List.sum subsizes) + size
cataFS fFile fDir fileSystemItem

And here are the results:

readme |> totalSize // 1
src |> totalSize // 16 = 10 + (1 + 2 + 3)
root |> totalSize // 31 = 5 + 16 + 10

File system domain: largestFile example

How about a more complicated function, such as "what is the largest file in the tree?"
Before we start this one, let's think about what it should return. That is, what is the 'r ?
You might think that it's just a File . But what if the subdirectory is empty and there are no
files?
So let's make 'r a File option instead.
The function for the File case should return Some file then:
let fFile (file:File) =
Some file

The function for the Directory case needs more thought:

If list of subfiles is empty, then return None
If list of subfiles is non-empty, then return the largest one
let fDir (name,size,subfiles) =
match subfiles with
| [] ->
None // empty directory
| subfiles ->
// return largest one

1487

Catamorphism examples

But remember that 'r is not a File but a File option . So that means that subfiles is
not a list of files, but a list of File option .
Now, how can we find the largest one of these? We probably want to use List.maxBy and
pass in the size. But what is the size of a File option ?
Let's write a helper function to provide the size of a File option , using this logic:
If the File option is None , return 0
Else return the size of the file inside the option
Here's the code:
// helper to provide a default if missing
let ifNone deflt opt =
defaultArg opt deflt
// get the file size of an option
let fileSize fileOpt =
fileOpt
|> Option.map (fun file -> file.fileSize)
|> ifNone 0

Putting it all together then, we have our largestFile function:

1488

Catamorphism examples

let largestFile fileSystemItem =

// helper to provide a default if missing
let ifNone deflt opt =
defaultArg opt deflt
// helper to get the size of a File option
let fileSize fileOpt =
fileOpt
|> Option.map (fun file -> file.fileSize)
|> ifNone 0
// handle File case
let fFile (file:File) =
Some file
// handle Directory case
let fDir (name,size,subfiles) =
match subfiles with
| [] ->
None // empty directory
| subfiles ->
// find the biggest File option using the helper
subfiles
|> List.maxBy fileSize
// call the catamorphism
cataFS fFile fDir fileSystemItem

If we test it, we get the results we expect:

readme |> largestFile
// Some {name = "readme.txt"; fileSize = 1}
src |> largestFile
// Some {name = "build.bat"; fileSize = 3}
bin |> largestFile
// None
root |> largestFile
// Some {name = "build.bat"; fileSize = 3}

Again, a little bit tricky to set up, but no more than if we had to write it from scratch without
using a catamorphism at all.

1489

Catamorphism examples

Catamorphism example: Product domain

Let's work with a slightly more complicated domain. This time, imagine that we make and
sell products of some kind:
Some products are bought, with an optional vendor.
Some products are made on our premises, built from subcomponents, where a
subcomponent is some quantity of another product.
Here's the domain modelled as types:
type Product =
| Bought of BoughtProduct
| Made of MadeProduct
and BoughtProduct = {
name : string
weight : int
vendor : string option }
and MadeProduct = {
name : string
weight : int
components:Component list }
and Component = {
qty : int
product : Product }

Note that the types are mutally recursive. Product references MadeProduct which
references Component which in turn references Product again.
Here are some example products:

1490

Catamorphism examples

Now to design the catamorphism, we need to do is replace the Product type with 'r in all
the constructors.
Just as with the previous example, the Bought case is easy:
// case constructor
Bought : BoughtProduct -> Product
// function parameter to handle Bought case
fBought : BoughtProduct -> 'r

The Made case is trickier. We need to expand the MadeProduct into a tuple. But that tuple
contains a Component , so we need to expand that as well. Finally we get to the inner
Product , and we can then mechanically replace that with 'r .

Here's the sequence of transformations:

1491

Catamorphism examples

// case constructor
Made : MadeProduct -> Product
// case constructor (MadeProduct unpacked as tuple)
Made : (string,int,Component list) -> Product
// case constructor (Component unpacked as tuple)
Made : (string,int,(int,Product) list) -> Product
// replace with 'r ===> ~~~~~~~ ~~~~~~~
// function parameter to handle Made case
fMade : (string,int,(int,'r) list) -> 'r

When implementing the cataProduct function we need to the same kind of mapping as
before, turning a list of Component into a list of (int,'r) .
We'll need a helper for that:
// Converts a Component into a (int * 'r) tuple
let convertComponentToTuple comp =
(comp.qty,recurse comp.product)

You can see that this uses the recurse function to turn the inner product ( comp.product )
into an 'r and then make a tuple int * 'r .
With convertComponentToTuple available, we can convert all the components to tuples using
List.map :

let componentTuples =
made.components
|> List.map convertComponentToTuple

componentTuples is a list of (int * 'r) , which is just what we need for the fMade function.

The complete implementation of cataProduct looks like this:

1492

Catamorphism examples

let rec cataProduct fBought fMade product :'r =

let recurse = cataProduct fBought fMade
// Converts a Component into a (int * 'r) tuple
let convertComponentToTuple comp =
(comp.qty,recurse comp.product)
match product with
| Bought bought ->
fBought bought
| Made made ->
let componentTuples = // (int * 'r) list
made.components
|> List.map convertComponentToTuple
fMade (made.name,made.weight,componentTuples)

Product domain: productWeight example

We can now use cataProduct to calculate the weight, say.
let productWeight product =
// handle Bought case
let fBought (bought:BoughtProduct) =
bought.weight
// handle Made case
let fMade (name,weight,componentTuples) =
// helper to calculate weight of one component tuple
let componentWeight (qty,weight) =
qty * weight
// add up the weights of all component tuples
let totalComponentWeight =
componentTuples
|> List.sumBy componentWeight
// and add the weight of the Made case too
totalComponentWeight + weight
// call the catamorphism
cataProduct fBought fMade product

Let's test it interactively to make sure it works:

label |> productWeight // 1
shampoo |> productWeight // 17 = 10 + (2x1 + 1x2 + 1x3)
twoPack |> productWeight // 39 = 5 + (2x17)

1493

Catamorphism examples

That's as we expect.
Try implementing productWeight from scratch, without using a helper function like
cataProduct . Again, it's do-able, but you'll probably waste quite bit of time getting the

recursion logic right.

Product domain: mostUsedVendor example

Let's do a more complex function now. What is the most used vendor?
The logic is simple: each time a product references a vendor, we'll give that vendor one
point, and the vendor with the highest score wins.
Again, let's think about what it should return. That is, what is the 'r ?
You might think that it's just a score of some kind, but we also need to know the vendor
name. Ok, a tuple then. But what if there are no vendors?
So let's make 'r a VendorScore option , where we are going to create a little type
VendorScore , rather than using a tuple.

type VendorScore = {vendor:string; score:int}

We'll also define some helpers to get data from a VendorScore easily:
let vendor vs = vs.vendor
let score vs = vs.score

Now, you can't determine the most used vendor over until you have results from the entire
tree, so both the Bought case and the Made case need to return a list which can added to
as we recurse up the tree. And then, after getting all the scores, we'll sort descending to find
the vendor with the highest one.
So we have to make 'r a VendorScore list , not just an option!
The logic for the Bought case is then:
If the vendor is present, return a VendorScore with score = 1, but as a one-element list
rather than as a single item.
If the vendor is missing, return an empty list.

1494

Catamorphism examples

let fBought (bought:BoughtProduct) =

// set score = 1 if there is a vendor
bought.vendor
|> Option.map (fun vendor -> {vendor = vendor; score = 1} )
// => a VendorScore option
|> Option.toList
// => a VendorScore list

The function for the Made case is more complicated.

If list of subscores is empty, then return an empty list.
If list of subscores is non-empty, we sum them by vendor and then return the new list.
But the list of subresults passed into the fMade function will not be a list of subscores, it will
be a list of tuples, qty * 'r where 'r is VendorScore list . Complicated!
What we need to do then is:
Turn qty * 'r into just 'r because we don't care about the qty in this case. We now
have a list of VendorScore list . We can use List.map snd to do this.
But now we would have a list of VendorScore list . We can flatten a list of lists into a
simple list using List.collect . And in fact, using List.collect snd can do both steps
in one go.
Group this list by vendor so that we have a list of key=vendor; values=VendorScore list
tuples.
Sum up the scores for each vendor ( values=VendorScore list ) into a single value, so
that we have a list of key=vendor; values=VendorScore tuples.
At this point the cata function will return a VendorScore list . To get the highest score, use
List.sortByDescending then List.tryHead . Note that maxBy won't work because the list

could be empty.
Here's the complete mostUsedVendor function:

1495

Catamorphism examples

let mostUsedVendor product =

let fBought (bought:BoughtProduct) =
// set score = 1 if there is a vendor
bought.vendor
|> Option.map (fun vendor -> {vendor = vendor; score = 1} )
// => a VendorScore option
|> Option.toList
// => a VendorScore list
let fMade (name,weight,subresults) =
// subresults are a list of (qty * VendorScore list)
// helper to get sum of scores
let totalScore (vendor,vendorScores) =
let totalScore = vendorScores |> List.sumBy score
{vendor=vendor; score=totalScore}
subresults
// => a list of (qty * VendorScore list)
|> List.collect snd // ignore qty part of subresult
// => a list of VendorScore
|> List.groupBy vendor
// second item is list of VendorScore, reduce to sum
|> List.map totalScore
// => list of VendorScores
// call the catamorphism
cataProduct fBought fMade product
|> List.sortByDescending score // find highest score
// return first, or None if list is empty
|> List.tryHead

Now let's test it:

label |> mostUsedVendor
// Some {vendor = "ACME"; score = 1}
formulation |> mostUsedVendor
// None
shampoo |> mostUsedVendor
// Some {vendor = "ACME"; score = 2}
twoPack |> mostUsedVendor
// Some {vendor = "ACME"; score = 2}

1496

Catamorphism examples

This isn't the only possible implementation of fMade , of course. I could have used
List.fold and done the whole thing in one pass, but this version seems like the most

obvious and readable implementation.

It's also true that I could have avoided using cataProduct altogether and written
mostUsedVendor from scratch. If performance is an issue, then that might be a better

approach, because the generic catamorphism creates intermediate values (such as the list
of qty * VendorScore option ) which are over general and potentially wasteful.
On other hand, by using the catamorphism, I could focus on the counting logic only and
ignore the recursion logic.
So as always, you should consider the pros and cons of reuse vs. creating from scratch; the
benefits of writing common code once and using it in a standardized way, versus the
performance but extra effort (and potential bugginess) of custom code.

Summary
We've seen in this post how to define a recursive type, and been introduced to
catamorphisms.
And we have also seen some uses for catamorphisms:
Any function that "collapses" a recursive type, such as Gift -> 'r , can be written in
terms of the catamorphism for that type.
Catamorphisms can be used to hide the internal structure of the type.
Catamorphisms can be used to create mappings from one type to another by tweaking
the functions that handle each case.
Catamorphisms can be used to create a clone of the original value by passing in the
type's case constructors.
But all is not perfect in the land of catamorphisms. In fact, all the catamorphism
implementations on this page have a potentially serious flaw.
In the next post we'll see what can go wrong with them, how to fix them, and in the process
look at the various kinds of "fold".
See you then!
The source code for this post is available at this gist.
UPDATE: Fixed logic error in mostUsedVendor as pointed out by Paul Schnapp in comments.
Thanks, Paul!

1497

Catamorphism examples

1498

Introducing Folds

Introducing Folds
This post is the third in a series.
In the first post, I introduced "catamorphisms", a way of creating functions for recursive
types, and in the second post, we created a few catamorphism implementations.
But at the end of the previous post, I noted that all the catamorphism implementations so far
have had a potentially serious flaw.
In this post, we'll look at the flaw and how to work around it, and in the process look at folds,
tail-recursion and the difference between "left fold" and "right fold".

1499

Introducing Folds

Making the Gift domain generic

Defining a generic Container type
A third way to implement the gift domain
Abstract or concrete? Comparing the three designs
Part 6: Trees in the real world
Defining a generic Tree type
The Tree type in the real world
Mapping the Tree type
Example: Creating a directory listing
Example: A parallel grep
Example: Storing the file system in a database
Example: Serializing a Tree to JSON
Example: Deserializing a Tree from JSON
Example: Deserializing a Tree from JSON - with error handling

A flaw in our catamorphism implementation

Before we look at the flaw, let's first review the recursive type Gift and the associated
catamorphism cataGift that we created for it.
Here's the domain:
type Book = {title: string; price: decimal}
type ChocolateType = Dark | Milk | SeventyPercent
type Chocolate = {chocType: ChocolateType ; price: decimal}
type WrappingPaperStyle =
| HappyBirthday
| HappyHolidays
| SolidColor
type Gift =
| Book of Book
| Chocolate of Chocolate
| Wrapped of Gift * WrappingPaperStyle
| Boxed of Gift
| WithACard of Gift * message:string

Here are some example values that we'll be using in this post:

1500

Introducing Folds

// A Book
let wolfHall = {title="Wolf Hall"; price=20m}
// A Chocolate
let yummyChoc = {chocType=SeventyPercent; price=5m}
// A Gift
let birthdayPresent = WithACard (Wrapped (Book wolfHall, HappyBirthday), "Happy Birthd
ay")
// A Gift
let christmasPresent = Wrapped (Boxed (Chocolate yummyChoc), HappyHolidays)

Here's the catamorphism:

and here is a totalCostUsingCata function built using cataGift :

let totalCostUsingCata gift =
let fBook (book:Book) =
book.price
let fChocolate (choc:Chocolate) =
choc.price
let fWrapped (innerCost,style) =
innerCost + 0.5m
let fBox innerCost =
innerCost + 1.0m
let fCard (innerCost,message) =
innerCost + 2.0m
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

What's the flaw?

So what is wrong with this implementation? Let's stress test it and find out!

1501

Introducing Folds

What we'll do is create a Box inside a Box inside a Box a very large number of times,
and see what happens.
Here's a little helper function to create nested boxes:
let deeplyNestedBox depth =
let rec loop depth boxSoFar =
match depth with
| 0 -> boxSoFar
| n -> loop (n-1) (Boxed boxSoFar)
loop depth (Book wolfHall)

Let's try it to make sure it works:

deeplyNestedBox 5
// Boxed (Boxed (Boxed (Boxed (Boxed (Book {title = "Wolf Hall"; price = 20M})))))
deeplyNestedBox 10
// Boxed(Boxed(Boxed(Boxed(Boxed
// (Boxed(Boxed(Boxed(Boxed(Boxed(Book {title = "Wolf Hall";price = 20M}))))))))))

Now try running totalCostUsingCata with these deeply nested boxes:

deeplyNestedBox 10 |> totalCostUsingCata // OK 30.0M
deeplyNestedBox 100 |> totalCostUsingCata // OK 120.0M
deeplyNestedBox 1000 |> totalCostUsingCata // OK 1020.0M

So far so good.
But if we use much larger numbers, we soon run into a stack overflow exception:
deeplyNestedBox 10000 |> totalCostUsingCata // Stack overflow?
deeplyNestedBox 100000 |> totalCostUsingCata // Stack overflow?

The exact number which causes an error depends on the environment, available memory,
and so on. But it is a certainty that you will run into it when you start using largish numbers.
Why is this happening?

The problem with deep recursion

Recall that the definition of the cost for the Boxed case ( fBox ) was innerCost + 1.0m . And
what is the inner cost? That's another box too, so we end up with a chain of calculations
looking like this:

1502

Introducing Folds

innerCost + 1.0m where innerCost =

innerCost2 + 1.0m where innerCost2 =
innerCost3 + 1.0m where innerCost3 =
innerCost4 + 1.0m where innerCost4 =
...
innerCost999 + 1.0m where innerCost999 =
innerCost1000 + 1.0m where innerCost1000 =
book.price

In other words, innerCost1000 has to be calculated before innerCost999 can be calculated,

and 999 other inner costs have to be calculated before the top level innerCost can be
calculated.
Every level is waiting for its inner cost to be calculated before doing the calculation for that
level.
All these unfinished calculations are stacked up waiting for the inner one to complete. And
when you have too many? Boom! Stack overflow!

The solution to stack overflows

The solution to this problem is simple. Rather than each level waiting for the inner cost to be
calculated, each level calculates the cost so far, using an accumulator, and passes that
down to the next inner level. When we get to the bottom level, we have the final answer.
costSoFar = 1.0m; evaluate calcInnerCost with costSoFar:
costSoFar = costSoFar + 1.0m; evaluate calcInnerCost with costSoFar:
costSoFar = costSoFar + 1.0m; evaluate calcInnerCost with costSoFar:
costSoFar = costSoFar + 1.0m; evaluate calcInnerCost with costSoFar:
...
costSoFar = costSoFar + 1.0m; evaluate calcInnerCost with costSoFar:
costSoFar = costSoFar + 1.0m; evaluate calcInnerCost with costSoFar:
finalCost = costSoFar + book.price // final answer

The big advantange of this approach is that all calculations at a particular level are
completely finished before the next lowel level is called. Which means that the level and its
associated data can be safely discarded from the stack. Which means no stack overflow!
An implementation like this, where the higher levels can be safely discarded, is called tail
recursive.

Reimplementating the totalCost function with an

accumulator

1503

Introducing Folds

Let's rewrite the total cost function from scratch, using an accumulator called costSoFar :
let rec totalCostUsingAcc costSoFar gift =
match gift with
| Book book ->
costSoFar + book.price // final result
| Chocolate choc ->
costSoFar + choc.price // final result
| Wrapped (innerGift,style) ->
let newCostSoFar = costSoFar + 0.5m
totalCostUsingAcc newCostSoFar innerGift
| Boxed innerGift ->
let newCostSoFar = costSoFar + 1.0m
totalCostUsingAcc newCostSoFar innerGift
| WithACard (innerGift,message) ->
let newCostSoFar = costSoFar + 2.0m
totalCostUsingAcc newCostSoFar innerGift

A few things to note:

The new version of the function has an extra parameter ( costSoFar ). We will have to
provide an initial value for this (such as zero) when we call it at the top level.
The non-recursive cases ( Book and Chocolate ) are the end points. The take the cost
so far and add it to their price, and then that is the final result.
The recursive cases calculate a new costSoFar based on the parameter that is passed
in. The new costSoFar is then passed down to the next lower level, just as in the
example above.
Let's stress test this version:
deeplyNestedBox 1000 |> totalCostUsingAcc 0.0m // OK 1020.0M
deeplyNestedBox 10000 |> totalCostUsingAcc 0.0m // OK 10020.0M
deeplyNestedBox 100000 |> totalCostUsingAcc 0.0m // OK 100020.0M
deeplyNestedBox 1000000 |> totalCostUsingAcc 0.0m // OK 1000020.0M

Excellent. Up to one million nested levels without a hiccup.

Introducing "fold"
Now let's apply the same design principle to the catamorphism implementation.
We'll create a new function foldGift . We'll introduce an accumulator acc that we will
thread through each level, and the non-recursive cases will return the final accumulator.

1504

Introducing Folds

let rec foldGift fBook fChocolate fWrapped fBox fCard acc gift :'r =
let recurse = foldGift fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
let finalAcc = fBook acc book
finalAcc // final result
| Chocolate choc ->
let finalAcc = fChocolate acc choc
finalAcc // final result
| Wrapped (innerGift,style) ->
let newAcc = fWrapped acc style
recurse newAcc innerGift
| Boxed innerGift ->
let newAcc = fBox acc
recurse newAcc innerGift
| WithACard (innerGift,message) ->
let newAcc = fCard acc message
recurse newAcc innerGift

If we look at the type signature, we can see that it is subtly different. The type of the
accumulator 'a is being used everywhere now. The only time where the final return type is
used is in the two non-recursive cases ( fBook and fChocolate ).
val foldGift :
fBook:('a -> Book -> 'r) ->
fChocolate:('a -> Chocolate -> 'r) ->
fWrapped:('a -> WrappingPaperStyle -> 'a) ->
fBox:('a -> 'a) ->
fCard:('a -> string -> 'a) ->
// accumulator
acc:'a ->
// input value
gift:Gift ->
// return value
'r

Let's look at this more closely, and compare the signatures of the original catamorphism
from the last post with the signatures of the new fold function.
First of all, the non-recursive cases:
// original catamorphism
fBook:(Book -> 'r)
fChocolate:(Chocolate -> 'r)
// fold
fBook:('a -> Book -> 'r)
fChocolate:('a -> Chocolate -> 'r)

1505

Introducing Folds

As you can see, with "fold", the non-recursive cases take an extra parameter (the
accumulator) and return the 'r type.
This is a very important point: the type of the accumulator does not need to be the same as
the return type. We will need to take advantage of this shortly.
What about the recursive cases? How did their signature change?
// original catamorphism
fWrapped:('r -> WrappingPaperStyle -> 'r)
fBox:('r -> 'r)
// fold
fWrapped:('a -> WrappingPaperStyle -> 'a)
fBox:('a -> 'a)

For the recursive cases, the structure is identical but all use of the 'r type has been
replaced with the 'a type. The recursive cases do not use the 'r type at all.

Reimplementating the totalCost function using fold

Once again, we can reimplement the total cost function, but this time using the foldGift
function:
let totalCostUsingFold gift =
let fBook costSoFar (book:Book) =
costSoFar + book.price
let fChocolate costSoFar (choc:Chocolate) =
costSoFar + choc.price
let fWrapped costSoFar style =
costSoFar + 0.5m
let fBox costSoFar =
costSoFar + 1.0m
let fCard costSoFar message =
costSoFar + 2.0m
// initial accumulator
let initialAcc = 0m
// call the fold
foldGift fBook fChocolate fWrapped fBox fCard initialAcc gift

And again, we can process very large numbers of nested boxes without a stack overflow:

1506

Introducing Folds

deeplyNestedBox 100000 |> totalCostUsingFold // no problem 100020.0M

deeplyNestedBox 1000000 |> totalCostUsingFold // no problem 1000020.0M

Problems with fold

So using fold solves all our problems, right?
Unfortunately, no.
Yes, there are no more stack overflows, but we have another problem now.

Reimplementation of the description function

To see what the problem is, let's revisit the description function that we created in the first
post.
The original one was not tail-recursive, so let's make it safer and reimplement it using
foldGift .

let descriptionUsingFold gift =

let fBook descriptionSoFar (book:Book) =
sprintf "'%s' %s" book.title descriptionSoFar
let fChocolate descriptionSoFar (choc:Chocolate) =
sprintf "%A chocolate %s" choc.chocType descriptionSoFar
let fWrapped descriptionSoFar style =
sprintf "%s wrapped in %A paper" descriptionSoFar style
let fBox descriptionSoFar =
sprintf "%s in a box" descriptionSoFar
let fCard descriptionSoFar message =
sprintf "%s with a card saying '%s'" descriptionSoFar message
// initial accumulator
let initialAcc = ""
// main call
foldGift fBook fChocolate fWrapped fBox fCard initialAcc gift

Let's see what the output is:

1507

Introducing Folds

birthdayPresent |> descriptionUsingFold

// "'Wolf Hall' with a card saying 'Happy Birthday' wrapped in HappyBirthday paper"
christmasPresent |> descriptionUsingFold
// "SeventyPercent chocolate wrapped in HappyHolidays paper in a box"

These outputs are wrong! The order of the decorations has been mixed up.
It's supposed to be a wrapped book with a card, not a book and a card wrapped together.
And it's supposed to be chocolate in a box, then wrapped, not wrapped chocolate in a box!
// OUTPUT: "'Wolf Hall' with a card saying 'Happy Birthday' wrapped in HappyBirthday
paper"
// CORRECT "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birth
day'"
// OUTPUT: "SeventyPercent chocolate wrapped in HappyHolidays paper in a box"
// CORRECT "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

What has gone wrong?

The answer is that the correct description for each layer depends on the description of the
layer below. We can't "pre-calculate" the description for a layer and pass it down to the next
layer using a descriptionSoFar accumulator.
But now we have a dilemma: a layer depends on information from the layer below, but we
want to avoid a stack overflow.

Using functions as accumulators

Remember that the accumulator type does not have to be the same as the return type. We
can use anything as an accumulator, even a function!
So what we'll do is, rather than passing a descriptionSoFar as the accumulator, we'll pass a
function ( descriptionGenerator say) that will build the appropriate description given the
value of the next layer down.
Here's the implementation for the non-recursive cases:

1508

Introducing Folds

let fBook descriptionGenerator (book:Book) =

descriptionGenerator (sprintf "'%s'" book.title)
// ~~~~~~~~~~~~~~~~~~~~ <= a function as an accumulator!
let fChocolate descriptionGenerator (choc:Chocolate) =
descriptionGenerator (sprintf "%A chocolate" choc.chocType)

The implementation for recursive cases is a bit more complicated:

We are given an accumulator ( descriptionGenerator ) as a parameter.
We need to create a new accumulator (a new descriptionGenerator ) to pass down to
the next lower layer.
The input to the description generator will be all the data accumulated from the lower
layers. We manipulate that to make a new description and then call the
descriptionGenerator passed in from the higher layer.

It's more complicated to talk about than to demonstrate, so here are implementations for two
of the cases:
let fWrapped descriptionGenerator style =
let newDescriptionGenerator innerText =
let newInnerText = sprintf "%s wrapped in %A paper" innerText style
descriptionGenerator newInnerText
newDescriptionGenerator
let fBox descriptionGenerator =
let newDescriptionGenerator innerText =
let newInnerText = sprintf "%s in a box" innerText
descriptionGenerator newInnerText
newDescriptionGenerator

We can simplify that code a little by using a lambda directly:

let fWrapped descriptionGenerator style =
fun innerText ->
let newInnerText = sprintf "%s wrapped in %A paper" innerText style
descriptionGenerator newInnerText
let fBox descriptionGenerator =
fun innerText ->
let newInnerText = sprintf "%s in a box" innerText
descriptionGenerator newInnerText

We could continue to make it more compact using piping and other things, but I think that
what we have here is a good balance between conciseness and obscurity.

1509

Introducing Folds

Here is the entire function:

let descriptionUsingFoldWithGenerator gift =
let fBook descriptionGenerator (book:Book) =
descriptionGenerator (sprintf "'%s'" book.title)
let fChocolate descriptionGenerator (choc:Chocolate) =
descriptionGenerator (sprintf "%A chocolate" choc.chocType)
let fWrapped descriptionGenerator style =
fun innerText ->
let newInnerText = sprintf "%s wrapped in %A paper" innerText style
descriptionGenerator newInnerText
let fBox descriptionGenerator =
fun innerText ->
let newInnerText = sprintf "%s in a box" innerText
descriptionGenerator newInnerText
let fCard descriptionGenerator message =
fun innerText ->
let newInnerText = sprintf "%s with a card saying '%s'" innerText message
descriptionGenerator newInnerText
// initial DescriptionGenerator
let initialAcc = fun innerText -> innerText
// main call
foldGift fBook fChocolate fWrapped fBox fCard initialAcc gift

Again, I'm using overly descriptive intermediate values to make it clear what is going on.
If we try descriptionUsingFoldWithGenerator now, we get the correct answers again:
birthdayPresent |> descriptionUsingFoldWithGenerator
// CORRECT "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birth
day'"
christmasPresent |> descriptionUsingFoldWithGenerator
// CORRECT "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

Introducing "foldback"
Now that we understand what to do, let's make a generic version that that handles the
generator function logic for us. This one we will call "foldback":

1510

Introducing Folds

By the way, I'm going to use term "generator" here. In other places, it is commonly referred
to as a "continuation" function, often abbreviated to just "k".
Here's the implementation:
let rec foldbackGift fBook fChocolate fWrapped fBox fCard generator gift :'r =
let recurse = foldbackGift fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
generator (fBook book)
| Chocolate choc ->
generator (fChocolate choc)
| Wrapped (innerGift,style) ->
let newGenerator innerVal =
let newInnerVal = fWrapped innerVal style
generator newInnerVal
recurse newGenerator innerGift
| Boxed innerGift ->
let newGenerator innerVal =
let newInnerVal = fBox innerVal
generator newInnerVal
recurse newGenerator innerGift
| WithACard (innerGift,message) ->
let newGenerator innerVal =
let newInnerVal = fCard innerVal message
generator newInnerVal
recurse newGenerator innerGift

You can see that it is just like the descriptionUsingFoldWithGenerator implementation,

except that we are using generic newInnerVal and generator values.
The type signatures are similar to the original catamorphism, except that every case works
with 'a only now. The only time 'r is used is in the generator function itself!
val foldbackGift :
fBook:(Book -> 'a) ->
fChocolate:(Chocolate -> 'a) ->
fWrapped:('a -> WrappingPaperStyle -> 'a) ->
fBox:('a -> 'a) ->
fCard:('a -> string -> 'a) ->
// accumulator
generator:('a -> 'r) ->
// input value
gift:Gift ->
// return value
'r

The foldback implementation above is written from scratch. If you want a fun exercise, see
if you can write foldback in terms of fold .
1511

Introducing Folds

Let's rewrite the description function using foldback :

let descriptionUsingFoldBack gift =
let fBook (book:Book) =
sprintf "'%s'" book.title
let fChocolate (choc:Chocolate) =
sprintf "%A chocolate" choc.chocType
let fWrapped innerText style =
sprintf "%s wrapped in %A paper" innerText style
let fBox innerText =
sprintf "%s in a box" innerText
let fCard innerText message =
sprintf "%s with a card saying '%s'" innerText message
// initial DescriptionGenerator
let initialAcc = fun innerText -> innerText
// main call
foldbackGift fBook fChocolate fWrapped fBox fCard initialAcc gift

And the results are still correct:

birthdayPresent |> descriptionUsingFoldBack
// CORRECT "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birth
day'"
christmasPresent |> descriptionUsingFoldBack
// CORRECT "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

Comparing foldback to the original catamorphism

The implementation of descriptionUsingFoldBack is almost identical to the version in the last
post that used the original catamorphism cataGift .
Here's the version using cataGift :
let descriptionUsingCata gift =
let fBook (book:Book) =
sprintf "'%s'" book.title
let fChocolate (choc:Chocolate) =
sprintf "%A chocolate" choc.chocType
let fWrapped (innerText,style) =
sprintf "%s wrapped in %A paper" innerText style
let fBox innerText =
sprintf "%s in a box" innerText
let fCard (innerText,message) =
sprintf "%s with a card saying '%s'" innerText message
// call the catamorphism
cataGift fBook fChocolate fWrapped fBox fCard gift

1512

Introducing Folds

And here's the version using foldbackGift :

let descriptionUsingFoldBack gift =
let fBook (book:Book) =
sprintf "'%s'" book.title
let fChocolate (choc:Chocolate) =
sprintf "%A chocolate" choc.chocType
let fWrapped innerText style =
sprintf "%s wrapped in %A paper" innerText style
let fBox innerText =
sprintf "%s in a box" innerText
let fCard innerText message =
sprintf "%s with a card saying '%s'" innerText message
// initial DescriptionGenerator
let initialAcc = fun innerText -> innerText // could be replaced with id
// main call
foldbackGift fBook fChocolate fWrapped fBox fCard initialAcc gift

All the handler functions are basically identical. The only change is the addition of an initial
generator function, which is just id in this case.
However, although the code looks the same in both cases, they differ in their recursion
safety. The foldbackGift version is still tail recursive, and can handle very large nesting
depths, unlike the cataGift version.
But this implementation is not perfect either. The chain of nested functions can get very slow
and generate a lot of garbage, and for this particular example, there is an even faster way,
which we'll look at in the next post.

Changing the type signature of foldback to avoid a

mixup
In foldGift the signature for fWrapped is:
fWrapped:('a -> WrappingPaperStyle -> 'a)

But in foldbackGift the signature for fWrapped is:

fWrapped:('a -> WrappingPaperStyle -> 'a)

Can you spot the difference? No, me neither.

The two functions are very similar, yet work very differently. In the foldGift version, the first
parameter is the accumulator from the outer levels, while in foldbackGift version, the first
parameter is the accumulator from the inner levels. Quite an important distinction!
1513

Introducing Folds

It is therefore common to change the signature of the foldBack version so that the
accumulator always comes last, while in the normal fold function, the accumulator always
comes first.
let rec foldbackGift fBook fChocolate fWrapped fBox fCard gift generator :'r =
//swapped => ~~~~~~~~~~~~~~
let recurse = foldbackGiftWithAccLast fBook fChocolate fWrapped fBox fCard
match gift with
| Book book ->
generator (fBook book)
| Chocolate choc ->
generator (fChocolate choc)
| Wrapped (innerGift,style) ->
let newGenerator innerVal =
let newInnerVal = fWrapped style innerVal
//swapped => ~~~~~~~~~~~~~~
generator newInnerVal
recurse innerGift newGenerator
//swapped => ~~~~~~~~~~~~~~~~~~~~~~
| Boxed innerGift ->
let newGenerator innerVal =
let newInnerVal = fBox innerVal
generator newInnerVal
recurse innerGift newGenerator
//swapped => ~~~~~~~~~~~~~~~~~~~~~~
| WithACard (innerGift,message) ->
let newGenerator innerVal =
let newInnerVal = fCard message innerVal
//swapped => ~~~~~~~~~~~~~~~~
generator newInnerVal
recurse innerGift newGenerator
//swapped => ~~~~~~~~~~~~~~~~~~~~~~

This change shows up in the type signature. The Gift value comes before the accumulator
now:

1514

Introducing Folds

val foldbackGift :
fBook:(Book -> 'a) ->
fChocolate:(Chocolate -> 'a) ->
fWrapped:(WrappingPaperStyle -> 'a -> 'a) ->
fBox:('a -> 'a) ->
fCard:(string -> 'a -> 'a) ->
// input value
gift:Gift ->
// accumulator
generator:('a -> 'r) ->
// return value
'r

and now we can tell the two versions apart easily.

// fold
fWrapped:('a -> WrappingPaperStyle -> 'a)
// foldback
fWrapped:(WrappingPaperStyle -> 'a -> 'a)

Rules for creating a fold

To finish up this post, let's summarize the rules for creating a fold.
In the first post we saw that creating a catamorphism is a mechanical process that follows
rules. The same is true for creating a iterative top-down fold. The process is:
Create a function parameter to handle each case in the structure.
Add an additional parameter as an accumulator.
For non-recursive cases, pass the function parameter the accumulator plus all the data
associated with that case.
For recursive cases, perform two steps:
First, pass the handler the accumulator plus all the data associated with that case
(except the inner recursive data). The result is a new accumulator value.
Then, call the fold recursively on the nested value using the new accumulator
value.
Note that each handler only "sees" (a) the data for that case, and (b) the accumulator
passed to it from the outer level. It does not have access to the results from the inner levels.

Summary
1515

Introducing Folds

We've seen in this post how to define a tail-recursive implementation of a catamorphism,

called "fold" and the reverse version "foldback".
In the next post we'll step back a bit and spend some time understanding what "fold" really
means, and at some guidelines for choosing between fold , foldback and cata .
We'll then see if we can apply these rules to another domain.
The source code for this post is available at this gist.

1516

Understanding Folds

Understanding Folds
This post is the fourth in a series.
In the previous post, I introduced "folds", a way of creating top-down iterative functions for
recursive types.
In this post, we'll spend some time understanding folds in more detail.

1517

Understanding Folds

Part 6: Trees in the real world

Iteration vs. recursion

We now have three different functions -- cata , fold and foldback .
So what exactly are the differences between them? We've seen that something that doesn't
work with fold will work with foldBack , but is there an easy way to remember the
difference?
One way to differentiate the three approaches is by remembering this:
fold is top-down iteration
cata is bottom-up recursion
foldBack is bottom-up iteration

What does this mean?

Well, for in fold , the accumulator was initialized at the top level, and was passed down to
each lower level until the lowest and last level was reached.
In code terms, each level did this:
accumulatorFromHigherLevel, combined with
stuffFromThisLevel
=> stuffToSendDownToNextLowerLevel

In an imperative language, this is exactly a "for loop" with a mutable variable storing the
accumulator.

1518

Understanding Folds

var accumulator = initialValue

foreach level in levels do
{
accumulator, combined with
stuffFromThisLevel
=> update accumulator
}

So, this kind of top-to-bottom folding can be thought of as iteration (and in fact, the F#
compiler will turn a tail-recursive function like this into an iteration behind the scenes).
On the other hand, in cata , the accumulator started at the bottom level, and was passed
up to each higher level until the top level was reached.
In code terms, each level did this:
accumulatorFromLowerLevel, combined with
stuffFromThisLevel
=> stuffToSendUpToNextHigherLevel

This is exactly a recursive loop:

let recurse (head::tail) =
if atBottomLevel then
return something
else // if not at bottom level
let accumulatorFromLowerLevel = recurse tail
return stuffFromThisLevel, combined with
accumulatorFromLowerLevel

Finally, foldback can be thought of as "reverse iteration". The accumulator is threaded

through all the levels, but starting at the bottom rather than at the top. It has the benefits of
cata in that the inner values are calculated first and passed back up, but because it is

iterative, there cannot be a stack overflow.

Many of the concepts we have discussed so far become clear when expressed in terms of
iteration vs. recursion. For example:
The iterative versions ( fold and foldback ) have no stack, and cannot cause a stack
overflow.
The "total cost" function needed no inner data, and so the top-down iterative version
( fold ) worked without problems.
The "description" function though, needed inner text for correct formatting, and so the
recursive version ( cata ) or bottom up iteration ( foldback ) was more suitable.

1519

Understanding Folds

Fold example: File system domain

In the last post, we described some rules for creating folds. Let's see if we can apply these
rules to create a fold in another domain, the "File System" domain from the second post in
the series.
As a reminder, here is the crude "file system" domain from that post:
type FileSystemItem =
| File of File
| Directory of Directory
and File = {name:string; fileSize:int}
and Directory = {name:string; dirSize:int; subitems:FileSystemItem list}

Note that each directory contains a list of subitems, so this is not a linear structure like
Gift , but a tree-like structure. Out implementation of fold will have to take this into account.

Here are some sample values:

We want to create a fold, foldFS , say. So, following the rules, let's add an extra
accumulator parameter acc and pass it to the File case:
let rec foldFS fFile fDir acc item :'r =
let recurse = foldFS fFile fDir
match item with
| File file ->
fFile acc file
| Directory dir ->
// to do

The Directory case is trickier. We are not supposed to know about the subitems, so that
means that the only data we can use is the name , dirSize , and the accumulator passed in
from a higher level. These are combined to make a new accumulator.
| Directory dir ->
let newAcc = fDir acc (dir.name,dir.dirSize)
// to do

1520

Understanding Folds

NOTE: I'm keeping the name and dirSize as a tuple for grouping purposes, but of course
you could pass them in as separate parameters.
Now we need to pass this new accumulator down to each subitem in turn, but each subitem
will return a new accumulator of its own, so we need to use the following approach:
Take the newly created accumulator and pass it to the first subitem.
Take the output of that (another accumulator) and pass it to the second subitem.
Take the output of that (another accumulator) and pass it to the third subitem.
And so on. The output of the last subitem is the final result.
That approach is already available to us though. It's exactly what List.fold does! So
here's the code for the Directory case:
| Directory dir ->
let newAcc = fDir acc (dir.name,dir.dirSize)
dir.subitems |> List.fold recurse newAcc

And here's the entire foldFS function:

let rec foldFS fFile fDir acc item :'r =
let recurse = foldFS fFile fDir
match item with
| File file ->
fFile acc file
| Directory dir ->
let newAcc = fDir acc (dir.name,dir.dirSize)
dir.subitems |> List.fold recurse newAcc

With this in place, we can rewrite the same two functions we implemented in the last post.
First, the totalSize function, which just sums up all the sizes:
let totalSize fileSystemItem =
let fFile acc (file:File) =
acc + file.fileSize
let fDir acc (name,size) =
acc + size
foldFS fFile fDir 0 fileSystemItem

And if we test it we get the same results as before:

readme |> totalSize // 1
src |> totalSize // 16 = 10 + (1 + 2 + 3)
root |> totalSize // 31 = 5 + 16 + 10

1521

Understanding Folds

File system domain: largestFile example

We can also reimplement the "what is the largest file in the tree?" function.
As before it will return a File option , because the tree might be empty. This means that the
accumulator will be a File option too.
This time it is the File case handler which is tricky:
If the accumulator being passed in is None , then this current file becomes the new
accumulator.
If the accumulator being passed in is Some file , then compare the size of that file with
this file. Whichever is bigger becomes the new accumulator.
Here's the code:
let fFile (largestSoFarOpt:File option) (file:File) =
match largestSoFarOpt with
| None ->
Some file
| Some largestSoFar ->
if largestSoFar.fileSize > file.fileSize then
Some largestSoFar
else
Some file

On the other hand, the Directory handler is trivial -- just pass the "largest so far"
accumulator down to the next level
let fDir largestSoFarOpt (name,size) =
largestSoFarOpt

Here's the complete implementation:

1522

Understanding Folds

let largestFile fileSystemItem =

let fFile (largestSoFarOpt:File option) (file:File) =
match largestSoFarOpt with
| None ->
Some file
| Some largestSoFar ->
if largestSoFar.fileSize > file.fileSize then
Some largestSoFar
else
Some file
let fDir largestSoFarOpt (name,size) =
largestSoFarOpt
// call the fold
foldFS fFile fDir None fileSystemItem

And if we test it we get the same results as before:

It is interesting to compare this implementation with the recursive version in the second post.
I think that this one is easier to implement, myself.

Tree traversal types

The various fold functions discussed so far correspond to various kinds of tree traversals:
A fold function (as implemented here) is more properly called a "pre-order depth-first"
tree traversal.
A foldback function would be a "post-order depth-first" tree traversal.
A cata function would not be a "traversal" at all, because each internal node deals with
a list of all the subresults at once.
By tweaking the logic, you can make other variants.
For a description of the various kinds of tree traversals, see Wikipedia.

1523

Understanding Folds

Do we need foldback ?
Do we need to implement a foldback function for the FileSystem domain?
I don't think so. If we need access to the inner data, we can just use the original "naive"
catamorphism implementation in the previous post.
But, hey wait, didn't I say at the beginning that we had to watch out for stack overflows?
Yes, if the recursive type is deeply nested. But consider a file system with only two
subdirectories per directory. How many directories would there be if there were 64 nested
levels? (Hint: you may be familiar with a similar problem. Something to do with grains on a
chessboard).
We saw earlier that the stack overflow issue only occurs with more than 1000 nested levels,
and that level of nesting generally only occurs with linear recursive types, not trees like the
FileSystem domain.

Common questions about "fold"

At this point you might be getting overwhelmed! All these different implementations with
different advantages and disadvantages.
So let's take a short break and address some common questions.

What's the difference between "left fold" and "right fold"

There is often quite a lot of confusion around the terminology of folds: "left" vs. "right",
"forward" vs. "backwards", etc.
A left fold or forward fold is what I have just called fold -- the top-down iterative
approach.
A right fold or backward fold is what I have called foldBack -- the bottom-up iterative
approach.
These terms, though, really only apply to linear recursive structures like Gift . When it
comes to more complex tree-like structures, these distinctions are too simple, because there
are many ways to traverse them: breadth-first, depth-first, pre-order and post-order, and so
on.

Which type of fold function should I use?

Here are some guidelines:

1524

Understanding Folds

If your recursive type is not going to be too deeply nested (less than 100 levels deep,
say), then the naive cata catamorphism we described in the first post is fine. It's really
easy to implement mechanically -- just replace the main recursive type with 'r .
If your recursive type is going to be deeply nested and you want to prevent stack
overflows, use the iterative fold .
If you are using an iterative fold but you need to have access to the inner data, pass a
continuation function as an accumulator.
Finally, the iterative approach is generally faster and uses less memory than the
recursive approach (but that advantage is lost if you pass around too many nested
continuations).
Another way to think about it is to look at your "combiner" function. At each step, you are
combining data from the different levels:
level1 data [combined with] level2 data [combined with] level3 data [combined with] le
vel4 data

If your combiner function is "left associative" like this:

(((level1 combineWith level2) combineWith level3) combineWith level4)

then use the iterative approach, but if your combiner function is "right associative" like this:
(level1 combineWith (level2 combineWith (level3 combineWith level4)))

then use the cata or foldback approach.

And if your combiner function doesn't care (like addition, for example), use whichever one is
more convenient.

How can I know whether code is tail-recursive or not?

It's not always obvious whether an implementation is tail-recursive or not. The easiest way to
be sure is to look at the very last expression for each case.
If the call to "recurse" is the very last expression, then it is tail-recursive. If there is any other
work after that, then it is not tail-recursive.
See for yourself with the three implementations that we have discussed.
First, here's the code for the WithACard case in the original cataGift implementation:

1525

Understanding Folds

| WithACard (gift,message) ->

fCard (recurse gift,message)
// ~~~~~~~ <= Call to recurse is not last expression.
// Tail-recursive? No!

The cataGift implementation is not tail-recursive.

Here's the code from the foldGift implementation:
| WithACard (innerGift,message) ->
let newAcc = fCard acc message
recurse newAcc innerGift
// ~~~~~~~ <= Call to recurse is last expression.
// Tail-recursive? Yes!

The foldGift implementation is tail-recursive.

And here's the code from the foldbackGift implementation:
| WithACard (innerGift,message) ->
let newGenerator innerVal =
let newInnerVal = fCard innerVal message
generator newInnerVal
recurse newGenerator innerGift
// ~~~~~~~ <= Call to recurse is last expression.
// Tail-recursive? Yes!

The foldbackGift implementation is also tail-recursive.

How do I short-circuit a fold?

In a language like C#, you can exit a iterative loop early using break like this:
foreach (var elem in collection)
{
// do something
if ( x == "error")
{
break;
}
}

So how do you do the same thing with a fold?

1526

Understanding Folds

The short answer is, you can't! A fold is designed to visit all elements in turn. The Visitor
Pattern has the same constraint.
There are three workarounds.
The first one is to not use fold at all and create your own recursive function that terminates
on the required condition:
In this example, the loop exits when the sum is larger than 100:
let rec firstSumBiggerThan100 sumSoFar listOfInts =
match listOfInts with
| [] ->
sumSoFar // exhausted all the ints!
| head::tail ->
let newSumSoFar = head + sumSoFar
if newSumSoFar > 100 then
newSumSoFar
else
firstSumBiggerThan100 newSumSoFar tail
// test
[30;40;50;60] |> firstSumBiggerThan100 0 // 120
[1..3..100] |> firstSumBiggerThan100 0 // 117

The second approach is to use fold but to add some kind of "ignore" flag to the
accumulator that is passed around. Once this flag is set, the remaining iterations do nothing.
Here's an example of calculating the sum, but the accumulator is actually a tuple with an
ignoreFlag in addition to the sumSoFar :

1527

Understanding Folds

let firstSumBiggerThan100 listOfInts =

let folder accumulator i =
let (ignoreFlag,sumSoFar) = accumulator
if not ignoreFlag then
let newSumSoFar = i + sumSoFar
let newIgnoreFlag = newSumSoFar > 100
(newIgnoreFlag, newSumSoFar)
else
// pass the accumulator along
accumulator
let initialAcc = (false,0)
listOfInts
|> List.fold folder initialAcc // use fold
|> snd // get the sumSoFar
/// test
[30;40;50;60] |> firstSumBiggerThan100 // 120
[1..3..100] |> firstSumBiggerThan100 // 117

The third version is a variant of the second -- create a special value to signal that the
remaining data should be ignored, but wrap it in a computation expression so that it looks
more natural.
This approach is documented on Tomas Petricek's blog and the code looks like this:
let firstSumBiggerThan100 listOfInts =
let mutable sumSoFar = 0
imperative {
for x in listOfInts do
sumSoFar <- x + sumSoFar
if sumSoFar > 100 then do! break
}
sumSoFar

Summary
The goal of this post was to help you understand folds better, and to show how they could be
applied to a tree structure like the file system. I hope it was helpful!
Up to this point in the series all the examples have been very concrete; we have
implemented custom folds for each domain we have encountered. Can we be a bit more
generic and build some reusable fold implementations?

1528

Understanding Folds

In the next post we'll look at generic recursive types, and how to work with them.
The source code for this post is available at this gist.

1529

Generic recursive types

This post is the fifth in a series.
In the previous post, we spent some time understanding folds for specific domain types.
In this post, we'll broaden our horizons and look at how to use generic recursive types.

1530

Generic recursive types

Defining a generic Tree type

The Tree type in the real world
Mapping the Tree type
Example: Creating a directory listing
Example: A parallel grep
Example: Storing the file system in a database
Example: Serializing a Tree to JSON
Example: Deserializing a Tree from JSON
Example: Deserializing a Tree from JSON - with error handling

LinkedList: A generic recursive type

Here's a question: if you only have algebraic types, and you can only combine them as
products (tuples, records) or sums (discriminated unions), then how can you make a list type
just by using these operations?
The answer is, of course, recursion!
Let's start with the most basic recursive type: the list.
I'm going to call my version LinkedList , but it is basically the same as the list type in F#.
So, how do you define a list in a recursive way?
Well, it's either empty, or it consists of an element plus another list. In other words we can
define it as a choice type ("discriminated union") like this:
type LinkedList<'a> =
| Empty
| Cons of head:'a * tail:LinkedList<'a>

The Empty case represents an empty list. The Cons case has a tuple: the head element,
and the tail, which is another list.
We can then define a particular LinkedList value like this:
let linkedList = Cons (1, Cons (2, Cons(3, Empty)))

Using the native F# list type, the equivalent definition would be:
let linkedList = 1 :: 2 :: 3 :: []

1531

Generic recursive types

which is just [1; 2; 3]

cata for LinkedList

Following the rules in the first post in this series, we can mechanically create a cata
function by replacing Empty and Cons with fEmpty and fCons :
module LinkedList =
let rec cata fCons fEmpty list :'r=
let recurse = cata fCons fEmpty
match list with
| Empty ->
fEmpty
| Cons (element,list) ->
fCons element (recurse list)

Note: We will be putting all the functions associated with LinkedList<'a> in a module called
LinkedList . One nice thing about using generic types is that the type name does not clash

with a similar module name!

As always, the signatures of the case handling functions are parallel to the signatures of the
type constructors, with LinkedList replaced by 'r .
val cata :
fCons:('a -> 'r -> 'r) ->
fEmpty:'r ->
list:LinkedList<'a>
-> 'r

fold for LinkedList

We can also create a top-down iterative fold function using the rules in the earlier post.

1532

Generic recursive types

module LinkedList =
let rec cata ...
let rec foldWithEmpty fCons fEmpty acc list :'r=
let recurse = foldWithEmpty fCons fEmpty
match list with
| Empty ->
fEmpty acc
| Cons (element,list) ->
let newAcc = fCons acc element
recurse newAcc list

This foldWithEmpty function is not quite the same as the standard List.fold function,
because it has an extra function parameter for the empty case ( fEmpty ). However, if we
eliminate that parameter and just return the accumulator we get this variant:
module LinkedList =
let rec fold fCons acc list :'r=
let recurse = fold fCons
match list with
| Empty ->
acc
| Cons (element,list) ->
let newAcc = fCons acc element
recurse newAcc list

If we compare the signature with the List.fold documentation we can see that they are
equivalent, with 'State replaced by 'r and 'T list replaced by LinkedList<'a> :
LinkedList.fold : ('r -> 'a -> 'r ) -> 'r -> LinkedList<'a> -> 'r
List.fold : ('State -> 'T -> 'State) -> 'State -> 'T list -> 'State

Let's test that fold works by doing a small sum:

let linkedList = Cons (1, Cons (2, Cons(3, Empty)))
linkedList |> LinkedList.fold (+) 0
// Result => 6

foldBack for LinkedList

Finally we can create a foldBack function, using the "function accumulator" approach
described in the previous post:

1533

Generic recursive types

module LinkedList =
let rec cata ...
let rec fold ...
let foldBack fCons list acc :'r=
let fEmpty' generator =
generator acc
let fCons' generator element=
fun innerResult ->
let newResult = fCons element innerResult
generator newResult
let initialGenerator = id
foldWithEmpty fCons' fEmpty' initialGenerator list

Again, if we compare the signature with the List.foldBack documentation, they are also
equivalent, with 'State replaced by 'r and 'T list replaced by LinkedList<'a> :
LinkedList.foldBack : ('a -> 'r -> 'r ) -> LinkedList<'a> -> 'r -> 'r
List.foldBack : ('T -> 'State -> 'State) -> 'T list -> 'State -> 'State

Using foldBack to convert between list types

In the first post we noted that catamorphisms could be used for converting between types of
similar structure.
Let's demonstrate that now by creating some functions that convert from LinkedList to the
native list type and back again.
To convert a LinkedList to a native list all we need to do is replace Cons with :: and
Empty with [] :

module LinkedList =
let toList linkedList =
let fCons head tail = head::tail
let initialState = []
foldBack fCons linkedList initialState

To convert the other way, we need to replace :: with Cons and [] with Empty :

1534

Generic recursive types

module LinkedList =
let ofList list =
let fCons head tail = Cons(head,tail)
let initialState = Empty
List.foldBack fCons list initialState

Simple! Let's test toList :

let linkedList = Cons (1, Cons (2, Cons(3, Empty)))
linkedList |> LinkedList.toList
// Result => [1; 2; 3]

and ofList :
let list = [1;2;3]
list |> LinkedList.ofList
// Result => Cons (1,Cons (2,Cons (3,Empty)))

Both work as expected.

Using foldBack to implement other functions

I said earlier that a catamorphism function (for linear lists, foldBack ) is the most basic
function available for a recursive type, and in fact is the only function you need!
Let's see for ourselves by implementing some other common functions using foldBack .
Here's map defined in terms of foldBack :
module LinkedList =
/// map a function "f" over all elements
let map f list =
// helper function
let folder head tail =
Cons(f head,tail)
foldBack folder list Empty

And here's a test:

1535

Generic recursive types

let linkedList = Cons (1, Cons (2, Cons(3, Empty)))

linkedList |> LinkedList.map (fun i -> i+10)
// Result => Cons (11,Cons (12,Cons (13,Empty)))

Here's filter defined in terms of foldBack :

module LinkedList =
/// return a new list of elements for which "pred" is true
let filter pred list =
// helper function
let folder head tail =
if pred head then
Cons(head,tail)
else
tail
foldBack folder list Empty

And here's a test:

let isOdd n = (n%2=1)
let linkedList = Cons (1, Cons (2, Cons(3, Empty)))
linkedList |> LinkedList.filter isOdd
// Result => Cons (1,Cons (3,Empty))

Finally, here's rev defined in terms of fold :

/// reverse the elements of the list
let rev list =
// helper function
let folder tail head =
Cons(head,tail)
fold folder Empty list

And here's a test:

let linkedList = Cons (1, Cons (2, Cons(3, Empty)))
linkedList |> LinkedList.rev
// Result => Cons (3,Cons (2,Cons (1,Empty)))

So, I hope you're convinced!

1536

Generic recursive types

Avoiding generator functions

I mentioned earlier that there was an alternative and (sometimes) more efficient way to
implement foldBack without using generators or continuations.
As we have seen, foldBack is reverse iteration, which means that it is the same as fold
applied to a reversed list!
So we could implement it like this:
let foldBack_ViaRev fCons list acc :'r=
let fCons' acc element =
// just swap the params!
fCons element acc
list
|> rev
|> fold fCons' acc

It involves making an extra copy of the list, but on the other hand there is no longer a large
set of pending continuations. It might be worth comparing the profile of the two versions in
your environment if performance is an issue.

Making the Gift domain generic

In the rest of this post, we'll look at the Gift type and see if we can make it more generic.
As a reminder, here is the original design:
type Gift =
| Book of Book
| Chocolate of Chocolate
| Wrapped of Gift * WrappingPaperStyle
| Boxed of Gift
| WithACard of Gift * message:string

Three of the cases are recursive and two are non-recursive.

Now, the focus of this particular design was on modelling the domain, which is why there are
so many separate cases.
But if we want to focus on reusability instead of domain modelling, then we should simplify
the design to the essentials, and all these special cases now become a hindrance.

1537

Generic recursive types

To make this ready for reuse, then, let's collapse all the non-recursive cases into one case,
say GiftContents , and all the recursive cases into another case, say GiftDecoration , like
this:
// unified data for non-recursive cases
type GiftContents =
| Book of Book
| Chocolate of Chocolate
// unified data for recursive cases
type GiftDecoration =
| Wrapped of WrappingPaperStyle
| Boxed
| WithACard of string
type Gift =
// non-recursive case
| Contents of GiftContents
// recursive case
| Decoration of Gift * GiftDecoration

The main Gift type has only two cases now: the non-recursive one and the recursive one.

Defining a generic Container type

Now that the type is simplified, we can "genericize" it by allowing any kind of contents and
any kind of decoration.
type Container<'ContentData,'DecorationData> =
| Contents of 'ContentData
| Decoration of 'DecorationData * Container<'ContentData,'DecorationData>

And as before, we can mechanically create a cata and fold and foldBack for it, using
the standard process:
module Container =
let rec cata fContents fDecoration (container:Container<'ContentData,'DecorationDa
ta>) :'r =
let recurse = cata fContents fDecoration
match container with
| Contents contentData ->
fContents contentData
| Decoration (decorationData,subContainer) ->
fDecoration decorationData (recurse subContainer)

1538

Generic recursive types

(*
val cata :
// function parameters
fContents:('ContentData -> 'r) ->
fDecoration:('DecorationData -> 'r -> 'r) ->
// input
container:Container<'ContentData,'DecorationData> ->
// return value
'r
*)
let rec fold fContents fDecoration acc (container:Container<'ContentData,'Decorati
onData>) :'r =
let recurse = fold fContents fDecoration
match container with
| Contents contentData ->
fContents acc contentData
| Decoration (decorationData,subContainer) ->
let newAcc = fDecoration acc decorationData
recurse newAcc subContainer
(*
val fold :
// function parameters
fContents:('a -> 'ContentData -> 'r) ->
fDecoration:('a -> 'DecorationData -> 'a) ->
// accumulator
acc:'a ->
// input
container:Container<'ContentData,'DecorationData> ->
// return value
'r
*)
let foldBack fContents fDecoration (container:Container<'ContentData,'DecorationDa
ta>) :'r =
let fContents' generator contentData =
generator (fContents contentData)
let fDecoration' generator decorationData =
let newGenerator innerValue =
let newInnerValue = fDecoration decorationData innerValue
generator newInnerValue
newGenerator
fold fContents' fDecoration' id container
(*
val foldBack :
// function parameters
fContents:('ContentData -> 'r) ->
fDecoration:('DecorationData -> 'r -> 'r) ->
// input
container:Container<'ContentData,'DecorationData> ->
// return value

1539

Generic recursive types

'r
*)

Converting the gift domain to use the Container type

Let's convert the gift type to this generic Container type:
type Gift = Container<GiftContents,GiftDecoration>

Now we need some helper methods to construct values while hiding the "real" cases of the
generic type:
let fromBook book =
Contents (Book book)
let fromChoc choc =
Contents (Chocolate choc)
let wrapInPaper paperStyle innerGift =
let container = Wrapped paperStyle
Decoration (container, innerGift)
let putInBox innerGift =
let container = Boxed
Decoration (container, innerGift)
let withCard message innerGift =
let container = WithACard message
Decoration (container, innerGift)

Finally we can create some test values:

1540

Generic recursive types

The totalCost function using the Container type

The "total cost" function can be written using fold , since it doesn't need any inner data.
Unlike the earlier implementations, we only have two function parameters, fContents and
fDecoration , so each of these will need some pattern matching to get at the "real" data.

Here's the code:

let totalCost gift =
let fContents costSoFar contentData =
match contentData with
| Book book ->
costSoFar + book.price
| Chocolate choc ->
costSoFar + choc.price
let fDecoration costSoFar decorationInfo =
match decorationInfo with
| Wrapped style ->
costSoFar + 0.5m
| Boxed ->
costSoFar + 1.0m
| WithACard message ->
costSoFar + 2.0m
// initial accumulator
let initialAcc = 0m
// call the fold
Container.fold fContents fDecoration initialAcc gift

And the code works as expected:

birthdayPresent |> totalCost
// 22.5m
christmasPresent |> totalCost
// 6.5m

The description function using the Container type

The "description" function needs to be written using foldBack , since it does need the inner
data. As with the code above, we need some pattern matching to get at the "real" data for
each case.

1541

Generic recursive types

let description gift =

let fContents contentData =
match contentData with
| Book book ->
sprintf "'%s'" book.title
| Chocolate choc ->
sprintf "%A chocolate" choc.chocType
let fDecoration decorationInfo innerText =
match decorationInfo with
| Wrapped style ->
sprintf "%s wrapped in %A paper" innerText style
| Boxed ->
sprintf "%s in a box" innerText
| WithACard message ->
sprintf "%s with a card saying '%s'" innerText message
// main call
Container.foldBack fContents fDecoration gift

And again the code works as we want:

birthdayPresent |> description
// CORRECT "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birth
day'"
christmasPresent |> description
// CORRECT "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

A third way to implement the gift domain

That all looks quite nice, doesn't it?
But I have to confess that I have been holding something back.
None of that code above was strictly necessary, because it turns out that there is yet another
way to model a Gift , without creating any new generic types at all!
The Gift type is basically a linear sequence of decorations, with some content as the final
step. We can just model this as a pair -- a Content and a list of Decoration . Or to make it a
little friendlier, a record with two fields: one for the content and one for the decorations.
type Gift = {contents: GiftContents; decorations: GiftDecoration list}

That's it! No other new types needed!

1542

Generic recursive types

Building values using the record type

As before, let's create some helpers to construct values using this type:
let fromBook book =
{ contents = (Book book); decorations = [] }
let fromChoc choc =
{ contents = (Chocolate choc); decorations = [] }
let wrapInPaper paperStyle innerGift =
let decoration = Wrapped paperStyle
{ innerGift with decorations = decoration::innerGift.decorations }
let putInBox innerGift =
let decoration = Boxed
{ innerGift with decorations = decoration::innerGift.decorations }
let withCard message innerGift =
let decoration = WithACard message
{ innerGift with decorations = decoration::innerGift.decorations }

With these helper functions, the way the values are constructed is identical to the previous
version. This is why it is good to hide your raw constructors, folks!
let wolfHall = {title="Wolf Hall"; price=20m}
let yummyChoc = {chocType=SeventyPercent; price=5m}
let birthdayPresent =
wolfHall
|> fromBook
|> wrapInPaper HappyBirthday
|> withCard "Happy Birthday"
let christmasPresent =
yummyChoc
|> fromChoc
|> putInBox
|> wrapInPaper HappyHolidays

The totalCost function using the record type

The totalCost function is even easier to write now.

1543

Generic recursive types

let totalCost gift =

let contentCost =
match gift.contents with
| Book book ->
book.price
| Chocolate choc ->
choc.price
let decorationFolder costSoFar decorationInfo =
match decorationInfo with
| Wrapped style ->
costSoFar + 0.5m
| Boxed ->
costSoFar + 1.0m
| WithACard message ->
costSoFar + 2.0m
let decorationCost =
gift.decorations |> List.fold decorationFolder 0m
// total cost
contentCost + decorationCost

The description function using the record type

Similarly, the description function is also easy to write.
let description gift =
let contentDescription =
match gift.contents with
| Book book ->
sprintf "'%s'" book.title
| Chocolate choc ->
sprintf "%A chocolate" choc.chocType
let decorationFolder decorationInfo innerText =
match decorationInfo with
| Wrapped style ->
sprintf "%s wrapped in %A paper" innerText style
| Boxed ->
sprintf "%s in a box" innerText
| WithACard message ->
sprintf "%s with a card saying '%s'" innerText message
List.foldBack decorationFolder gift.decorations contentDescription

1544

Generic recursive types

Abstract or concrete? Comparing the three

designs
If you are confused by this plethora of designs, I don't blame you!
But as it happens, the three different definitions are actually interchangable:
The original version
type Gift =
| Book of Book
| Chocolate of Chocolate
| Wrapped of Gift * WrappingPaperStyle
| Boxed of Gift
| WithACard of Gift * message:string

The generic container version

The record version

type GiftContents =
| Book of Book
| Chocolate of Chocolate
type GiftDecoration =
| Wrapped of WrappingPaperStyle
| Boxed
| WithACard of string
type Gift = {contents: GiftContents; decorations: GiftDecoration list}

1545

Generic recursive types

If this is not obvious, it might be helpful to read my post on data type sizes. It explains how
two types can be "equivalent", even though they appear to be completely different at first
glance.

Picking a design
So which design is best? The answer is, as always, "it depends".
For modelling and documenting a domain, I like the first design with the five explicit cases.
Being easy for other people to understand is more important to me than introducing
abstraction for the sake of reusability.
If I wanted a reusable model that was applicable in many situations, I'd probably choose the
second ("Container") design. It seems to me that this type does represent a commonly
encountered situation, where the contents are one kind of thing and the wrappers are
another kind of thing. This abstraction is therefore likely to get some use.
The final "pair" model is fine, but by separating the two components, we've over-abstracted
the design for this scenario. In other situations, this design might be a great fit (e.g. the
decorator pattern), but not here, in my opinion.
There is one further choice which gives you the best of all worlds.
As I noted above, all the designs are logically equivalent, which means there are "lossless"
mappings between them. In that case, your "public" design can be the domain-oriented one,
like the first one, but behind the scenes you can map it to a more efficient and reusable
"private" type.
Even the F# list implementation itself does this. For example, some of the functions in the
List module, such foldBack and sort , convert the list into an array, do the operations,

and then convert it back to a list again.

Summary
In this post we looked at some ways of modelling the Gift as a generic type, and the pros
and cons of each approach.
In the next post we'll look at real-world examples of using a generic recursive type.
The source code for this post is available at this gist.

1546

Generic recursive types

1547

Trees in the real world

This post is the sixth in a series.
In the previous post, we briefly looked at some generic types.
In this post, we'll do some deeper dives into some real-world examples of using trees and
folds.

1548

Trees in the real world

Part 6: Trees in the real world

Defining a generic Tree type

In this post, we'll be working with a generic Tree inspired by the FileSystem domain that
we explored earlier.
Here was the original design:
type FileSystemItem =
| File of FileInfo
| Directory of DirectoryInfo
and FileInfo = {name:string; fileSize:int}
and DirectoryInfo = {name:string; dirSize:int; subitems:FileSystemItem list}

We can separate out the data from the recursion, and create a generic Tree type like this:
type Tree<'LeafData,'INodeData> =
| LeafNode of 'LeafData
| InternalNode of 'INodeData * Tree<'LeafData,'INodeData> seq

Notice that I have used seq to represent the subitems rather than list . The reason for
this will become apparent shortly.
The file system domain can then be modelled using Tree by specifying FileInfo as data
associated with a leaf node and DirectoryInfo as data associated with an internal node:
type FileInfo = {name:string; fileSize:int}
type DirectoryInfo = {name:string; dirSize:int}
type FileSystemItem = Tree<FileInfo,DirectoryInfo>

1549

Trees in the real world

cata and fold for Tree

We can define cata and fold in the usual way:
module Tree =
let rec cata fLeaf fNode (tree:Tree<'LeafData,'INodeData>) :'r =
let recurse = cata fLeaf fNode
match tree with
| LeafNode leafInfo ->
fLeaf leafInfo
| InternalNode (nodeInfo,subtrees) ->
fNode nodeInfo (subtrees |> Seq.map recurse)
let rec fold fLeaf fNode acc (tree:Tree<'LeafData,'INodeData>) :'r =
let recurse = fold fLeaf fNode
match tree with
| LeafNode leafInfo ->
fLeaf acc leafInfo
| InternalNode (nodeInfo,subtrees) ->
// determine the local accumulator at this level
let localAccum = fNode acc nodeInfo
// thread the local accumulator through all the subitems using Seq.fold
let finalAccum = subtrees |> Seq.fold recurse localAccum
// ... and return it
finalAccum

Note that I am not going to implement foldBack for the Tree type, because it's unlikely
that the tree will get so deep as to cause a stack overflow. Functions that need inner data
can use cata .

Modelling the File System domain with Tree

Let's test it with the same values that we used before:
let fromFile (fileInfo:FileInfo) =
LeafNode fileInfo
let fromDir (dirInfo:DirectoryInfo) subitems =
InternalNode (dirInfo,subitems)
let readme = fromFile {name="readme.txt"; fileSize=1}
let config = fromFile {name="config.xml"; fileSize=2}
let build = fromFile {name="build.bat"; fileSize=3}
let src = fromDir {name="src"; dirSize=10} [readme; config; build]
let bin = fromDir {name="bin"; dirSize=10} []
let root = fromDir {name="root"; dirSize=5} [src; bin]

1550

Trees in the real world

The totalSize function is almost identical to the one in the previous post:
let totalSize fileSystemItem =
let fFile acc (file:FileInfo) =
acc + file.fileSize
let fDir acc (dir:DirectoryInfo)=
acc + dir.dirSize
Tree.fold fFile fDir 0 fileSystemItem
readme |> totalSize // 1
src |> totalSize // 16 = 10 + (1 + 2 + 3)
root |> totalSize // 31 = 5 + 16 + 10

And so is the largestFile function:

let largestFile fileSystemItem =
let fFile (largestSoFarOpt:FileInfo option) (file:FileInfo) =
match largestSoFarOpt with
| None ->
Some file
| Some largestSoFar ->
if largestSoFar.fileSize > file.fileSize then
Some largestSoFar
else
Some file
let fDir largestSoFarOpt dirInfo =
largestSoFarOpt
// call the fold
Tree.fold fFile fDir None fileSystemItem
readme |> largestFile
// Some {name = "readme.txt"; fileSize = 1}
src |> largestFile
// Some {name = "build.bat"; fileSize = 3}
bin |> largestFile
// None
root |> largestFile
// Some {name = "build.bat"; fileSize = 3}

The source code for this section is available at this gist.

The Tree type in the real world

1551

Trees in the real world

We can use the Tree to model the real file system too! To do this, just set the leaf node
type to System.IO.FileInfo and the internal node type to System.IO.DirectoryInfo .
open System
open System.IO
type FileSystemTree = Tree<IO.FileInfo,IO.DirectoryInfo>

And let's create some helper methods to create the various nodes:
let fromFile (fileInfo:FileInfo) =
LeafNode fileInfo
let rec fromDir (dirInfo:DirectoryInfo) =
let subItems = seq{
yield! dirInfo.EnumerateFiles() |> Seq.map fromFile
yield! dirInfo.EnumerateDirectories() |> Seq.map fromDir
}
InternalNode (dirInfo,subItems)

Now you can see why I used seq rather than list for the subitems. The seq is lazy,
which means that we can create nodes without actually hitting the disk.
Here's the totalSize function again, this time using the real file information:
let totalSize fileSystemItem =
let fFile acc (file:FileInfo) =
acc + file.Length
let fDir acc (dir:DirectoryInfo)=
acc
Tree.fold fFile fDir 0L fileSystemItem

Let's see what the size of the current directory is:

// set the current directory to the current source directory
Directory.SetCurrentDirectory __SOURCE_DIRECTORY__
// get the current directory as a Tree
let currentDir = fromDir (DirectoryInfo("."))
// get the size of the current directory
currentDir |> totalSize

Similarly, we can get the largest file:

1552

Trees in the real world

let largestFile fileSystemItem =

let fFile (largestSoFarOpt:FileInfo option) (file:FileInfo) =
match largestSoFarOpt with
| None ->
Some file
| Some largestSoFar ->
if largestSoFar.Length > file.Length then
Some largestSoFar
else
Some file
let fDir largestSoFarOpt dirInfo =
largestSoFarOpt
// call the fold
Tree.fold fFile fDir None fileSystemItem
currentDir |> largestFile

So that's one big benefit of using generic recursive types. If we can turn a real-world
hierarchy into our tree structure, we can get all the benefits of fold "for free".

Mapping with generic types

One other advantage of using generic types is that you can do things like map -- converting
every element to a new type without changing the structure.
We can see this in action with the real world file system. But first we need to define map for
the Tree type!
The implementation of map can also be done mechanically, using the following rules:
Create a function parameter to handle each case in the structure.
For non-recursive cases
First, use the function parameter to transform the non-recursive data associated
with that case
Then wrap the result in the same case constructor
For recursive cases, perform two steps:
First, use the function parameter to transform the non-recursive data associated
with that case
Next, recursively map the nested values.
Finally, wrap the results in the same case constructor
Here's the implementation of map for Tree , created by following those rules:

1553

Trees in the real world

module Tree =
let rec cata ...
let rec fold ...
let rec map fLeaf fNode (tree:Tree<'LeafData,'INodeData>) =
let recurse = map fLeaf fNode
match tree with
| LeafNode leafInfo ->
let newLeafInfo = fLeaf leafInfo
LeafNode newLeafInfo
| InternalNode (nodeInfo,subtrees) ->
let newNodeInfo = fNode nodeInfo
let newSubtrees = subtrees |> Seq.map recurse
InternalNode (newNodeInfo, newSubtrees)

If we look at the signature of Tree.map , we can see that all the leaf data is transformed to
type 'a , all the node data is transformed to type 'b , and the final result is a Tree<'a,'b> .
val map :
fLeaf:('LeafData -> 'a) ->
fNode:('INodeData -> 'b) ->
tree:Tree<'LeafData,'INodeData> ->
Tree<'a,'b>

We can define Tree.iter in a similar way:

module Tree =
let rec map ...
let rec iter fLeaf fNode (tree:Tree<'LeafData,'INodeData>) =
let recurse = iter fLeaf fNode
match tree with
| LeafNode leafInfo ->
fLeaf leafInfo
| InternalNode (nodeInfo,subtrees) ->
subtrees |> Seq.iter recurse
fNode nodeInfo

Example: Creating a directory listing

1554

Trees in the real world

Let's say we want to use map to transform the file system into a directory listing - a tree of
strings where each string has information about the corresponding file or directory. Here's
how we could do it:
let dirListing fileSystemItem =
let printDate (d:DateTime) = d.ToString()
let mapFile (fi:FileInfo) =
sprintf "%10i %s %-s" fi.Length (printDate fi.LastWriteTime) fi.Name
let mapDir (di:DirectoryInfo) =
di.FullName
Tree.map mapFile mapDir fileSystemItem

And then we can print the strings out like this:

currentDir
|> dirListing
|> Tree.iter (printfn "%s") (printfn "\n%s")

The results will look something like this:

8315 10/08/2015 23:37:41 Fold.fsx
3680 11/08/2015 23:59:01 FoldAndRecursiveTypes.fsproj
1010 11/08/2015 01:19:07 FoldAndRecursiveTypes.sln
1107 11/08/2015 23:59:01 HtmlDom.fsx
79 11/08/2015 01:21:54 LinkedList.fsx

The source code for this example is available at this gist.

Example: Creating a parallel grep

Let's look at a more complex example. I'll demonstrate how to create a parallel "grep" style
search using fold .
The logic will be like this:
Use fold to iterate through the files.
For each file, if its name doesn't match the desired file pattern, return None .
If the file is to be processed, then return an async that returns all the line matches in the
file.
Next, all these asyncs -- the output of the fold -- are aggregated into a sequence.
The sequence of asyncs is transformed into a single one using Async.Parallel which
returns a list of results.

1555

Trees in the real world

Before we start writing the main code, we'll need some helper functions.
First, a generic function that folds over the lines in a file asynchronously. This will be the
basis of the pattern matching.
/// Fold over the lines in a file asynchronously
/// passing in the current line and line number tothe folder function.
///
/// Signature:
/// folder:('a -> int -> string -> 'a) ->
/// acc:'a ->
/// fi:FileInfo ->
/// Async<'a>
let foldLinesAsync folder acc (fi:FileInfo) =
async {
let mutable acc = acc
let mutable lineNo = 1
use sr = new StreamReader(path=fi.FullName)
while not sr.EndOfStream do
let! lineText = sr.ReadLineAsync() |> Async.AwaitTask
acc <- folder acc lineNo lineText
lineNo <- lineNo + 1
return acc
}

Next, a little helper that allows us to map over Async values:

let asyncMap f asyncX = async {
let! x = asyncX
return (f x) }

Now for the central logic. We will create a function that, given a textPattern and a
FileInfo , will return a list of lines that match the textPattern, but asynchronously:

1556

Trees in the real world

/// return the matching lines in a file, as an async<string list>

let matchPattern textPattern (fi:FileInfo) =
// set up the regex
let regex = Text.RegularExpressions.Regex(pattern=textPattern)
// set up the function to use with "fold"
let folder results lineNo lineText =
if regex.IsMatch lineText then
let result = sprintf "%40s:%-5i %s" fi.Name lineNo lineText
result :: results
else
// pass through
results
// main flow
fi
|> foldLinesAsync folder []
// the fold output is in reverse order, so reverse it
|> asyncMap List.rev

And now for the grep function itself:

let grep filePattern textPattern fileSystemItem =
let regex = Text.RegularExpressions.Regex(pattern=filePattern)
/// if the file matches the pattern
/// do the matching and return Some async, else None
let matchFile (fi:FileInfo) =
if regex.IsMatch fi.Name then
Some (matchPattern textPattern fi)
else
None
/// process a file by adding its async to the list
let fFile asyncs (fi:FileInfo) =
// add to the list of asyncs
(matchFile fi) :: asyncs
// for directories, just pass through the list of asyncs
let fDir asyncs (di:DirectoryInfo) =
asyncs
fileSystemItem
|> Tree.fold fFile fDir [] // get the list of asyncs
|> Seq.choose id // choose the Somes (where a file was processed)
|> Async.Parallel // merge all asyncs into a single async
|> asyncMap (Array.toList >> List.collect id) // flatten array of lists into a si
ngle list

Let's test it!

1557

Trees in the real world

currentDir
|> grep "fsx" "LinkedList"
|> Async.RunSynchronously

The result will look something like this:

" SizeOfTypes.fsx:120 type LinkedList<'a> = ";
" SizeOfTypes.fsx:122 | Cell of head:'a * tail:LinkedList<'a>
";
" SizeOfTypes.fsx:125 let S = size(LinkedList<'a>)";
" RecursiveTypesAndFold-3.fsx:15 // LinkedList";
" RecursiveTypesAndFold-3.fsx:18 type LinkedList<'a> = ";
" RecursiveTypesAndFold-3.fsx:20 | Cons of head:'a * tail:LinkedList<'a>
";
" RecursiveTypesAndFold-3.fsx:26 module LinkedList = ";
" RecursiveTypesAndFold-3.fsx:39 list:LinkedList<'a> ";
" RecursiveTypesAndFold-3.fsx:64 list:LinkedList<'a> -> ";

That's not bad for about 40 lines of code. This conciseness is because we are using various
kinds of fold and map which hide the recursion, allowing us to focus on the pattern
matching logic itself.
Of course, this is not at all efficient or optimized (an async for every line!), and so I wouldn't
use it as a real implementation, but it does give you an idea of the power of fold.
The source code for this example is available at this gist.

Example: Storing the file system in a database

For the next example, let's look at how to store a file system tree in a database. I don't really
know why you would want to do that, but the principles would work equally well for storing
any hierarchical structure, so I will demonstrate it anyway!
To model the file system hierarchy in the database, say that we have four tables:
DbDir stores information about each directory.
DbFile stores information about each file.
DbDir_File stores the relationship between a directory and a file.
DbDir_Dir stores the relationship between a parent directory and a child directory.

Here are the database table definitions:

1558

Trees in the real world

CREATE TABLE DbDir (

DirId int IDENTITY NOT NULL,
Name nvarchar(50) NOT NULL
)
CREATE TABLE DbFile (
FileId int IDENTITY NOT NULL,
Name nvarchar(50) NOT NULL,
FileSize int NOT NULL
)
CREATE TABLE DbDir_File (
DirId int NOT NULL,
FileId int NOT NULL
)
CREATE TABLE DbDir_Dir (
ParentDirId int NOT NULL,
ChildDirId int NOT NULL
)

That's simple enough. But note that in order to save a directory completely along with its
relationships to its child items, we first need the ids of all its children, and each child
directory needs the ids of its children, and so on.
This implies that we should use cata instead of fold , so that we have access to the data
from the lower levels of the hierarchy.

Implementing the database functions

We're not wise enough to be using the SQL Provider and so we have written our own table
insertion functions, like this dummy one:
/// Insert a DbFile record
let insertDbFile name (fileSize:int64) =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s size:%i" "DbFile" id name fileSize

In a real database, the identity column would be automatically generated for you, but for this
example, I'll use a little helper function nextIdentity :

1559

Trees in the real world

let nextIdentity =
let id = ref 0
fun () ->
id := !id + 1
!id
// test
nextIdentity() // 1
nextIdentity() // 2
nextIdentity() // 3

Now in order to insert a directory, we need to first know all the ids of the files in the directory.
This implies that the insertDbFile function should return the id that was generated.
/// Insert a DbFile record and return the new file id
let insertDbFile name (fileSize:int64) =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s size:%i" "DbFile" id name fileSize
id

But that logic applies to the directories too:

/// Insert a DbDir record and return the new directory id
let insertDbDir name =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s" "DbDir" id name
id

But that's still not good enough. When the child ids are passed to the parent directory, it
needs to distinguish between files and directories, because the relations are stored in
different tables.
No problem -- we'll just use a choice type to distinguish between them!
type PrimaryKey =
| FileId of int
| DirId of int

With this in place, we can complete the implementation of the database functions:

1560

Trees in the real world

/// Insert a DbFile record and return the new PrimaryKey

let insertDbFile name (fileSize:int64) =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s size:%i" "DbFile" id name fileSize
FileId id
/// Insert a DbDir record and return the new PrimaryKey
let insertDbDir name =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s" "DbDir" id name
DirId id
/// Insert a DbDir_File record
let insertDbDir_File dirId fileId =
printfn "%10s: inserting parentDir:%i childFile:%i" "DbDir_File" dirId fileId
/// Insert a DbDir_Dir record
let insertDbDir_Dir parentDirId childDirId =
printfn "%10s: inserting parentDir:%i childDir:%i" "DbDir_Dir" parentDirId childDi
rId

Working with the catamorphism

As noted above, we need to use cata instead of fold , because we need the inner ids at
each step.
The function to handle the File case is easy -- just insert it and return the PrimaryKey .
let fFile (fi:FileInfo) =
insertDbFile fi.Name fi.Length

The function to handle the Directory case will be passed the DirectoryInfo and a
sequence of PrimaryKey s from the children that have already been inserted.
It should insert the main directory record, then insert the children, and then return the
PrimaryKey for the next higher level:

let fDir (di:DirectoryInfo) childIds =

let dirId = insertDbDir di.Name
// insert the children
// return the id to the parent
dirId

After inserting the directory record and getting its id, for each child id, we insert either into
the DbDir_File table or the DbDir_Dir , depending on the type of the childId .

1561

Trees in the real world

let fDir (di:DirectoryInfo) childIds =

let dirId = insertDbDir di.Name
let parentPK = pkToInt dirId
childIds |> Seq.iter (fun childId ->
match childId with
| FileId fileId -> insertDbDir_File parentPK fileId
| DirId childDirId -> insertDbDir_Dir parentPK childDirId
)
// return the id to the parent
dirId

Note that I've also created a little helper function pkToInt that extracts the integer id from
the PrimaryKey type.
Here is all the code in one chunk:
open System
open System.IO
let nextIdentity =
let id = ref 0
fun () ->
id := !id + 1
!id
type PrimaryKey =
| FileId of int
| DirId of int
/// Insert a DbFile record and return the new PrimaryKey
let insertDbFile name (fileSize:int64) =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s size:%i" "DbFile" id name fileSize
FileId id
/// Insert a DbDir record and return the new PrimaryKey
let insertDbDir name =
let id = nextIdentity()
printfn "%10s: inserting id:%i name:%s" "DbDir" id name
DirId id
/// Insert a DbDir_File record
let insertDbDir_File dirId fileId =
printfn "%10s: inserting parentDir:%i childFile:%i" "DbDir_File" dirId fileId
/// Insert a DbDir_Dir record
let insertDbDir_Dir parentDirId childDirId =
printfn "%10s: inserting parentDir:%i childDir:%i" "DbDir_Dir" parentDirId childDi
rId

1562

Trees in the real world

let pkToInt primaryKey =

match primaryKey with
| FileId fileId -> fileId
| DirId dirId -> dirId
let insertFileSystemTree fileSystemItem =
let fFile (fi:FileInfo) =
insertDbFile fi.Name fi.Length
let fDir (di:DirectoryInfo) childIds =
let dirId = insertDbDir di.Name
let parentPK = pkToInt dirId
childIds |> Seq.iter (fun childId ->
match childId with
| FileId fileId -> insertDbDir_File parentPK fileId
| DirId childDirId -> insertDbDir_Dir parentPK childDirId
)
// return the id to the parent
dirId
fileSystemItem
|> Tree.cata fFile fDir

Now let's test it:

// get the current directory as a Tree
let currentDir = fromDir (DirectoryInfo("."))
// insert into the database
currentDir
|> insertFileSystemTree

The output should look something like this:

DbDir: inserting id:41 name:FoldAndRecursiveTypes
DbFile: inserting id:42 name:Fold.fsx size:8315
DbDir_File: inserting parentDir:41 childFile:42
DbFile: inserting id:43 name:FoldAndRecursiveTypes.fsproj size:3680
DbDir_File: inserting parentDir:41 childFile:43
DbFile: inserting id:44 name:FoldAndRecursiveTypes.sln size:1010
DbDir_File: inserting parentDir:41 childFile:44
...
DbDir: inserting id:57 name:bin
DbDir: inserting id:58 name:Debug
DbDir_Dir: inserting parentDir:57 childDir:58
DbDir_Dir: inserting parentDir:41 childDir:57

1563

Trees in the real world

You can see that the ids are being generated as the files are iterated over, and that each
DbFile insert is followed by a DbDir_File insert.

The source code for this example is available at this gist.

Example: Serializing a Tree to JSON

Let's look at another common challenge: serializing and deserializing a tree to JSON, XML,
or some other format.
Let's use the Gift domain again, but this time, we'll model the Gift type as a tree. That
means we get to put more than one thing in a box!

Modelling the Gift domain as a tree

Here are the main types again, but notice that the final Gift type is defined as a tree:
type Book = {title: string; price: decimal}
type ChocolateType = Dark | Milk | SeventyPercent
type Chocolate = {chocType: ChocolateType ; price: decimal}
type WrappingPaperStyle =
| HappyBirthday
| HappyHolidays
| SolidColor
// unified data for non-recursive cases
type GiftContents =
| Book of Book
| Chocolate of Chocolate
// unified data for recursive cases
type GiftDecoration =
| Wrapped of WrappingPaperStyle
| Boxed
| WithACard of string
type Gift = Tree<GiftContents,GiftDecoration>

As usual, we can create some helper functions to assist with constructing a Gift :

1564

Trees in the real world

let fromBook book =

LeafNode (Book book)
let fromChoc choc =
LeafNode (Chocolate choc)
let wrapInPaper paperStyle innerGift =
let container = Wrapped paperStyle
InternalNode (container, [innerGift])
let putInBox innerGift =
let container = Boxed
InternalNode (container, [innerGift])
let withCard message innerGift =
let container = WithACard message
InternalNode (container, [innerGift])
let putTwoThingsInBox innerGift innerGift2 =
let container = Boxed
InternalNode (container, [innerGift;innerGift2])

And we can create some sample data:

let wolfHall = {title="Wolf Hall"; price=20m}
let yummyChoc = {chocType=SeventyPercent; price=5m}
let birthdayPresent =
wolfHall
|> fromBook
|> wrapInPaper HappyBirthday
|> withCard "Happy Birthday"
let christmasPresent =
yummyChoc
|> fromChoc
|> putInBox
|> wrapInPaper HappyHolidays
let twoBirthdayPresents =
let thing1 = wolfHall |> fromBook
let thing2 = yummyChoc |> fromChoc
putTwoThingsInBox thing1 thing2
|> wrapInPaper HappyBirthday
let twoWrappedPresentsInBox =
let thing1 = wolfHall |> fromBook |> wrapInPaper HappyHolidays
let thing2 = yummyChoc |> fromChoc |> wrapInPaper HappyBirthday
putTwoThingsInBox thing1 thing2

1565

Trees in the real world

Functions like description now need to handle a list of inner texts, rather than one. We'll
just concat the strings together with an & separator:
let description gift =
let fLeaf leafData =
match leafData with
| Book book ->
sprintf "'%s'" book.title
| Chocolate choc ->
sprintf "%A chocolate" choc.chocType
let fNode nodeData innerTexts =
let innerText = String.concat " & " innerTexts
match nodeData with
| Wrapped style ->
sprintf "%s wrapped in %A paper" innerText style
| Boxed ->
sprintf "%s in a box" innerText
| WithACard message ->
sprintf "%s with a card saying '%s'" innerText message
// main call
Tree.cata fLeaf fNode gift

Finally, we can check that the function still works as before, and that multiple items are
handled correctly:
birthdayPresent |> description
// "'Wolf Hall' wrapped in HappyBirthday paper with a card saying 'Happy Birthday'"
christmasPresent |> description
// "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"
twoBirthdayPresents |> description
// "'Wolf Hall' & SeventyPercent chocolate in a box
// wrapped in HappyBirthday paper"
twoWrappedPresentsInBox |> description
// "'Wolf Hall' wrapped in HappyHolidays paper
// & SeventyPercent chocolate wrapped in HappyBirthday paper
// in a box"

Step 1: Defining GiftDto

Our Gift type consists of many discriminated unions. In my experience, these do not
serialize well. In fact, most complex types do not serialize well!

1566

Trees in the real world

So what I like to do is define DTO types that are explicitly designed to be serialized well. In
practice this means that the DTO types are constrained as follows:
Only record types should be used.
The record fields should consist only primitive values such as int , string and bool .
By doing this, we also get some other advantages:
We gain control of the serialization output. These kinds of data types are handled the
same by most serializers, while "strange" things such as unions can be interpreted
differently by different libraries.
We have better control of error handling. My number one rule when dealing with
serialized data is "trust no one". It's very common that the data is structured correctly but is
invalid for the domain: supposedly non-null strings are null, strings are too long, integers are
outside the correct bounds, and so on.
By using DTOs, we can be sure that the deserialization step itself will work. Then, when we
convert the DTO to a domain type, we can do proper validation.
So, let's define some DTO types for out domain. Each DTO type will correspond to a domain
type, so let's start with GiftContents . We'll define a corresponding DTO type called
GiftContentsDto as follows:

[<CLIMutableAttribute>]
type GiftContentsDto = {
discriminator : string // "Book" or "Chocolate"
// for "Book" case only
bookTitle: string
// for "Chocolate" case only
chocolateType : string // one of "Dark" "Milk" "SeventyPercent"
// for all cases
price: decimal
}

Obviously, this quite different from the original GiftContents , so let's look at the differences:
First, it has the CLIMutableAttribute , which allows deserializers to construct them using
reflection.
Second, it has a discriminator which indicates which case of the original union type is
being used. Obviously, this string could be set to anything, so when converting from the
DTO back to the domain type, we'll have to check that carefully!
Next is a series of fields, one for every possible item of data that needs to be stored. For
example, in the Book case, we need a bookTitle , while in the Chocolate case, we
need the chocolate type. And finally the price field which is in both types. Note that
the chocolate type is stored as a string as well, and so will also need special treatment
1567

Trees in the real world

when we convert from DTO to domain.

The GiftDecorationDto type is created in the same way, with a discriminator and strings
rather than unions.
[<CLIMutableAttribute>]
type GiftDecorationDto = {
discriminator: string // "Wrapped" or "Boxed" or "WithACard"
// for "Wrapped" case only
wrappingPaperStyle: string // "HappyBirthday" or "HappyHolidays" or "SolidColor"

// for "WithACard" case only
message: string
}

Finally, we can define a GiftDto type as being a tree that is composed of the two DTO
types:
type GiftDto = Tree<GiftContentsDto,GiftDecorationDto>

Step 2: Transforming a Gift to a GiftDto

Now that we have this DTO type, all we need to do is use Tree.map to convert from a Gift
to a GiftDto . And in order to do that, we need to create two functions: one that converts
from GiftContents to GiftContentsDto and one that converts from GiftDecoration to
GiftDecorationDto .

Here's the complete code for giftToDto , which should be self-explanatory:

1568

Trees in the real world

let giftToDto (gift:Gift) :GiftDto =

let fLeaf leafData :GiftContentsDto =
match leafData with
| Book book ->
{discriminator= "Book"; bookTitle=book.title; chocolateType=null; price=bo
ok.price}
| Chocolate choc ->
let chocolateType = sprintf "%A" choc.chocType
{discriminator= "Chocolate"; bookTitle=null; chocolateType=chocolateType;
price=choc.price}
let fNode nodeData :GiftDecorationDto =
match nodeData with
| Wrapped style ->
let wrappingPaperStyle = sprintf "%A" style
{discriminator= "Wrapped"; wrappingPaperStyle=wrappingPaperStyle; message=
null}
| Boxed ->
{discriminator= "Boxed"; wrappingPaperStyle=null; message=null}
| WithACard message ->
{discriminator= "WithACard"; wrappingPaperStyle=null; message=message}
// main call
Tree.map fLeaf fNode gift

You can see that the case ( Book , Chocolate , etc.) is turned into a discriminator string
and the chocolateType is also turned into a string, just as explained above.

Step 3: Defining a TreeDto

I said above that a good DTO should be a record type. Well we have converted the nodes of
the tree, but the tree itself is a union type! We need to transform the Tree type as well, into
say a TreeDto type.
How can we do this? Just as for the gift DTO types, we will create a record type which
contains all the data for both cases. We could use a discriminator field as we did before, but
this time, since there are only two choices, leaf and internal node, I'll just check whether the
values are null or not when deserializing. If the leaf value is not null, then the record must
represent the LeafNode case, otherwise the record must represent the InternalNode case.
Here's the definition of the data type:

1569

Trees in the real world

/// A DTO that represents a Tree

/// The Leaf/Node choice is turned into a record
[<CLIMutableAttribute>]
type TreeDto<'LeafData,'NodeData> = {
leafData : 'LeafData
nodeData : 'NodeData
subtrees : TreeDto<'LeafData,'NodeData>[] }

As before, the type has the CLIMutableAttribute . And as before, the type has fields to store
the data from all possible choices. The subtrees are stored as an array rather than a seq -this makes the serializer happy!
To create a TreeDto , we use our old friend cata to assemble the record from a regular
Tree .

/// Transform a Tree into a TreeDto

let treeToDto tree : TreeDto<'LeafData,'NodeData> =
let fLeaf leafData =
let nodeData = Unchecked.defaultof<'NodeData>
let subtrees = [||]
{leafData=leafData; nodeData=nodeData; subtrees=subtrees}
let fNode nodeData subtrees =
let leafData = Unchecked.defaultof<'NodeData>
let subtrees = subtrees |> Seq.toArray
{leafData=leafData; nodeData=nodeData; subtrees=subtrees}
// recurse to build up the TreeDto
Tree.cata fLeaf fNode tree

Note that in F#, records are not nullable, so I am using Unchecked.defaultof<'NodeData>

rather than null to indicate missing data.
Note also that I am assuming that LeafData or NodeData are reference types. If LeafData
or NodeData are ever value types like int or bool , then this approach will break down,
because you won't be able to tell the difference between a default value and a missing
value. In which case, I'd switch to a discriminator field as before.
Alternatively, I could have used an IDictionary . That would be less convenient to
deserialize, but would avoid the need for null-checking.

Step 4: Serializing a TreeDto

Finally we can serialize the TreeDto using a JSON serializer.

1570

Trees in the real world

For this example, I am using the built-in DataContractJsonSerializer so that I don't need to
take a dependency on a NuGet package. There are other JSON serializers that might be
better for a serious project.
#r "System.Runtime.Serialization.dll"
open System.Runtime.Serialization
open System.Runtime.Serialization.Json
let toJson (o:'a) =
let serializer = new DataContractJsonSerializer(typeof<'a>)
let encoding = System.Text.UTF8Encoding()
use stream = new System.IO.MemoryStream()
serializer.WriteObject(stream,o)
stream.Close()
encoding.GetString(stream.ToArray())

Step 5: Assembling the pipeline

So, putting it all together, we have the following pipeline:
Transform Gift to GiftDto using giftToDto ,
that is, use Tree.map to go from Tree<GiftContents,GiftDecoration> to
Tree<GiftContentsDto,GiftDecorationDto>

Transform Tree to TreeDto using treeToDto ,

that is, use Tree.cata to go from Tree<GiftContentsDto,GiftDecorationDto> to
TreeDto<GiftContentsDto,GiftDecorationDto>

Serialize TreeDto to a JSON string

Here's some example code:
let goodJson = christmasPresent |> giftToDto |> treeToDto |> toJson

And here is what the JSON output looks like:

1571

Trees in the real world

{
"leafData@": null,
"nodeData@": {
"discriminator@": "Wrapped",
"message@": null,
"wrappingPaperStyle@": "HappyHolidays"
},
"subtrees@": [
{
"leafData@": null,
"nodeData@": {
"discriminator@": "Boxed",
"message@": null,
"wrappingPaperStyle@": null
},
"subtrees@": [
{
"leafData@": {
"bookTitle@": null,
"chocolateType@": "SeventyPercent",
"discriminator@": "Chocolate",
"price@": 5
},
"nodeData@": null,
"subtrees@": []
}
]
}
]
}

The ugly @ signs on the field names are an artifact of serializing the F# record type. This
can be corrected with a bit of effort, but I'm not going to bother right now!
The source code for this example is available at this gist

Example: Deserializing a Tree from JSON

Now that we have created the JSON, what about going the other way and loading it into a
Gift ?

Simple! We just need to reverse the pipeline:

Deserialize a JSON string into a TreeDto .
Transform a TreeDto into a Tree to using dtoToTree ,
that is, go from TreeDto<GiftContentsDto,GiftDecorationDto> to

1572

Trees in the real world

Tree<GiftContentsDto,GiftDecorationDto> . We can't use cata for this -- we'll have to

create a little recursive loop.

Transform GiftDto to Gift using dtoToGift ,
that is, use Tree.map to go from Tree<GiftContentsDto,GiftDecorationDto> to
Tree<GiftContents,GiftDecoration> .

Step 1: Deserializing a TreeDto

We can deserialize the TreeDto using a JSON serializer.
let fromJson<'a> str =
let serializer = new DataContractJsonSerializer(typeof<'a>)
let encoding = System.Text.UTF8Encoding()
use stream = new System.IO.MemoryStream(encoding.GetBytes(s=str))
let obj = serializer.ReadObject(stream)
obj :?> 'a

What if the deserialization fails? For now, we will ignore any error handling and let the
exception propagate.

Step 2: Transforming a TreeDto into a Tree

To transform a TreeDto into a Tree we recursively loop through the record and its
subtrees, turning each one into a InternalNode or a LeafNode , based on whether the
appropriate field is null or not.
let rec dtoToTree (treeDto:TreeDto<'Leaf,'Node>) :Tree<'Leaf,'Node> =
let nullLeaf = Unchecked.defaultof<'Leaf>
let nullNode = Unchecked.defaultof<'Node>
// check if there is nodeData present
if treeDto.nodeData <> nullNode then
if treeDto.subtrees = null then
failwith "subtrees must not be null if node data present"
else
let subtrees = treeDto.subtrees |> Array.map dtoToTree
InternalNode (treeDto.nodeData,subtrees)
// check if there is leafData present
elif treeDto.leafData <> nullLeaf then
LeafNode (treeDto.leafData)
// if both missing then fail
else
failwith "expecting leaf or node data"

As you can see, a number of things could go wrong:

1573

Trees in the real world

What if the leafData and nodeData fields are both null?

What if the nodeData field is not null but the subtrees field is null?
Again, we will ignore any error handling and just throw exceptions (for now).
Question: Could we create a cata for TreeDto that would make this code simpler? Would
it be worth it?

Step 3: Transforming a GiftDto into Gift

Now we have a proper tree, we can use Tree.map again to convert each leaf and internal
node from a DTO to the proper domain type.
That means we need functions that map a GiftContentsDto into a GiftContents and a
GiftDecorationDto into a GiftDecoration .

Here's the complete code -- it's a lot more complicated than going in the other direction!
The code can be grouped as follows:
Helper methods (such as strToChocolateType ) that convert a string into a proper
domain type and throw an exception if the input is invalid.
Case converter methods (such as bookFromDto ) that convert an entire DTO into a case.
And finally, the dtoToGift function itself. It looks at the discriminator field to see
which case converter to call, and throws an exception if the discriminator value is not
recognized.
let strToBookTitle str =
match str with
| null -> failwith "BookTitle must not be null"
| _ -> str
let strToChocolateType str =
match str with
| "Dark" -> Dark
| "Milk" -> Milk
| "SeventyPercent" -> SeventyPercent
| _ -> failwithf "ChocolateType %s not recognized" str
let strToWrappingPaperStyle str =
match str with
| "HappyBirthday" -> HappyBirthday
| "HappyHolidays" -> HappyHolidays
| "SolidColor" -> SolidColor
| _ -> failwithf "WrappingPaperStyle %s not recognized" str
let strToCardMessage str =
match str with

1574

Trees in the real world

| null -> failwith "CardMessage must not be null"

| _ -> str
let bookFromDto (dto:GiftContentsDto) =
let bookTitle = strToBookTitle dto.bookTitle
Book {title=bookTitle; price=dto.price}
let chocolateFromDto (dto:GiftContentsDto) =
let chocType = strToChocolateType dto.chocolateType
Chocolate {chocType=chocType; price=dto.price}
let wrappedFromDto (dto:GiftDecorationDto) =
let wrappingPaperStyle = strToWrappingPaperStyle dto.wrappingPaperStyle
Wrapped wrappingPaperStyle
let boxedFromDto (dto:GiftDecorationDto) =
Boxed
let withACardFromDto (dto:GiftDecorationDto) =
let message = strToCardMessage dto.message
WithACard message
/// Transform a GiftDto to a Gift
let dtoToGift (giftDto:GiftDto) :Gift=
let fLeaf (leafDto:GiftContentsDto) =
match leafDto.discriminator with
| "Book" -> bookFromDto leafDto
| "Chocolate" -> chocolateFromDto leafDto
| _ -> failwithf "Unknown leaf discriminator '%s'" leafDto.discriminator
let fNode (nodeDto:GiftDecorationDto) =
match nodeDto.discriminator with
| "Wrapped" -> wrappedFromDto nodeDto
| "Boxed" -> boxedFromDto nodeDto
| "WithACard" -> withACardFromDto nodeDto
| _ -> failwithf "Unknown node discriminator '%s'" nodeDto.discriminator
// map the tree
Tree.map fLeaf fNode giftDto

Step 4: Assembling the pipeline

We can now assemble the pipeline that takes a JSON string and creates a Gift .
let goodGift = goodJson |> fromJson |> dtoToTree |> dtoToGift
// check that the description is unchanged
goodGift |> description
// "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

1575

Trees in the real world

This works fine, but the error handling is terrible!

Look what happens if we corrupt the JSON a little:
let badJson1 = goodJson.Replace("leafData","leafDataXX")
let badJson1_result = badJson1 |> fromJson |> dtoToTree |> dtoToGift
// Exception "The data contract type 'TreeDto' cannot be deserialized because the requ
ired data member 'leafData@' was not found."

We get an ugly exception.

Or what if a discriminator is wrong?
let badJson2 = goodJson.Replace("Wrapped","Wrapped2")
let badJson2_result = badJson2 |> fromJson |> dtoToTree |> dtoToGift
// Exception "Unknown node discriminator 'Wrapped2'"

or one of the values for the WrappingPaperStyle DU?

let badJson3 = goodJson.Replace("HappyHolidays","HappyHolidays2")
let badJson3_result = badJson3 |> fromJson |> dtoToTree |> dtoToGift
// Exception "WrappingPaperStyle HappyHolidays2 not recognized"

We get lots of exceptions, and as as functional programmers, we should try to remove them
whenever we can.
How we can do that will be discussed in the next section.
The source code for this example is available at this gist.

Example: Deserializing a Tree from JSON - with

error handling
To address the error handling issue, we're going use the Result type shown below:
type Result<'a> =
| Success of 'a
| Failure of string list

1576

Trees in the real world

I'm not going to explain how it works here. If you are not familar with this approach, please
read my post or watch my talk on the topic of functional error handling.
Let's revisit all the steps from the previous section, and use Result rather than throwing
exceptions.

Step 1: Deserializing a TreeDto

When we deserialize the TreeDto using a JSON serializer we will trap exceptions and turn
them into a Result .
let fromJson<'a> str =
try
let serializer = new DataContractJsonSerializer(typeof<'a>)
let encoding = System.Text.UTF8Encoding()
use stream = new System.IO.MemoryStream(encoding.GetBytes(s=str))
let obj = serializer.ReadObject(stream)
obj :?> 'a
|> Result.retn
with
| ex ->
Result.failWithMsg ex.Message

The signature of fromJson is now string -> Result<'a> .

Step 2: Transforming a TreeDto into a Tree

As before, we transform a TreeDto into a Tree by recursively looping through the record
and its subtrees, turning each one into a InternalNode or a LeafNode . This time, though,
we use Result to handle any errors.

1577

Trees in the real world

let rec dtoToTreeOfResults (treeDto:TreeDto<'Leaf,'Node>) :Tree<Result<'Leaf>,Result<'

Node>> =
let nullLeaf = Unchecked.defaultof<'Leaf>
let nullNode = Unchecked.defaultof<'Node>
// check if there is nodeData present
if treeDto.nodeData <> nullNode then
if treeDto.subtrees = null then
LeafNode <| Result.failWithMsg "subtrees must not be null if node data pre
sent"
else
let subtrees = treeDto.subtrees |> Array.map dtoToTreeOfResults
InternalNode (Result.retn treeDto.nodeData,subtrees)
// check if there is leafData present
elif treeDto.leafData <> nullLeaf then
LeafNode <| Result.retn (treeDto.leafData)
// if both missing then fail
else
LeafNode <| Result.failWithMsg "expecting leaf or node data"
// val dtoToTreeOfResults :
// treeDto:TreeDto<'Leaf,'Node> -> Tree<Result<'Leaf>,Result<'Node>>

But uh-oh, we now have a Tree where every internal node and leaf is wrapped in a
Result . It's a tree of Results ! The actual ugly signature is this:
Tree<Result<'Leaf>,Result<'Node>> .

But this type is useless as it stands -- what we really want is to merge all the errors together
and return a Result containing a Tree .
How can we transform a Tree of Results into a Result of Tree?
The answer is to use a sequence function which "swaps" the two types. You can read much
more about sequence in my series on elevated worlds.
Note that we could also use the slightly more complicated traverse variant to combine the
map and sequence into one step, but for the purposes of this demonstration, it's easier to

understand if the steps are kept separate.

We need to create our own sequence function for the Tree/Result combination. Luckily the
creation of a sequence function is a mechanical process:
For the lower type ( Result ) we need to define apply and return functions. See here
for more details on what apply means.
For the higher type ( Tree ) we need to have a cata function, which we do.
In the catamorphism, each constructor of the higher type ( LeafNode and InternalNode
in this case) is replaced by an equivalent that is "lifted" to the Result type (e.g. retn

1578

Trees in the real world

LeafNode <*> data )

Here is the actual code -- don't worry if you can't understand it immediately. Luckily, we only
need to write it once for each combination of types, so for any kind of Tree/Result
combination in the future, we're set!
/// Convert a tree of Results into a Result of tree
let sequenceTreeOfResult tree =
// from the lower level
let (<*>) = Result.apply
let retn = Result.retn
// from the traversable level
let fLeaf data =
retn LeafNode <*> data
let fNode data subitems =
let makeNode data items = InternalNode(data,items)
let subItems = Result.sequenceSeq subitems
retn makeNode <*> data <*> subItems
// do the traverse
Tree.cata fLeaf fNode tree
// val sequenceTreeOfResult :
// tree:Tree<Result<'a>,Result<'b>> -> Result<Tree<'a,'b>>

Finally, the actual dtoToTree function is simple -- just send the treeDto through
dtoToTreeOfResults and then use sequenceTreeOfResult to convert the final result into a
Result<Tree<..>> , which is just what we need.

let dtoToTree treeDto =

treeDto |> dtoToTreeOfResults |> sequenceTreeOfResult
// val dtoToTree : treeDto:TreeDto<'a,'b> -> Result<Tree<'a,'b>>

Step 3: Transforming a GiftDto into a Gift

Again we can use Tree.map to convert each leaf and internal node from a DTO to the
proper domain type.
But our functions will handle errors, so they need to map a GiftContentsDto into a
Result<GiftContents> and a GiftDecorationDto into a Result<GiftDecoration> . This results

in a Tree of Results again, and so we'll have to use sequenceTreeOfResult again to get it
back into the correct Result<Tree<..>> shape.

1579

Trees in the real world

Let's start with the helper methods (such as strToChocolateType ) that convert a string into a
proper domain type. This time, they return a Result rather than throwing an exception.
let strToBookTitle str =
match str with
| null -> Result.failWithMsg "BookTitle must not be null"
| _ -> Result.retn str
let strToChocolateType str =
match str with
| "Dark" -> Result.retn Dark
| "Milk" -> Result.retn Milk
| "SeventyPercent" -> Result.retn SeventyPercent
| _ -> Result.failWithMsg (sprintf "ChocolateType %s not recognized" str)
let strToWrappingPaperStyle str =
match str with
| "HappyBirthday" -> Result.retn HappyBirthday
| "HappyHolidays" -> Result.retn HappyHolidays
| "SolidColor" -> Result.retn SolidColor
| _ -> Result.failWithMsg (sprintf "WrappingPaperStyle %s not recognized" str)
let strToCardMessage str =
match str with
| null -> Result.failWithMsg "CardMessage must not be null"
| _ -> Result.retn str

The case converter methods have to build a Book or Chocolate from parameters that are
Result s rather than normal values. This is where lifting functions like Result.lift2 can

help. For details on how this works, see this post on lifting and this one on validation with
applicatives.

1580

Trees in the real world

let bookFromDto (dto:GiftContentsDto) =

let book bookTitle price =
Book {title=bookTitle; price=price}
let bookTitle = strToBookTitle dto.bookTitle
let price = Result.retn dto.price
Result.lift2 book bookTitle price
let chocolateFromDto (dto:GiftContentsDto) =
let choc chocType price =
Chocolate {chocType=chocType; price=price}
let chocType = strToChocolateType dto.chocolateType
let price = Result.retn dto.price
Result.lift2 choc chocType price
let wrappedFromDto (dto:GiftDecorationDto) =
let wrappingPaperStyle = strToWrappingPaperStyle dto.wrappingPaperStyle
Result.map Wrapped wrappingPaperStyle
let boxedFromDto (dto:GiftDecorationDto) =
Result.retn Boxed
let withACardFromDto (dto:GiftDecorationDto) =
let message = strToCardMessage dto.message
Result.map WithACard message

And finally, the dtoToGift function itself is changed to return a Result if the
discriminator is invalid.

As before, this mapping creates a Tree of Results, so we pipe the output of the Tree.map
through sequenceTreeOfResult ...
`Tree.map fLeaf fNode giftDto |> sequenceTreeOfResult`

... to return a Result of Tree.

Here's the complete code for dtoToGift :

1581

Trees in the real world

open TreeDto_WithErrorHandling
/// Transform a GiftDto to a Result<Gift>
let dtoToGift (giftDto:GiftDto) :Result<Gift>=
let fLeaf (leafDto:GiftContentsDto) =
match leafDto.discriminator with
| "Book" -> bookFromDto leafDto
| "Chocolate" -> chocolateFromDto leafDto
| _ -> Result.failWithMsg (sprintf "Unknown leaf discriminator '%s'" leafDto.d
iscriminator)
let fNode (nodeDto:GiftDecorationDto) =
match nodeDto.discriminator with
| "Wrapped" -> wrappedFromDto nodeDto
| "Boxed" -> boxedFromDto nodeDto
| "WithACard" -> withACardFromDto nodeDto
| _ -> Result.failWithMsg (sprintf "Unknown node discriminator '%s'" nodeDto.d
iscriminator)
// map the tree
Tree.map fLeaf fNode giftDto |> sequenceTreeOfResult

The type signature of dtoToGift has changed -- it now returns a Result<Gift> rather than
just a Gift .
// val dtoToGift : GiftDto -> Result<GiftUsingTree.Gift>

Step 4: Assembling the pipeline

We can now reassemble the pipeline that takes a JSON string and creates a Gift .
But changes are needed to work with the new error handling code:
The fromJson function returns a Result<TreeDto> but the next function in the pipeline
( dtoToTree ) expects a regular TreeDto as input.
Similarly dtoToTree returns a Result<Tree> but the next function in the pipeline
( dtoToGift ) expects a regular Tree as input.
In both case, Result.bind can be used to solve that problem of mis-matched output/input.
See here for a more detailed discussion of bind.
Ok, let's try deserializing the goodJson string we created earlier.

1582

Trees in the real world

let goodGift = goodJson |> fromJson |> Result.bind dtoToTree |> Result.bind dtoToGift
// check that the description is unchanged
goodGift |> description
// Success "SeventyPercent chocolate in a box wrapped in HappyHolidays paper"

That's fine.
Let's see if the error handling has improved now. We'll corrupt the JSON again:
let badJson1 = goodJson.Replace("leafData","leafDataXX")
let badJson1_result = badJson1 |> fromJson |> Result.bind dtoToTree |> Result.bind dto
ToGift
// Failure ["The data contract type 'TreeDto' cannot be deserialized because the requi
red data member 'leafData@' was not found."]

Great! We get an nice Failure case.

Or what if a discriminator is wrong?
let badJson2 = goodJson.Replace("Wrapped","Wrapped2")
let badJson2_result = badJson2 |> fromJson |> Result.bind dtoToTree |> Result.bind dto
ToGift
// Failure ["Unknown node discriminator 'Wrapped2'"]

or one of the values for the WrappingPaperStyle DU?

let badJson3 = goodJson.Replace("HappyHolidays","HappyHolidays2")
let badJson3_result = badJson3 |> fromJson |> Result.bind dtoToTree |> Result.bind dto
ToGift
// Failure ["WrappingPaperStyle HappyHolidays2 not recognized"]

Again, nice Failure cases.

What's very nice (and this is something that the exception handling approach can't offer) is
that if there is more than one error, the various errors can be aggregated so that we get a list
of all the things that went wrong, rather than just one error at a time.
Let's see this in action by introducing two errors into the JSON string:

1583

Trees in the real world

// create two errors

let badJson4 = goodJson.Replace("HappyHolidays","HappyHolidays2")
.Replace("SeventyPercent","SeventyPercent2")
let badJson4_result = badJson4 |> fromJson |> Result.bind dtoToTree |> Result.bind dto
ToGift
// Failure ["WrappingPaperStyle HappyHolidays2 not recognized";
// "ChocolateType SeventyPercent2 not recognized"]

So overall, I'd say that's a success!

The source code for this example is available at this gist.

Summary
We've seen in this series how to define catamorphisms, folds, and in this post in particular,
how to use them to solve real world problems. I hope these posts have been useful, and
have provided you with some tips and insights that you can apply to your own code.
This series turned out to be a lot longer that I intended, so thanks for making it to the end!
Cheers!

1584

The "A functional approach to authorization" Series

In this series of posts, I'll look at how you might handle the common security challenge of
authorization. That is, how can you ensure that clients of your code can only do what you
want them to do?
This series will sketch out two different approaches, first using an approach called capability
based security, and second using statically checked types to emulate access tokens.
Interestingly, both approaches tend to produce a cleaner, more modular design as a side
effect, which is why I like them!
A functional approach to authorization. Capability based security and more.
Constraining capabilities based on identity and role. A functional approach to
authorization, part 2.
Using types as access tokens. A functional approach to authorization, part 3.

1585

A functional approach to authorization

UPDATE: Slides and video from my talk on this topic
In this series of posts, I'll look at how you might handle the common security challenge of
authorization. That is, how can you ensure that clients of your code can only do what you
want them to do?
This series will sketch out two different approaches, first using an approach called capability
based security, and second using statically checked types to emulate access tokens.
Interestingly, both approaches tend to produce a cleaner, more modular design as a side
effect, which is why I like them!
Before I start, I must mention a major caveat. In a .NET environment, you can generally use
reflection to bypass compile-time checking, so the approaches shown here are not about
preventing truly malicious attacks so much as helping you create designs that reduce
unintentional security vulnerabilities.
Finally, I'm no expert on security -- I'm just putting down some of my own thoughts and
suggestions. This post is certainly not meant to substitute for a proper full-fledged security
design, nor is it a serious study of security practices. If you want to know more, there are
links to further reading at the bottom of the post.

Part 1: A configuration example

First, let's start with a simple scenario:
You have a configuration option that can be set by one part of the the code. Let's say it
is a boolean called DontShowThisMessageAgain .
We have a component of the application (the UI say) that wants to set this option.
In addition, we're also going to assume that the component was written by a malicious
developer and is going to try to cause trouble if possible.
So, how should we expose this configuration setting to a potentially malicious caller?
Attempt 1: Give the caller the name of the configuration file
Let's start with a really bad idea. We'll just provide the name of the config file to the caller,
and let them change the file themselves.
Here's how this might be written in C# pseudocode:

1586

A functional approach to authorization

interface IConfiguration
{
string GetConfigFilename();
}

and the caller code would be

var filename = config.GetConfigFilename();
// open file
// write new config
// close file

Obviously, this is not good! In order for this to work, we have to give the caller the ability to
write to any file on the filesystem, and then a malicious caller could delete or corrupt all sorts
of things.
You could avoid this to some extent by having strict permissions on the file system, but we're
still giving way too much control to the caller.
Attempt 2: Give the caller a TextWriter
Ok, so let's open the file ourselves and just give the caller the opened file stream as a
TextWriter . That way the caller doesn't need permission to access the file system at all.

But of course, a malicious caller could still corrupt the config file by writing garbage to the
file. Again, we're giving way too much control to the caller.
Attempt 3: Give the caller a key/value interface
Let's lock this down by providing the caller an interface that forces them to treat the config
file as a key/value store, like this:
interface IConfiguration
{
void SetConfig(string key, string value);
}

The caller code is then something like this:

config.SetConfig("DontShowThisMessageAgain", "True");

That's much better, but because it is a stringly-typed interface, a malicious caller could still
corrupt the configuration by setting the value to a non-boolean which would not parse. They
could also corrupt all the other configuration keys if they wanted to.

1587

A functional approach to authorization

Attempt 4: Give the caller a domain-centric interface

Ok, so rather than having a generic config interface, let's provide an interface that provides
specific methods for each configuration setting.
enum MessageFlag {
ShowThisMessageAgain,
DontShowThisMessageAgain
}
interface IConfiguration
{
void SetMessageFlag(MessageFlag value);
void SetConnectionString(ConnectionString value);
void SetBackgroundColor(Color value);
}

Now the caller can't possibly corrupt the config, because each option is statically typed.
But we still have a problem! What's to stop a malicious caller changing the connection string
when they were only supposed to change the message flag?
Attempt 5: Give the caller only the interface they need
Ok, so let's define a new interface that contains only the methods the caller should have
access to, with all the other methods hidden.
interface IWarningMessageConfiguration
{
void SetMessageFlag(MessageFlag value);
}

That's about as locked down as we can get! The caller can only do the thing we allow them
to do.
In other words, we have just created a design using the Principle Of Least Authority,
normally abbreviated to "POLA".

Security as good design

What's interesting about this approach is that it exactly parallels what you would do for good
design anyway, regardless of a malicious caller.
Here's how I might think about designing this, basing my decisions only on core design
principles such information hiding and decoupling.

1588

A functional approach to authorization

If we give the caller a filename, we would be limiting ourselves to file-based config files.
By giving the caller a TextWriter, we can make the design more mockable.
But if we give the caller a TextWriter, we are exposing a specific storage format (XML,
JSON, etc) and are also limiting ourselves to text-based storage. By giving the caller a
generic KeyValue store, we hide the format and make the implementation choices more
flexible.
But if we give the caller a generic KeyValue store using strings, we are still exposing
ourselves to bugs where the value is not a boolean, and we'd have to write validation
and tests for that. If we use a statically typed interface instead, we don't have to write
any corruption checking code.
But if we give the caller an interface with too many methods, we are not following the
Interface Segregation Principle. Hence, we should reduce the number of available
methods to the absolute minimum needed by the caller.
Working through a thought process like this, using good design practices only, we end up
with exactly the same result as if we had been worried about security!
That is: designing the most minimal interface that the caller needs will both avoid accidental
complexity (good design) and increase security (POLA).
Of course, we don't normally have to deal with malicious callers, but we should treat
ourselves, as developers, as unintentionally malicious. For example, if there is a extra
method in the interface, it might well be used in a different context, which then increases
coupling between the two contexts and makes refactoring harder.
So, here's a tip: design for malicious callers and you will probably end up with more
modular code!

Introducing capability-based security

What we have done above is gradually reduce the surface area to the caller so that by the
final design, the caller can only do exactly one thing.
That "one thing" is a "capability". The caller has a capability to set the message flag, and
that's all.
"Capability-based" security is a security model that is based on this idea:
The system provides "capabilities" to clients (in our case, via an implementation of an
interface).
These capabilities encapsulate any access rights that are needed. For example, the
very fact that I have access to an implementation of the interface means that I can set
that flag. If I did not have permission to set that flag, I would have not been given the

1589

A functional approach to authorization

capability (interface) in the first place. (I'll talk more about authorization in the next post).
Finally, the capabilities can be passed around. For example, I can acquire the capability
at startup and then later pass it to the UI layer which can use it as needed.
In other words, we have a "just-in-time" rather than a "just-in-case" model; we pass in the
minimal amount of authority as and when needed, rather than having excess "ambient"
authority available globally to everyone.
The capability-based model is often focused on operating systems, but it can be mapped to
programming languages very nicely, where it is called the object-capability model.
I hope to demonstrate in this post that by using a capability-based approach in your code,
you can create better designed and more robust code. In addition, potential security errors
will be detectable at compile-time rather than at run-time.
As I mentioned above, if your app is trusted, you can always use .NET reflection to "forge"
capabilities that you are not entitled to. So, again, the approach shown here is not about
preventing truly malicious attacks so much as it about creating a more robust design that
reduces unintentional security vulnerabilities.

Authority vs. permission

A capability-based security model tends to use the term "authority" rather than "permission".
There is a distinction between the two:
In an authority based system, once I have been granted authority to do something, I can
pass some or all of that authority to others, add additional constraints of my own, and so
on.
In a permission based system, I can ask for permission to do something, but I cannot
pass that around to others.
It might seem that an authority based system is more open and "dangerous" than a
permission based system. But in a permission based system, if others have access to me
and I cooperate with them, I can act as proxy for anything they want to do so, so third-parties
can still get authority indirectly. Permissions don't really make things more secure -- an
attacker just has to use a more convoluted approach.
Here's a concrete example. Let's say Alice trusts me to drive her car, and she is willing to let
me borrow it, but she doesn't trust Bob. If I'm friends with Bob, I can let Bob drive the car
anyway when Alice is not looking. So if Alice trusts me, she also implicitly trusts anyone that
I trust. An authority-based system just makes this explicit. Alice giving me her car keys is
giving me the "capability" to drive her car, with full knowledge that I might give the car keys
to someone else.

1590

A functional approach to authorization

Of course, when I act as a proxy in a permission based system, I can stop cooperating with
the third-party if I want to, at which point the third-party loses their access.
The equivalent of that in an authority based system is "revokable authority", which we will
see an example of later. In the car key analogy, this might be like having car keys that selfdestruct on demand!

Modelling capabilities as functions

An interface with one method can be better realized as a function. So this interface:
interface IWarningMessageConfiguration
{
void SetMessageFlag(MessageFlag value);
}

becomes just this function:

Action<MessageFlag> messageFlagCapability = // get function;

or in F#:
let messageFlagCapability = // get function;

In a functional approach to capability-based security, each capability is represented by a

function rather than an interface.
The nice thing about using functions to represent capabilities is that we can use all the
standard functional programming techniques: we can compose them, combine them with
combinators, and so on.

The object-capability model vs. the functional

programming model
Many of the other requirements of the object-capability model fit well within a functional
programming framework. Here is a comparison table:

1591

A functional approach to authorization

Object-capability model

Functional programming

No global mutable state is allowed.

Capabilities are always passed around

explicitly from parent to child, or from a
sender to a receiver.

Functions are values that can be passed

as parameters.

Capabilities are never extracted out of the

environment ("ambient authority").

Pure functions have all "dependencies"

passed in explicitly.

Capabilities cannot be tampered with.

Data is immutable.

Capabilities cannot be forged or cast to

other capabilities.

In a uncompromising FP language, there

is no reflection or casting available (of
course, F# is not strict in this way).

Capabilities should "fail safe". If a

capability cannot be obtained, or doesn't
work, we must not allow any progress on
paths that assumed that it was successful.

In a statically typed language such as F#,

we can embed these kinds of control-flow
rules into the type system. The use of
Option is an example of this.

You can see that there is quite a lot of overlap.

One of the unofficial goals of the object-capability model is make security user-friendly by
making the security invisible. I think that this is a great idea, and by passing capabilities
as functions, is quite easily achievable.
It's important to note there is one important aspect in which a capability-based model does
not overlap with a true functional model.
Capabilities are mostly all about (side) effects -- reading or writing the file system, the
network, etc. A true functional model would try to wrap them somehow (e.g. in a monad).
Personally, using F#, I would generally just allow the side-effects rather than constructing a
more complex framework.
But again, as I noted above, the goal of this post is to not to force you into a 100% strict
object-capability model, but to borrow some of the same ideas in order to create better
designs.

Getting capabilities
A natural question at this point is: where do these capability functions come from?
The answer is, some sort of service that can authorize you to have that capability. In the
configuration example, we generally don't do serious authorization, so the configuration
service itself will normally provide the capabilities without checking your identity, role or other
claims.

1592

A functional approach to authorization

But now I need a capability to access the configuration service. Where does that come
from? The buck has to stop somewhere!
In OO designs, there is typically a bootstrap/startup stage where all the dependencies are
constructed and an IoC container is configured. In a capability based system, a so-called
Powerbox plays a similar role of being the starting point for all authority.
Here's the code for a service that provides configuration capabilities:
interface IConfigurationCapabilities
{
Action<MessageFlag> SetMessageFlag();
Action<ConnectionString> SetConnectionString();
Action<Color> SetBackgroundColor();
}

This code might look very similar to the interface defined earlier, but the difference is that
this one will be initialized at startup to return capabilities that are then passed around.
The actual users of the capabilities will not have access to the configuration system at all,
just the capabilities they have been given. That is, the capability will be injected into the
clients in the same way as a one method interface would be injected in an OO model.
Here's some sample C# pseudocode to demonstrate:
The capability is obtained at startup.
The capability is injected into the main window ( ApplicationWindow ) via the constructor.
The ApplicationWindow creates a checkbox.
The event handler for the checkbox calls the capability.

1593

A functional approach to authorization

// at startup
var messageFlagCapability =
configurationCapabilities.SetMessageFlag()
var appWindow = new ApplicationWindow(messageFlagCapability)
// and in the UI class
class ApplicationWindow
{
// pass in capability in the constructor
// just as you would an interface
ApplicationWindow(Action<MessageFlag> messageFlagCapability)
{
// set fields
}
// setup the check box and register the "OnCheckboxChecked" handler
// use the capability when the event happens
void OnCheckboxChecked(CheckBox sender)
{
messageFlagCapability(sender.IsChecked)
}
}

A complete example in F#
Here's the code to a complete example in F# (also available as a gist here).
This example consists of a simple window with a main region and some extra buttons.
If you click in the main area, an annoying dialog pops up with a "don't show this
message again" option.
One of the buttons allows you to change the background color using the system color
picker, and store it in the config.
The other button allows you to reset the "don't show this message again" option back to
false.
It's very crude and very ugly -- no UI designers were hurt in the making of it -- but it should
demonstrate the main points so far.

1594

A functional approach to authorization

The configuration system

We start with the configuration system. Here's an overview:
The custom types MessageFlag , ConnectionString , and Color are defined.
The record type ConfigurationCapabilities is defined to hold all the capabilities.
An in-memory store ( ConfigStore ) is created for the purposes of the demo
Finally, the configurationCapabilities are created using functions that read and write
to the ConfigStore

1595

A functional approach to authorization

module Config =
type MessageFlag = ShowThisMessageAgain | DontShowThisMessageAgain
type ConnectionString = ConnectionString of string
type Color = System.Drawing.Color
type ConfigurationCapabilities = {
GetMessageFlag : unit -> MessageFlag
SetMessageFlag : MessageFlag -> unit
GetBackgroundColor : unit -> Color
SetBackgroundColor : Color -> unit
GetConnectionString : unit -> ConnectionString
SetConnectionString : ConnectionString -> unit
}
// a private store for demo purposes
module private ConfigStore =
let mutable MessageFlag = ShowThisMessageAgain
let mutable BackgroundColor = Color.White
let mutable ConnectionString = ConnectionString ""
// public capabilities
let configurationCapabilities = {
GetMessageFlag = fun () -> ConfigStore.MessageFlag
SetMessageFlag = fun flag -> ConfigStore.MessageFlag <- flag
GetBackgroundColor = fun () -> ConfigStore.BackgroundColor
SetBackgroundColor = fun color -> ConfigStore.BackgroundColor <- color
SetConnectionString = fun _ -> () // ignore
GetConnectionString = fun () -> ConfigStore.ConnectionString
SetConnectionString = fun connStr -> ConfigStore.ConnectionString <- connStr
}

The annoying popup dialog

Next, we'll create the annoying popup dialog. This will be triggered whenever you click on
the main window, unless the "Don't show this message again" option is checked.
The dialog consists of a label control, the message flag checkbox, and the OK button.
Notice that the createMessageFlagCheckBox function, which creates the checkbox control, is
passed only the two capabilities it needs -- to get and set the flag.
This requires in turn that the main form creation function ( createForm ) is also passed the
capabilities. These capabilities, and these capabilities only are passed in to the form. The
capabilities for setting the background color or connection string are not passed in, and thus
not available to be (mis)used.

1596

A functional approach to authorization

module AnnoyingPopupMessage =
open System.Windows.Forms
let createLabel() =
new Label(Text="You clicked the main window", Dock=DockStyle.Top)
let createMessageFlagCheckBox capabilities =
let getFlag,setFlag = capabilities
let ctrl= new CheckBox(Text="Don't show this annoying message again", Dock=Doc
kStyle.Bottom)
ctrl.Checked <- getFlag()
ctrl.CheckedChanged.Add (fun _ -> ctrl.Checked |> setFlag)
ctrl // return new control
let createOkButton (dialog:Form) =
let ctrl= new Button(Text="OK",Dock=DockStyle.Bottom)
ctrl.Click.Add (fun _ -> dialog.Close())
ctrl
let createForm capabilities =
let form = new Form(Text="Annoying Popup Message", Width=300, Height=150)
form.FormBorderStyle <- FormBorderStyle.FixedDialog
form.StartPosition <- FormStartPosition.CenterParent
let label = createLabel()
let messageFlag = createMessageFlagCheckBox capabilities
let okButton = createOkButton form
form.Controls.Add label
form.Controls.Add messageFlag
form.Controls.Add okButton
form

The main application window

We can now create a main window for our rather silly "application". It consists of:
A label control that can be clicked to produce the annoying popup ( createClickMeLabel )
A button that brings up a color picking dialog to change the background color
( createChangeBackColorButton )
A button that resets the message flag to "show" again ( createResetMessageFlagButton )
All three of these constructor functions are passed capabilities, but capabilities are different
in each case.
The label control is only passed getFlag and setFlag capabilities
The color picking dialog is only passed getColor and setColor capabilities
The button that resets the message flag is only passed the setFlag capability

1597

A functional approach to authorization

In the main form ( createMainForm ) the complete set of capabilities are passed in, and they
are recombined in various ways as needed for the child controls ( popupMessageCapabilities ,
colorDialogCapabilities ).

In addition, the capability functions are modified:

A new "SetColor" capability is created from the existing one, with the addition of
changing the form's background as well.
The flag capabilities are converted from the domain type ( MessageFlag ) to bools that
can be used directly with the checkbox.
Here's the code:
module UserInterface =
open System.Windows.Forms
open System.Drawing
let showPopupMessage capabilities owner =
let getFlag,setFlag = capabilities
let popupMessage = AnnoyingPopupMessage.createForm (getFlag,setFlag)
popupMessage.Owner <- owner
popupMessage.ShowDialog() |> ignore // don't care about result
let showColorDialog capabilities owner =
let getColor,setColor = capabilities
let dlg = new ColorDialog(Color=getColor())
let result = dlg.ShowDialog(owner)
if result = DialogResult.OK then
dlg.Color |> setColor
let createClickMeLabel capabilities owner =
let getFlag,_ = capabilities
let ctrl= new Label(Text="Click me", Dock=DockStyle.Fill, TextAlign=ContentAli
gnment.MiddleCenter)
ctrl.Click.Add (fun _ ->
if getFlag() then showPopupMessage capabilities owner)
ctrl // return new control
let createChangeBackColorButton capabilities owner =
let ctrl= new Button(Text="Change background color", Dock=DockStyle.Bottom)
ctrl.Click.Add (fun _ -> showColorDialog capabilities owner)
ctrl
let createResetMessageFlagButton capabilities =
let setFlag = capabilities
let ctrl= new Button(Text="Show popup message again", Dock=DockStyle.Bottom)
ctrl.Click.Add (fun _ -> setFlag Config.ShowThisMessageAgain)
ctrl
let createMainForm capabilities =
// get the individual component capabilities from the parameter

1598

A functional approach to authorization

let getFlag,setFlag,getColor,setColor = capabilities

let form = new Form(Text="Capability example", Width=500, Height=300)
form.BackColor <- getColor() // update the form from the config
// transform color capability to change form as well
let newSetColor color =
setColor color // change config
form.BackColor <- color // change form as well
// transform flag capabilities from domain type to bool
let getBoolFlag() =
getFlag() = Config.ShowThisMessageAgain
let setBoolFlag bool =
if bool
then setFlag Config.ShowThisMessageAgain
else setFlag Config.DontShowThisMessageAgain
// set up capabilities for child objects
let colorDialogCapabilities = getColor,newSetColor
let popupMessageCapabilities = getBoolFlag,setBoolFlag
// setup controls with their different capabilities
let clickMeLabel = createClickMeLabel popupMessageCapabilities form
let changeColorButton = createChangeBackColorButton colorDialogCapabilities fo
rm
let resetFlagButton = createResetMessageFlagButton setFlag
// add controls
form.Controls.Add clickMeLabel
form.Controls.Add changeColorButton
form.Controls.Add resetFlagButton
form // return form

The startup code

Finally, the top-level module, here called Startup , gets some of the capabilities from the
Configuration subsystem, and combines them into a tuple that can be passed to the main
form. The ConnectionString capabilities are not passed in though, so there is no way the
form can accidentally show it to a user or update it.

1599

A functional approach to authorization

module Startup =
// set up capabilities
let configCapabilities = Config.configurationCapabilities
let formCapabilities =
configCapabilities.GetMessageFlag,
configCapabilities.SetMessageFlag,
configCapabilities.GetBackgroundColor,
configCapabilities.SetBackgroundColor
// start
let form = UserInterface.createMainForm formCapabilities
form.ShowDialog() |> ignore

Summary of Part 1
As you can see, this code is very similar to an OO system designed with dependency
injection. There is no global access to capabilities, only those passed in from the parent.

Of course, the use of functions to parameterize behavior like this is nothing special. It's one
of the most fundamental functional programming techniques. So this code is not really
showing any new ideas, rather it is just demonstrating how a standard functional
programming approach can be applied to enforce access paths.
Some common questions at this point:
Question: This seems like extra work. Why do I need to do this at all?
If you have a simple system, you certainly don't need to do this. But here's where it might be
useful:
You have a system which uses fine-grained authorization already, and you want to
make this more explicit and easier to use in practice.
You have a system which runs at a high privilege but has strict requirements about
leaking data or performing actions in an unauthorized context.

1600

A functional approach to authorization

In these situations, I believe that is very important to be explicit about what the capabilities
are at all points in the codebase, not just in the UI layer. This not only helps with compliance
and auditing needs, but also has the practical benefit that it makes the code more modular
and easier to maintain.
Question: What's the difference between this approach and dependency injection?
Dependency injection and a capability-based model have different goals. Dependency
injection is all about decoupling, while capabilities are all about controlling access. As we
have seen, both approaches end up promoting similar designs.
Question: What happens if I have hundreds of capabilities that I need to pass around?
It seems like this should be a problem, but in practice it tends not to be. For one thing,
judicious use of partial application means that capabilities can be baked in to a function
before passing it around, so that child objects are not even aware of them.
Secondly, it is very easy -- just a few lines -- to create simple record types that contain a
group of capabilities (as I did with the ConfigurationCapabilities type) and pass those
around if needed.
Question: What's to stop someone accessing global capabilities without following
this approach?
Nothing in C# or F# can stop you accessing global public functions. Just like other best
practices, such as avoiding global variables, we have to rely on self-discipline (and maybe
code reviews) to keep us on the straight and narrow path!
But in the third part of this series, we'll look at a way to prevent access to global functions by
using access tokens.
Question: Aren't these just standard functional programming techniques?
Yes. I'm not claiming to be doing anything clever here!
Question: These capability functions have side-effects. What's up with that?
Yes, these capability functions are not pure. The goal here is not about being pure -- it's
about being explicit about the provision of capabilities.
Even if we used a pure IO context (e.g. in Haskell) it would not help control access to
capabilities. That is, in the context of security, there's a big difference between the capability
to change a password or credit card vs. the capability to change a background color
configuration, even though they are both just "IO" from a computation point of view.
Creating pure capabilities is possible but not very easy to do in F#, so I'm going to keep it
out of scope for this post.

1601

A functional approach to authorization

Question: What's your response to what (some person) wrote? And why didn't you
cite (some paper)?
This is a blog post, not an academic paper. I'm not an expert in this area, but just doing
some experiments of my own.
More importantly, as I said earlier, my goal here is very different from security experts -- I'm
not attempting to develop a ideal security model. Rather, I'm just trying to encourage some
good design practices that can help pragmatic developers avoid accidental vulnerabilities in
their code.
I've got more questions...
Some additional questions are answered at the end of part 2, so read those answers first.
Otherwise please add your question in the comments below, and I'll try to address it.

Further reading
The ideas on capability-based security here are mostly derived from the work of Mark Miller
and Marc Stiegler, and the erights.org website, although my version is cruder and simpler.
For a more complete understanding, I suggest you follow up on the links below:
The Wikipedia articles on Capability-based security and Object-capability model are a
good starting point.
What is a Capability, Anyway? by Jonathan Shapiro of the EROS project. He also
discusses ACL-based security vs. a capability-based model.
"The Lazy Programmer's Guide to Secure Computing", a great video on capabilitybased security by Marc Stiegler. Don't miss the last 5 mins (starting around
1h:02m:10s)!
"Object Capabilities for Security", a good talk by David Wagner.
A lot of work has been done on hardening languages for security and safety. For example
the E Language and Mark Miller's thesis on the E Language(PDF); the Joe-E Language built
on top of Java; Google's Caja built over JavaScript; Emily, a capability based language
derived from OCaml; and Safe Haskell(PDF).
My approach is not about strict safeness so much as proactively designing to avoid
unintentional breaches, and the references above do not focus on very much on design
specifically. The most useful thing I have found is a section on capability patterns in E.
Also, if you like this kind of thing, then head over to LtU where there are a number of
discussions, such as this one and this one and this paper.

1602

A functional approach to authorization

Coming up next
In the next post, we'll look at how to constrain capabilities based on claims such as the
current user's identity and role.
NOTE: All the code for this post is available as a gist here.

1603

Constraining capabilities based on identity and role

Constraining capabilities based on identity

and role
UPDATE: Slides and video from my talk on this topic
In the previous post, we started looking at "capabilities" as the basis for ensuring that code
could not do any more than it was supposed to do. And I demonstrated this with a simple
application that changed a configuration flag.
In this post, we'll look at how to constrain capabilities based on the current user's identity
and role.
So let's switch from the configuration example to a typical situation where stricter
authorization is required.

Database capabilities example

Consider a website and call-centre with a backing database. We have the following security
rules:
A customer can only view or update their own record in the database (via the website)
A call-centre operator can view or update any record in the database
This means that at some point, we'll have to do some authorization based on the identity
and role of the user. (We'll assume that the user has been authenticated successfully).
The tendency in many web frameworks is to put the authorization in the UI layer, often in the
controller. My concern about this approach is that once you are "inside" (past the gateway),
any part of the app has full authority to access the database, and it is all to easy for code to
do the wrong thing by mistake, resulting in a security breach.
Not only that, but because the authority is everywhere ("ambient"), it is hard to review the
code for potential security issues.
To avoid these issues, let's instead put the access logic as "low" as possible, in the database
access layer in this case.
We'll start with an obvious approach. We'll add the identity and role to each database call
and then do authorization there.

1604

Constraining capabilities based on identity and role

The following method assumes that there is a CustomerIdBelongsToPrincipal function that

checks whether the customer id being accessed is owned by the principal requesting
access. Then, if the customerId does belong to the principal, or the principal has the role of
"CustomerAgent", the access is granted.
public class CustomerDatabase
{
public CustomerData GetCustomer(CustomerId id, IPrincipal principal)
{
if ( CustomerIdBelongsToPrincipal(id, principal) ||
principal.IsInRole("CustomerAgent") )
{
// get customer data
}
else
{
// throw authorization exception
}
}
}

Note that I have deliberately added the IPrincipal to the method signature -- we are not
allowing any "magic" where the principal is fetched from a global context. As with the use of
any global, having implicit access hides the dependencies and makes it hard to test in
isolation.
Here's the F# equivalent, using a Success/Failure return value rather than throwing
exceptions:
let getCustomer id principal =
if customerIdBelongsToPrincipal id principal ||
principal.IsInRole("CustomerAgent")
then
// get customer data
Success "CustomerData"
else
Failure AuthorizationFailed

This "inline" authorization approach is all too common, but unfortunately it has many
problems.
It mixes up security concerns with the database logic. If the authorization logic gets
more complicated, the code will also get more complicated.
It throws an exception (C#) or returns an error (F#) if the authorization fails. It would be
nice if we could tell in advance if we had the authorization rather than waiting until the
last minute.

1605

Constraining capabilities based on identity and role

Let's compare this with a capability-based approach. Instead of directly getting a customer,
we first obtain the capability of doing it.
class CustomerDatabase
{
// "real" code is hidden from public view
private CustomerData GetCustomer(CustomerId id)
{
// get customer data
}
// Get the capability to call GetCustomer
public Func<CustomerId,CustomerData> GetCustomerCapability(CustomerId id, IPrincip
al principal)
{
if ( CustomerIdBelongsToPrincipal(id, principal) ||
principal.IsInRole("CustomerAgent") )
{
// return the capability (the real method)
return GetCustomer;
}
else
{
// throw authorization exception
}
}
}

As you can see, if the authorization succeeds, a reference to the GetCustomer method is
returned to the caller.
It might not be obvious, but the code above has a rather large security hole. I can request
the capability for a particular customer id, but I get back a function that can called for any
customer id! That's not very safe, is it?
What we need to is "bake in" the customer id to the capability, so that it can't be misused.
The return value will now be a Func<CustomerData> , with the customer id not available to be
passed in any more.

1606

Constraining capabilities based on identity and role

class CustomerDatabase
{
// "real" code is hidden from public view
private CustomerData GetCustomer(CustomerId id)
{
// get customer data
}
// Get the capability to call GetCustomer
public Func<CustomerData> GetCustomerCapability(CustomerId id, IPrincipal principal
)
{
if ( CustomerIdBelongsToPrincipal(id, principal) ||
principal.IsInRole("CustomerAgent") )
{
// return the capability (the real method)
return ( () => GetCustomer(id) );
}
else
{
// throw authorization exception
}
}
}

With this separation of concerns in place, we can now handle failure nicely, by returning an
optional value which is present if we get the capability, or absent if not. That is, we know
whether we have the capability at the time of trying to obtain it, not later on when we try to
use it.

1607

Constraining capabilities based on identity and role

class CustomerDatabase
{
// "real" code is hidden from public view
// and doesn't need any checking of identity or role
private CustomerData GetCustomer(CustomerId id)
{
// get customer data
}
// Get the capability to call GetCustomer. If not allowed, return None.
public Option<Func<CustomerData>> GetCustomerCapability(CustomerId id, IPrincipal
principal)
{
if (CustomerIdBelongsToPrincipal(id, principal) ||
principal.IsInRole("CustomerAgent"))
{
// return the capability (the real method)
return Option<Func<CustomerData>>.Some( () => GetCustomer(id) );
}
else
{
return Option<Func<CustomerData>>.None();
}
}
}

This assumes that we're using some sort of Option type in C# rather than just returning
null!
Finally, we can put the authorization logic into its own class (say
CustomerDatabaseCapabilityProvider ), to keep the authorization concerns separate from the
CustomerDatabase .

We'll have to find some way of keeping the "real" database functions private to all other
callers though. For now, I'll just assume the database code is in a different assembly, and
mark the code internal .

1608

Constraining capabilities based on identity and role

// not accessible to the business layer

internal class CustomerDatabase
{
// "real" code is hidden from public view
private CustomerData GetCustomer(CustomerId id)
{
// get customer data
}
}
// accessible to the business layer
public class CustomerDatabaseCapabilityProvider
{
CustomerDatabase _customerDatabase;
// Get the capability to call GetCustomer
public Option<Func<CustomerData>> GetCustomerCapability(CustomerId id, IPrincipal
principal)
{
if (CustomerIdBelongsToPrincipal(id, principal) ||
principal.IsInRole("CustomerAgent"))
{
// return the capability (the real method)
return Option<Func<CustomerData>>.Some( () => _customerDatabase.GetCustome
r(id) );
}
else
{
return Option<Func<CustomerData>>.None();
}
}
}

And here's the F# version of the same code:

/// not accessible to the business layer
module internal CustomerDatabase =
let getCustomer (id:CustomerId) :CustomerData =
// get customer data
/// accessible to the business layer
module CustomerDatabaseCapabilityProvider =
// Get the capability to call getCustomer
let getCustomerCapability (id:CustomerId) (principal:IPrincipal) =
let principalId = GetIdForPrincipal(principal)
if (principalId = id) || principal.IsInRole("CustomerAgent") then
Some ( fun () -> CustomerDatabase.getCustomer id )
else
None

1609

Constraining capabilities based on identity and role

Here's a diagram that represents this design:

Problems with this model

In this model, the caller is isolated from the CustomerDatabase , and the
CustomerDatabaseCapabilityProvider acts as a proxy between them.

Which means, as currently designed, for every function available in CustomerDatabase there
must be a parallel function available in CustomerDatabaseCapabilityProvider as well. We can
see that this approach will not scale well.
It would be nice if we had a way to generally get capabilities for a whole set of database
functions rather than one at a time. Let's see if we can do that!

Restricting and transforming capabilities

The getCustomer function in CustomerDatabase can be thought of as a capability with no
restrictions, while the getCustomerCapability returns a capability restricted by identity and
role.
But note that the two function signatures are similar ( CustomerId -> CustomerData vs unit > CustomerData ), and so they are almost interchangeable from the callers point of view. In a

sense, then, the second capability is a transformed version of the first, with additional
restrictions.
Transforming functions to new functions! This is something we can easily do.
So, let's write a transformer that, given any function of type CustomerId -> 'a , we return a
function with the customer id baked in ( unit -> 'a ), but only if the authorization
requirements are met.

1610

Constraining capabilities based on identity and role

module CustomerCapabilityFilter =
// Get the capability to use any function that has a CustomerId parameter
// but only if the caller has the same customer id or is a member of the
// CustomerAgent role.
let onlyForSameIdOrAgents (id:CustomerId) (principal:IPrincipal) (f:CustomerId ->
'a) =
let principalId = GetIdForPrincipal(principal)
if (principalId = id) || principal.IsInRole("CustomerAgent") then
Some (fun () -> f id)
else
None

The type signature for the onlyForSameIdOrAgents function is (CustomerId -> 'a) -> (unit > 'a) option . It accepts any CustomerId based function and returns, maybe, the same

function with the customer id already applied if the authorization succeeds. If the
authorization does not succeed, None is returned instead.
You can see that this function will work generically with any function that has a CustomerId
as the first parameter. That could be "get", "update", "delete", etc.
So for example, given:
module internal CustomerDatabase =
let getCustomer (id:CustomerId) =
// get customer data
let updateCustomer (id:CustomerId) (data:CustomerData) =
// update customer data

We can create restricted versions now, for example at the top level bootstrapper or
controller:
let principal = // from context
let id = // from context
// attempt to get the capabilities
let getCustomerOnlyForSameIdOrAgents =
onlyForSameIdOrAgents id principal CustomerDatabase.getCustomer
let updateCustomerOnlyForSameIdOrAgents =
onlyForSameIdOrAgents id principal CustomerDatabase.updateCustomer

The types of getCustomerOnlyForSameIdOrAgents and updateCustomerOnlyForSameIdOrAgents

are similar to the original functions in the database module, but with CustomerId replaced
with unit :

1611

Constraining capabilities based on identity and role

val getCustomerOnlyForSameIdOrAgents :
(unit -> CustomerData) option
val updateCustomerOnlyForSameIdOrAgents :
(unit -> CustomerData -> unit) option

The updateCustomerOnlyForSameIdOrAgents has a extra CustomerData parameter, so the

extra unit where the CustomerId used to be is a bit ugly. If this is too annoying, you could
easily create other versions of the function which handle this more elegantly. I'll leave that as
an exercise for the reader!
So now we have an option value that might or might not contain the capability we wanted. If
it does, we can create a child component and pass in the capability. If it does not, we can
return some sort of error, or hide a element from a view, depending on the type of
application.
match getCustomerOnlyForSameIdOrAgents with
| Some cap -> // create child component and pass in the capability
| None -> // return error saying that you don't have the capability to get the data

Here's a diagram that represents this design:

More transforms on capabilities

Because capabilities are functions, we can easily create new capabilities by chaining or
combining transformations.
For example, we could create a separate filter function for each business rule, like this:

1612

Constraining capabilities based on identity and role

module CustomerCapabilityFilter =
let onlyForSameId (id:CustomerId) (principal:IPrincipal) (f:CustomerId -> 'a) =
if customerIdBelongsToPrincipal id principal then
Some (fun () -> f id)
else
None
let onlyForAgents (id:CustomerId) (principal:IPrincipal) (f:CustomerId -> 'a) =
if principal.IsInRole("CustomerAgent") then
Some (fun () -> f id)
else
None

For the first business rule, onlyForSameId , we return a capability with the customer id baked
in, as before.
The second business rule, onlyForAgents , doesn't mention customer ids anywhere, so why
do we restrict the function parameter to CustomerId -> 'a ? The reason is that it enforces
that this rule only applies to customer centric capabilities, not ones relating to products or
payments, say.
But now, to make the output of this filter compatible with the first rule ( unit -> 'a ), we need
to pass in a customer id and partially apply it too. It's a bit of a hack but it will do for now.
We can also write a generic combinator that returns the first valid capability from a list.
// given a list of capability options,
// return the first good one, if any
let first capabilityList =
capabilityList |> List.tryPick id

It's a trivial implementation really -- this is the kind of helper function that is just to help the
code be a little more self-documenting.
With this in place, we can apply the rules separately, take the two filters and combine them
into one.
let getCustomerOnlyForSameIdOrAgents =
let f = CustomerDatabase.getCustomer
let cap1 = onlyForSameId id principal f
let cap2 = onlyForAgents id principal f
first [cap1; cap2]
// val getCustomerOnlyForSameIdOrAgents : (CustomerId -> CustomerData) option

1613

Constraining capabilities based on identity and role

Or let's say we have some sort of restriction; the operation can only be performed during
business hours, say.
let onlyIfDuringBusinessHours (time:DateTime) f =
if time.Hour >= 8 && time.Hour <= 17 then
Some f
else
None

We can write another combinator that restricts the original capability. This is just a version of
"bind".
// given a capability option, restrict it
let restrict filter originalCap =
originalCap
|> Option.bind filter

With this in place, we can restrict the "agentsOnly" capability to business hours:
let getCustomerOnlyForAgentsInBusinessHours =
let f = CustomerDatabase.getCustomer
let cap1 = onlyForAgents id principal f
let restriction f = onlyIfDuringBusinessHours (DateTime.Now) f
cap1 |> restrict restriction

So now we have created a new capability, "Customer agents can only access customer data
during business hours", which tightens the data access logic a bit more.
We can combine this with the previous onlyForSameId filter to build a compound capability
which can access customer data:
if you have the same customer id (at any time of day)
if you are a customer agent (only during business hours)
let getCustomerOnlyForSameId =
let f = CustomerDatabase.getCustomer
onlyForSameId id principal f
let getCustomerOnlyForSameId_OrForAgentsInBusinessHours =
let cap1 = getCustomerOnlyForSameId
let cap2 = getCustomerOnlyForAgentsInBusinessHours
first [cap1; cap2]

As you can see, this approach is a useful way to build complex capabilities from simpler
ones.

1614

Constraining capabilities based on identity and role

Additional transforms
It should be obvious that you can easily create additional transforms which can extend
capabilities in other ways. Some examples:
a capability that writes to an audit log on each execution.
a capability that can only be performed once.
a capability that can be revoked when needed.
a capability that is throttled and can only be performed a limited number of times in a
given time period (such as password change attempts).
And so on.
Here are implementations of the first three of them:
/// Uses of the capability will be audited
let auditable capabilityName f =
fun x ->
// simple audit log!
printfn "AUDIT: calling %s with %A" capabilityName x
// use the capability
f x
/// Allow the function to be called once only
let onlyOnce f =
let allow = ref true
fun x ->
if !allow then //! is dereferencing not negation!
allow := false
f x
else
Failure OnlyAllowedOnce
/// Return a pair of functions: the revokable capability,
/// and the revoker function
let revokable f =
let allow = ref true
let capability = fun x ->
if !allow then //! is dereferencing not negation!
f x
else
Failure Revoked
let revoker() =
allow := false
capability, revoker

Let's say that we have an updatePassword function, such as this:

1615

Constraining capabilities based on identity and role

module internal CustomerDatabase =

let updatePassword (id,password) =
Success "OK"

We can then create a auditable version of updatePassword :

let updatePasswordWithAudit x =
auditable "updatePassword" CustomerDatabase.updatePassword x

And then test it:

updatePasswordWithAudit (1,"password")
updatePasswordWithAudit (1,"new password")

The results are:

AUDIT: calling updatePassword with (1, "password")
AUDIT: calling updatePassword with (1, "new password")

Or, we could create a one-time only version:

let updatePasswordOnce =
onlyOnce CustomerDatabase.updatePassword

And then test it:

updatePasswordOnce (1,"password") |> printfn "Result 1st time: %A"
updatePasswordOnce (1,"password") |> printfn "Result 2nd time: %A"

The results are:

Result 1st time: Success "OK"
Result 2nd time: Failure OnlyAllowedOnce

Finally, we can create a revokable function:

let revokableUpdatePassword, revoker =
revokable CustomerDatabase.updatePassword

And then test it:

1616

Constraining capabilities based on identity and role

revokableUpdatePassword (1,"password") |> printfn "Result 1st time before revoking: %A"
revokableUpdatePassword (1,"password") |> printfn "Result 2nd time before revoking: %A"
revoker()
revokableUpdatePassword (1,"password") |> printfn "Result 3nd time after revoking: %A"

With the following results:

Result 1st time before revoking: Success "OK"
Result 2nd time before revoking: Success "OK"
Result 3nd time after revoking: Failure Revoked

The code for all these F# examples is available as a gist here.

A complete example in F#
Here's the code to a complete application in F# (also available as a gist here).
This example consists of a simple console app that allows you to get and update customer
records.
The first step is to login as a user. "Alice" and "Bob" are normal users, while "Zelda" has
a customer agent role.
Once logged in, you can select a customer to edit. Again, you are limited to a choice
between "Alice" and "Bob". (I'm sure you can hardly contain your excitement)
Once a customer is selected, you are presented with some (or none) of the following
options:
Get a customer's data.
Update a customer's data.
Update a customer's password.
Which options are shown depend on which capabilities you have. These in turn are based
on who you are logged in as, and which customer is selected.

Implementing the domain

We'll start with the core domain types that are shared across the application:

1617

Constraining capabilities based on identity and role

module Domain =
open Rop
type CustomerId = CustomerId of int
type CustomerData = CustomerData of string
type Password = Password of string
type FailureCase =
| AuthenticationFailed of string
| AuthorizationFailed
| CustomerNameNotFound of string
| CustomerIdNotFound of CustomerId
| OnlyAllowedOnce
| CapabilityRevoked

The FailureCase type documents all possible things that can go wrong at the top-level of
the application. See the "Railway Oriented Programming" talk for more discussion on this.

Defining the capabilities

Next, we document all the capabilities that are available in the application. To add clarity to
the code, each capability is given a name (i.e. a type alias).
type GetCustomerCap = unit -> SuccessFailure<CustomerData,FailureCase>

Finally, the CapabilityProvider is a record of functions, each of which accepts a customer

id and principal, and returns an optional capability of the specified type. This record is
created in the top level model and then passed around to the child components.
Here's the complete code for this module:

1618

Constraining capabilities based on identity and role

module Capabilities =
open Rop
open Domain
// capabilities
type GetCustomerCap = unit -> SuccessFailure<CustomerData,FailureCase>
type UpdateCustomerCap = unit -> CustomerData -> SuccessFailure<unit,FailureCase>
type UpdatePasswordCap = Password -> SuccessFailure<unit,FailureCase>
type CapabilityProvider = {
/// given a customerId and IPrincipal, attempt to get the GetCustomer capabili
ty
getCustomer : CustomerId -> IPrincipal -> GetCustomerCap option
/// given a customerId and IPrincipal, attempt to get the UpdateCustomer capab
ility
updateCustomer : CustomerId -> IPrincipal -> UpdateCustomerCap option
/// given a customerId and IPrincipal, attempt to get the UpdatePassword capab
ility
updatePassword : CustomerId -> IPrincipal -> UpdatePasswordCap option
}

This module references a SuccessFailure result type similar to the one discussed here, but
which I won't show.

Implementing authentication
Next, we'll roll our own little authentication system. Note that when the user "Zelda" is
authenticated, the role is set to "CustomerAgent".

1619

Constraining capabilities based on identity and role

The customerIdForName function attempts to find the customer id associated with a particular
name, while the customerIdOwnedByPrincipal compares this id with another one.

Implementing authorization
Here are the functions related to authorization, very similar to what was discussed above.
module Authorization =
open Rop
open Domain
let onlyForSameId (id:CustomerId) (principal:IPrincipal) (f:CustomerId -> 'a) =
if Authentication.customerIdOwnedByPrincipal id principal then
Some (fun () -> f id)
else
None

1620

Constraining capabilities based on identity and role

let onlyForAgents (id:CustomerId) (principal:IPrincipal) (f:CustomerId -> 'a) =

if principal.IsInRole(Authentication.customerAgentRole) then
Some (fun () -> f id)
else
None
let onlyIfDuringBusinessHours (time:DateTime) f =
if time.Hour >= 8 && time.Hour <= 17 then
Some f
else
None
// constrain who can call a password update function
let passwordUpdate (id:CustomerId) (principal:IPrincipal) (f:CustomerId*Password > 'a) =
if Authentication.customerIdOwnedByPrincipal id principal then
Some (fun password -> f (id,password))
else
None
// return the first good capability, if any
let first capabilityList =
capabilityList |> List.tryPick id
// given a capability option, restrict it
let restrict filter originalCap =
originalCap
|> Option.bind filter
/// Uses of the capability will be audited
let auditable capabilityName principalName f =
fun x ->
// simple audit log!
let timestamp = DateTime.UtcNow.ToString("u")
printfn "AUDIT: User %s used capability %s at %s" principalName capability
Name timestamp
// use the capability
f x
/// Return a pair of functions: the revokable capability,
/// and the revoker function
let revokable f =
let allow = ref true
let capability = fun x ->
if !allow then //! is dereferencing not negation!
f x
else
Failure CapabilityRevoked
let revoker() =
allow := false
capability, revoker

1621

Constraining capabilities based on identity and role

Implementing the database

The functions related to database access are similar to those in the earlier examples, only
this time we have implemented a crude in-memory database (just a Dictionary ).
module CustomerDatabase =
open Rop
open System.Collections.Generic
open Domain
let private db = Dictionary<CustomerId,CustomerData>()
let getCustomer id =
match db.TryGetValue id with
| true, value -> Success value
| false, _ -> Failure (CustomerIdNotFound id)
let updateCustomer id data =
db.[id] <- data
Success ()
let updatePassword (id:CustomerId,password:Password) =
Success () // dummy implementation

Implementing the business services

Next we have the "business services" (for lack of better word) where all the work gets done.

1622

Constraining capabilities based on identity and role

module BusinessServices =
open Rop
open Domain
// use the getCustomer capability
let getCustomer capability =
match capability() with
| Success data -> printfn "%A" data
| Failure err -> printfn ".. %A" err
// use the updateCustomer capability
let updateCustomer capability =
printfn "Enter new data: "
let customerData = Console.ReadLine() |> CustomerData
match capability () customerData with
| Success _ -> printfn "Data updated"
| Failure err -> printfn ".. %A" err
// use the updatePassword capability
let updatePassword capability =
printfn "Enter new password: "
let password = Console.ReadLine() |> Password
match capability password with
| Success _ -> printfn "Password updated"
| Failure err -> printfn ".. %A" err

Note that each of these functions is passed in only the capability needed to do its job. This
code knows nothing about databases, or anything else.
Yes, in this crude example, the code is reading and writing directly to the console. Obviously
in a more complex (and less crude!) design, the inputs to these functions would be passed in
as parameters.
Here's a simple exercise: replace the direct access to the console with a capability such as
getDataWithPrompt ?

Implementing the user interface

Now for the user interface module, where most of the complex code lies.
First up is a type ( CurrentState ) that represents the state of the user interface.
When we're LoggedOut there is no IPrincipal available.
When we're LoggedIn there is a IPrincipal available, but no selected customer.
When we're in the CustomerSelected state there is both a IPrincipal and a
CustomerId available.

Finally, the Exit state is a signal to the app to shutdown.

1623

Constraining capabilities based on identity and role

I very much like using a "state" design like this, because it ensures that we can't accidentally
access data that we shouldn't. For example, we literally cannot access a customer when
none is selected, because there is no customer id in that state!
For each state, there is a corresponding function.
loggedOutActions is run when we are in the LoggedOut state. It presents the available

actions to you, and changes the state accordingly. You can log in as a user, or exit. If the
login is successful ( authenticate name worked) then the state is changed to LoggedIn .
loggedInActions is run when we are in the LoggedIn state. You can select a customer, or

log out. If the customer selection is successful ( customerIdForName customerName worked)

then the state is changed to CustomerSelected .
selectedCustomerActions is run when we are in the CustomerSelected state. This works as

follows:
First, find out what capabilities we have.
Next convert each capability into a corresponding menu text (using Option.map
because the capability might be missing), then remove the ones that are None.
Next, read a line from input, and depending on what it is, call one of the "business
services" ( getCustomer , updateCustomer , or updatePassword ).
Finally the mainUiLoop function loops around until the state is set to Exit .
module UserInterface =
open Rop
open Domain
open Capabilities
type CurrentState =
| LoggedOut
| LoggedIn of IPrincipal
| CustomerSelected of IPrincipal * CustomerId
| Exit
/// do the actions available while you are logged out. Return the new state
let loggedOutActions originalState =
printfn "[Login] enter Alice, Bob, Zelda, or Exit: "
let action = Console.ReadLine()
match action with
| "Exit" ->
// Change state to Exit
Exit
| name ->
// otherwise try to authenticate the name
match Authentication.authenticate name with
| Success principal ->
LoggedIn principal

1624

Constraining capabilities based on identity and role

| Failure err ->

printfn ".. %A" err
originalState
/// do the actions available while you are logged in. Return the new state
let loggedInActions originalState (principal:IPrincipal) =
printfn "[%s] Pick a customer to work on. Enter Alice, Bob, or Logout: " princ
ipal.Identity.Name
let action = Console.ReadLine()
match action with
| "Logout" ->
// Change state to LoggedOut
LoggedOut
// otherwise treat it as a customer name
| customerName ->
// Attempt to find customer
match Authentication.customerIdForName customerName with
| Success customerId ->
// found -- change state
CustomerSelected (principal,customerId)
| Failure err ->
// not found -- stay in originalState
printfn ".. %A" err
originalState
let getAvailableCapabilities capabilityProvider customerId principal =
let getCustomer = capabilityProvider.getCustomer customerId principal
let updateCustomer = capabilityProvider.updateCustomer customerId principal
let updatePassword = capabilityProvider.updatePassword customerId principal
getCustomer,updateCustomer,updatePassword
/// do the actions available when a selected customer is available. Return the new
state
let selectedCustomerActions originalState capabilityProvider customerId principal
=
// get the individual component capabilities from the provider
let getCustomerCap,updateCustomerCap,updatePasswordCap =
getAvailableCapabilities capabilityProvider customerId principal
// get the text for menu options based on capabilities that are present
let menuOptionTexts =
[
getCustomerCap |> Option.map (fun _ -> "(G)et");
updateCustomerCap |> Option.map (fun _ -> "(U)pdate");
updatePasswordCap |> Option.map (fun _ -> "(P)assword");
]
|> List.choose id
// show the menu
let actionText =
match menuOptionTexts with

1625

Constraining capabilities based on identity and role

| [] -> " (no other actions available)"

| texts -> texts |> List.reduce (fun s t -> s + ", " + t)
printfn "[%s] (D)eselect customer, %s" principal.Identity.Name actionText
// process the user action
let action = Console.ReadLine().ToUpper()
match action with
| "D" ->
// revert to logged in with no selected customer
LoggedIn principal
| "G" ->
// use Option.iter in case we don't have the capability
getCustomerCap
|> Option.iter BusinessServices.getCustomer
originalState // stay in same state
| "U" ->
updateCustomerCap
|> Option.iter BusinessServices.updateCustomer
originalState
| "P" ->
updatePasswordCap
|> Option.iter BusinessServices.updatePassword
originalState
| _ ->
// unknown option
originalState
let rec mainUiLoop capabilityProvider state =
match state with
| LoggedOut ->
let newState = loggedOutActions state
mainUiLoop capabilityProvider newState
| LoggedIn principal ->
let newState = loggedInActions state principal
mainUiLoop capabilityProvider newState
| CustomerSelected (principal,customerId) ->
let newState = selectedCustomerActions state capabilityProvider customerId
principal
mainUiLoop capabilityProvider newState
| Exit ->
() // done
let start capabilityProvider =
mainUiLoop capabilityProvider LoggedOut

Implementing the top-level module

With all this in place, we can implement the top-level module.
This module fetches all the capabilities, adds restrictions as explained previously, and
creates a capabilities record.

1626

Constraining capabilities based on identity and role

The capabilities record is then passed into the user interface when the app is started.
module Application=
open Rop
open Domain
open CustomerDatabase
open Authentication
open Authorization

let capabilities =
let getCustomerOnlyForSameId id principal =
onlyForSameId id principal CustomerDatabase.getCustomer
let getCustomerOnlyForAgentsInBusinessHours id principal =
let f = CustomerDatabase.getCustomer
let cap1 = onlyForAgents id principal f
let restriction f = onlyIfDuringBusinessHours (DateTime.Now) f
cap1 |> restrict restriction
let getCustomerOnlyForSameId_OrForAgentsInBusinessHours id principal =
let cap1 = getCustomerOnlyForSameId id principal
let cap2 = getCustomerOnlyForAgentsInBusinessHours id principal
first [cap1; cap2]
let updateCustomerOnlyForSameId id principal =
onlyForSameId id principal CustomerDatabase.updateCustomer
let updateCustomerOnlyForAgentsInBusinessHours id principal =
let f = CustomerDatabase.updateCustomer
let cap1 = onlyForAgents id principal f
let restriction f = onlyIfDuringBusinessHours (DateTime.Now) f
cap1 |> restrict restriction
let updateCustomerOnlyForSameId_OrForAgentsInBusinessHours id principal =
let cap1 = updateCustomerOnlyForSameId id principal
let cap2 = updateCustomerOnlyForAgentsInBusinessHours id principal
first [cap1; cap2]
let updatePasswordOnlyForSameId id principal =
let cap = passwordUpdate id principal CustomerDatabase.updatePassword
cap
|> Option.map (auditable "UpdatePassword" principal.Identity.Name)
// create the record that contains the capabilities
{
getCustomer = getCustomerOnlyForSameId_OrForAgentsInBusinessHours
updateCustomer = updateCustomerOnlyForSameId_OrForAgentsInBusinessHours
updatePassword = updatePasswordOnlyForSameId
}

1627

Constraining capabilities based on identity and role

let start() =
// pass capabilities to UI
UserInterface.start capabilities

The complete code for this example is available as a gist here.

Summary of Part 2
In part 2, we added authorization and other transforms as a separate concern that could be
applied to restrict authority. Again, there is nothing particularly clever about using functions
like this, but I hope that this has given you some ideas that might be useful.
Question: Why go to all this trouble? What's the benefit over just testing an
"IsAuthorized" flag or something?
Here's a typical use of a authorization flag:
if user.CanUpdate() then
doTheAction()

Recall the quote from the previous post: "Capabilities should 'fail safe'. If a capability cannot
be obtained, or doesn't work, we must not allow any progress on paths that assumed that it
was successful."
The problem with testing a flag like this is that it's easy to forget, and the compiler won't
complain if you do. And then you have a possible security breach, as in the following code.
if user.CanUpdate() then
// ignore
// now do the action anyway!
doTheAction()

Not only that, but by "inlining" the test like this, we're mixing security concerns into our main
code, as pointed out earlier.
In contrast, a simple capability approach looks like this:
let updateCapability = // attempt to get the capability
match updateCapability with
| Some update -> update() // call the function
| None -> () // can't call the function

1628

Constraining capabilities based on identity and role

In this example, it is not possible to accidentally use the capability if you are not allowed
to, as you literally don't have a function to call! And this has to be handled at compile-time,
not at runtime.
Furthermore, as we have just seen, capabilities are just functions, so we get all the benefits
of filtering, etc., which are not available with the inlined boolean test version.
Question: In many situations, you don't know whether you can access a resource
until you try. So aren't capabilities just extra work?
This is indeed true. For example, you might want to test whether a file exists first, and only
then try to access it. The IT gods are always ruthless in these cases, and in the time
between checking the file's existence and trying to open it, the file will probably be deleted!
So since we will have to check for exceptions anyway, why do two slow I/O operations when
one would have sufficed?
The answer is that the capability model is not about physical or system-level authority, but
logical authority -- only having the minimum you need to accomplish a task.
For example, a web service process may be operating at a high level of system authority,
and can access any database record. But we don't want to expose that to most of our code.
We want to make sure that any failures in programming logic cannot accidentally expose
unauthorized data.
Yes, of course, the capability functions themselves must do error handling, and as you can
see in the snippets above, I'm using the Success/Failure result type as described here. As
a result, we will need to merge failures from core functions (e.g. database errors) with
capability-specific failures such as Failure OnlyAllowedOnce .
Question: You've created a whole module with types defined for each capability. I
might have hundreds of capabilities. Do you really expect me to do all this extra
work?
There are two points here, so let's address each one in turn:
First, do you have a system that already uses fine-grained authorization, or has businesscritical requirements about not leaking data, or performing actions in an unauthorized
context, or needs a security audit?
If none of these apply, then indeed, this approach is complete overkill!
But if you do have such a system, that raises some new questions:
should the capabilities that are authorized be explicitly described in the code
somewhere?

1629

Constraining capabilities based on identity and role

and if so, should the capabilities be explicit throughout the code, or only at the top-level
(e.g. in the controller) and implicit everywhere else.
The question comes to down to whether you want to be explicit or implicit.
Personally, I prefer things like this to be explicit. It may be a little extra work initially, just a
few lines to define each capability, but I find that it generally stops problems from occurring
further down the line.
And it has the benefit of acting as a single place to document all the security-related
capabilities that you support. Any new requirements would require a new entry here, so can
be sure that no capabilities can sneak in under the radar (assuming developers follow these
practices).
Question: In this code, you've rolled your own authorization. Wouldn't you use a
proper authorization provider instead?
Yes. This code is just an example. The authorization logic is completely separate from the
domain logic, so it should be easy to substitute any authorization provider, such as
ClaimsAuthorizationManager class, or something like XACML.

I've got more questions...

If you missed them, some additional questions are answered at the end of part 1. Otherwise
please add your question in the comments below, and I'll try to address it.

Coming up
In the next post, we'll look at how to use types to emulate access tokens and prevent
unauthorized access to global functions.
NOTE: All the code for this post is available as a gist here and here.

1630

Using types as access tokens

UPDATE: Slides and video from my talk on this topic
In the previous posts (link, link) we looked at "capabilities" as the basis for locking down
code.
But in most of the examples so far, we've been relying on self-discipline to avoid using the
global capabilities, or by trying to hide the "raw" capabilities using the internal keyword.
It's a bit ugly -- can we do better?
In this post, we'll show that we can by using types to emulate "access tokens".

Real-world authorization
First, let's step back and look at how authorization works in the real world.
Here's a simplified diagram of a basic authorization system (such as OAuth 2.0).

The steps, in their crudest form, are:

The client presents some claims to the Authorization Service, including identity and the
id and scope (capability) of the service it wants to access.
The Authorization Service checks whether the client is authorized, and if so, creates an
access token which is returned to the client.
The client then presents this access token to the Resource Service (the service the
client wants to use).
In general, the access token will only let the client do certain things. In our terminology,
it has been granted a limited set of capabilities.
Obviously, there's a lot more to it than that, but it will be enough to give us some ideas.

1631

Using types as access tokens

Implementing an Access Token

If we want to emulate this in our design, it's clear that we need some sort of "access token".
Since we're running in a single process, and the primary goal is to stop accidental errors, we
don't need to do cryptographic signatures and all that. All we need is some object that can
only be created by an authorization service.
That's easy. We can just use a type with a private constructor!
We'll set it up so that the type can only be created by an Authorization Service, but is
required to be passed in to the database service.
For example, here's an F# implementation of the AccessToken type. The constructor is
private, and there's a static member that returns an instance if authorization is allowed.
type AccessToken private() =
// create an AccessToken that allows access to a particular customer
static member getAccessToCustomer id principal =
let principalId = GetIdForPrincipal(principal)
if (principalId = id) || principal.IsInRole("CustomerAgent") then
Some <| AccessToken()
else
None

Next, in the database module, we will add an extra parameter to each function, which is the
AccessToken.
Because the AccessToken token is required, we can safely make the database module
public now, as no unauthorized client can call the functions.
let getCustomer (accessToken:AccessToken) (id:CustomerId) =
// get customer data
let updateCustomer (accessToken:AccessToken) (id:CustomerId) (data:CustomerData) =
// update database

Note that the accessToken is not actually used in the implementation. It is just there to force
callers to obtain a token at compile time.
So let's look at how this might be used in practice.

1632

Using types as access tokens

let principal = // from context

let id = // from context
// attempt to get an access token
let accessToken = AuthorizationService.AccessToken.getAccessToCustomer id principal

At this point we have an optional access token. Using Option.map , we can apply it to
CustomerDatabase.getCustomer to get an optional capability. And by partially applying the

access token, the user of the capability is isolated from the authentication process.
let getCustomerCapability =
accessToken |> Option.map CustomerDatabase.getCustomer

And finally, we can attempt to use the capability, if present.

match getCustomerCapability with
| Some getCustomer -> getCustomer id
| None -> Failure AuthorizationFailed // error

So now we have a statically typed authorization system that will prevent us from accidentally
getting too much access to the database.

Oops! We have made a big mistake...

This design looks fine on the surface, but we haven't actually made anything more secure.
The first problem is that the AccessToken type is too broad. If I can somehow get hold of an
access token for innocently writing to a config file, I might also be able to use it to maliciously
update passwords as well.
The second problem is that the AccessToken throws away the context of the operation. For
example, I might get an access token for updating CustomerId 1 , but when I actually use
the capability, I could pass in CustomerId 2 as the the customer id instead!
The answer to both these issues is to store information in the access token itself, at the point
when the authorization is granted.
For example, if the token stores the operation that was requested, the service can check that
the token matches the operation being called, which ensures that the token can only be
used for that particular operation. In fact, as we'll see in a minute, we can be lazy and have
the compiler do this checking for us!

1633

Using types as access tokens

And, if we also store any data (such as the customer id) that was part of the authorization
request, then we don't need to ask for it again in the service.
What's more, we can trust that the information stored in the token is not forged or tampered
with because only the Authorization Service can create the token. In other words, this is the
equivalent of the token being "signed".

Revisiting the Access Token design

So let's revisit the design and fix it up.
First we will define a distinct type for each capability. The type will also contain any data
needed at authorization time, such as the customer id.
For example, here are two types that represent access to capabilities, one for accessing a
customer (both read and update), and another one updating a password. Both of these will
store the CustomerId that was provided at authorization time.
type AccessCustomer = AccessCustomer of CustomerId
type UpdatePassword = UpdatePassword of CustomerId

Next, the AccessToken type is redefined to be a generic container with a data field. The
constructor is still private, but a public getter is added so clients can access the data field.
type AccessToken<'data> = private {data:'data} with
// but do allow read access to the data
member this.Data = this.data

The authorization implementation is similar to the previous examples, except that this time
the capability type and customer id are stored in the token.

1634

Using types as access tokens

// create an AccessToken that allows access to a particular customer

let getAccessCustomerToken id principal =
if customerIdBelongsToPrincipal id principal ||
principal.IsInRole("CustomerAgent")
then
Some {data=AccessCustomer id}
else
None
// create an AccessToken that allows access to UpdatePassword
let getUpdatePasswordToken id principal =
if customerIdBelongsToPrincipal id principal then
Some {data=UpdatePassword id}
else
None

Using Access Tokens in the database

With these access token types in place the database functions can be rewritten to require a
token of a particular type. The customerId is no longer needed as an explicit parameter,
because it will be passed in as part of the access token's data.
Note also that both getCustomer and updateCustomer can use the same type of token
( AccessCustomer ), but updatePassword requires a different type ( UpdatePassword ).
let getCustomer (accessToken:AccessToken<AccessCustomer>) =
// get customer id
let (AccessCustomer id) = accessToken.Data
// now get customer data using the id
match db.TryGetValue id with
| true, value -> Success value
| false, _ -> Failure (CustomerIdNotFound id)
let updateCustomer (accessToken:AccessToken<AccessCustomer>) (data:CustomerData) =
// get customer id
let (AccessCustomer id) = accessToken.Data
// update database
db.[id] <- data
Success ()
let updatePassword (accessToken:AccessToken<UpdatePassword>) (password:Password) =
Success () // dummy implementation

Putting it all together

1635

Using types as access tokens

So now let's see all this in action.

The steps to getting a customer are:
Attempt to get the access token from the authorization service
If you have the access token, get the getCustomer capability from the database
Finally, if you have the capability, you can use it.
Note that, as always, the getCustomer capability does not take a customer id parameter. It
was baked in when the capability was created.
let principal = // from context
let customerId = // from context
// attempt to get a capability
let getCustomerCap =
// attempt to get a token
let accessToken = AuthorizationService.getAccessCustomerToken customerId principal
match accessToken with
// if token is present pass the token to CustomerDatabase.getCustomer,
// and return a unit->CustomerData
| Some token ->
Some (fun () -> CustomerDatabase.getCustomer token)
| None -> None
// use the capability, if available
match getCustomerCap with
| Some getCustomer -> getCustomer()
| None -> Failure AuthorizationFailed // error

Now what happens if we accidentally get the wrong type of access token? For example, let
us try to access the updatePassword function with an AccessCustomer token.
// attempt to get a capability
let getUpdatePasswordCap =
let accessToken = AuthorizationService.getAccessCustomerToken customerId principal
match accessToken with
| Some token ->
Some (fun password -> CustomerDatabase.updatePassword token password)
| None -> None
match getUpdatePasswordCap with
| Some updatePassword ->
let password = Password "p@ssw0rd"
updatePassword password
| None ->
Failure AuthorizationFailed // error

1636

Using types as access tokens

This code will not even compile! The line CustomerDatabase.updatePassword token password
has an error.
error FS0001: Type mismatch. Expecting a
AccessToken<Capabilities.UpdatePassword>
but given a
AccessToken<Capabilities.AccessCustomer>
The type 'Capabilities.UpdatePassword' does not match the type 'Capabilities.AccessCus
tomer'

We have accidentally fetched the wrong kind of Access Token, but we have been stopped
from accessing the wrong database method at compile time.
Using types in this way is a nice solution to the problem of global access to a potentially
dangerous capability.

A complete example in F#
In the last post, I showed a complete console application in F# that used capabilities to
update a database.
Now let's update it to use access tokens as well. (The code is available as a gist here).
Since this is an update of the example, I'll focus on just the changes.

Defining the capabilities

The capabilities are as before except that we have defined the two new types
( AccessCustomer and UpdatePassword ) to be stored inside the access tokens.

1637

Using types as access tokens

module Capabilities =
open Rop
open Domain
// each access token gets its own type
type AccessCustomer = AccessCustomer of CustomerId
type UpdatePassword = UpdatePassword of CustomerId
// capabilities
type GetCustomerCap = unit -> SuccessFailure<CustomerData,FailureCase>
type UpdateCustomerCap = CustomerData -> SuccessFailure<unit,FailureCase>
type UpdatePasswordCap = Password -> SuccessFailure<unit,FailureCase>
type CapabilityProvider = {
/// given a customerId and IPrincipal, attempt to get the GetCustomer capabili
ty
getCustomer : CustomerId -> IPrincipal -> GetCustomerCap option
/// given a customerId and IPrincipal, attempt to get the UpdateCustomer capab
ility
updateCustomer : CustomerId -> IPrincipal -> UpdateCustomerCap option
/// given a customerId and IPrincipal, attempt to get the UpdatePassword capab
ility
updatePassword : CustomerId -> IPrincipal -> UpdatePasswordCap option
}

Implementing authorization
The authorization implementation must be changed to return AccessTokens now. The
onlyIfDuringBusinessHours restriction applies to capabilities, not access tokens, so it is

unchanged.

1638

Using types as access tokens

// the constructor is protected

type AccessToken<'data> = private {data:'data} with
// but do allow read access to the data
member this.Data = this.data
let onlyForSameId (id:CustomerId) (principal:IPrincipal) =
if Authentication.customerIdOwnedByPrincipal id principal then
Some {data=AccessCustomer id}
else
None
let onlyForAgents (id:CustomerId) (principal:IPrincipal) =
if principal.IsInRole(Authentication.customerAgentRole) then
Some {data=AccessCustomer id}
else
None
let onlyIfDuringBusinessHours (time:DateTime) f =
if time.Hour >= 8 && time.Hour <= 17 then
Some f
else
None
// constrain who can call a password update function
let passwordUpdate (id:CustomerId) (principal:IPrincipal) =
if Authentication.customerIdOwnedByPrincipal id principal then
Some {data=UpdatePassword id}
else
None

Implementing the database

Compared with the example from the previous post, the database functions have the
CustomerId parameter replaced with an AccessToken instead.

Here's what the database implementation looked like before using access tokens:
let getCustomer id =
// code
let updateCustomer id data =
// code
let updatePassword (id:CustomerId,password:Password) =
// code

And here's what the code looks like after using access tokens:

1639

Using types as access tokens

let getCustomer (accessToken:AccessToken<AccessCustomer>) =

// get customer id
let (AccessCustomer id) = accessToken.Data
// now get customer data using the id
// as before
let updateCustomer (accessToken:AccessToken<AccessCustomer>) (data:CustomerData) =
// get customer id
let (AccessCustomer id) = accessToken.Data
// update database
// as before
let updatePassword (accessToken:AccessToken<UpdatePassword>) (password:Password) =
// as before

Implementing the business services and user interface

The code relating to the business services and UI is completely unchanged.
Because these functions have been passed capabilities only, they are decoupled from both
the lower levels and higher levels of the application, so any change in the authorization logic
has no effect on these layers.

Implementing the top-level module

The major change in the top-level module is how the capabilities are fetched. We now have
an additional step of getting the access token first.
Here's what the code looked like before using access tokens:
let getCustomerOnlyForSameId id principal =
onlyForSameId id principal CustomerDatabase.getCustomer
let getCustomerOnlyForAgentsInBusinessHours id principal =
let cap1 = onlyForAgents id principal CustomerDatabase.getCustomer
let restriction f = onlyIfDuringBusinessHours (DateTime.Now) f
cap1 |> restrict restriction

And here's what the code looks like after using access tokens:

1640

Using types as access tokens

let getCustomerOnlyForSameId id principal =

let accessToken = Authorization.onlyForSameId id principal
accessToken |> tokenToCap CustomerDatabase.getCustomer
let getCustomerOnlyForAgentsInBusinessHours id principal =
let accessToken = Authorization.onlyForAgents id principal
let cap1 = accessToken |> tokenToCap CustomerDatabase.getCustomer
let restriction f = onlyIfDuringBusinessHours (DateTime.Now) f
cap1 |> restrict restriction

The tokenToCap function is a little utility that applies the (optional) token to a given function
as the first parameter. The output is an (equally optional) capability.
let tokenToCap f token =
token
|> Option.map (fun token ->
fun () -> f token)

And that's it for the changes needed to support access tokens. You can see all the code for
this example here.

Summary of Part 3
In this post, we used types to represent access tokens, as follows:
The AccessToken type is the equivalent of a signed ticket in a distributed authorization
system. It has a private constructor and can only be created by the Authorization
Service (ignoring reflection, of course!).
A specific type of AccessToken is needed to access a specific operation, which ensures
that we can't accidentally do unauthorized activities.
Each specific type of AccessToken can store custom data collected at authorization
time, such as a CustomerId .
Global functions, such as the database, are modified so that they cannot be accessed
without an access token. This means that they can safely be made public.
Question: Why not also store the caller in the access token, so that no other client
can use it?
This is not needed because of the authority-based approach we're using. As discussed in
the first post, once a client has a capability, they can pass it around to other people to use,
so there is no point limiting it to a specific caller.

1641

Using types as access tokens

Question: The authorization module needs to know about the capability and access
token types now. Isn't that adding extra coupling?
If the authorization service is going to do its job, it has to know something about what
capabilities are available, so there is always some coupling, whether it is implicit
("resources" and "actions" in XACML) or explicit via types, as in this model.
So yes, the authorization service and database service both have a dependency on the set
of capabilities, but they are not coupled to each other directly.
Question: How do you use this model in a distributed system?
This model is really only designed to be used in a single codebase, so that type checking
can occur.
You could probably hack it so that types are turned into tickets at the boundary, and
conversely, but I haven't looked at that at all.
Question: Where can I read more on using types as access tokens?
This type-oriented version of an access token is my own design, although I very much doubt
that I'm the first person to think of using types this way. There are some related things for
Haskell (example) but I don't know of any directly analogous work that's accessible to
mainstream developers.
I've got more questions...
Some additional questions are answered at the end of part 1 and part 2, so read those
answers first. Otherwise please add your question in the comments below, and I'll try to
address it.

Conclusion
Thanks for making it all the way to the end!
As I said at the beginning, the goal is not to create an absolutely safe system, but instead
encourage you to think about and integrate authorization constraints into the design of your
system from the beginning, rather than treating it as an afterthought.
What's more, the point of doing this extra work is not just to improve security, but also to
improve the general design of your code. If you follow the principle of least authority, you get
modularity, decoupling, explicit dependencies, etc., for free!
In my opinion, a capability-based system works very well for this:
Functions map well to capabilities, and the need to pass capabilities around fits in very

1642

Using types as access tokens

well with standard functional programming patterns.

Once created, capabilities hide all the ugliness of authorization from the client, and so
the model succeeds in "making security user-friendly by making the security invisible".
Finally, with the addition of type-checked access tokens, we can have high confidence
that no part of our code can access global functions to do unauthorized operations.
I hope you found this series useful, and might inspire you to investigate some of these ideas
more fully.
NOTE: All the code for this post is available as a gist here and here.

1643

An introduction to property-based testing

This post is part of the F# Advent Calendar in English 2014 project. Check out all the
other great posts there! And special thanks to Sergey Tihon for organizing this.
UPDATE: I did a talk on property-based testing based on these posts. Slides and video here.
Let's start with a discussion that I hope never to have:
Me to co-worker: "We need a function that will add two numbers together, would you min
d implementing it?"
(a short time later)
Co-worker: "I just finished implementing the 'add' function"
Me: "Great, have you written unit tests for it?"
Co-worker: "You want tests as well?" (rolls eyes) "Ok."
(a short time later)
Co-worker: "I just wrote a test. Look! 'Given 1 + 2, I expect output is 3'. "
Co-worker: "So can we call it done now?"
Me: "Well that's only *one* test. How do you know that it doesn't fail for other input
s?"
Co-worker: "Ok, let me do another one."
(a short time later)
Co-worker: "I just wrote a another awesome test. 'Given 2 + 2, I expect output is 4'"
Me: "Yes, but you're still only testing for special cases. How do you know that it doe
sn't fail for other inputs you haven't thought of?"
Co-worker: "You want even *more* tests?"
(mutters "slavedriver" under breath and walks away)

But seriously, my imaginary co-worker's complaint has some validity: How many tests are
enough?
So now imagine that rather than being a developer, you are a test engineer who is
responsible for testing that the "add" function is implemented correctly.
Unfortunately for you, the implementation is being written by a burned-out, always lazy and
often malicious programmer, who I will call The Enterprise Developer From Hell, or "EDFH".
(The EDFH has a cousin who you might have heard of).
You are practising test-driven-development, enterprise-style, which means that you write a
test, and then the EDFH implements code that passes the test.
So you start with a test like this (using vanilla NUnit style):

1644

An introduction to property-based testing

[<Test>]
let ``When I add 1 + 2, I expect 3``()=
let result = add 1 2
Assert.AreEqual(3,result)

The EDFH then implements the add function like this:

let add x y =
if x=1 && y=2 then
3
else
0

And your test passes!

When you complain to the EDFH, they say that they are doing TDD properly, and only
writing the minimal code that will make the test pass.
Fair enough. So you write another test:
[<Test>]
let ``When I add 2 + 2, I expect 4``()=
let result = add 2 2
Assert.AreEqual(4,result)

The EDFH then changes the implementation of the add function to this:
let add x y =
if x=1 && y=2 then
3
else if x=2 && y=2 then
4
else
0

When you again complain to the EDFH, they point out that this approach is actually a best
practice. Apparently it's called "The Transformation Priority Premise".
At this point, you start thinking that the EDFH is being malicious, and that this back-and-forth
could go on forever!

Beating the malicious programmer

1645

An introduction to property-based testing

So the question is, what kind of test could you write so that a malicious programmer could
not create an incorrect implementation, even if they wanted to?
Well, you could start with a much larger list of known results, and mix them up a bit.
[<Test>]
let ``When I add two numbers, I expect to get their sum``()=
for (x,y,expected) in [ (1,2,3); (2,2,4); (3,5,8); (27,15,42); ]
let actual = add x y
Assert.AreEqual(expected,actual)

But the EDFH is tireless, and will update the implementation to include all of these cases as
well.
A much better approach is to generate random numbers and use those for inputs, so that a
malicious programmer could not possibly know what to do in advance.
let rand = System.Random()
let randInt() = rand.Next()
[<Test>]
let ``When I add two random numbers, I expect their sum``()=
let x = randInt()
let y = randInt()
let expected = x + y
let actual = add x y
Assert.AreEqual(expected,actual)

If the test looks like this, then the EDFH will be forced to implement the add function
correctly!
One final improvement -- the EDFH might just get lucky and have picked numbers that work
by chance, so let's repeat the random number test a number of times, say 100 times.
[<Test>]
let ``When I add two random numbers (100 times), I expect their sum``()=
for _ in [1..100] do
let x = randInt()
let y = randInt()
let expected = x + y
let actual = add x y
Assert.AreEqual(expected,actual)

So now we're done!

Or are we?

1646

An introduction to property-based testing

Property based testing

There's just one problem. In order to test the add function, you're making use of the +
function. In other words, you are using one implementation to test another.
In some cases that is acceptable (see the use of "test oracles" in a following post), but in
general, it's a bad idea to have your tests duplicate the code that you are testing! It's a waste
of time and effort, and now you have two implementations to build and keep up to date.
So if you can't test by using + , how can you test?
The answer is to create tests that focus on the properties of the function -- the
"requirements". These properties should be things that are true for any correct
implementation.
So let's think about what the properties of an add function are.
One way of getting started is to think about how add differs from other similar functions.
So for example, what is the difference between add and subtract ? Well, for subtract ,
the order of the parameters makes a difference, while for add it doesn't.
So there's a good property to start with. It doesn't depend on addition itself, but it does
eliminate a whole class of incorrect implementations.
[<Test>]
let ``When I add two numbers, the result should not depend on parameter order``()=
for _ in [1..100] do
let x = randInt()
let y = randInt()
let result1 = add x y
let result2 = add y x // reversed params
Assert.AreEqual(result1,result2)

That's a good start, but it doesn't stop the EDFH. The EDFH could still implement add
using x * y and this test would pass!
So now what about the difference between add and multiply ? What does addition really
mean?
We could start by testing with something like this, which says that x + x should the same
as x * 2 :
let result1 = add x x
let result2 = x * 2
Assert.AreEqual(result1,result2)

1647

An introduction to property-based testing

But now we are assuming the existence of multiplication! Can we define a property that only
depends on add itself?
One very useful approach is to see what happens when the function is repeated more than
once. That is, what if you add and then add to the result of that?
That leads to the idea that two add 1 s is the same as one add 2 . Here's the test:
[<Test>]
let ``Adding 1 twice is the same as adding 2``()=
for _ in [1..100] do
let x = randInt()
let y = randInt()
let result1 = x |> add 1 |> add 1
let result2 = x |> add 2
Assert.AreEqual(result1,result2)

That's great! add works perfectly with this test, while multiply doesn't.
However, note that the EDFH could still implement add using y - x and this test would
pass!
Luckily, we have the "parameter order" test above as well. So the combination of both of
these tests should narrow it down so that there is only one correct implementation, surely?
After submitting this test suite we find out the EDFH has written an implementation that
passes both these tests. Let's have a look:
let add x y = 0 // malicious implementation

Aarrghh! What happened? Where did our approach go wrong?

Well, we forgot to force the implementation to actually use the random numbers we were
generating!
So we need to ensure that the implementation does indeed do something with the
parameters that are passed into it. We're going to have to check that the result is somehow
connected to the input in a specific way.
Is there a trivial property of add that we know the answer to without reimplementing our
own version?
Yes!
What happens when you add zero to a number? You always get the same number back.

1648

An introduction to property-based testing

[<Test>]
let ``Adding zero is the same as doing nothing``()=
for _ in [1..100] do
let x = randInt()
let result1 = x |> add 0
let result2 = x
Assert.AreEqual(result1,result2)

So now we have a set of properties that can be used to test any implementation of add ,
and that force the EDFH to create a correct implementation:

Refactoring the common code

There's quite a bit of duplicated code in these three tests. Let's do some refactoring.
First, we'll write a function called propertyCheck that does the work of generating 100 pairs
of random ints.
propertyCheck will also need a parameter for the property itself. This will be a function that

takes two ints and returns a bool:

let propertyCheck property =
// property has type: int -> int -> bool
for _ in [1..100] do
let x = randInt()
let y = randInt()
let result = property x y
Assert.IsTrue(result)

With this in place, we can redefine one of the tests by pulling out the property into a separate
function, like this:
let commutativeProperty x y =
let result1 = add x y
let result2 = add y x // reversed params
result1 = result2
[<Test>]
let ``When I add two numbers, the result should not depend on parameter order``()=
propertyCheck commutativeProperty

We can also do the same thing for the other two properties.
After the refactoring, the complete code looks like this:

1649

An introduction to property-based testing

let rand = System.Random()

let randInt() = rand.Next()
let add x y = x + y // correct implementation
let propertyCheck property =
// property has type: int -> int -> bool
for _ in [1..100] do
let x = randInt()
let y = randInt()
let result = property x y
Assert.IsTrue(result)
let commutativeProperty x y =
let result1 = add x y
let result2 = add y x // reversed params
result1 = result2
[<Test>]
let ``When I add two numbers, the result should not depend on parameter order``()=
propertyCheck commutativeProperty
let adding1TwiceIsAdding2OnceProperty x _ =
let result1 = x |> add 1 |> add 1
let result2 = x |> add 2
result1 = result2
[<Test>]
let ``Adding 1 twice is the same as adding 2``()=
propertyCheck adding1TwiceIsAdding2OnceProperty
let identityProperty x _ =
let result1 = x |> add 0
result1 = x
[<Test>]
let ``Adding zero is the same as doing nothing``()=
propertyCheck identityProperty

Reviewing what we have done so far

We have defined a set of properties that any implementation of add should satisfy:
The parameter order doesn't matter ("commutativity" property)
Doing add twice with 1 is the same as doing add once with 2
Adding zero does nothing ("identity" property)

1650

An introduction to property-based testing

What's nice about these properties is that they work with all inputs, not just special magic
numbers. But more importantly, they show us the core essence of addition.
In fact, you can take this approach to the logical conclusion and actually define addition as
anything that has these properties.
This is exactly what mathematicians do. If you look up addition on Wikipedia, you'll see that
it is defined entirely in terms of commutativity, associativity, identity, and so on.
You'll note that in our experiment, we missed defining "associativity", but instead created a
weaker property ( x+1+1 = x+2 ). We'll see later that the EDFH can indeed write a malicious
implementation that satisfies this property, and that associativity is better.
Alas, it's hard to get properties perfect on the first attempt, but even so, by using the three
properties we came up with, we have got a much higher confidence that the implementation
is correct, and in fact, we have learned something too -- we have understood the
requirements in a deeper way.

Specification by properties
A collection of properties like this can be considered a specification.
Historically, unit tests, as well as being functional tests, have been used as a sort of
specification as well. But an approach to specification using properties instead of tests with
"magic" data is an alternative which I think is often shorter and less ambiguous.
You might be thinking that only mathematical kinds of functions can be specified this way,
but in future posts, we'll see how this approach can be used to test web services and
databases too.
Of course, not every business requirement can be expressed as properties like this, and we
must not neglect the social component of software development. Specification by example
and domain driven design can play a valuable role when working with non-technical
customers.
You also might be thinking that designing all these properties is a lot of work -- and you'd be
right! It is the hardest part. In a follow-up post, I'll present some tips for coming up with
properties which might reduce the effort somewhat.
But even with the extra effort involved upfront (the technical term for this activity is called
"thinking about the problem", by the way) the overall time saved by having automated tests
and unambigous specifications will more than pay for the upfront cost later.

1651

An introduction to property-based testing

In fact, the arguments that are used to promote the benefits of unit testing can equally well
be applied to property-based testing! So if a TDD fan tells you that they don't have the time
to come up with property-based tests, then they might not be looking at the big picture.

Introducing QuickCheck and FsCheck

We have implemented our own property checking system, but there are quite a few
problems with it:
It only works with integer functions. It would be nice if we could use the same approach
for functions that had string parameters, or in fact any type of parameter, including ones
we defined ourselves.
It only works with two parameter functions (and we had to ignore one of them for the
adding1TwiceIsAdding2OnceProperty and identity properties). It would be nice if we

could use the same approach for functions with any number of parameters.
When there is a counter-example to the property, we don't know what it is! Not very
helpful when the tests fail!
There's no logging of the random numbers that we generated, and there's no way to set
the seed, which means that we can't debug and reproduce errors easily.
It's not configurable. For example, we can't easily change the number of loops from 100
to something else.
It would be nice if there was a framework that did all that for us!
Thankfully there is! The "QuickCheck" library was originally developed for Haskell by Koen
Claessen and John Hughes, and has been ported to many other languages.
The version of QuickCheck used in F# (and C# too) is the excellent "FsCheck" library
created by Kurt Schelfthout. Although based on the Haskell QuickCheck, it has some nice
additional features, including integration with test frameworks such as NUnit and xUnit.
So let's look at how FsCheck would do the same thing as our homemade property-testing
system.

Using FsCheck to test the addition properties

First, you need to install FsCheck and load the DLL (FsCheck can be a bit finicky -- see the
bottom of this page for instructions and troubleshooting).
The top of your script file should look something like this:

1652

An introduction to property-based testing

System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
#I @"Packages\FsCheck.1.0.3\lib\net45"
//#I @"Packages\FsCheck.0.9.2.0\lib\net40-Client" // use older version for VS2012
#I @"Packages\NUnit.2.6.3\lib"
#r @"FsCheck.dll"
#r @"nunit.framework.dll"
open System
open FsCheck
open NUnit.Framework

Once FsCheck is loaded, you can use Check.Quick and pass in any "property" function. For
now, let's just say that a "property" function is any function (with any parameters) that returns
a boolean.
let add x y = x + y // correct implementation
let commutativeProperty (x,y) =
let result1 = add x y
let result2 = add y x // reversed params
result1 = result2
// check the property interactively
Check.Quick commutativeProperty
let adding1TwiceIsAdding2OnceProperty x =
let result1 = x |> add 1 |> add 1
let result2 = x |> add 2
result1 = result2
// check the property interactively
Check.Quick adding1TwiceIsAdding2OnceProperty
let identityProperty x =
let result1 = x |> add 0
result1 = x
// check the property interactively
Check.Quick identityProperty

If you check one of the properties interactively, say with Check.Quick commutativeProperty ,
you'll see the message:
Ok, passed 100 tests.

Using FsCheck to find unsatified properties

1653

An introduction to property-based testing

Let's see what happens when we have a malicious implementation of add . In the code
below, the EDFH implements add as multiplication!
That implementation will satisfy the commutative property, but what about the
adding1TwiceIsAdding2OnceProperty ?

let add x y =
x * y // malicious implementation
let adding1TwiceIsAdding2OnceProperty x =
let result1 = x |> add 1 |> add 1
let result2 = x |> add 2
result1 = result2
// check the property interactively
Check.Quick adding1TwiceIsAdding2OnceProperty

The result from FsCheck is:

Falsifiable, after 1 test (1 shrink) (StdGen (1657127138,295941511)):
1

That means that using 1 as the input to adding1TwiceIsAdding2OnceProperty will result in

false , which you can easily see that it does.

The return of the malicious EDFH

By using random testing, we have made it harder for a malicious implementor. They will
have to change tactics now!
The EDFH notes that we are still using some magic numbers in the
adding1TwiceIsAdding2OnceProperty -- namely 1 and 2, and decides to create an

implementation that exploits this. They'll use a correct implementation for low input values
and an incorrect implementation for high input values:
let add x y =
if (x < 10) || (y < 10) then
x + y // correct for low values
else
x * y // incorrect for high values

Oh no! If we retest all our properties, they all pass now!

That'll teach us to use magic numbers in our tests!

1654

An introduction to property-based testing

What's the alternative? Well, let's steal from the mathematicians and create an associative
property test.
let associativeProperty x y z =
let result1 = add x (add y z) // x + (y + z)
let result2 = add (add x y) z // (x + y) + z
result1 = result2
// check the property interactively
Check.Quick associativeProperty

Aha! Now we get a falsification:

Falsifiable, after 38 tests (4 shrinks) (StdGen (127898154,295941554)):
8
2
10

That means that using (8+2)+10 is not the same as 8+(2+10) .

Note that not only has FsCheck found some inputs that break the property, but it has found a
lowest example. It knows that the inputs 8,2,9 pass but going one higher ( 8,2,10 ) fails.
That's very nice!

Understanding FsCheck: Generators

Now that we have used FsCheck for real, let's pause and have a look at how it works.
The first thing that FsCheck does is generate random inputs for you. This is called
"generation", and for each type, there is an associated generator.
For example, to generate a list of sample data, you use the generator along with two
parameters: the number of elements in the list and a "size". The precise meaning of "size"
depends on the type being generated and the context. Examples of things "size" is used for
are: the maximum value of an int; the length of a list; the depth of a tree; etc.
Here's some code that generates ints:

1655

An introduction to property-based testing

// get the generator for ints

let intGenerator = Arb.generate<int>
// generate three ints with a maximum size of 1
Gen.sample 1 3 intGenerator // e.g. [0; 0; -1]
// generate three ints with a maximum size of 10
Gen.sample 10 3 intGenerator // e.g. [-4; 8; 5]
// generate three ints with a maximum size of 100
Gen.sample 100 3 intGenerator // e.g. [-37; 24; -62]

In this example, the ints are not generated uniformly, but clustered around zero. You can see
this for yourself with a little code:
// see how the values are clustered around the center point
intGenerator
|> Gen.sample 10 1000
|> Seq.groupBy id
|> Seq.map (fun (k,v) -> (k,Seq.length v))
|> Seq.sortBy (fun (k,v) -> k)
|> Seq.toList

The result is something like this:

[(-10, 3); (-9, 14); (-8, 18); (-7, 10); (-6, 27); (-5, 42); (-4, 49);
(-3, 56); (-2, 76); (-1, 119); (0, 181); (1, 104); (2, 77); (3, 62);
(4, 47); (5, 44); (6, 26); (7, 16); (8, 14); (9, 12); (10, 3)]

You can see that most of the values are in the center (0 is generated 181 times, 1 is
generated 104 times), and the outlying values are rare (10 is generated only 3 times).
You can repeat with larger samples too. This one generates 10000 elements in the range
[-30,30]
intGenerator
|> Gen.sample 30 10000
|> Seq.groupBy id
|> Seq.map (fun (k,v) -> (k,Seq.length v))
|> Seq.sortBy (fun (k,v) -> k)
|> Seq.toList

There are plenty of other generator functions available as well as Gen.sample (more
documentation here).

1656

An introduction to property-based testing

Understanding FsCheck: Generating all sorts

of types automatically
What's great about the generator logic is that it will automatically generate compound values
as well.
For example, here is a generator for a tuple of three ints:
let tupleGenerator = Arb.generate<int*int*int>
// generate 3 tuples with a maximum size of 1
Gen.sample 1 3 tupleGenerator
// result: [(0, 0, 0); (0, 0, 0); (0, 1, -1)]
// generate 3 tuples with a maximum size of 10
Gen.sample 10 3 tupleGenerator
// result: [(-6, -4, 1); (2, -2, 8); (1, -4, 5)]
// generate 3 tuples with a maximum size of 100
Gen.sample 100 3 tupleGenerator
// result: [(-2, -36, -51); (-5, 33, 29); (13, 22, -16)]

Once you have a generator for a base type, option and list generators follow. Here is a
generator for int option s:
let intOptionGenerator = Arb.generate<int option>
// generate 10 int options with a maximum size of 5
Gen.sample 5 10 intOptionGenerator
// result: [Some 0; Some -1; Some 2; Some 0; Some 0;
// Some -4; null; Some 2; Some -2; Some 0]

And here is a generator for int list s:

let intListGenerator = Arb.generate<int list>
// generate 10 int lists with a maximum size of 5
Gen.sample 5 10 intListGenerator
// result: [ []; []; [-4]; [0; 3; -1; 2]; [1];
// [1]; []; [0; 1; -2]; []; [-1; -2]]

And of course you can generate random strings too!

1657

An introduction to property-based testing

let stringGenerator = Arb.generate<string>

// generate 3 strings with a maximum size of 1
Gen.sample 1 3 stringGenerator
// result: [""; "!"; "I"]
// generate 3 strings with a maximum size of 10
Gen.sample 10 3 stringGenerator
// result: [""; "eiX$a^"; "U%0Ika&r"]

The best thing is that the generator will work with your own user-defined types too!
type Color = Red | Green of int | Blue of bool
let colorGenerator = Arb.generate<Color>
// generate 10 colors with a maximum size of 50
Gen.sample 50 10 colorGenerator
// result: [Green -47; Red; Red; Red; Blue true;
// Green 2; Blue false; Red; Blue true; Green -12]

Here's one that generates a user-defined record type containing another user-defined type.
type Point = {x:int; y:int; color: Color}
let pointGenerator = Arb.generate<Point>
// generate 10 points with a maximum size of 50
Gen.sample 50 10 pointGenerator
(* result
[{x = -8; y = 12; color = Green -4;};
{x = 28; y = -31; color = Green -6;};
{x = 11; y = 27; color = Red;};
{x = -2; y = -13; color = Red;};
{x = 6; y = 12; color = Red;};
// etc
*)

There are ways to have more fine-grained control over how your types are generated, but
that will have to wait for another post!

Understanding FsCheck: Shrinking

1658

An introduction to property-based testing

Creating minimum counter-examples is one of the cool things about QuickCheck-style

testing.
How does it do this?
There are two parts to the process that FsCheck uses:
First it generates a sequence of random inputs, starting small and getting bigger. This is the
"generator" phase as described above.
If any inputs cause the property to fail, it starts "shrinking" the first parameter to find a
smaller number. The exact process for shrinking varies depending on the type (and you can
override it too), but let's say that for numbers, they get smaller in a sensible way.
For example, let's say that you have a silly property isSmallerThan80 :
let isSmallerThan80 x = x < 80

You have generated random numbers and found that then property fails for 100 , and you
want to try a smaller number. Arb.shrink will generate a sequence of ints, all of which are
smaller than 100. Each one of these is tried with the property in turn until the property fails
again.
isSmallerThan80 100 // false, so start shrinking
Arb.shrink 100 |> Seq.toList
// [0; 50; 75; 88; 94; 97; 99]

For each element in the list, test the property against it until you find another failure:
isSmallerThan80 0 // true
isSmallerThan80 50 // true
isSmallerThan80 75 // true
isSmallerThan80 88 // false, so shrink again

The property failed with 88 , so shrink again using that as a starting point:
Arb.shrink 88 |> Seq.toList
// [0; 44; 66; 77; 83; 86; 87]
isSmallerThan80 0 // true
isSmallerThan80 44 // true
isSmallerThan80 66 // true
isSmallerThan80 77 // true
isSmallerThan80 83 // false, so shrink again

1659

An introduction to property-based testing

The property failed with 83 now, so shrink again using that as a starting point:
Arb.shrink 83 |> Seq.toList
// [0; 42; 63; 73; 78; 81; 82]
// smallest failure is 81, so shrink again

The property failed with 81 , so shrink again using that as a starting point:
Arb.shrink 81 |> Seq.toList
// [0; 41; 61; 71; 76; 79; 80]
// smallest failure is 80

After this point, shrinking on 80 doesn't work -- no smaller value will be found.
In this case then, FsCheck will report that 80 falsifies the property and that 4 shrinks were
needed.
Just as with generators, FsCheck will generate shrink sequences for almost any type:
Arb.shrink (1,2,3) |> Seq.toList
// [(0, 2, 3); (1, 0, 3); (1, 1, 3); (1, 2, 0); (1, 2, 2)]
Arb.shrink "abcd" |> Seq.toList
// ["bcd"; "acd"; "abd"; "abc"; "abca"; "abcb"; "abcc"; "abad"; "abbd"; "aacd"]
Arb.shrink [1;2;3] |> Seq.toList
// [[2; 3]; [1; 3]; [1; 2]; [1; 2; 0]; [1; 2; 2]; [1; 0; 3]; [1; 1; 3]; [0; 2; 3]]

And, as with generators, there are ways to customize how shrinking works if needed.

Configuring FsCheck: Changing the number of

tests
I mentioned a silly property isSmallerThan80 -- let's see how FsCheck does with it.
// silly property to test
let isSmallerThan80 x = x < 80
Check.Quick isSmallerThan80
// result: Ok, passed 100 tests.

Oh dear! FsCheck didn't find a counter-example!

At this point, we can try a few things. First, we can try increasing the number of tests.
1660

An introduction to property-based testing

We do this by changing the default ("Quick") configuration. There is a field called MaxTest
that we can set. The default is 100, so let's increase it to 1000.
Finally, to use a specific config, you'll need to use Check.One(config,property) rather than
just Check.Quick(property) .
let config = {
Config.Quick with
MaxTest = 1000
}
Check.One(config,isSmallerThan80 )
// result: Ok, passed 1000 tests.

Oops! FsCheck didn't find a counter-example with 1000 tests either! Let's try once more with
10000 tests:
let config = {
Config.Quick with
MaxTest = 10000
}
Check.One(config,isSmallerThan80 )
// result: Falsifiable, after 8660 tests (1 shrink) (StdGen (539845487,295941658)):
// 80

Ok, so we finally got it to work. But why did it take so many tests?
The answer lies in some other configuration settings: StartSize and EndSize .
Remember that the generators start with small numbers and gradually increase them. This is
controlled by the StartSize and EndSize settings. By default, StartSize is 1 and
EndSize is 100. So at the end of the test, the "size" parameter to the generator will be 100.

But, as we saw, even if the size is 100, very few numbers are generated at the extremes. In
this case it means that numbers greater than 80 are unlikely to be generated.
So let's change the EndSize to something larger and see what happens!
let config = {
Config.Quick with
EndSize = 1000
}
Check.One(config,isSmallerThan80 )
// result: Falsifiable, after 21 tests (4 shrinks) (StdGen (1033193705,295941658)):
// 80

That's more like it! Only 21 tests needed now rather than 8660 tests!

1661

An introduction to property-based testing

Configuring FsCheck: Verbose mode and

logging
I mentioned that one of the benefits of FsCheck over a home-grown solution is the logging
and reproducibility, so let's have a look at that.
We'll tweak the malicious implementation to have a boundary of 25 . Let's see how
FsCheck detects this boundary via logging.
let add x y =
if (x < 25) || (y < 25) then
x + y // correct for low values
else
x * y // incorrect for high values
let associativeProperty x y z =
let result1 = add x (add y z) // x + (y + z)
let result2 = add (add x y) z // (x + y) + z
result1 = result2
// check the property interactively
Check.Quick associativeProperty

The result is:

Falsifiable, after 66 tests (12 shrinks) (StdGen (1706196961,295941556)):
1
24
25

Again, FsCheck has found that 25 is the exact boundary point quite quickly. But how did it
do it?
First, the simplest way to see what FsCheck is doing is to use "verbose" mode. That is, use
Check.Verbose rather than Check.Quick :

// check the property interactively

Check.Quick associativeProperty
// with tracing/logging
Check.Verbose associativeProperty

When do this, you'll see an output like that shown below. I've added all the comments to
explain the various elements.

1662

An introduction to property-based testing

0: // test 1
-1 // param 1
-1 // param 2
0 // param 3
// associativeProperty -1 -1 0 => true, keep going
1: // test 2
0
0
0 // associativeProperty 0 0 0 => true, keep going
2: // test 3
-2
0
-3 // associativeProperty -2 0 -3 => true, keep going
3: // test 4
1
2
0 // associativeProperty 1 2 0 => true, keep going
// etc
49: // test 50
46
-4
50 // associativeProperty 46 -4 50 => false, start shrinking
// etc
shrink:
35
-4
50 // associativeProperty 35 -4 50 => false, keep shrinking
shrink:
27
-4
50 // associativeProperty 27 -4 50 => false, keep shrinking
// etc
shrink:
25
1
29 // associativeProperty 25 1 29 => false, keep shrinking
shrink:
25
1
26 // associativeProperty 25 1 26 => false, keep shrinking
// next shrink fails
Falsifiable, after 50 tests (10 shrinks) (StdGen (995282583,295941602)):
25
1
26

This display takes up a lot of space! Can we make it more compact?

Yes -- you can control how each test and shrink is displayed by writing your own custom
functions, and telling FsCheck to use them via its Config structure.

1663

An introduction to property-based testing

These functions are generic, and the list of parameters is represented by a list of unknown
length ( obj list ). But since I know I am testing a three parameter property I can hard-code
a three-element list parameter and print them all on one line.
The configuration also has a slot called Replay which is normally None , which means that
each run will be different.
If you set Replay to Some seed , then the test will be replayed exactly the same way. The
seed looks like StdGen (someInt,someInt) and is printed on each run, so if you want to
preserve a run all you need to do is paste that seed into the config.
And again, to use a specific config, you'll need to use Check.One(config,property) rather
than just Check.Quick(property) .
Here's the code with the default tracing functions changed, and the replay seed set explicitly.
// create a function for displaying a test
let printTest testNum [x;y;z] =
sprintf "#%-3i %3O %3O %3O\n" testNum x y z
// create a function for displaying a shrink
let printShrink [x;y;z] =
sprintf "shrink %3O %3O %3O\n" x y z
// create a new FsCheck configuration
let config = {
Config.Quick with
Replay = Random.StdGen (995282583,295941602) |> Some
Every = printTest
EveryShrink = printShrink
}
// check the given property with the new configuration
Check.One(config,associativeProperty)

The output is now much more compact, and looks like this:

1664

An introduction to property-based testing

#0 -1 -1 0
#1 0 0 0
#2 -2 0 -3
#3 1 2 0
#4 -4 2 -3
#5 3 0 -3
#6 -1 -1 -1
// etc
#46 -21 -25 29
#47 -10 -7 -13
#48 -4 -19 23
#49 46 -4 50
// start shrinking first parameter
shrink 35 -4 50
shrink 27 -4 50
shrink 26 -4 50
shrink 25 -4 50
// start shrinking second parameter
shrink 25 4 50
shrink 25 2 50
shrink 25 1 50
// start shrinking third parameter
shrink 25 1 38
shrink 25 1 29
shrink 25 1 26
Falsifiable, after 50 tests (10 shrinks) (StdGen (995282583,295941602)):
25
1
26

So there you go -- it's quite easy to customize the FsCheck logging if you need to.
Let's look at how the shrinking was done in detail. The last set of inputs (46,-4,50) was false,
so shrinking started.
// The last set of inputs (46,-4,50) was false, so shrinking started
associativeProperty 46 -4 50 // false, so shrink
// list of possible shrinks starting at 46
Arb.shrink 46 |> Seq.toList
// result [0; 23; 35; 41; 44; 45]

We'll loop through the list [0; 23; 35; 41; 44; 45] stopping at the first element that causes
the property to fail:

1665

An introduction to property-based testing

// find the next test that fails when shrinking the x parameter
let x,y,z = (46,-4,50)
Arb.shrink x
|> Seq.tryPick (fun x -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (35, -4, 50)

The first element that caused a failure was x=35 , as part of the inputs (35, -4, 50) .
So now we start at 35 and shrink that:
// find the next test that fails when shrinking the x parameter
let x,y,z = (35,-4,50)
Arb.shrink x
|> Seq.tryPick (fun x -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (27, -4, 50)

The first element that caused a failure was now x=27 , as part of the inputs (27, -4, 50) .
So now we start at 27 and keep going:
// find the next test that fails when shrinking the x parameter
let x,y,z = (27,-4,50)
Arb.shrink x
|> Seq.tryPick (fun x -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (26, -4, 50)
// find the next test that fails when shrinking the x parameter
let x,y,z = (26,-4,50)
Arb.shrink x
|> Seq.tryPick (fun x -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, -4, 50)
// find the next test that fails when shrinking the x parameter
let x,y,z = (25,-4,50)
Arb.shrink x
|> Seq.tryPick (fun x -> if associativeProperty x y z then None else Some (x,y,z) )
// answer None

At this point, x=25 is as low as you can go. None of its shrink sequence caused a failure.
So we're finished with the x parameter!
Now we just repeat this process with the y parameter

1666

An introduction to property-based testing

// find the next test that fails when shrinking the y parameter
let x,y,z = (25,-4,50)
Arb.shrink y
|> Seq.tryPick (fun y -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 4, 50)
// find the next test that fails when shrinking the y parameter
let x,y,z = (25,4,50)
Arb.shrink y
|> Seq.tryPick (fun y -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 2, 50)
// find the next test that fails when shrinking the y parameter
let x,y,z = (25,2,50)
Arb.shrink y
|> Seq.tryPick (fun y -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 1, 50)
// find the next test that fails when shrinking the y parameter
let x,y,z = (25,1,50)
Arb.shrink y
|> Seq.tryPick (fun y -> if associativeProperty x y z then None else Some (x,y,z) )
// answer None

At this point, y=1 is as low as you can go. None of its shrink sequence caused a failure. So
we're finished with the y parameter!
Finally, we repeat this process with the z parameter

1667

An introduction to property-based testing

// find the next test that fails when shrinking the z parameter
let x,y,z = (25,1,50)
Arb.shrink z
|> Seq.tryPick (fun z -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 1, 38)
// find the next test that fails when shrinking the z parameter
let x,y,z = (25,1,38)
Arb.shrink z
|> Seq.tryPick (fun z -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 1, 29)
// find the next test that fails when shrinking the z parameter
let x,y,z = (25,1,29)
Arb.shrink z
|> Seq.tryPick (fun z -> if associativeProperty x y z then None else Some (x,y,z) )
// answer (25, 1, 26)
// find the next test that fails when shrinking the z parameter
let x,y,z = (25,1,26)
Arb.shrink z
|> Seq.tryPick (fun z -> if associativeProperty x y z then None else Some (x,y,z) )
// answer None

And now we're finished with all the parameters!

The final counter-example after shrinking is (25,1,26) .

Adding pre-conditions
Let's say that we have a new idea for a property to check. We'll create a property called
addition is not multiplication which will help to stop any malicious (or even accidental)

mixup in the implementations.

Here's our first attempt:
let additionIsNotMultiplication x y =
x + y <> x * y

Bt when we run this test, we get a failure!

Check.Quick additionIsNotMultiplication
// Falsifiable, after 3 tests (0 shrinks) (StdGen (2037191079,295941699)):
// 0
// 0

1668

An introduction to property-based testing

Well duh, obviously 0+0 and 0*0 are equal. But how can we tell FsCheck to ignore just
those inputs and leave all the other ones alone?
This is done via a "condition" or filter expression that is prepended to the property function
using ==> (an operator defined by FsCheck).
Here's an example:
let additionIsNotMultiplication x y =
x + y <> x * y
let preCondition x y =
(x,y) <> (0,0)
let additionIsNotMultiplication_withPreCondition x y =
preCondition x y ==> additionIsNotMultiplication x y

The new property is additionIsNotMultiplication_withPreCondition and can be passed to

Check.Quick just like any other property.

Check.Quick additionIsNotMultiplication_withPreCondition
// Falsifiable, after 38 tests (0 shrinks) (StdGen (1870180794,295941700)):
// 2
// 2

Oops! We forgot another case! Let's fix up our precondition again:

let preCondition x y =
(x,y) <> (0,0)
&& (x,y) <> (2,2)
let additionIsNotMultiplication_withPreCondition x y =
preCondition x y ==> additionIsNotMultiplication x y

And now this works.

Check.Quick additionIsNotMultiplication_withPreCondition
// Ok, passed 100 tests.

This kind of precondition should only be used if you want to filter out a small number of
cases.
If most of the inputs will be invalid, then this filtering will be expensive. In this case there is a
better way to do it, which will be discussed in a future post.

1669

An introduction to property-based testing

The FsCheck documentation has more on how you can tweak properties here.

Naming convention for properties

These properties functions have a different purpose from "normal" functions, so how should
we name them?
In the Haskell and Erlang world, properties are given a prop_ prefix by convention. In the
.NET world, it is more common to use a suffix like AbcProperty .
Also, in F# we have namespaces, modules, and attributes (like [<Test>] ) that we can use
to organize properties and distinguish them from other functions.

Combining multiple properties

Once you have a set of properties, you can combine them into a group (or even, gasp, a
specification!), by adding them as static members of a class type.
You can then do Check.QuickAll and pass in the name of the class.
For example, here are our three addition properties:
let add x y = x + y // good implementation
let commutativeProperty x y =
add x y = add y x
let associativeProperty x y z =
add x (add y z) = add (add x y) z
let leftIdentityProperty x =
add x 0 = x
let rightIdentityProperty x =
add 0 x = x

And here's the corresponding static class to be used with Check.QuickAll :

1670

An introduction to property-based testing

type AdditionSpecification =
static member ``Commutative`` x y = commutativeProperty x y
static member ``Associative`` x y z = associativeProperty x y z
static member ``Left Identity`` x = leftIdentityProperty x
static member ``Right Identity`` x = rightIdentityProperty x
Check.QuickAll<AdditionSpecification>()

Combining property-based tests with examplebased tests

At the beginning of this post, I was dismissive of tests that used "magic" numbers to test a
very small part of the input space.
However, I do think that example-based tests have a role that complements property-based
tests.
An example-based test is often easier to understand because it is less abstract, and so
provides a good entry point and documentation in conjuction with the properties.
Here's an example:
type AdditionSpecification =
static member ``Commutative`` x y = commutativeProperty x y
static member ``Associative`` x y z = associativeProperty x y z
static member ``Left Identity`` x = leftIdentityProperty x
static member ``Right Identity`` x = rightIdentityProperty x
// some examples as well
static member ``1 + 2 = 3``() =
add 1 2 = 3
static member ``1 + 2 = 2 + 1``() =
add 1 2 = add 2 1
static member ``42 + 0 = 0 + 42``() =
add 42 0 = add 0 42

Using FsCheck from NUnit

You can use FsCheck from NUnit and other test frameworks, with an extra plugin (e.g.
FsCheck.NUnit for Nunit).

1671

An introduction to property-based testing

Rather than marking a test with Test or Fact , you use the Property attribute. And unlike
normal tests, these tests can have parameters!
Here's an example of some tests.
open NUnit.Framework
open FsCheck
open FsCheck.NUnit
[<Property(QuietOnSuccess = true)>]
let ``Commutative`` x y =
commutativeProperty x y
[<Property(Verbose= true)>]
let ``Associative`` x y z =
associativeProperty x y z
[<Property(EndSize=300)>]
let ``Left Identity`` x =
leftIdentityProperty x

As you can see, you can change the configuration for each test (such as Verbose and
EndSize ) via properties of the annotation.

And the QuietOnSuccess flag is available to make FsCheck compatible with standard test
frameworks, which are silent on success and only show messages if something goes wrong.

Summary
In this post I've introduced you to the basics of property-based checking.
There's much more to cover though! In future posts I will cover topics such as:
How to come up with properties that apply to your code. The properties don't have
to be mathematical. We'll look at more general properties such as inverses (for testing
serialization/deserialization), idempotence (for safe handling of multiple updates or
duplicate messages), and also look at test oracles.
How to create your own generators and shrinkers. We've seen that FsCheck can
generate random values nicely. But what about values with constraints such as positive
numbers, or valid email addresses, or phone numbers. FsCheck gives you the tools to
build your own.
How to do model-based testing, and in particular, how to test for concurrency issues.
I've also introduced the notion of an evil malicious programmer. You might think that such a
malicious programmer is unrealistic and over-the-top.

1672

An introduction to property-based testing

But in many cases you act like an unintentionally malicious programmer. You happily create
a implementation that works for some special cases, but doesn't work more generally, not
out of evil intent, but out of unawareness and blindness.
Like fish unaware of water, we are often unaware of the assumptions we make. Propertybased testing can force us to become aware of them.
Until next time -- happy testing!
The code samples used in this post are available on GitHub.
Want more? I have written a follow up post on choosing properties for property-based
testing
UPDATE: I did a talk on property-based testing based on these posts. Slides and video here.

Appendix: Installing and troubleshooting

FsCheck
The easiest way to make FsCheck available to you is to create an F# project and add the
NuGet package "FsCheck.NUnit". This will install both FsCheck and NUnit in the packages
directory.
If you are using a FSX script file for interactive development, you'll need to load the DLLs
from the appropriate package location, like this:
// sets the current directory to be same as the script directory
System.IO.Directory.SetCurrentDirectory (__SOURCE_DIRECTORY__)
// assumes nuget install FsCheck.Nunit has been run
// so that assemblies are available under the current directory
#I @"Packages\FsCheck.1.0.3\lib\net45"
//#I @"Packages\FsCheck.0.9.2.0\lib\net40-Client" // use older version for VS2012
#I @"Packages\NUnit.2.6.3\lib"
#r @"FsCheck.dll"
#r @"nunit.framework.dll"
open System
open FsCheck
open NUnit.Framework

Next, test that FsCheck is working correctly by running the following:

1673

An introduction to property-based testing

let revRevIsOrig (xs:list<int>) = List.rev(List.rev xs) = xs

Check.Quick revRevIsOrig

If you get no errors, then everything is good.

If you do get errors, it's probably because you are on an older version of Visual Studio.
Upgrade to VS2013 or failing that, do the following:
First make sure you have the latest F# core installed (currently 3.1).
Make sure your that your app.config has the appropriate binding redirects.
Make sure that your NUnit assemblies are being referenced locally rather than from the
GAC.
These steps should ensure that compiled code works.
With F# interactive, it can be trickier. If you are not using VS2013, you might run into errors
such as System.InvalidCastException: Unable to cast object of type 'Arrow' .
The best cure for this is to upgrade to VS2013! Failing that, you can use an older version of
FsCheck, such as 0.9.2 (which I have tested successfully with VS2012)

1674

Choosing properties for property-based testing

Choosing properties for property-based

testing
UPDATE: I did a talk on property-based testing based on these posts. Slides and video here.
In the previous post, I described the basics of property-based testing, and showed how it
could save a lot of time by generating random tests.
But here's a common problem. Everyone who sees a property-based testing tool like
FsCheck or QuickCheck thinks that it is amazing... but when it times come to start creating
your own properties, the universal complaint is: "what properties should I use? I can't think of
any!"
The goal of this post is to show some common patterns that can help you discover the
properties that are applicable to your code.

Categories for properties

In my experience, many properties can be discovered by using one of the seven approaches
listed below.
"Different paths, same destination"
"There and back again"
"Some things never change"
"The more things change, the more they stay the same"
"Solve a smaller problem first"
"Hard to prove, easy to verify"
"The test oracle"
This is by no means a comprehensive list, just the ones that have been most useful to me.
For a different perspective, check out the list of patterns that the PEX team at Microsoft have
compiled.

"Different paths, same destination"

These kinds of properties are based on combining operations in different orders, but getting
the same result. For example, in the diagram below, doing X then Y gives the same result
as doing Y followed by X .

1675

Choosing properties for property-based testing

The commutative property of addition is an obvious example of this pattern. For example,
the result of add 1 then add 2 is the same as the result of add 2 followed by add 1 .
This pattern, generalized, can produce a wide range of useful properties. We'll see some
more uses of this pattern later in this post.

"There and back again"

These kinds of properties are based on combining an operation with its inverse, ending up
with the same value you started with.
In the diagram below, doing X serializes ABC to some kind of binary format, and the
inverse of X is some sort of deserialization that returns the same ABC value again.

In addition to serialization/deserialization, other pairs of operations can be checked this way:

addition / subtraction , write / read , setProperty / getProperty , and so on.

Other pair of functions fit this pattern too, even though they are not strict inverses, pairs such
as insert / contains , create / exists , etc.

"Some things never change"

These kinds of properties are based on an invariant that is preserved after some
transformation.
In the diagram below, the transform changes the order of the items, but the same four items
are still present afterwards.

1676

Choosing properties for property-based testing

Common invariants include size of a collection (for map say), the contents of a collection
(for sort say), the height or depth of something in proportion to size (e.g. balanced trees).

"The more things change, the more they stay the same"
These kinds of properties are based on "idempotence" -- that is, doing an operation twice is
the same as doing it once.
In the diagram below, using distinct to filter the set returns two items, but doing distinct
twice returns the same set again.

Idempotence properties are very useful, and can be extended to things like database
updates and message processing.

"Solve a smaller problem first"

These kinds of properties are based on "structural induction" -- that is, if a large thing can be
broken into smaller parts, and some property is true for these smaller parts, then you can
often prove that the property is true for a large thing as well.
In the diagram below, we can see that the four-item list can be partitioned into an item plus a
three-item list, which in turn can be partitioned into an item plus a two-item list. If we can
prove the property holds for two-item list, then we can infer that it holds for the three-item
list, and for the four-item list as well.

Induction properties are often naturally applicable to recursive structures such as lists and
trees.

"Hard to prove, easy to verify"

1677

Choosing properties for property-based testing

Often an algorithm to find a result can be complicated, but verifying the answer is easy.
In the diagram below, we can see that finding a route through a maze is hard, but checking
that it works is trivial!

Many famous problems are of this sort, such as prime number factorization. But this
approach can be used for even simple problems.
For example, you might check that a string tokenizer works by just concatenating all the
tokens again. The resulting string should be the same as what you started with.

"The test oracle"

In many situations you often have an alternate version of an algorithm or process (a "test
oracle") that you can use to check your results.

For example, you might have a high-performance algorithm with optimization tweaks that
you want to test. In this case, you might compare it with a brute force algorithm that is much
slower but is also much easier to write correctly.
Similarly, you might compare the result of a parallel or concurrent algorithm with the result of
a linear, single thread version.

Putting the categories to work with some real

examples
1678

Choosing properties for property-based testing

In this section, we'll apply these categories to see if we can come up with properties for
some simple functions such as "sort a list" and "reverse a list".

"Different paths, same destination" applied to a list sort

Let's start with "different paths, same destination" and apply it to a "list sort" function.
Can we think of any way of combining an operation before List.sort , and another
operation after List.sort , so that you should end up with the same result? That is, so that
"going up then across the top" is the same as "going across the bottom then up".

How about this?

Path 1: We add one to each element of the list, then sort.
Path 2: We sort, then add one to each element of the list.
Both lists should be equal.

Here's some code that implements that property:

1679

Choosing properties for property-based testing

let ``+1 then sort should be same as sort then +1`` sortFn aList =
let add1 x = x + 1
let result1 = aList |> sortFn |> List.map add1
let result2 = aList |> List.map add1 |> sortFn
result1 = result2
// test
let goodSort = List.sort
Check.Quick (``+1 then sort should be same as sort then +1`` goodSort)
// Ok, passed 100 tests.

Well, that works, but it also would work for a lot of other transformations too. For example, if
we implemented List.sort as just the identity, then this property would be satisfied equally
well! You can test this for yourself:
let badSort aList = aList
Check.Quick (``+1 then sort should be same as sort then +1`` badSort)
// Ok, passed 100 tests.

The problem with this property is that it is not exploiting any of the "sortedness". We know
that a sort will probably reorder a list, and certainly, the smallest element should be first.
How about adding an item that we know will come at the front of the list after sorting?
Path 1: We append Int32.MinValue to the end of the list, then sort.
Path 2: We sort, then prepend Int32.MinValue to the front of the list.
Both lists should be equal.

Here's the code:

1680

Choosing properties for property-based testing

let ``append minValue then sort should be same as sort then prepend minValue`` sortFn
aList =
let minValue = Int32.MinValue
let appendThenSort = (aList @ [minValue]) |> sortFn
let sortThenPrepend = minValue :: (aList |> sortFn)
appendThenSort = sortThenPrepend
// test
Check.Quick (``append minValue then sort should be same as sort then prepend minValue`
` goodSort)
// Ok, passed 100 tests.

The bad implementation fails now!

Check.Quick (``append minValue then sort should be same as sort then prepend minValue`
` badSort)
// Falsifiable, after 1 test (2 shrinks)
// [0]

In other words, the bad sort of [0; minValue] is not the same as [minValue; 0] .
So that's good!
But... we've got some hard coded things in there that the Enterprise Developer From Hell
(see previous post) could take advantage of! The EDFH will exploit the fact that we always
use Int32.MinValue and that we always prepend or append it to the test list.
In other words, the EDFH can identify which path we are on and have special cases for each
one:
// The Enterprise Developer From Hell strikes again
let badSort2 aList =
match aList with
| [] -> []
| _ ->
let last::reversedTail = List.rev aList
if (last = Int32.MinValue) then
// if min is last, move to front
let unreversedTail = List.rev reversedTail
last :: unreversedTail
else
aList // leave alone

And when we check it...

1681

Choosing properties for property-based testing

// Oh dear, the bad implementation passes!

Check.Quick (``append minValue then sort should be same as sort then prepend minValue`
` badSort2)
// Ok, passed 100 tests.

We could fix this by (a) picking a random number smaller than any number in the list and (b)
inserting it at a random location rather than always appending it. But rather than getting too
complicated, let's stop and reconsider.
An alternative approach which also exploits the "sortedness" is to first negate all the values,
then on the path that negates after the sort, add an extra reverse as well.

let ``negate then sort should be same as sort then negate then reverse`` sortFn aList
=
let negate x = x * -1
let negateThenSort = aList |> List.map negate |> sortFn
let sortThenNegateAndReverse = aList |> sortFn |> List.map negate |> List.rev
negateThenSort = sortThenNegateAndReverse

This property is harder for the EDFH to beat because there are no magic numbers to help
identify which path you are on:

1682

Choosing properties for property-based testing

// test
Check.Quick ( ``negate then sort should be same as sort then negate then reverse`` goo
dSort)
// Ok, passed 100 tests.
// test
Check.Quick ( ``negate then sort should be same as sort then negate then reverse`` ba
dSort)
// Falsifiable, after 1 test (1 shrinks)
// [1; 0]
// test
Check.Quick ( ``negate then sort should be same as sort then negate then reverse`` ba
dSort2)
// Falsifiable, after 5 tests (3 shrinks)
// [1; 0]

You might argue that we are only testing sorting for lists of integers. But the List.sort
function is generic and knows nothing about integers per se, so I have high confidence that
this property does test the core sorting logic.

Applying "different paths, same destination" to a list

reversal function
Ok, enough of List.sort . What about applying the same ideas to the list reversal function?
We can do the same append/prepend trick:

Here's the code for the property:

let ``append any value then reverse should be same as reverse then prepend same value`
` revFn anyValue aList =
let appendThenReverse = (aList @ [anyValue]) |> revFn
let reverseThenPrepend = anyValue :: (aList |> revFn)
appendThenReverse = reverseThenPrepend

1683

Choosing properties for property-based testing

Here are the test results for the correct function and for two incorrect functions:
// test
let goodReverse = List.rev
Check.Quick (`àppend any value then reverse should be same as reverse then prepend sa
me value`` goodReverse)
// Ok, passed 100 tests.
// bad implementation fails
let badReverse aList = []
Check.Quick (`àppend any value then reverse should be same as reverse then prepend sa
me value`` badReverse)
// Falsifiable, after 1 test (2 shrinks)
// true, []
// bad implementation fails
let badReverse2 aList = aList
Check.Quick (`àppend any value then reverse should be same as reverse then prepend sa
me value`` badReverse2)
// Falsifiable, after 1 test (1 shrinks)
// true, [false]

You might notice something interesting here. I never specified the type of the list. The
property works with any list.
In cases like these, FsCheck will generate random lists of bools, strings, ints, etc.
In both failing cases, the anyValue is a bool. So FsCheck is using lists of bools to start with.
Here's an exercise for you: Is this property good enough? Is there some way that the EDFH
can create an implementation that will pass?

"There and back again"

Sometimes the multi-path style properties are not available or too complicated, so let's look
at some other approaches.
We'll start with properties involving inverses.
Let's start with list sorting again. Is there an inverse to sorting? Hmmm, not really. So we'll
skip sorting for now.
What about list reversal? Well, as it happens, reversal is its own inverse!

1684

Choosing properties for property-based testing

Let's turn that into a property:

let ``reverse then reverse should be same as original`` revFn aList =
let reverseThenReverse = aList |> revFn |> revFn
reverseThenReverse = aList

And it passes:
let goodReverse = List.rev
Check.Quick (``reverse then reverse should be same as original`` goodReverse)
// Ok, passed 100 tests.

Unfortunately, a bad implementation satisfies the property too!

let badReverse aList = aList
Check.Quick (``reverse then reverse should be same as original`` badReverse)
// Ok, passed 100 tests.

Nevertheless, the use of properties involving inverses can be very useful to verify that your
inverse function (such as deserialization) does indeed "undo" the primary function (such as
serialization).
We'll see some real examples of using this in the next post.

"Hard to prove, easy to verify"

So far we've been testing properties without actually caring about the end result of an
operation.
But of course in practice, we do care about the end result!
Now we normally can't really tell if the result is right without duplicating the function under
test. But often we can tell that the result is wrong quite easily. In the maze diagram from
above, we can easily check whether the path works or not.

1685

Choosing properties for property-based testing

If we are looking for the shortest path, we might not be able to check it, but at least we know
that we have some valid path.
This principle can be applied quite generally.
For example, let's say that we want to check whether a string split function is working.
We don't have to write a tokenizer -- all we have to do is ensure that the tokens, when
concatenated, give us back the original string!

Here's the core code from that property:

let concatWithComma s t = s + "," + t
let tokens = originalString.Split [| ',' |]
let recombinedString =
// can use reduce safely because there is always at least one token
tokens |> Array.reduce concatWithComma
// compare the result with the original
originalString = recombinedString

But how can we create an original string? The random strings generated by FsCheck are
unlikely to contain many commas!
There are ways that you can control exactly how FsCheck generates random data, which
we'll look at later.
For now though, we'll use a trick. The trick is to let FsCheck generate a list of random
strings, and then we'll build an originalString from them by concatting them together.
So here's the complete code for the property:

1686

Choosing properties for property-based testing

let ``concatting the elements of a string split by commas recreates the original strin
g`` aListOfStrings =
// helper to make a string
let addWithComma s t = s + "," + t
let originalString = aListOfStrings |> List.fold addWithComma ""
// now for the property
let tokens = originalString.Split [| ',' |]
let recombinedString =
// can use reduce safely because there is always at least one token
tokens |> Array.reduce addWithComma
// compare the result with the original
originalString = recombinedString

When we test this we are happy:

Check.Quick ``concatting the elements of a string split by commas recreates the origin
al string``
// Ok, passed 100 tests.

"Hard to prove, easy to verify" for list sorting

So how can we apply this principle to a sorted list? What property is easy to verify?
The first thing that pops into my mind is that for each pair of elements in the list, the first one
will be smaller than the second.

So let's make that into a property:

let ``adjacent pairs from a list should be ordered`` sortFn aList =
let pairs = aList |> sortFn |> Seq.pairwise
pairs |> Seq.forall (fun (x,y) -> x <= y )

But something funny happens when we try to check it. We get an error!

1687

Choosing properties for property-based testing

let goodSort = List.sort

Check.Quick (``adjacent pairs from a list should be ordered`` goodSort)

System.Exception: Geneflect: type not handled System.IComparable

at FsCheck.ReflectArbitrary.reflectObj@102-4.Invoke(String message)
at Microsoft.FSharp.Core.PrintfImpl.go@523-3[b,c,d](String fmt, Int32 len, FSharpFu
nc`2 outputChar, FSharpFunc`2 outa, b os, FSharpFunc`2 finalize, FSharpList`1 args, In
t32 i)
at Microsoft.FSharp.Core.PrintfImpl.run@521[b,c,d](FSharpFunc`2 initialize, String
fmt, Int32 len, FSharpList`1 args)

What does System.Exception: type not handled System.IComparable mean? It means that
FsCheck is trying to generate a random list, but all it knows is that the elements must be
IComparable . But IComparable is not a type than can be instantiated, so FsCheck throws

an error.
How can we prevent this from happening? The solution is to specify a particular type for the
property, such as int list , like this:
let ``adjacent pairs from a list should be ordered`` sortFn (aList:int list) =
let pairs = aList |> sortFn |> Seq.pairwise
pairs |> Seq.forall (fun (x,y) -> x <= y )

This code works now.

let goodSort = List.sort
Check.Quick (``adjacent pairs from a list should be ordered`` goodSort)
// Ok, passed 100 tests.

Note that even though the property has been constrained, the property is still a very general
one. We could have used string list instead, for example, and it would work just the
same.
let ``adjacent pairs from a string list should be ordered`` sortFn (aList:string list)
=
let pairs = aList |> sortFn |> Seq.pairwise
pairs |> Seq.forall (fun (x,y) -> x <= y )
Check.Quick (``adjacent pairs from a string list should be ordered`` goodSort)
// Ok, passed 100 tests.

TIP: If FsCheck throws "type not handled", add explicit type constraints to your
property

1688

Choosing properties for property-based testing

Are we done now? No! One problem with this property is that it doesn't catch malicious
implementations by the EDFH.
// bad implementation passes
let badSort aList = []
Check.Quick (``adjacent pairs from a list should be ordered`` badSort)
// Ok, passed 100 tests.

Is it a surprise to you that a silly implementation also works?

Hmmm. That tells us that there must be some property other than pairwise ordering
associated with sorting that we've overlooked. What are we missing here?
This is a good example of how doing property-based testing can lead to insights about
design. We thought we knew what sorting meant, but we're being forced to be a bit stricter in
our definition.
As it happens, we'll fix this particular problem by using the next principle!

"Some things never change"

A useful kind of property is based on an invariant that is preserved after some
transformation, such as preserving length or contents.
They are not normally sufficient in themselves to ensure a correct implementation, but they
do often act as a counter-check to more general properties.
For example, in the previous post, we created commutative and associative properties for
addition, but then noticed that simply having an implementation that returned zero would
satisfy them just as well! It was only when we added x + 0 = x as a property that we could
eliminate that particular malicious implementation.
And in the "list sort" example above, we could satisfy the "pairwise ordered" property with a
function that just returned an empty list! How could we fix that?
Our first attempt might be to check the length of the sorted list. If the lengths are different,
then the sort function obviously cheated!
let ``sort should have same length as original`` sortFn (aList:int list) =
let sorted = aList |> sortFn
List.length sorted = List.length aList

We check it and it works:

1689

Choosing properties for property-based testing

let goodSort = List.sort

Check.Quick (``sort should have same length as original`` goodSort )
// Ok, passed 100 tests.

And yes, the bad implementation fails:

let badSort aList = []
Check.Quick (``sort should have same length as original`` badSort )
// Falsifiable, after 1 test (1 shrink)
// [0]

Unfortunately, the BDFH is not defeated and can come up with another compliant
implementation! Just repeat the first element N times!
// bad implementation has same length
let badSort2 aList =
match aList with
| [] -> []
| head::_ -> List.replicate (List.length aList) head
// for example
// badSort2 [1;2;3] => [1;1;1]

Now when we test this, it passes:

Check.Quick (``sort should have same length as original`` badSort2)
// Ok, passed 100 tests.

What's more, it also satisfies the pairwise property too!

Check.Quick (``adjacent pairs from a list should be ordered`` badSort2)
// Ok, passed 100 tests.

Sort invariant - 2nd attempt

So now we have to try again. What is the difference between the real result [1;2;3] and
the fake result [1;1;1] ?
Answer: the fake result is throwing away data. The real result always contains the same
contents as the original list, but just in a different order.

1690

Choosing properties for property-based testing

That leads us to a new property: a sorted list is always a permutation of the original list. Aha!
Let's write the property in terms of permutations now:
let ``a sorted list is always a permutation of the original list`` sortFn (aList:int l
ist) =
let sorted = aList |> sortFn
let permutationsOfOriginalList = permutations aList
// the sorted list must be in the seq of permutations
permutationsOfOriginalList
|> Seq.exists (fun permutation -> permutation = sorted)

Great, now all we need is a permutation function.

Let's head over to StackOverflow and steal borrow an implementation. Here it is:

1691

Choosing properties for property-based testing

/// given aList and anElement to insert,

/// generate all possible lists with anElement
/// inserted into aList
let rec insertElement anElement aList =
// From http://stackoverflow.com/a/4610704/1136133
seq {
match aList with
// empty returns a singleton
| [] -> yield [anElement]
// not empty?
| first::rest ->
// return anElement prepended to the list
yield anElement::aList
// also return first prepended to all the sublists
for sublist in insertElement anElement rest do
yield first::sublist
}
/// Given a list, return all permutations of it
let rec permutations aList =
seq {
match aList with
| [] -> yield []
| first::rest ->
// for each sub-permutation,
// return the first inserted into it somewhere
for sublist in permutations rest do
yield! insertElement first sublist
}

Some quick interactive tests confirm that it works as expected:

permutations ['a';'b';'c'] |> Seq.toList
// [['a'; 'b'; 'c']; ['b'; 'a'; 'c']; ['b'; 'c'; 'a']; ['a'; 'c'; 'b'];
// ['c'; 'a'; 'b']; ['c'; 'b'; 'a']]
permutations ['a';'b';'c';'d'] |> Seq.toList
// [['a'; 'b'; 'c'; 'd']; ['b'; 'a'; 'c'; 'd']; ['b'; 'c'; 'a'; 'd'];
// ['b'; 'c'; 'd'; 'a']; ['a'; 'c'; 'b'; 'd']; ['c'; 'a'; 'b'; 'd'];
// ['c'; 'b'; 'a'; 'd']; ['c'; 'b'; 'd'; 'a']; ['a'; 'c'; 'd'; 'b'];
// ['c'; 'a'; 'd'; 'b']; ['c'; 'd'; 'a'; 'b']; ['c'; 'd'; 'b'; 'a'];
// ['a'; 'b'; 'd'; 'c']; ['b'; 'a'; 'd'; 'c']; ['b'; 'd'; 'a'; 'c'];
// ['b'; 'd'; 'c'; 'a']; ['a'; 'd'; 'b'; 'c']; ['d'; 'a'; 'b'; 'c'];
// ['d'; 'b'; 'a'; 'c']; ['d'; 'b'; 'c'; 'a']; ['a'; 'd'; 'c'; 'b'];
// ['d'; 'a'; 'c'; 'b']; ['d'; 'c'; 'a'; 'b']; ['d'; 'c'; 'b'; 'a']]
permutations [3;3] |> Seq.toList
// [[3; 3]; [3; 3]]

Excellent! Now let's run FsCheck:

1692

Choosing properties for property-based testing

Check.Quick (``a sorted list is always a permutation of the original list`` goodSort)

Hmmm. That's funny, nothing seems to be happening. And my CPU is maxing out for some
reason. What's going on?
What's going on is that you are going to be sitting there for a long time! If you are following
along at home, I suggest you right-click and cancel the interactive session now.
The innocent looking permutations is really really slow for any normal sized list. For
example, a list of just 10 items has 3,628,800 permutations. While with 20 items, you are
getting to astronomical numbers.
And of course, FsCheck will be doing hundreds of these tests! So this leads to an important
tip:
TIP: Make sure your property checks are very fast. You will be running them a LOT!
We've already seen that even in the best case, FsCheck will evaluate the property 100
times. And if shrinking is needed, even more. So make sure your tests are fast to run!
But what happens if you are dealing with real systems such as databases, networks, or
other slow dependencies?
In his (highly recommended) video on using QuickCheck, John Hughes tells of when his
team was trying to detect flaws in a distributed data store that could be caused by network
partitions and node failures.
Of course, killing real nodes thousands of times was too slow, so they extracted the core
logic into a virtual model, and tested that instead. As a result, the code was later refactored
to make this kind of testing easier. In other words, property-based testing influenced the
design of the code, just as TDD would.

Sort invariant - 3rd attempt

Ok, so we can't use permutations by just looping through them. So let's use the same idea
but write a function that is specific for this case, a isPermutationOf function.
let ``a sorted list has same contents as the original list`` sortFn (aList:int list) =

let sorted = aList |> sortFn
isPermutationOf aList sorted

Here's the code for isPermutationOf and its associated helper functions:

1693

Choosing properties for property-based testing

/// Given an element and a list, and other elements previously skipped,
/// return a new list without the specified element.
/// If not found, return None
let rec withoutElementRec anElement aList skipped =
match aList with
| [] -> None
| head::tail when anElement = head ->
// matched, so create a new list from the skipped and the remaining
// and return it
let skipped' = List.rev skipped
Some (skipped' @ tail)
| head::tail ->
// no match, so prepend head to the skipped and recurse
let skipped' = head :: skipped
withoutElementRec anElement tail skipped'
/// Given an element and a list
/// return a new list without the specified element.
/// If not found, return None
let withoutElement x aList =
withoutElementRec x aList []
/// Given two lists, return true if they have the same contents
/// regardless of order
let rec isPermutationOf list1 list2 =
match list1 with
| [] -> List.isEmpty list2 // if both empty, true
| h1::t1 ->
match withoutElement h1 list2 with
| None -> false
| Some t2 ->
isPermutationOf t1 t2

Let's try the test again. And yes, this time it completes before the heat death of the universe.
Check.Quick (``a sorted list has same contents as the original list`` goodSort)
// Ok, passed 100 tests.

What's also great is that the malicious implementation now fails to satisfy this property!
Check.Quick (``a sorted list has same contents as the original list`` badSort2)
// Falsifiable, after 2 tests (5 shrinks)
// [1; 0]

In fact, these two properties, adjacent pairs from a list should be ordered and a sorted
list has same contents as the original list should indeed ensure that any implementation

is correct.

1694

Choosing properties for property-based testing

Sidebar: Combining properties

Just above, we noted that there were two properties needed to define the "is sorted"
property. It would be nice if we could combine them into one property is sorted so that we
can have a single test.
Well, of course we can always merge the two sets of code into one function, but it's
preferable to keep functions as small as possible. Furthermore, a property like has same
contents might be reusable in other contexts as well.

What we want then, is an equivalent to AND and OR that is designed to work with
properties.
FsCheck to the rescue! There are built in operators to combine properties: .&. for AND
and .|. for OR .
Here is an example of them in use:
let ``list is sorted``sortFn (aList:int list) =
let prop1 = ``adjacent pairs from a list should be ordered`` sortFn aList
let prop2 = ``a sorted list has same contents as the original list`` sortFn aList
prop1 .&. prop2

When we test the combined property with a good implementation of sort , everything works
as expected.
let goodSort = List.sort
Check.Quick (``list is sorted`` goodSort )
// Ok, passed 100 tests.

And if we test a bad implementation, the combined property fails as well.

let badSort aList = []
Check.Quick (``list is sorted`` badSort )
// Falsifiable, after 1 test (0 shrinks)
// [0]

But there's a problem now. Which of the two properties failed?

What we would like to do is add a "label" to each property so that we can tell them apart. In
FsCheck, this is done with the |@ operator:

1695

Choosing properties for property-based testing

let ``list is sorted (labelled)``sortFn (aList:int list) =

let prop1 = ``adjacent pairs from a list should be ordered`` sortFn aList
|@ "adjacent pairs from a list should be ordered"
let prop2 = ``a sorted list has same contents as the original list`` sortFn aList
|@ "a sorted list has same contents as the original list"
prop1 .&. prop2

And now, when we test with the bad sort, we get a message Label of failing property: a
sorted list has same contents as the original list :

Check.Quick (``list is sorted (labelled)`` badSort )

// Falsifiable, after 1 test (2 shrinks)
// Label of failing property: a sorted list has same contents as the original list
// [0]

For more on these operators, see the FsCheck documentation under "And, Or and Labels".
And now, back to the property-divising strategies.

"Solving a smaller problem"

Sometimes you have a recursive data structure or a recursive problem. In these cases, you
can often find a property that is true of a smaller part.
For example, for a sort, we could say something like:
A list is sorted if:
* The first element is smaller (or equal to) the second.
* The rest of the elements after the first element are also sorted.

Here is that logic expressed in code:

let rec ``First element is <= than second, and tail is also sorted`` sortFn (aList:int
list) =
let sortedList = aList |> sortFn
match sortedList with
| [] -> true
| [first] -> true
| [first;second] ->
first <= second
| first::second::tail ->
first <= second &&
let subList = second::tail
``First element is <= than second, and tail is also sorted`` sortFn subList

1696

Choosing properties for property-based testing

This property is satisfied by the real sort function:

let goodSort = List.sort
Check.Quick (``First element is <= than second, and tail is also sorted`` goodSort )
// Ok, passed 100 tests.

But unfortunately, just like previous examples, the malicious implementations also pass.
let badSort aList = []
Check.Quick (``First element is <= than second, and tail is also sorted`` badSort )
// Ok, passed 100 tests.
let badSort2 aList =
match aList with
| [] -> []
| head::_ -> List.replicate (List.length aList) head
Check.Quick (``First element is <= than second, and tail is also sorted`` badSort2)
// Ok, passed 100 tests.

So as before, we'll need another property (such as the has same contents invariant) to
ensure that the code is correct.
If you do have a recursive data structure, then try looking for recursive properties. They are
pretty obvious and low hanging, when you get the hang of it.

Is the EDFH really a problem?

In the last few examples, I've noted that trivial but wrong implementations often satisfy the
properties as well as good implementations.
But should we really spend time worrying about this? I mean, if we ever really released a
sort algorithm that just duplicated the first element it would be obvious immediately, surely?
So yes, it's true that truly malicious implementations are unlikely to be a problem. On the
other hand, you should think of property-based testing not as a testing process, but as a
design process -- a technique that helps you clarify what your system is really trying to do.
And if a key aspect of your design is satisfied with just a simple implementation, then
perhaps there is something you have overlooked -- something that, when you discover it, will
make your design both clearer and more robust.

1697

Choosing properties for property-based testing

"The more things change, the more they stay

the same"
Our next type of property is "idempotence". Idempotence simply means that doing
something twice is the same as doing it once. If I tell you to "sit down" and then tell you to
"sit down" again, the second command has no effect.
Idempotence is essential for reliable systems and is a key aspect of service oriented and
message-based architectures.
If you are designing these kinds of real-world systems it is well worth ensuring that your
requests and processes are idempotent.
I won't go too much into this right now, but let's look at two simple examples.
First, our old friend sort is idempotent (ignoring stability) while reverse is not, obviously.
let ``sorting twice gives the same result as sorting once`` sortFn (aList:int list) =
let sortedOnce = aList |> sortFn
let sortedTwice = aList |> sortFn |> sortFn
sortedOnce = sortedTwice
// test
let goodSort = List.sort
Check.Quick (``sorting twice gives the same result as sorting once`` goodSort )
// Ok, passed 100 tests.

In general, any kind of query should be idempotent, or to put it another way: "asking a
question should not change the answer".
In the real world, this may not be the case. A simple query on a datastore run at different
times may give different results.
Here's a quick demonstration.
First we'll create a NonIdempotentService that gives different results on each query.

1698

Choosing properties for property-based testing

type NonIdempotentService() =
let mutable data = 0
member this.Get() =
data
member this.Set value =
data <- value
let ``querying NonIdempotentService after update gives the same result`` value1 value2
=
let service = NonIdempotentService()
service.Set value1
// first GET
let get1 = service.Get()
// another task updates the data store
service.Set value2
// second GET called just like first time
let get2 = service.Get()
get1 = get2

But if we test it now, we find that it does not satisfy the required idempotence property:
Check.Quick ``querying NonIdempotentService after update gives the same result``
// Falsifiable, after 2 tests

As an alternative, we can create a (crude) IdempotentService that requires a timestamp for

each transaction. In this design, multiple GETs using the same timestamp will always
retrieve the same data.

1699

Choosing properties for property-based testing

type IdempotentService() =
let mutable data = Map.empty
member this.GetAsOf (dt:DateTime) =
data |> Map.find dt
member this.SetAsOf (dt:DateTime) value =
data <- data |> Map.add dt value
let ``querying IdempotentService after update gives the same result`` value1 value2 =
let service = IdempotentService()
let dt1 = DateTime.Now.AddMinutes(-1.0)
service.SetAsOf dt1 value1
// first GET
let get1 = service.GetAsOf dt1
// another task updates the data store
let dt2 = DateTime.Now
service.SetAsOf dt2 value2
// second GET called just like first time
let get2 = service.GetAsOf dt1
get1 = get2

And this one works:

Check.Quick ``querying IdempotentService after update gives the same result``
// Ok, passed 100 tests.

So, if you are building a REST GET handler or a database query service, and you want
idempotence, you should consider using techniques such as etags, "as-of" times, date
ranges, etc.
If you need tips on how to do this, searching for idempotency patterns will turn up some
good results.

"Two heads are better than one"

And finally, last but not least, we come to the "test oracle". A test oracle is simply an
alternative implementation that gives the right answer, and that you can check your results
against.
Often the test oracle implementation is not suitable for production -- it's too slow, or it doesn't
parallelize, or it's too poetic, etc., but that doesn't stop it being very useful for testing.

1700

Choosing properties for property-based testing

So for "list sort", there are many simple but slow implementations around. For example,
here's a quick implementation of insertion sort:
module InsertionSort =
// Insert a new element into a list by looping over the list.
// As soon as you find a larger element, insert in front of it
let rec insert newElem list =
match list with
| head::tail when newElem > head ->
head :: insert newElem tail
| other -> // including empty list
newElem :: other
// Sorts a list by inserting the head into the rest of the list
// after the rest have been sorted
let rec sort list =
match list with
| [] -> []
| head::tail ->
insert head (sort tail)
// test
// insertionSort [5;3;2;1;1]

With this in place, we can write a property that tests the result against insertion sort.
let ``sort should give same result as insertion sort`` sortFn (aList:int list) =
let sorted1 = aList |> sortFn
let sorted2 = aList |> InsertionSort.sort
sorted1 = sorted2

When we test the good sort, it works. Good!

let goodSort = List.sort
Check.Quick (``sort should give same result as insertion sort`` goodSort)
// Ok, passed 100 tests.

And when we test a bad sort, it doesn't. Even better!

let badSort aList = aList
Check.Quick (``sort should give same result as insertion sort`` badSort)
// Falsifiable, after 4 tests (6 shrinks)
// [1; 0]

1701

Choosing properties for property-based testing

Generating Roman numerals in two different

ways
We can also use the test oracle approach to cross-check two different implementations
when you're not sure that either implementation is right!
For example, in my post "Commentary on 'Roman Numerals Kata with Commentary'" I came
up with two completely different algorithms for generating Roman Numerals. Can we
compare them to each other and test them both in one fell swoop?
The first algorithm was based on understanding that Roman numerals were based on
tallying, leading to this simple code:
let arabicToRomanUsingTallying arabic =
(String.replicate arabic "I")
.Replace("IIIII","V")
.Replace("VV","X")
.Replace("XXXXX","L")
.Replace("LL","C")
.Replace("CCCCC","D")
.Replace("DD","M")
// optional substitutions
.Replace("IIII","IV")
.Replace("VIV","IX")
.Replace("XXXX","XL")
.Replace("LXL","XC")
.Replace("CCCC","CD")
.Replace("DCD","CM")

Another way to think about Roman numerals is to imagine an abacus. Each wire has four
"unit" beads and one "five" bead.
This leads to the so-called "bi-quinary" approach:

1702

Choosing properties for property-based testing

let biQuinaryDigits place (unit,five,ten) arabic =

let digit = arabic % (10*place) / place
match digit with
| 0 -> ""
| 1 -> unit
| 2 -> unit + unit
| 3 -> unit + unit + unit
| 4 -> unit + five // changed to be one less than five
| 5 -> five
| 6 -> five + unit
| 7 -> five + unit + unit
| 8 -> five + unit + unit + unit
| 9 -> unit + ten // changed to be one less than ten
| _ -> failwith "Expected 0-9 only"
let arabicToRomanUsingBiQuinary arabic =
let units = biQuinaryDigits 1 ("I","V","X") arabic
let tens = biQuinaryDigits 10 ("X","L","C") arabic
let hundreds = biQuinaryDigits 100 ("C","D","M") arabic
let thousands = biQuinaryDigits 1000 ("M","?","?") arabic
thousands + hundreds + tens + units

We now have two completely different algorithms, and we can cross-check them with each
other to see if they give the same result.
let ``biquinary should give same result as tallying`` arabic =
let tallyResult = arabicToRomanUsingTallying arabic
let biquinaryResult = arabicToRomanUsingBiQuinary arabic
tallyResult = biquinaryResult

But if we try running this code, we get a ArgumentException: The input must be non-negative
due to the String.replicate call.
Check.Quick ``biquinary should give same result as tallying``
// ArgumentException: The input must be non-negative.

So we need to only include inputs that are positive. We also need to exclude numbers that
are greater than 4000, say, since the algorithms break down there too.
How can we implement this filter?
We saw in the previous post that we could use preconditions. But for this example, we'll try
something different and change the generator.

1703

Choosing properties for property-based testing

First we'll define a new arbitrary integer called arabicNumber which is filtered as we want (an
"arbitrary" is a combination of a generator algorithm and a shrinker algorithm, as described
in the previous post).
let arabicNumber = Arb.Default.Int32() |> Arb.filter (fun i -> i > 0 && i <= 4000)

Next, we create a new property which is constrained to only use "arabicNumber" by using
the Prop.forAll helper.
We'll give the property the rather clever name of "for all values of arabicNumber, biquinary
should give same result as tallying".
let ``for all values of arabicNumber biquinary should give same result as tallying`` =

Prop.forAll arabicNumber ``biquinary should give same result as tallying``

Now finally, we can do the cross-check test:

Check.Quick ``for all values of arabicNumber biquinary should give same result as tall
ying``
// Ok, passed 100 tests.

And we're good! Both algorithms work correctly, it seems.

"Model-based" testing
"Model-based" testing, which we will discuss in more detail in a later post, is a variant on
having a test oracle.
The way it works is that, in parallel with your (complex) system under test, you create a
simplified model.
Then, when you do something to the system under test, you do the same (but simplified)
thing to your model.
At the end, you compare your model's state with the state of the system under test. If they
are the same, you're done. If not, either your SUT is buggy or your model is wrong and you
have to start over!

Interlude: A game based on finding properties

1704

Choosing properties for property-based testing

With that, we have come to the end of the various property categories. We'll go over them
one more time in a minute -- but first, an interlude.
If you sometimes feel that trying to find properties is a mental challenge, you're not alone.
Would it help to pretend that it is a game?
As it happens, there is a game based on property-based testing.
It's called Zendo and it involves placing sets of objects (such as plastic pyramids) on a table,
such that each layout conforms to a pattern -- a rule -- or as we would now say, a property!.
The other players then have to guess what the rule (property) is, based on what they can
see.
Here's a picture of a Zendo game in progress:

The white stones mean the property has been satisfied, while black stones mean failure.
Can you guess the rule here? I'm going to guess that it's something like "a set must have a
yellow pyramid that's not touching the ground".
Alright, I suppose Zendo wasn't really inspired by property-based testing, but it is a fun
game, and it has even been known to make an appearance at programming conferences.
If you want to learn more about Zendo, the rules are here.

Applying the categories one more time

1705

Choosing properties for property-based testing

With all these categories in hand, let's look at one more example problem, and see if we can
find properties for it.
This sample is based on the well-known Dollar example described in Kent Beck's "TDD By
Example" book.
Nat Pryce, of Growing Object-Oriented Software Guided by Tests fame, wrote a blog post
about property-based testing a while ago ("Exploring Test-Driven Development with
QuickCheck").
In it, he expressed some frustration about property-based testing being useful in practice. So
let's revisit the example he referenced and see what we can do with it.
We're not going to attempt to critique the design itself and make it more type-driven -- others
have done that. Instead, we'll take the design as given and see what properties we can
come up with.
So what do we have?
A Dollar class that stores an Amount .
Methods Add and Times that transform the amount in the obvious way.
// OO style class with members
type Dollar(amount:int) =
member val Amount = amount with get, set
member this.Add add =
this.Amount <- this.Amount + add
member this.Times multiplier =
this.Amount <- this.Amount * multiplier
static member Create amount =
Dollar amount

So, first let's try it out interactively to make sure it works as expected:
let d = Dollar.Create 2
d.Amount // 2
d.Times 3
d.Amount // 6
d.Add 1
d.Amount // 7

But that's just playing around, not real testing. So what kind of properties can we think of?
Let's run through them all again:
Different paths to same result
Inverses

1706

Choosing properties for property-based testing

Invariants
Idempotence
Structural induction
Easy to verify
Test oracle
Let's skip the "different paths" one for now. What about inverses? Are there any inverses we
can use?
Yes, the setter and getter form an inverse that we can create a property from:
let ``set then get should give same result`` value =
let obj = Dollar.Create 0
obj.Amount <- value
let newValue = obj.Amount
value = newValue
Check.Quick ``set then get should give same result``
// Ok, passed 100 tests.

Idempotence is relevant too. For example, doing two sets in a row should be the same as
doing just one. Here's a property for that:
let ``set amount is idempotent`` value =
let obj = Dollar.Create 0
obj.Amount <- value
let afterFirstSet = obj.Amount
obj.Amount <- value
let afterSecondSet = obj.Amount
afterFirstSet = afterSecondSet
Check.Quick ``set amount is idempotent``
// Ok, passed 100 tests.

Any "structural induction" properties? No, not relevant to this case.

Any "easy to verify" properties? Not anything obvious.
Finally, is there a test oracle? No. Again not relevant, although if we really were designing a
complex currency management system, it might be very useful to cross-check our results
with a third party system.

Properties for an immutable Dollar

A confession! I cheated a bit in the code above and created a mutable class, which is how
most OO objects are designed.

1707

Choosing properties for property-based testing

But in "TDD by Example" , Kent quickly realizes the problems with that and changes it to an
immutable class, so let me do the same.
Here's the immutable version:
type Dollar(amount:int) =
member val Amount = amount
member this.Add add =
Dollar (amount + add)
member this.Times multiplier =
Dollar (amount * multiplier)
static member Create amount =
Dollar amount
// interactive test
let d1 = Dollar.Create 2
d1.Amount // 2
let d2 = d1.Times 3
d2.Amount // 6
let d3 = d2.Add 1
d3.Amount // 7

What's nice about immutable code is that we can eliminate the need for testing of setters, so
the two properties we just created have now become irrelevant!
To tell the truth they were pretty trivial anyway, so it's no great loss.
So then, what new properties can we devise now?
Let's look at the Times method. How can we test that? Which one of the strategies can we
use?
I think the "different paths to same result" is very applicable. We can do the same thing we
did with "sort" and do a times operation both "inside" and "outside" and see if they give the
same result.

Here's that property expressed in code:

1708

Choosing properties for property-based testing

let ``create then times should be same as times then create`` start multiplier =
let d0 = Dollar.Create start
let d1 = d0.Times(multiplier)
let d2 = Dollar.Create (start * multiplier)
d1 = d2

Great! Let's see if it works!

Check.Quick ``create then times should be same as times then create``
// Falsifiable, after 1 test

Oops -- it doesn't work!

Why not? Because we forgot that Dollar is a reference type and doesn't compare equal by
default!
As a result of this mistake, we have discovered a property that we might have overlooked!
Let's encode that before we forget.
let ``dollars with same amount must be equal`` amount =
let d1 = Dollar.Create amount
let d2 = Dollar.Create amount
d1 = d2
Check.Quick ``dollars with same amount must be equal``
// Falsifiable, after 1 test

So now we need to fix this by adding support for IEquatable and so on.
You can do that if you like -- I'm going to switch to F# record types and get equality for free!

Dollar properties -- version 3

Here's the Dollar rewritten again:
type Dollar = {amount:int }
with
member this.Add add =
{amount = this.amount + add }
member this.Times multiplier =
{amount = this.amount * multiplier }
static member Create amount =
{amount=amount}

And now our two properties are satisfied:

1709

Choosing properties for property-based testing

Check.Quick ``dollars with same amount must be equal``

// Ok, passed 100 tests.
Check.Quick ``create then times should be same as times then create``
// Ok, passed 100 tests.

We can extend this approach for different paths. For example, we can extract the amount
and compare it directly, like this:

The code looks like this:

let ``create then times then get should be same as times`` start multiplier =
let d0 = Dollar.Create start
let d1 = d0.Times(multiplier)
let a1 = d1.amount
let a2 = start * multiplier
a1 = a2
Check.Quick ``create then times then get should be same as times``
// Ok, passed 100 tests.

And we can also include Add in the mix as well.

For example, we can do a Times followed by an Add via two different paths, like this:

And here's the code:

1710

Choosing properties for property-based testing

let ``create then times then add should be same as times then add then create`` start
multiplier adder =
let d0 = Dollar.Create start
let d1 = d0.Times(multiplier)
let d2 = d1.Add(adder)
let directAmount = (start * multiplier) + adder
let d3 = Dollar.Create directAmount
d2 = d3
Check.Quick ``create then times then add should be same as times then add then create`
`
// Ok, passed 100 tests.

So this "different paths, same result" approach is very fruitful, and we can generate lots of
paths this way.

Dollar properties -- version 4

Shall we call it done then? I would say not!
We are beginning to get a whiff of a code smell. All this (start * multiplier) + adder code
seems like a bit of duplicated logic, and could end up being brittle.
Can we abstract out some commonality that is present all these cases?
If we think about it, our logic is really just this:
Transform the amount on the "inside" in some way.
Transform the amount on the "outside" in the same way.
Make sure that the results are the same.
But to test this, the Dollar class is going to have to support an arbitrary transform! Let's call
it Map !
Now all our tests can be reduced to this one property:

1711

Choosing properties for property-based testing

Let's add a Map method to Dollar . And we can also rewrite Times and Add in terms of
Map :

type Dollar = {amount:int }

with
member this.Map f =
{amount = f this.amount}
member this.Times multiplier =
this.Map (fun a -> a * multiplier)
member this.Add adder =
this.Map (fun a -> a + adder)
static member Create amount =
{amount=amount}

Now the code for our property looks like this:

let ``create then map should be same as map then create`` start f =
let d0 = Dollar.Create start
let d1 = d0.Map f
let d2 = Dollar.Create (f start)
d1 = d2

But how can we test it now? What functions should we pass in?
Don't worry! FsCheck has you covered! In cases like this, FsCheck will actually generate
random functions for you too!
Try it -- it just works!
Check.Quick ``create then map should be same as map then create``
// Ok, passed 100 tests.

Our new "map" property is much more general than the original property using "times", so
we can eliminate the latter safely.

Logging the function parameter

There's a little problem with the property as it stands. If you want to see what the function is
that FsCheck is generating, then Verbose mode is not helpful.
Check.Verbose ``create then map should be same as map then create``

Gives the output:

1712

Choosing properties for property-based testing

0:
18
<fun:Invoke@3000>
1:
7
<fun:Invoke@3000>
-- etc
98:
47
<fun:Invoke@3000>
99:
36
<fun:Invoke@3000>
Ok, passed 100 tests.

We can't tell what the function values actually were.

However, you can tell FsCheck to show more useful information by wrapping your function in
a special F case, like this:
let ``create then map should be same as map then create2`` start (F (_,f)) =
let d0 = Dollar.Create start
let d1 = d0.Map f
let d2 = Dollar.Create (f start)
d1 = d2

And now when you use Verbose mode...

Check.Verbose ``create then map should be same as map then create2``

... you get a detailed log of each function that was used:

1713

Choosing properties for property-based testing

0:
0
{ 0->1 }
1:
0
{ 0->0 }
2:
2
{ 2->-2 }
-- etc
98:
-5
{ -5->-52 }
99:
10
{ 10->28 }
Ok, passed 100 tests.

Each { 2->-2 } , { 10->28 } , etc., represents the function that was used for that iteration.

TDD vs. property-based testing

How does property-based testing (PBT) fit in with TDD? This is a common question, so let
me quickly give you my take on it.
First off, TDD works with specific examples, while PBT works with universal properties.
As I said in the previous post, I think examples are useful as a way into a design, and can be
a form of documentation. But in my opinion, relying only on example-based tests would be a
mistake.
Property-based approaches have a number of advantages over example-based tests:
Property-based tests are more general, and thus are less brittle.
Property-based tests provide a better and more concise description of requirements
than a bunch of examples.
As a consequence, one property-based test can replace many, many, example-based
tests.
By generating random input, property-based tests often reveal issues that you have
overlooked, such as dealing with nulls, missing data, divide by zero, negative numbers,
etc.
Property-based tests force you to think.
Property-based tests force you to have a clean design.

1714

Choosing properties for property-based testing

These last two points are the most important for me. Programming is not a matter of writing
lines of code, it is about creating a design that meets the requirements.
So, anything that helps you think deeply about the requirements and what can go wrong
should be a key tool in your personal toolbox!
For example, in the Roman Numeral section, we saw that accepting int was a bad idea
(the code broke!). We had a quick fix, but really we should model the concept of a
PositiveInteger in our domain, and then change our code to use that type rather than just

an int . This demonstrates how using PBT can actually improve your domain model, not
just find bugs.
Similarly, introducing a Map method in the Dollar scenario not only made testing easier, but
actually improved the usefulness of the Dollar "api".
Stepping back to look at the big picture, though, TDD and property-based testing are not at
all in conflict. They share the same goal of building correct programs, and both are really
more about design than coding (think "Test-driven design" rather than "Test-driven
development").

The end, at last

So that brings us to the end of another long post on property-based testing!
I hope that you now have some useful approaches that you can take away and apply to your
own code base.
Next time, we'll look at some real-world examples, and how you can create custom
generators that match your domain.
The code samples used in this post are available on GitHub.

1715

Commentary on 'Roman Numerals Kata with Commentary'

Commentary on 'Roman Numerals Kata

with Commentary'
I recently watched a video called "Roman Numerals Kata with Commentary". In it, Corey
Haines demonstrates how to implement the Arabic to Roman Numerals Kata in Ruby using
a TDD approach.
This video annoyed me intensely.
I mean no disrespect to Corey Haines' programming skills, and many people seem to have
found the video useful, but I just found it exasperating.
I thought that in this post I'd try to explain why I got annoyed, and to present my alternative
approach to problems like this.

Where are the requirements?

"Few programmers write even a rough sketch of what their programs will do before they
start coding. Most programmers regard anything that doesn't generate code to be a
waste of time."
Leslie Lamport, "Why We Should Build Software Like We Build Houses"
In standard TDD fashion, the video starts with implementing a initial failing case (handling
zero), then making that work, then adding a test case for handling "1", then making that
work, and so on.
This was the first thing that irritated me -- diving into code without really understanding the
requirements.
A programming kata) is so called because the goal is to practice your skills as a developer.
But for me, coding skills are just one aspect of a being a software developer, and not always
the most important.
If there is anything that needs practicing by most developers, it is listening to and
understanding the needs of the customer (a.k.a. requirements). We should never forget that
our goal is to deliver value, not just to write code.
In this case, even though there is a wiki page for the kata, the requirements are still
somewhat fuzzy, and so I view this as an excellent opportunity to drill down into them, and
maybe learn something new.

1716

Commentary on 'Roman Numerals Kata with Commentary'

Becoming a domain expert

In fact, I believe that going as deep as possible into the requirements has some important
benefits.
It's fun. It's fun to really understand a new domain. I like to learn new things -- it is one of
the perks of being a developer.
It's not just me. Dan North tells of how much fun he had working very closely with domain
experts in his "accelerating agile" presentation. Part of that team's success was that the
developers studied the domain (trading) right along with the traders themselves, so that
communication was easy and confusion minimized.
Good design. I do believe that in order to produce good software you have to become
reasonably expert in the domain you are attempting to model. This is the thesis behind
Domain Driven Design, of course, but also is a key component of an Agile process: the "on
site customer" who works very closely with the developers at all stages.
And almost always, understanding the requirements properly will lead you into the right* way
to implement a solution. No amount of shallow iterations will make up for a lack of deep
insight.
* Of course there is not really a "right" way, but there are plenty of wrong ways. So here I just
mean not horribly complicated and unmaintainable.
Good tests. You can't create good tests without understanding the requirements. A process
like BDD makes this explicit; the requirements are written in such a way that they actually
become the tests.

Understanding Roman numerals

"During an inception, when we are most ignorant about most aspects of the project, the
best use we can possibly make of the time available is to attempt to identify and reduce
our ignorance across all the axes we can think of." -- Dan North, "Deliberate Discovery"
So, how does this apply to the Roman Numerals Kata? Should we seriously become domain
experts before we write a line of code?
I would say yes!
I know it is a trivial problem, and it seems like overkill, but then again, this is a kata, so you
should be practising all the steps carefully and mindfully.
So, what can we find out about Roman numerals?

1717

Commentary on 'Roman Numerals Kata with Commentary'

First, a little background reading from a reliable source shows that they probably originated
from something similar to tally marks.

This explains the simple strokes for "I" to "IIII" and then the different symbol for "V".
As it evolved, symbols were added for ten and fifty, one hundred and five hundred, and so
on.
This system of counting with ones and fives can be seen in the design of the abacus, old
and new.

In fact, this system even has a name which I'd never heard of -- "bi-quinary coded decimal".
Isn't that fun? I shall now attempt to drop that phrase into casual conversation wherever
possible. (And by the way, the little stones used as counters are called "calculi", whence the
name for the bane of high school students everywhere.)
Much later, in the 13th century, certain abbreviations were added -- substituting "IV" for "IIII"
and "IX" for "VIIII". This subtractive notation means that the order of the symbols becomes
important, something that is not required for a pure tally-based system.
These new requirements show us that nothing has changed in the development biz...
Pope: "We need to add subtractive notation ASAP -- the Arabs are beating us on features."
You: "But it's not backwards compatible, sir. It's a breaking change!"
Pope: "Tough. I need it by next week."
So now that we know all about Roman numerals, do we have enough information to create
the requirements?
Alas, no. As we investigate further, it becomes clear that there is a lot of inconsistency.
There is no ISO or ANSI standard for Roman numerals!

1718

Commentary on 'Roman Numerals Kata with Commentary'

This is not unusual of course. A fuzziness around requirements affects most software
projects. Indeed, part of our job as a developer is to help clarify things and eliminate
ambiguity. So, let's create some requirements based on what we know so far.

The requirements for this kata

"Programmers are not to be measured by their ingenuity and their logic but by the
completeness of their case analysis." -- Alan Perlis, Epigrams
I think we would all agree that having unambiguous and testable requirements is a critical
step towards having a successful project.
Now when I talk about "requirements", I'm not talking about a 200 page document that takes
six months to write. I'm just talking about a few bullet points that take 5 or 10 minutes to
write down.
But... it is important to have them. Thinking carefully before coding is an essential skill that
needs to be practiced, and so I would recommend doing this step as part of the discipline for
any code kata.
So here are the requirements as I see them:
The output will be generated by tallying 1, 5, 10, 50, 100, 500, and 1000, using the
symbols "I", "V", "X", "L", "C", "D" and "M" respectively.
The symbols must be written in descending order: "M" before "D" before "C" before "L",
etc.
Using the tallying logic, it's clear that we can only have up to four repetitions of "I", "X",
"C" and "M". And only one "V", "L" or "D". Any more than that and the multiple tally
marks are abbreviated to the next "higher" tally mark.
Finally, we have the six (optional) substitution rules: "IIII"=>"IV", "VIIII"=>"IX",
"XXXX"=>"XL", "LXXXX"=>"XC", "CCCC"=>"CD", "DCCCC"=>"CM". These are
exceptions to the descending order rule.
There's one other very important requirement that isn't on this list.
What is the range of valid inputs?
If we don't explicitly document this, we could easily assume that all integers are valid,
including zero and negative numbers.
And what about large numbers, in the millions or billions? Are they allowed? Probably not.
So let's be explicit and say that the valid input ranges from 0 to 4000. Then what should
happen if the input is not valid? Return an empty string? Throw an exception?

1719

Commentary on 'Roman Numerals Kata with Commentary'

In a functional programming language like F#, the most common approach is to return an
Option type, or to return a Success/Failure Choice type. Let's just use an Option , so to

finish off the requirements, we have:

The Arabic number 0 is mapped to the empty string.
If the input is < 0 or > 4000 return None otherwise return Some(roman) , where roman
is the Arabic number converted to Roman numerals as described above.
So to sum up this step, we have read about Roman numerals, learned a few fun things, and
come up with some clear requirements for the next stage. The whole thing took only 5-10
mins. In my opinion, that was time well spent.

Writing the tests

"Unit tests have been compared with shining a flashlight into a dark room in search of a
monster. Shine the light into the room and then into all the scary corners. It doesn't
mean the room is monster free --- just that the monster isn't standing where you've
shined your flashlight."
Now that we have the requirements, we can start writing the tests.
In the original video, the tests were developed incrementally, starting with 0, then 1, and so
on.
Personally, I think that there are a number of problems with that approach.
First, as we should know, a major goal of TDD is not testing but design.
But this micro, incremental approach to design does not seem to me to lead to a particularly
good end result.
For example, in the video, there is a big jump in the implementation complexity from testing
the "I" case to testing the "II" case. But the rationale is a bit hard to understand, and to me it
smacks a little of sleight-of-hand, of someone who already knows the answer, rather than
naturally evolving from the previous case.
Unfortunately, I have seen this happen a lot with a strict TDD approach. You might be
cruising along nicely and then bump into a huge road block which forces a huge rethink and
refactoring.
A strict TDD'er using Uncle Bob's "Transformation Priority Premise" approach would say that
that is fine and good, and part of the process.
Personally, I'd rather start with the trickiest requirements first, and front-load the risk rather
than leaving it till the end.

1720

Commentary on 'Roman Numerals Kata with Commentary'

Second, I don't like testing individual cases. I'd prefer that my tests cover all inputs. This is
not always feasible, but when you can do it, as in this case, I think you should.

Two tests compared

To demonstrate what I mean, let's compare the test suite developed in the video with a more
general requirements-based test.
The test suite developed in the video checks only the obvious inputs, plus the case 3497 "for
good measure". Here's the Ruby code ported to F#:
[<Test>]
let ``For certain inputs, expect certain outputs``() =
let testpairs = [
(1,"I")
(2,"II")
(4,"IV")
(5,"V")
(9,"IX")
(10,"X")
// etc
(900,"CM")
(1000,"M")
(3497,"MMMCDXCVII")
]
for (arabic,expectedRoman) in testpairs do
let roman = arabicToRoman arabic
Assert.AreEqual(expectedRoman, roman)

With this set of inputs, how confident are we that the code meets the requirements?
In a simple case like this, I might be reasonably confident, but this approach to testing
worries me because of the use of "magic" test inputs that are undocumented.
For example, why was 3497 plucked out of nowhere? Because (a) it is bigger than a
thousand and (b) it has some 4's and 9's in it. But the reason it was picked is not
documented in the test code.
Furthermore, if we compare this test suite with the requirements, we can see that the second
and third requirements are not explicitly tested for at all. True, the test with 3497 implicitly
checks the ordering requirement ("M" before "C" before "X"), but that is never made explicit.
Now compare that test with this one:

1721

Commentary on 'Roman Numerals Kata with Commentary'

[<Test>]
let ``For all valid inputs, there must be a max of four "I"s in a row``() =
for i in [1..4000] do
let roman = arabicToRoman i
roman |> assertMaxRepetition "I" 4

This test checks the requirement that you can only have four repetitions of "I".
Unlike the one in the TDD video, this test case covers all possible inputs, not just one. If it
passes, I will have complete confidence that the code meets this particular requirement.

Property-based testing
If you are not familiar with this approach to testing, it is called "property-based testing". You
define a "property" that must be true in general, and then you generate as many inputs as
possible in order to find cases where the property is not true.
In this case, we can test all 4000 inputs. In general though, our problems have a much larger
range of possible inputs, so we generally just test on some representative sample of the
inputs.
Most property-based testing tools are modelled after Haskell's QuickCheck, which is a tool
that automatically generates "interesting" inputs for you, in order to find edge cases as
quickly as possible. These inputs would include things like nulls, negative numbers, empty
lists, strings with non-ascii characters in them, and so on.
An equivalent to QuickCheck is available for most languages now, including FsCheck for F#.
The advantage of property-based testing is that it forces you to think about the requirements
in general terms, rather than as lots of special cases.
That is, rather than a test that says the input "4" maps to "IV" , we have a more general
test that says any input with 4 in the units place has "IV" as the last two characters .

Implementing a property-based test

To switch to property-based testing for the requirement above, I would refactor the code so
that (a) I create a function that defines a property and then (b) I check that property against a
range of inputs.
Here's the refactored code:

1722

Commentary on 'Roman Numerals Kata with Commentary'

// Define a property that should be true for all inputs

let ``has max rep of four Is`` arabic =
let roman = arabicToRoman arabic
roman |> assertMaxRepetition "I" 4
// Explicitly enumerate all inputs...
[<Test>]
let ``For all valid inputs, there must be a max of four "I"s``() =
for i in [1..4000] do
//check that the property holds
``has max rep of four Is`` i
// ...Or use FsCheck to generate inputs for you
let isInRange i = (i >= 1) && (i <= 4000)
// input is in range implies has max of four Is
let prop i = isInRange i ==> ``has max rep of four Is`` i
// check all inputs for this property
Check.Quick prop

Or for example, let's say that I want to test the substitution rule for 40 => "XL".
// Define a property that should be true for all inputs
let `ìf arabic has 4 tens then roman has one XL otherwise none`` arabic =
let roman = arabicToRoman arabic
let has4Tens = (arabic % 100 / 10) = 4
if has4Tens then
assertMaxOccurs "XL" 1 roman
else
assertMaxOccurs "XL" 0 roman
// Explicitly enumerate all inputs...
[<Test>]
let ``For all valid inputs, check the XL substitution``() =
for i in [1..4000] do
`ìf arabic has 4 tens then roman has one XL otherwise none`` i
// ...Or again use FsCheck to generate inputs for you
let isInRange i = (i >= 1) && (i <= 4000)
let prop i = isInRange i ==> `ìf arabic has 4 tens then roman has one XL otherwise no
ne`` i
Check.Quick prop

I'm not going to go into property-based testing any more here, but I think you can see the
benefits over hand-crafted cases with magic inputs.
The code for this post has a full property-based test suite.

Requirements Driven Design?

1723

Commentary on 'Roman Numerals Kata with Commentary'

At this point, we can start on the implementation.

Unlike the TDD video, I'd rather build the implementation by iterating on the requirements,
not on the test cases. I need a catchy phrase for this, so I'll call it Requirements Driven
Design?. Watch out for a Requirements Driven Design Manifesto coming soon.
And rather than implementing code that handles individual inputs one by one, I prefer my
implementations to cover as many input cases as possible -- preferably all of them. As each
new requirement is added the implementation is modified or refined, using the tests to
ensure that it still meets the requirements.
But isn't this exactly TDD as demonstrated in the video?
No, I don't think so. The TDD demonstration was test-driven, but not requirements driven.
Mapping 1 to "I" and 2 to "II" are tests, but are not true requirements in my view. A good
requirement is based on insight into the domain. Just testing that 2 maps to "II" does not
provide that insight.

A very simple implementation

After criticizing someone else's implementation, time for me to put up or shut up.
So, what is the simplest implementation I can think of that would work?
How about just converting our arabic number to tally marks? 1 becomes "I", 2 becomes "II",
and so on.
let arabicToRoman arabic =
String.replicate arabic "I"

Here it is in action:
arabicToRoman 1 // "I"
arabicToRoman 5 // "IIIII"
arabicToRoman 10 // "IIIIIIIIII"

This code actually meets the first and second requirements already, and for all inputs!
Of course, having 4000 tally marks is not very helpful, which is no doubt why the Romans
started abbreviating them.
This is where insight into the domain comes in. If we understand that the tally marks are
being abbreviated, we can emulate that in our code.
So let's convert all runs of five tally marks into a "V".

1724

Commentary on 'Roman Numerals Kata with Commentary'

let arabicToRoman arabic =

(String.replicate arabic "I")
.Replace("IIIII","V")
// test
arabicToRoman 1 // "I"
arabicToRoman 5 // "V"
arabicToRoman 6 // "VI"
arabicToRoman 10 // "VV"

But now we can have runs of "V"s. Two "V"s need to be collapsed into an "X".
let arabicToRoman arabic =
(String.replicate arabic "I")
.Replace("IIIII","V")
.Replace("VV","X")
// test
arabicToRoman 1 // "I"
arabicToRoman 5 // "V"
arabicToRoman 6 // "VI"
arabicToRoman 10 // "X"
arabicToRoman 12 // "XII"
arabicToRoman 16 // "XVI"

I think you get the idea. We can go on adding abbreviations...

let arabicToRoman arabic =
(String.replicate arabic "I")
.Replace("IIIII","V")
.Replace("VV","X")
.Replace("XXXXX","L")
.Replace("LL","C")
.Replace("CCCCC","D")
.Replace("DD","M")
// test
arabicToRoman 1 // "I"
arabicToRoman 5 // "V"
arabicToRoman 6 // "VI"
arabicToRoman 10 // "X"
arabicToRoman 12 // "XII"
arabicToRoman 16 // "XVI"
arabicToRoman 3497 // "MMMCCCCLXXXXVII"

And now we're done. We've met the first three requirements.

1725

Commentary on 'Roman Numerals Kata with Commentary'

If we want to add the optional abbreviations for the fours and nines, we can do that at the
end, after all the tally marks have been accumulated.
let arabicToRoman arabic =
(String.replicate arabic "I")
.Replace("IIIII","V")
.Replace("VV","X")
.Replace("XXXXX","L")
.Replace("LL","C")
.Replace("CCCCC","D")
.Replace("DD","M")
// optional substitutions
.Replace("IIII","IV")
.Replace("VIV","IX")
.Replace("XXXX","XL")
.Replace("LXL","XC")
.Replace("CCCC","CD")
.Replace("DCD","CM")

// test
arabicToRoman 1 // "I"
arabicToRoman 4 // "IV"
arabicToRoman 5 // "V"
arabicToRoman 6 // "VI"
arabicToRoman 10 // "X"
arabicToRoman 12 // "XII"
arabicToRoman 16 // "XVI"
arabicToRoman 40 // "XL"
arabicToRoman 946 // "CMXLVI"
arabicToRoman 3497 // "MMMCDXCVII"

Here is what I like about this approach:

It is derived from understanding the domain model (tally marks) rather than jumping
right into a recursive design.
As a result, the implementation follows the requirements very closely. In fact it basically
writes itself.
By following this step-by-step approach, someone else would have high confidence in
the code being correct just by examining the code. There is no recursion or special
tricks that would confuse anyone.
The implementation generates output for all inputs at all times. In the intermediate
stages, when it doesn't meet all the requirements, it at least generates output (e.g. 10
mapped to "VV") that tells us what we need to do next.

1726

Commentary on 'Roman Numerals Kata with Commentary'

Yes, this might not be the most efficient code, creating strings with 4000 "I"s in them! And of
course, a more efficient approach would subtract the large tallies ("M", then "D", then "C")
straight from the input, leading to the recursive solution demonstrated in the TDD video.
But on the other hand, this implementation might well be efficient enough. The requirements
don't say anything about performance constraints -- YAGNI anyone? -- so I'm tempted to
leave it at this.

A bi-quinary coded decimal implementation

I can't resist another implementation, just so that I can use the word "bi-quinary" again.
The implementation will again be based on our understanding of the domain, in this case,
the Roman abacus.
In the abacus, each row or wire represents a decimal place, just as our common Arabic
notation does. But the number in that place can be encoded by two different symbols,
depending on the number.
Some examples:
1 in the tens place is encoded by "X"
2 in the tens place is encoded by "XX"
5 in the tens place is encoded by "L"
6 in the tens place is encoded by "LX"
and so on.
This leads directly to an algorithm based on converting the beads on the abacus into a string
representation.
Split the input number into units, tens, hundreds and thousands. These represent each
row or wire on the abacus.
Encode the digit for each place into a string using the "bi-quinary" representation and
the appropriate symbols for that place.
Concat the representations for each place together to make single output string.
Here's an implementation that is a direct translation of that algorithm:

1727

Commentary on 'Roman Numerals Kata with Commentary'

let biQuinaryDigits place (unit,five) arabic =

let digit = arabic % (10*place) / place
match digit with
| 0 -> ""
| 1 -> unit
| 2 -> unit + unit
| 3 -> unit + unit + unit
| 4 -> unit + unit + unit + unit
| 5 -> five
| 6 -> five + unit
| 7 -> five + unit + unit
| 8 -> five + unit + unit + unit
| 9 -> five + unit + unit + unit + unit
| _ -> failwith "Expected 0-9 only"
let arabicToRoman arabic =
let units = biQuinaryDigits 1 ("I","V") arabic
let tens = biQuinaryDigits 10 ("X","L") arabic
let hundreds = biQuinaryDigits 100 ("C","D") arabic
let thousands = biQuinaryDigits 1000 ("M","?") arabic
thousands + hundreds + tens + units

Note that the above code does not produce the abbreviations for the four and nine cases.
We can easily modify it to do this though. We just need to pass in the symbol for ten, and
tweak the mapping for the 4 and 9 case, as follows:
let biQuinaryDigits place (unit,five,ten) arabic =
let digit = arabic % (10*place) / place
match digit with
| 0 -> ""
| 1 -> unit
| 2 -> unit + unit
| 3 -> unit + unit + unit
| 4 -> unit + five // changed to be one less than five
| 5 -> five
| 6 -> five + unit
| 7 -> five + unit + unit
| 8 -> five + unit + unit + unit
| 9 -> unit + ten // changed to be one less than ten
| _ -> failwith "Expected 0-9 only"
let arabicToRoman arabic =
let units = biQuinaryDigits 1 ("I","V","X") arabic
let tens = biQuinaryDigits 10 ("X","L","C") arabic
let hundreds = biQuinaryDigits 100 ("C","D","M") arabic
let thousands = biQuinaryDigits 1000 ("M","?","?") arabic
thousands + hundreds + tens + units

1728

Commentary on 'Roman Numerals Kata with Commentary'

Again, both these implementations are very straightforward and easy to verify. There are no
subtle edge cases lurking in the code.

Review
I started off this post being annoyed at a TDD demonstration. Let me review the reasons
why I was annoyed, and how my approach differs.
Requirements
The TDD demonstration video did not make any attempt to document the requirements at
all. I would say that this a dangerous thing to do, especially if you are learning.
I would prefer that before you start coding you always make an effort to be explicit about
what you are trying to do.
With only a tiny bit of effort I came up with some explicit requirements that I could use for
verification later.
I also explicitly documented the range of valid inputs -- something that was unfortunately
lacking in the TDD demonstration.
Understanding the domain
Even if the requirements have been made explicit for you, I think that it is always worthwhile
spending time to really understand the domain you are working in.
In this case, understanding that Roman numerals were a tally-based system helped with the
design later. (Plus I learned what "bi-quinary" means and got to use it in this post!)
Unit tests
The unit tests in the TDD demonstration were built one single case at a time. First zero, then
one, and so on.
As I note above, I feel very uncomfortable with this approach because (a) I don't think it
leads to a good design and (b) the single cases don't cover all possible inputs.
I would strongly recommend that you write tests that map directly to the requirements. If the
requirements are any good, this will mean that the tests cover many inputs at once, so you
can then test as many inputs as you can.
Ideally, you would use a property-based testing tool like QuickCheck. Not only does it make
this approach much easier to implement, but it forces you to identify what the properties of
your design should be, which in turn helps you clarify any fuzzy requirements.

1729

Commentary on 'Roman Numerals Kata with Commentary'

Implementation
Finally, I described two implementations, both completely different from the recursive one
demonstrated in the TDD video.
Both designs were derived directly from an understanding of the domain. The first from using
tally marks, and the second from using an abacus.
To my mind, both of these designs were also easier to understand -- no recursion! -- and
thus easier to have confidence in.

Summary
(Added based on comments I made below.)
Let me be clear that I have absolutely no problem with TDD. And I don't have a problem with
katas either.
But here's my concern about these kinds of "dive-in" demos, namely that novices and
learners might unintentionally learn the following (implicit) lessons:
It is OK to accept requirements as given without asking questions.
It is OK to work without a clear idea of the goal.
It is OK to start coding immediately.
It is OK to create tests that are extremely specific (e.g. with magic numbers).
It is OK to consider only the happy path.
It is OK to do micro refactoring without looking at the bigger picture.
Personally, I think that if you are practicing to be a professional developer, you should:
Practice asking for as much information as possible before you start coding.
Practice writing requirements (from unclear input) in such a way that they can be tested.
Practice thinking (analyzing and designing) rather than immediately coding.
Practice creating general tests rather than specific ones.
Practice thinking about and handling bad inputs, corner cases, and errors.
Practice major refactoring (rather than micro refactoring) so as to develop an intuition
about where the shearing layers should be.
These principles are all completely compatible with TDD (or at least the "London" school of
TDD) and programming katas. There is no conflict, and I cannot see why they would be
controversial.

What do you think?

1730

Commentary on 'Roman Numerals Kata with Commentary'

I'm sure many of you will disagree with this post. I'm up for a (civilized) debate. Please leave
comments below or on Reddit.
If you'd like to see the complete code for this post, it is available as a gist here. The gist also
includes full property-based tests for both implementations.

1731

Calculator Walkthrough: Part 1

One comment I hear often is a complaint about the gap between theory and practice in F#
and functional programming in general. In other words, you know the theory, but how do you
actually design and implement an application using FP principles?
So I thought it might be useful to show you how I personally would go about designing and
implementing some little applications from beginning to end.
These will be sort of annotated "live coding" sessions. I'll take a problem and start coding it,
taking you through my thought process at each stage. I will make mistakes too, so you'll see
how I deal with that, and do backtracking and refactoring.
Please be aware that I'm not claiming that this is production ready code. The code I'm going
to show you is more like a exploratory sketch, and as a result I will do certain bad things (like
not testing!) which I would not do for more critical code.
For this first post in the series, I'll be developing a simple pocket calculator app, like this:

My development approach
My approach to software development is eclectic and pragmatic -- I like to mix different
techniques and alternate between top-down and bottom-up approaches.

1732

Calculator Walkthrough: Part 1

Typically I start with the requirements -- I'm a fan of requirements-driven design! Ideally, I
would aim to become an expert in the domain as well.
Next, I work on modelling the domain, using domain-driven design with a focus on domain
events ("event storming"), not just static data ("aggregates" in DDD terminology).
As part of the modelling process, I sketch a design using type-first development to create
types that represent both the domain data types ("nouns") and the domain activities
("verbs").
After doing a first draft of the domain model, I typically switch to a "bottom up" approach and
code a small prototype that exercises the model that I have defined so far.
Doing some real coding at this point acts as a reality check. It ensures that the domain
model actually makes sense and is not too abstract. And of course, it often drives more
questions about the requirements and domain model, so I go back to step 1, do some
refining and refactoring, and rinse and repeat until I am happy.
(Now if I was working with a team on a large project, at this point we could also start building
a real system incrementally and start on the user interface (e.g. with paper prototypes). Both
of these activities will typically generate yet more questions and changes in requirements
too, so the whole process is cyclical at all levels.)
So this would be my approach in a perfect world. In practice, of course, the world is not
perfect. There is bad management to contend with, a lack of requirements, silly deadlines
and more, all of which mean that I rarely get to use an ideal process.
But in this example, I'm the boss, so if I don't like the result, I've only myself to blame!

Getting started
So, let's get started. What should we do first?
Normally I would start with requirements. But do I really need to spend a lot of time writing
up requirements for a calculator?
I'm going to be lazy and say no. Instead I'm just to dive in -- I'm confident that I know how a
calculator works. (As you'll see later, I was wrong! Trying to write up the requirements would
have been a good exercise, as there are some interesting edge cases.)
So let's start with the type-first design instead.
In my designs, every use-case is a function, with one input and one output.

1733

Calculator Walkthrough: Part 1

For this example then, we need to model the public interface to the Calculator as a function.
Here's the signature:
type Calculate = CalculatorInput -> CalculatorOutput

That was easy! The first question then is: are there any other use-cases that we need to
model? I think for now, no. We'll just start with a single case that handles all the inputs.

Defining the input and output to the function

But now we have created two new types, CalculatorInput and CalculatorOutput , that are
undefined (and if you type this into a F# script file, you'll have red squigglies to remind you).
We'd better define those now.
Before moving on, I should make it very clear that the input and output types for this function
are going to be pure and clean. When designing our domain we never want to be dealing
with the messy world of strings, primitive datatypes, validation, and so on.
Instead there will typically be a validation/transformation function that converts from the
messy untrusted world into our lovely, pristine domain on the way in, and another similar
function that does the reverse on the way out.

1734

Calculator Walkthrough: Part 1

Ok, let's work on the CalculatorInput first. What would the structure of the input look like?
First, obviously, there will be some keystrokes, or some other way of communicating the
intent of the user. But also, since the calculator is stateless, we need to pass in some state
as well. This state would contain, for example, the digits typed in so far.
As to the output, the function will have to emit a new, updated state, of course.
But do we need anything else, such as a structure containing formatted output for display? I
don't think we do. We want to isolate ourselves from the display logic, so we'll just let the UI
turn the state into something that can be displayed.
What about errors? In other posts, I have spent a lot of time talking about error handling. Is it
needed in this case?
In this case, I think not. In a cheap pocket calculator, any errors are shown right in the
display, so we'll stick with that approach for now.
So here's the new version of the function:
type Calculate = CalculatorInput * CalculatorState -> CalculatorState

1735

Calculator Walkthrough: Part 1

CalculatorInput now means the keystrokes or whatever, and CalculatorState is the state.

Notice that I have defined this function using a tuple ( CalculatorInput * CalculatorState ) as
input, rather than as two separate parameters (which would look like CalculatorInput ->
CalculatorState -> CalculatorState ). I did this because both parameters are always needed

and a tuple makes this clear -- I don't want to be partially applying the input, for example.
In fact I do this for all functions when doing type-first design. Every function has one input
and one output. This doesn't mean that there might not be potential for doing partial
application later, just that, at the design stage, I only want one parameter.
Also note that things that are not part of the pure domain (such as configuration and
connection strings) will never be shown at this stage, although, at implementation time, they
will of course be added to the functions that implement the design.

Defining the CalculatorState type

Now let's look at the CalculatorState . All I can think of that we need right now is something
to hold the information to display.
type Calculate = CalculatorInput * CalculatorState -> CalculatorState
and CalculatorState = {
display: CalculatorDisplay
}

I've defined a type CalculatorDisplay , firstly as documentation to make it clear what the
field value is used for, and secondly, so I can postpone deciding what the display actually is!
So what should the type of the display be? A float? A string? A list of characters? A record
with multiple fields?
Well, I'm going to go for string , because, as I said above, we might need to display errors.
type Calculate = CalculatorInput * CalculatorState -> CalculatorState
and CalculatorState = {
display: CalculatorDisplay
}
and CalculatorDisplay = string

Notice that I am using and to connect the type definitions together. Why?
Well, F# compiles from top to bottom, so you must define a type before it is used. The
following code will not compile:

1736

Calculator Walkthrough: Part 1

type Calculate = CalculatorInput * CalculatorState -> CalculatorState

type CalculatorState = {
display: CalculatorDisplay
}
type CalculatorDisplay = string

I could fix this by changing the order of the declarations, but since I am in "sketch" mode,
and I don't want to reorder things all the time, I will just append new declarations to the
bottom and use and to connect them.
In the final production code though, when the design has stabilized, I would reorder these
types to avoid using and . The reason is that and can hide cycles between types and
prevent refactoring.

Defining the CalculatorInput type

Some people might say: why not use a char as the input? But as I explained above, in my
domain I only want to deal with ideal data. By using a limited set of choices like this, I never
have to deal with unexpected input.
Also, a side benefit of using abstract types rather than chars is that DecimalSeparator is not
assumed to be ".". The actual separator should be obtained by first getting the current
culture ( System.Globalization.CultureInfo.CurrentCulture ) and then using
CurrentCulture.NumberFormat.CurrencyDecimalSeparator to get the separator. By hiding this

implementation detail from the design, changing the actual separator used will have minimal
effect on the code.

Refining the design: handling digits

So that's a first pass at the design done. Now let's dig deeper and define some of the
internal processes.

1737

Calculator Walkthrough: Part 1

Let's start with how the digits are handled.

When a digit key is pressed, we want to append the digit to the current display. Let's define a
function type that represents that:
type UpdateDisplayFromDigit = CalculatorDigit * CalculatorDisplay -> CalculatorDisplay

The CalculatorDisplay type is the one we defined earlier, but what is this new
CalculatorDigit type?

So the next question is, how do we get a value of this type? Do we need a function that
maps a CalculatorInput to a CalculatorDigit type, like this?
let convertInputToDigit (input:CalculatorInput) =
match input with
| Zero -> CalculatorDigit.Zero
| One -> CalculatorDigit.One
| etc
| Add -> ???
| Clear -> ???

While we're at it, let's classify the other buttons as well.

I would classify Add | Subtract | Multiply | Divide as math operations, and as for Equals
| Clear , I'll just call them "actions" for lack of better word.

1738

Calculator Walkthrough: Part 1

Here's the complete refactored design with new types CalculatorDigit , CalculatorMathOp
and CalculatorAction :
type Calculate = CalculatorInput * CalculatorState -> CalculatorState
and CalculatorState = {
display: CalculatorDisplay
}
and CalculatorDisplay = string
and CalculatorInput =
| Digit of CalculatorDigit
| Op of CalculatorMathOp
| Action of CalculatorAction
and CalculatorDigit =
| Zero | One | Two | Three | Four
| Five | Six | Seven | Eight | Nine
| DecimalSeparator
and CalculatorMathOp =
| Add | Subtract | Multiply | Divide
and CalculatorAction =
| Equals | Clear
type UpdateDisplayFromDigit = CalculatorDigit * CalculatorDisplay -> CalculatorDisplay

This is not the only approach. I could have easily left Equals and Clear as separate
choices.
Now let's revisit UpdateDisplayFromDigit again. Do we need any other parameters? For
example, do we need any other part of the state?
No, I can't think of anything else. When defining these functions, I want to be as minimal as
possible. Why pass in the whole calculator state if you only need the display?
Also, would UpdateDisplayFromDigit ever return an error? For example, surely we can't add
digits indefinitely -- what happens when we are not allowed to? And is there some other
combination of inputs that might cause an error? For example, inputting nothing but decimal
separators! What happens then?
For this little project, I will assume that neither of these will create an explicit error, but
instead, bad input will be rejected silently. In other words, after 10 digits, say, other digits will
be ignored. And after the first decimal separator, subsequent ones will be ignored as well.
Alas, I cannot encode these requirements in the design. But that fact that
UpdateDisplayFromDigit does not return any explicit error type does at least tell me that

errors will be handled silently.

Refining the design: the math operations

1739

Calculator Walkthrough: Part 1

Now let's move on to the math operations.

These are all binary operations, taking two numbers and spitting out a new result.
A function type to represent this would look like this:
type DoMathOperation = CalculatorMathOp * Number * Number -> Number

If there were unary operations as well, such as 1/x , we would need a different type for
those, but we don't, so we can keep things simple.
Next decision: what numeric type should we use? Should we make it generic?
Again, let's just keep it simple and use float . But we'll keep the Number alias around to
decouple the representation a bit. Here's the updated code:
type DoMathOperation = CalculatorMathOp * Number * Number -> Number
and Number = float

Now let's ponder DoMathOperation , just as we did for UpdateDisplayFromDigit above.

Question 1: Is this the minimal set of parameters? For example, do we need any other part
of the state?
Answer: No, I can't think of anything else.
Question 2: Can DoMathOperation ever return an error?
Answer: Yes! What about dividing by zero?
So how should we handle errors? Let's create a new type that represents a result of a math
operation, and make that the output of DoMathOperation :
The new type, MathOperationResult will have two choices (discriminated union) between
Success and Failure .

type DoMathOperation = CalculatorMathOp * Number * Number -> MathOperationResult

and Number = float
and MathOperationResult =
| Success of Number
| Failure of MathOperationError
and MathOperationError =
| DivideByZero

1740

Calculator Walkthrough: Part 1

We could have also used the built-in generic Choice type, or even a full "railway oriented
programming" approach, but since this is a sketch of the design, I want the design to stand
alone, without a lot of dependencies, so I'll just define the specific type right here.
Any other errors? NaNs or underflows or overflows? I'm not sure. We have the
MathOperationError type, and it would be easy to extend it as needed.

Where do numbers come from?

We've defined DoMathOperation to use Number values as input. But where does a Number
come from?
Well they come from the sequence of digits that have been entered -- converting the digits
into a float.
One approach would be to store a Number in the state along with the string display, and
update it as each digit comes in.
I'm going to take a simpler approach, and just get the number from the display directly. In
other words, we need a function that looks like this:
type GetDisplayNumber = CalculatorDisplay -> Number

Thinking about it though, the function could fail, because the display string could be "error"
or something. So let's return an option instead.
type GetDisplayNumber = CalculatorDisplay -> Number option

Similarly, when we do have a successful result, we will want to display it, so we need a
function that works in the other direction:
type SetDisplayNumber = Number -> CalculatorDisplay

This function can never error (I hope), so we don't need the option .

Refining the design: handling a math operation

input
We're not done with math operations yet, though!

1741

Calculator Walkthrough: Part 1

What is the visible effect when the input is Add ? None!

The Add event needs another number to be entered later, so the Add event is somehow
kept pending, waiting for the next number.
If you think about, we not only have to keep the Add event pending, but also the previous
number, ready to be added to the latest number that is input.
Where will we keep track of this? In the CalculatorState of course!
Here's our first attempt to add the new fields:
and CalculatorState = {
display: CalculatorDisplay
pendingOp: CalculatorMathOp
pendingNumber: Number
}

But sometimes there isn't a pending operation, so we have to make it optional:

and CalculatorState = {
display: CalculatorDisplay
pendingOp: CalculatorMathOp option
pendingNumber: Number option
}

But this is wrong too! Can we have a pendingOp without a pendingNumber , or vice versa?
No. They live and die together.
This implies that the state should contain a pair, and the whole pair is optional, like this:
and CalculatorState = {
display: CalculatorDisplay
pendingOp: (CalculatorMathOp * Number) option
}

But now we are still missing a piece. If the operation is added to the state as pending, when
does the operation actually get run and the result displayed?
Answer: when the Equals button is pushed, or indeed any another math op button. We'll
deal with that later.

Refining the design: handling the Clear button

We've got one more button to handle, the Clear button. What does it do?
1742

Calculator Walkthrough: Part 1

Well, it obviously just resets the state so that the display is empty and any pending
operations are removed.
I'm going to call this function InitState rather than "clear", and here is its signature:
type InitState = unit -> CalculatorState

Defining the services

At this point, we have everything we need to switch to bottom up development. I'm eager to
try building a trial implementation of the Calculate function, to see if the design is usable,
and if we've missed anything.
But how can I create a trial implementation without implementing the whole thing?
This is where all these types come in handy. We can define a set of "services" that the
calculate function will use, but without actually implementing them!

Here's what I mean:

type CalculatorServices = {
updateDisplayFromDigit: UpdateDisplayFromDigit
doMathOperation: DoMathOperation
getDisplayNumber: GetDisplayNumber
setDisplayNumber: SetDisplayNumber
initState: InitState
}

We've created a set of services that can be injected into an implementation of the
Calculate function. With these in place, we can code the Calculate function immediately

and deal with the implementation of the services later.

At this point, you might be thinking that this seems like overkill for a tiny project.
It's true -- we don't want this to turn into FizzBuzz Enterprise Edition!
But I'm demonstrating a principle here. By separating the "services" from the core code, you
can start prototyping immediately. The goal is not to make a production ready codebase, but
to find any issues in the design. We are still in the requirements discovery phase.
This approach should not be unfamiliar to you -- it is directly equivalent to the OO principle of
creating a bunch of interfaces for services and then injecting them into the core domain.

1743

Calculator Walkthrough: Part 1

Review
So let's review -- with the addition of the services, our initial design is complete. Here is all
the code so far:
type Calculate = CalculatorInput * CalculatorState -> CalculatorState
and CalculatorState = {
display: CalculatorDisplay
pendingOp: (CalculatorMathOp * Number) option
}
and CalculatorDisplay = string
and CalculatorInput =
| Digit of CalculatorDigit
| Op of CalculatorMathOp
| Action of CalculatorAction
and CalculatorDigit =
| Zero | One | Two | Three | Four
| Five | Six | Seven | Eight | Nine
| DecimalSeparator
and CalculatorMathOp =
| Add | Subtract | Multiply | Divide
and CalculatorAction =
| Equals | Clear
and UpdateDisplayFromDigit =
CalculatorDigit * CalculatorDisplay -> CalculatorDisplay
and DoMathOperation =
CalculatorMathOp * Number * Number -> MathOperationResult
and Number = float
and MathOperationResult =
| Success of Number
| Failure of MathOperationError
and MathOperationError =
| DivideByZero
type GetDisplayNumber =
CalculatorDisplay -> Number option
type SetDisplayNumber =
Number -> CalculatorDisplay
type InitState =
unit -> CalculatorState
type CalculatorServices = {
updateDisplayFromDigit: UpdateDisplayFromDigit
doMathOperation: DoMathOperation
getDisplayNumber: GetDisplayNumber
setDisplayNumber: SetDisplayNumber
initState: InitState
}

1744

Calculator Walkthrough: Part 1

Summary
I think that this is quite nice. We haven't written any "real" code yet, but with a bit of thought,
we have already built quite a detailed design.
In the next post, I'll put this design to the test by attempting to create an implementation.
The code for this post is available in this gist on GitHub.

1745

Calculator Walkthrough: Part 2

In this post, I'll continue developing a simple pocket calculator app, like this:

In the previous post, we completed a first draft of the design, using only types (no UML
diagrams!).
Now it's time to create a trial implementation that uses the design.
Doing some real coding at this point acts as a reality check. It ensures that the domain
model actually makes sense and is not too abstract. And of course, it often drives more
questions about the requirements and domain model.

First implementation
So let's try implementing the main calculator function, and see how we do.
First, we can immediately create a skeleton that matches each kind of input and processes it
accordingly.

1746

Calculator Walkthrough: Part 2

let createCalculate (services:CalculatorServices) :Calculate =

fun (input,state) ->
match input with
| Digit d ->
let newState = // do something
newState //return
| Op op ->
let newState = // do something
newState //return
| Action Clear ->
let newState = // do something
newState //return
| Action Equals ->
let newState = // do something
newState //return

You can see that this skeleton has a case for each type of input to handle it appropriately.
Note that in all cases, a new state is returned.
This style of writing a function might look strange though. Let's look at it a bit more closely.
First, we can see that createCalculate is the not the calculator function itself, but a function
that returns another function. The returned function is a value of type Calculate -- that's
what the :Calculate at the end means.
Here's just the top part:
let createCalculate (services:CalculatorServices) :Calculate =
fun (input,state) ->
match input with
// code

Since it is returning a function, I chose to write it using a lambda. That's what the fun
(input,state) -> is for.

But I could have also written it using an inner function, like this
let createCalculate (services:CalculatorServices) :Calculate =
let innerCalculate (input,state) =
match input with
// code
innerCalculate // return the inner function

Both approaches are basically the same* -- take your pick!

* Although there might be some performance differences.

1747

Calculator Walkthrough: Part 2

Dependency injection of services

But createCalculate doesn't just return a function, it also has a services parameter. This
parameter is used for doing the "dependency injection" of the services.
That is, the services are only used in createCalculate itself, and are not visible in the
function of type Calculate that is returned.
The "main" or "bootstrapper" code that assembles all the components for the application
would look something like this:
// create the services
let services = CalculatorServices.createServices()
// inject the services into the "factory" method
let calculate = CalculatorImplementation.createCalculate services
// the returned "calculate" function is of type Calculate
// and can be passed into the UI, for example
// create the UI and run it
let form = new CalculatorUI.CalculatorForm(calculate)
form.Show()

Implementation: handling digits

Now let's start implementing the various parts of the calculation function. We'll start with the
digits handling logic.
To keep the main function clean, let's pass the reponsibility for all the work to a helper
function updateDisplayFromDigit , like this:
let createCalculate (services:CalculatorServices) :Calculate =
fun (input,state) ->
match input with
| Digit d ->
let newState = updateDisplayFromDigit services d state
newState //return

Note that I'm creating a newState value from the result of updateDisplayFromDigit and then
returning it as a separate step.
I could have done the same thing in one step, without an explicit newState value, as shown
below:

1748

Calculator Walkthrough: Part 2

let createCalculate (services:CalculatorServices) :Calculate =

fun (input,state) ->
match input with
| Digit d ->
updateDisplayFromDigit services d state

Neither approach is automatically best. I would pick one or the other depending on the
context.
For simple cases, I would avoid the extra line as being unnecessary, but sometimes having
an explicit return value is more readable. The name of the value tells you an indication of the
return type, and it gives you something to watch in the debugger, if you need to.
Alright, let's implement updateDisplayFromDigit now. It's pretty straightforward.
first use the updateDisplayFromDigit in the services to actually update the display
then create a new state from the new display and return it.
let updateDisplayFromDigit services digit state =
let newDisplay = services.updateDisplayFromDigit (digit,state.display)
let newState = {state with display=newDisplay}
newState //return

Implementation: handling Clear and Equals

Before we move onto the implementation of the math operations, lets look at handling
Clear and Equals , as they are simpler.

For Clear , just init the state, using the provided initState service.
For Equals , we check if there is a pending math op. If there is, run it and update the
display, otherwise do nothing. We'll put that logic in a helper function called
updateDisplayFromPendingOp .

So here's what createCalculate looks like now:

1749

Calculator Walkthrough: Part 2

let createCalculate (services:CalculatorServices) :Calculate =

fun (input,state) ->
match input with
| Digit d -> // as above
| Op op -> // to do
| Action Clear ->
let newState = services.initState()
newState //return
| Action Equals ->
let newState = updateDisplayFromPendingOp services state
newState //return

Now to updateDisplayFromPendingOp . I spent a few minutes thinking about, and I've come up
with the following algorithm for updating the display:
First, check if there is any pending op. If not, then do nothing.
Next, try to get the current number from the display. If you can't, then do nothing.
Next, run the op with the pending number and the current number from the display. If
you get an error, then do nothing.
Finally, update the display with the result and return a new state.
The new state also has the pending op set to None , as it has been processed.
And here's what that logic looks like in imperative style code:
// First version of updateDisplayFromPendingOp
// * very imperative and ugly
let updateDisplayFromPendingOp services state =
if state.pendingOp.IsSome then
let op,pendingNumber = state.pendingOp.Value
let currentNumberOpt = services.getDisplayNumber state.display
if currentNumberOpt.IsSome then
let currentNumber = currentNumberOpt.Value
let result = services.doMathOperation (op,pendingNumber,currentNumber)
match result with
| Success resultNumber ->
let newDisplay = services.setDisplayNumber resultNumber
let newState = {display=newDisplay; pendingOp=None}
newState //return
| Failure error ->
state // original state is untouched
else
state // original state is untouched
else
state // original state is untouched

Ewww! Don't try that at home!

1750

Calculator Walkthrough: Part 2

That code does follow the algorithm exactly, but is really ugly and also error prone (using
.Value on an option is a code smell).

On the plus side, we did make extensive use of our "services", which has isolated us from
the actual implementation details.
So, how can we rewrite it to be more functional?

Bumping into bind

The trick is to recognize that the pattern "if something exists, then act on that value" is
exactly the bind pattern discussed here and here.
In order to use the bind pattern effectively, it's a good idea to break the code into many small
chunks.
First, the code if state.pendingOp.IsSome then do something can be replaced by
Option.bind .

let updateDisplayFromPendingOp services state =

let result =
state.pendingOp
|> Option.bind ???

But remember that the function has to return a state. If the overall result of the bind is None ,
then we have not created a new state, and we must return the original state that was passed
in.
This can be done with the built-in defaultArg function which, when applied to an option,
returns the option's value if present, or the second parameter if None .
let updateDisplayFromPendingOp services state =
let result =
state.pendingOp
|> Option.bind ???
defaultArg result state

You can also tidy this up a bit as well by piping the result directly into defaultArg , like this:
let updateDisplayFromPendingOp services state =
state.pendingOp
|> Option.bind ???
|> defaultArg <| state

1751

Calculator Walkthrough: Part 2

I admit that the reverse pipe for state looks strange -- it's definitely an acquired taste!
Onwards! Now what about the parameter to bind ? When this is called, we know that
pendingOp is present, so we can write a lambda with those parameters, like this:
let result =
state.pendingOp
|> Option.bind (fun (op,pendingNumber) ->
let currentNumberOpt = services.getDisplayNumber state.display
// code
)

Alternatively, we could create a local helper function instead, and connect it to the bind, like
this:
let executeOp (op,pendingNumber) =
let currentNumberOpt = services.getDisplayNumber state.display
/// etc
let result =
state.pendingOp
|> Option.bind executeOp

I myself generally prefer the second approach when the logic is complicated, as it allows a
chain of binds to be simple. That is, I try to make my code look like:
let doSomething input = return an output option
let doSomethingElse input = return an output option
let doAThirdThing input = return an output option
state.pendingOp
|> Option.bind doSomething
|> Option.bind doSomethingElse
|> Option.bind doAThirdThing

Note that in this approach, each helper function has a non-option for input but always must
output an option.

Using bind in practice

Once we have the pending op, the next step is to get the current number from the display so
we can do the addition (or whatever).

1752

Calculator Walkthrough: Part 2

Rather than having a lot of logic, I'm going keep the helper function ( getCurrentNumber )
simple.
The input is the pair (op,pendingNumber)
The output is the triple (op,pendingNumber,currentNumber) if currentNumber is Some ,
otherwise None .
In other words, the signature of getCurrentNumber will be pair -> triple option , so we can
be sure that is usable with the Option.bind function.
How to convert the pair into the triple? This can be done just by using Option.map to convert
the currentNumber option to a triple option. If the currentNumber is Some , then the output of
the map is Some triple . On the other hand, if the currentNumber is None , then the output
of the map is None also.
let getCurrentNumber (op,pendingNumber) =
let currentNumberOpt = services.getDisplayNumber state.display
currentNumberOpt
|> Option.map (fun currentNumber -> (op,pendingNumber,currentNumber))
let result =
state.pendingOp
|> Option.bind getCurrentNumber
|> Option.bind ???

We can rewrite getCurrentNumber to be a bit more idiomatic by using pipes:

let getCurrentNumber (op,pendingNumber) =
state.display
|> services.getDisplayNumber
|> Option.map (fun currentNumber -> (op,pendingNumber,currentNumber))

Now that we have a triple with valid values, we have everything we need to write a helper
function for the math operation.
It takes a triple as input (the output of getCurrentNumber )
It does the math operation
It then pattern matches the Success/Failure result and outputs the new state if
applicable.

1753

Calculator Walkthrough: Part 2

let doMathOp (op,pendingNumber,currentNumber) =

let result = services.doMathOperation (op,pendingNumber,currentNumber)
match result with
| Success resultNumber ->
let newDisplay = services.setDisplayNumber resultNumber
let newState = {display=newDisplay; pendingOp=None}
Some newState //return something
| Failure error ->
None // failed

Note that, unlike the earlier version with nested ifs, this version returns Some on success
and None on failure.

Displaying errors
Writing the code for the Failure case made me realize something. If there is a failure, we
are not displaying it at all, just leaving the display alone. Shouldn't we show an error or
something?
Hey, we just found a requirement that got overlooked! This is why I like to create an
implementation of the design as soon as possible. Writing real code that deals with all the
cases will invariably trigger a few "what happens in this case?" moments.
So how are we going to implement this new requirement?
In order to do this, we'll need a new "service" that accepts a MathOperationError and
generates a CalculatorDisplay .
type SetDisplayError = MathOperationError -> CalculatorDisplay

and we'll need to add it to the CalculatorServices structure too:

type CalculatorServices = {
// as before
setDisplayNumber: SetDisplayNumber
setDisplayError: SetDisplayError
initState: InitState
}

doMathOp can now be altered to use the new service. Both Success and Failure cases

now result in a new display, which in turn is wrapped in a new state.

1754

Calculator Walkthrough: Part 2

let doMathOp (op,pendingNumber,currentNumber) =

let result = services.doMathOperation (op,pendingNumber,currentNumber)
let newDisplay =
match result with
| Success resultNumber ->
services.setDisplayNumber resultNumber
| Failure error ->
services.setDisplayError error
let newState = {display=newDisplay;pendingOp=None}
Some newState //return something

I'm going to leave the Some in the result, so we can stay with Option.bind in the result
pipeline*.
* An alternative would be to not return Some , and then use Option.map in the result pipeline
Putting it all together, we have the final version of updateDisplayFromPendingOp . Note that
I've also added a ifNone helper that makes defaultArg better for piping.

1755

Calculator Walkthrough: Part 2

// helper to make defaultArg better for piping

let ifNone defaultValue input =
// just reverse the parameters!
defaultArg input defaultValue
// Third version of updateDisplayFromPendingOp
// * Updated to show errors on display in Failure case
// * replaces awkward defaultArg syntax
let updateDisplayFromPendingOp services state =
// helper to extract CurrentNumber
let getCurrentNumber (op,pendingNumber) =
state.display
|> services.getDisplayNumber
|> Option.map (fun currentNumber -> (op,pendingNumber,currentNumber))
// helper to do the math op
let doMathOp (op,pendingNumber,currentNumber) =
let result = services.doMathOperation (op,pendingNumber,currentNumber)
let newDisplay =
match result with
| Success resultNumber ->
services.setDisplayNumber resultNumber
| Failure error ->
services.setDisplayError error
let newState = {display=newDisplay;pendingOp=None}
Some newState //return something
// connect all the helpers
state.pendingOp
|> Option.bind getCurrentNumber
|> Option.bind doMathOp
|> ifNone state // return original state if anything fails

Using a "maybe" computation expression

instead of bind
So far, we've being using "bind" directly. That has helped by removing the cascading
if/else .

But F# allows you to hide the complexity in a different way, by creating computation
expressions.
Since we are dealing with Options, we can create a "maybe" computation expression that
allows clean handling of options. (If we were dealing with other types, we would need to
create a different computation expression for each type).
Here's the definition -- only four lines!

1756

Calculator Walkthrough: Part 2

type MaybeBuilder() =
member this.Bind(x, f) = Option.bind f x
member this.Return(x) = Some x
let maybe = new MaybeBuilder()

With this computation expression available, we can use maybe instead of bind, and our
code would look something like this:
let doSomething input = return an output option
let doSomethingElse input = return an output option
let doAThirdThing input = return an output option
let finalResult = maybe {
let! result1 = doSomething
let! result2 = doSomethingElse result1
let! result3 = doAThirdThing result2
return result3
}

In our case, then we can write yet another version of updateDisplayFromPendingOp -- our
fourth!
// Fourth version of updateDisplayFromPendingOp
// * Changed to use "maybe" computation expression
let updateDisplayFromPendingOp services state =
// helper to do the math op
let doMathOp (op,pendingNumber,currentNumber) =
let result = services.doMathOperation (op,pendingNumber,currentNumber)
let newDisplay =
match result with
| Success resultNumber ->
services.setDisplayNumber resultNumber
| Failure error ->
services.setDisplayError error
{display=newDisplay;pendingOp=None}
// fetch the two options and combine them
let newState = maybe {
let! (op,pendingNumber) = state.pendingOp
let! currentNumber = services.getDisplayNumber state.display
return doMathOp (op,pendingNumber,currentNumber)
}
newState |> ifNone state

1757

Calculator Walkthrough: Part 2

Note that in this implementation, I don't need the getCurrentNumber helper any more, as I
can just call services.getDisplayNumber directly.
So, which of these variants do I prefer?
It depends.
If there is a very strong "pipeline" feel, as in the ROP approach, then I prefer using an
explicit bind .
On the other hand, if I am pulling options from many different places, and I want to
combine them in various ways, the maybe computation expression makes it easier.
So, in this case, I'll go for the last implementation, using maybe .

Implementation: handling math operations

Now we are ready to do the implementation of the math operation case.
First, if there is a pending operation, the result will be shown on the display, just as for the
Equals case. But in addition, we need to push the new pending operation onto the state as

well.
For the math operation case, then, there will be two state transformations, and
createCalculate will look like this:

let createCalculate (services:CalculatorServices) :Calculate =

fun (input,state) ->
match input with
| Digit d -> // as above
| Op op ->
let newState1 = updateDisplayFromPendingOp services state
let newState2 = addPendingMathOp services op newState1
newState2 //return

We've already defined updateDisplayFromPendingOp above. So we just need

addPendingMathOp as a helper function to push the operation onto the state.

The algorithm for addPendingMathOp is:

Try to get the current number from the display. If you can't, then do nothing.
Update the state with the op and current number.
Here's the ugly version:

1758

Calculator Walkthrough: Part 2

// First version of addPendingMathOp

// * very imperative and ugly
let addPendingMathOp services op state =
let currentNumberOpt = services.getDisplayNumber state.display
if currentNumberOpt.IsSome then
let currentNumber = currentNumberOpt.Value
let pendingOp = Some (op,currentNumber)
let newState = {state with pendingOp=pendingOp}
newState //return
else
state // original state is untouched

Again, we can make this more functional using exactly the same techniques we used for
updateDisplayFromPendingOp .

So here's the more idiomatic version using Option.map and a newStateWithPending helper
function:
// Second version of addPendingMathOp
// * Uses "map" and helper function
let addPendingMathOp services op state =
let newStateWithPending currentNumber =
let pendingOp = Some (op,currentNumber)
{state with pendingOp=pendingOp}
state.display
|> services.getDisplayNumber
|> Option.map newStateWithPending
|> ifNone state

And here's one using maybe :

// Third version of addPendingMathOp
// * Uses "maybe"
let addPendingMathOp services op state =
maybe {
let! currentNumber =
state.display |> services.getDisplayNumber
let pendingOp = Some (op,currentNumber)
return {state with pendingOp=pendingOp}
}
|> ifNone state // return original state if anything fails

As before, I'd probably go for the last implementation using maybe . But the Option.map one
is fine too.

1759

Calculator Walkthrough: Part 2

Implementation: review
Now we're done with the implementation part. Let's review the code:

1760

Calculator Walkthrough: Part 2

let updateDisplayFromDigit services digit state =

let newDisplay = services.updateDisplayFromDigit (digit,state.display)
let newState = {state with display=newDisplay}
newState //return
let updateDisplayFromPendingOp services state =
// helper to do the math op
let doMathOp (op,pendingNumber,currentNumber) =
let result = services.doMathOperation (op,pendingNumber,currentNumber)
let newDisplay =
match result with
| Success resultNumber ->
services.setDisplayNumber resultNumber
| Failure error ->
services.setDisplayError error
{display=newDisplay;pendingOp=None}
// fetch the two options and combine them
let newState = maybe {
let! (op,pendingNumber) = state.pendingOp
let! currentNumber = services.getDisplayNumber state.display
return doMathOp (op,pendingNumber,currentNumber)
}
newState |> ifNone state
let addPendingMathOp services op state =
maybe {
let! currentNumber =
state.display |> services.getDisplayNumber
let pendingOp = Some (op,currentNumber)
return {state with pendingOp=pendingOp}
}
|> ifNone state // return original state if anything fails
let createCalculate (services:CalculatorServices) :Calculate =
fun (input,state) ->
match input with
| Digit d ->
let newState = updateDisplayFromDigit services d state
newState //return
| Op op ->
let newState1 = updateDisplayFromPendingOp services state
let newState2 = addPendingMathOp services op newState1
newState2 //return
| Action Clear ->
let newState = services.initState()
newState //return
| Action Equals ->
let newState = updateDisplayFromPendingOp services state
newState //return

1761

Calculator Walkthrough: Part 2

Not bad -- the whole implementation is less than 60 lines of code.

Summary
We have proved that our design is reasonable by making an implementation -- plus we
found a missed requirement.
In the next post, we'll implement the services and the user interface to create a complete
application.
The code for this post is available in this gist on GitHub.

1762

Calculator Walkthrough: Part 3

In this post, I'll continue developing a simple pocket calculator app.
In the first post, we completed a first draft of the design, using only types (no UML
diagrams!). and in the previous post, we created an initial implementation that exercised the
design and revealed a missing requirement.
Now it's time to build the remaining components and assemble them into a complete
application

Creating the services

We have a implementation. But the implementation depends on some services, and we
haven't created the services yet.
In practice though, this bit is very easy and straightforward. The types defined in the domain
enforce constraints such there is really only one way of writing the code.
I'm going to show all the code at once (below), and I'll add some comments afterwards.
// ================================================
// Implementation of CalculatorConfiguration
// ================================================
module CalculatorConfiguration =
// A record to store configuration options
// (e.g. loaded from a file or environment)
type Configuration = {
decimalSeparator : string
divideByZeroMsg : string
maxDisplayLength: int
}
let loadConfig() = {
decimalSeparator =
System.Globalization.CultureInfo.CurrentCulture.NumberFormat.CurrencyDecim
alSeparator
divideByZeroMsg = "ERR-DIV0"
maxDisplayLength = 10
}
// ================================================
// Implementation of CalculatorServices
// ================================================

1763

Calculator Walkthrough: Part 3

module CalculatorServices =
open CalculatorDomain
open CalculatorConfiguration
let updateDisplayFromDigit (config:Configuration) :UpdateDisplayFromDigit =
fun (digit, display) ->
// determine what character should be appended to the display
let appendCh=
match digit with
| Zero ->
// only allow one 0 at start of display
if display="0" then "" else "0"
| One -> "1"
| Two -> "2"
| Three-> "3"
| Four -> "4"
| Five -> "5"
| Six-> "6"
| Seven-> "7"
| Eight-> "8"
| Nine-> "9"
| DecimalSeparator ->
if display="" then
// handle empty display with special case
"0" + config.decimalSeparator
else if display.Contains(config.decimalSeparator) then
// don't allow two decimal separators
""
else
config.decimalSeparator
// ignore new input if there are too many digits
if (display.Length > config.maxDisplayLength) then
display // ignore new input
else
// append the new char
display + appendCh
let getDisplayNumber :GetDisplayNumber = fun display ->
match System.Double.TryParse display with
| true, d -> Some d
| false, _ -> None
let setDisplayNumber :SetDisplayNumber = fun f ->
sprintf "%g" f
let setDisplayError divideByZeroMsg :SetDisplayError = fun f ->
match f with
| DivideByZero -> divideByZeroMsg
let doMathOperation :DoMathOperation = fun (op,f1,f2) ->
match op with

1764

Calculator Walkthrough: Part 3

| Add -> Success (f1 + f2)

| Subtract -> Success (f1 - f2)
| Multiply -> Success (f1 * f2)
| Divide ->
try
Success (f1 / f2)
with
| :? System.DivideByZeroException ->
Failure DivideByZero
let initState :InitState = fun () ->
{
display=""
pendingOp = None
}
let createServices (config:Configuration) = {
updateDisplayFromDigit = updateDisplayFromDigit config
doMathOperation = doMathOperation
getDisplayNumber = getDisplayNumber
setDisplayNumber = setDisplayNumber
setDisplayError = setDisplayError (config.divideByZeroMsg)
initState = initState
}

Some comments:
I have created a configuration record that stores properties that are used to
parameterize the services, such as the decimal separator.
The configuration record is passed into the createServices function, which in turn
passes the configuration on those services that need it.
All the functions use the same approach of returning one of the types defined in the
design, such as UpdateDisplayFromDigit or DoMathOperation .
There are only a few tricky edge cases, such as trapping exceptions in division, or
preventing more than one decimal separator being appended.

Creating the user interface

For the user interface, I'm going to use WinForms rather than WPF or a web-based
approach. It's simple and should work on Mono/Xamarin as well as Windows. And it should
be easy to port to other UI frameworks as well.
As is typical with UI development I spent more time on this than on any other part of the
process! I'm going to spare you all the painful iterations and just go directly to the final
version.

1765

Calculator Walkthrough: Part 3

I won't show all the code, as it is about 200 lines (and you can see it in the gist), but here are
some highlights:
module CalculatorUI =
open CalculatorDomain
type CalculatorForm(initState:InitState, calculate:Calculate) as this =
inherit Form()
// initialization before constructor
let mutable state = initState()
let mutable setDisplayedText =
fun text -> () // do nothing

The CalculatorForm is a subclass of Form , as usual.

There are two parameters for its constructor. One is initState , the function that creates an
empty state, and calculate , the function that transforms the state based on the input. In
other words, I'm using standard constructor based dependency injection here.
There are two mutable fields (shock horror!).
One is the state itself. Obviously, it will be modified after each button is pressed.
The second is a function called setDisplayedText . What's that all about?
Well, after the state has changed, we need to refresh the control (a Label) that displays the
text.
The standard way to do it is to make the label control a field in the form, like this:
type CalculatorForm(initState:InitState, calculate:Calculate) as this =
inherit Form()
let displayControl :Label = null

and then set it to an actual control value when the form has been initialized:
member this.CreateDisplayLabel() =
let display = new Label(Text="",Size=displaySize,Location=getPos(0,0))
display.TextAlign <- ContentAlignment.MiddleRight
display.BackColor <- Color.White
this.Controls.Add(display)
// traditional style - set the field when the form has been initialized
displayControl <- display

1766

Calculator Walkthrough: Part 3

But this has the problem that you might accidentally try to access the label control before it is
initialized, causing a NRE. Also, I'd prefer to focus on the desired behavior, rather than
having a "global" field that can be accessed by anyone anywhere.
By using a function, we (a) encapsulate the access to the real control and (b) avoid any
possibility of a null reference.
The mutable function starts off with a safe default implementation ( fun text -> () ), and is
then changed to a new implementation when the label control is created:
member this.CreateDisplayLabel() =
let display = new Label(Text="",Size=displaySize,Location=getPos(0,0))
this.Controls.Add(display)
// update the function that sets the text
setDisplayedText < (fun text -> display.Text <- text)

Creating the buttons

The buttons are laid out in a grid, and so I create a helper function getPos(row,col) that
gets the physical position from a logical (row,col) on the grid.
Here's an example of creating the buttons:
member this.CreateButtons() =
let sevenButton = new Button(Text="7",Size=buttonSize,Location=getPos(1,0),BackCol
or=DigitButtonColor)
sevenButton |> addDigitButton Seven
let eightButton = new Button(Text="8",Size=buttonSize,Location=getPos(1,1),BackCol
or=DigitButtonColor)
eightButton |> addDigitButton Eight
let nineButton = new Button(Text="9",Size=buttonSize,Location=getPos(1,2),BackColo
r=DigitButtonColor)
nineButton |> addDigitButton Nine
let clearButton = new Button(Text="C",Size=buttonSize,Location=getPos(1,3),BackCol
or=DangerButtonColor)
clearButton |> addActionButton Clear
let addButton = new Button(Text="+",Size=doubleHeightSize,Location=getPos(1,4),Bac
kColor=OpButtonColor)
addButton |> addOpButton Add

1767

Calculator Walkthrough: Part 3

And since all the digit buttons have the same behavior, as do all the math op buttons, I just
created some helpers that set the event handler in a generic way:
let addDigitButton digit (button:Button) =
button.Click.AddHandler(EventHandler(fun _ _ -> handleDigit digit))
this.Controls.Add(button)
let addOpButton op (button:Button) =
button.Click.AddHandler(EventHandler(fun _ _ -> handleOp op))
this.Controls.Add(button)

I also added some keyboard support:

Button clicks and keyboard presses are eventually routed into the key function handleInput ,
which does the calculation.
let handleInput input =
let newState = calculate(input,state)
state <- newState
setDisplayedText state.display
let handleDigit digit =
Digit digit |> handleInput
let handleOp op =
Op op |> handleInput

As you can see, the implementation of handleInput is trivial. It calls the calculation function
that was injected, sets the mutable state to the result, and then updates the display.
So there you have it -- a complete calculator!
Let's try it now -- get the code from this gist and try running it as a F# script.

Disaster strikes!

1768

Calculator Walkthrough: Part 3

Let's start with a simple test. Try entering 1 Add 2 Equals . What would you expect?
I don't know about you, but what I wouldn't expect is that the calculator display shows 12 !
What's going on? Some quick experimenting shows that I have forgotten something really
important -- when an Add or Equals operation happens, any subsequent digits should not
be added to the current buffer, but instead start a new one. Oh no! We've got a showstopper
bug!
Remind me again, what idiot said "if it compiles, it probably works".*
* Actually, that idiot would be me (among many others).
So what went wrong then?
Well the code did compile, but it didn't work as expected, not because the code was buggy,
but because my design was flawed.
In other words, the use of the types from the type-first design process means that I do have
high confidence that the code I wrote is a correct implementation of the design. But if the
requirements and design are wrong, all the correct code in the world can't fix that.
We'll revisit the requirements in the next post, but meanwhile, is there a patch we can make
that will fix the problem?

Fixing the bug

Let's think of the circumstances when we start a new set of digits, vs. when we just append
to the existing ones. As we noted above, a math operation or Equals will force the reset.
So why not set a flag when those operations happen? If the flag is set, then start a new
display buffer, and after that, unset the flag so that characters are appended as before.
What changes do we need to make to the code?
First, we need to store the flag somewhere. We'll store it in the CalculatorState of course!
type CalculatorState = {
display: CalculatorDisplay
pendingOp: (CalculatorMathOp * Number) option
allowAppend: bool
}

(This might seem like a good solution for now, but using flags like this is really a design
smell. In the next post, I'll use a different approach which doesn't involve flags)

1769

Calculator Walkthrough: Part 3

Fixing the implementation

With this change made, compiling the CalculatorImplementation code now breaks
everywhere a new state is created.
Actually, that's what I like about using F# -- something like adding a new field to a record is a
breaking change, rather than something that can be overlooked by mistake.
We'll make the following tweaks to the code:
For updateDisplayFromDigit , we return a new state with allowAppend set to true.
For updateDisplayFromPendingOp and addPendingMathOp , we return a new state with
allowAppend set to false.

Fixing the services

Most of the services are fine. The only service that is broken now is initState , which just
needs to be tweaked to have allowAppend be true when starting.
let initState :InitState = fun () ->
{
display=""
pendingOp = None
allowAppend = true
}

Fixing the user interface

The CalculatorForm class continues to work with no changes.
But this change does raise the question of how much the CalculatorForm should know
about the internals of the CalculatorDisplay type.
Should CalculatorDisplay be transparent, in which case the form might break every time
we change the internals?
Or should CalculatorDisplay be an opaque type, in which case we will need to add another
"service" that extracts the buffer from the CalculatorDisplay type so that the form can
display it?
For now, I'm happy to tweak the form if there are changes. But in a bigger or more long-term
project, when we are trying to reduce dependencies, then yes, I would make the domain
types opaque as much as possible to reduce the fragility of the design.

1770

Calculator Walkthrough: Part 3

Testing the patched version

Let's try out the patched version now (you can get the code for the patched version from this
gist).
Does it work now?
Yes. Entering 1 Add 2 Equals results in 3 , as expected.
So that fixes the major bug. Phew.
But if you keep playing around with this implementation, you will encounter other bugs
undocumented features too.
For example:
1.0 / 0.0 displays Infinity . What happened to our divide by zero error?

You get strange behaviors if you enter operations in unusual orders. For example,
entering 2 + + - shows 8 on the display!
So obviously, this code is not yet fit for purpose.

What about Test-Driven Development?

At this point, you might be saying to yourself: "if only he had used TDD this wouldn't have
happened".
It's true -- I wrote all this code, and yet I didn't even bother to write a test that checked
whether you could add two numbers properly!
If I had started out by writing tests, and letting that drive the design, then surely I wouldn't
have run into this problem.
Well in this particular example, yes, I would probably would have caught the problem
immediately. In a TDD approach, checking that 1 + 2 = 3 would have been one of the first
tests I wrote! But on the other hand, for obvious flaws like this, any interactive testing will
reveal the issue too.
To my mind, the advantages of test-driven development are that:
it drives the design of the code, not just the implementation.
it provides guarantees that code stays correct during refactoring.

1771

Calculator Walkthrough: Part 3

So the real question is, would test-driven development help us find missing requirements or
subtle edge cases? Not necessarily. Test-driven development will only be effective if we can
think of every possible case that could happen in the first place. In that sense, TDD would
not make up for a lack of imagination!
And if do have good requirements, then hopefully we can design the types to make illegal
states unrepresentable and then we won't need the tests to provide correctness guarantees.
Now I'm not saying that I am against automated testing. In fact, I do use it all the time to
verify certain requirements, and especially for integration and testing in the large.
So, for example, here is how I might test this code:

1772

Calculator Walkthrough: Part 3

module CalculatorTests =
open CalculatorDomain
open System
let config = CalculatorConfiguration.loadConfig()
let services = CalculatorServices.createServices config
let calculate = CalculatorImplementation.createCalculate services
let emptyState = services.initState()
/// Given a sequence of inputs, start with the empty state
/// and apply each input in turn. The final state is returned
let processInputs inputs =
// helper for fold
let folder state input =
calculate(input,state)
inputs
|> List.fold folder emptyState
/// Check that the state contains the expected display value
let assertResult testLabel expected state =
let actual = state.display
if (expected <> actual) then
printfn "Test %s failed: expected=%s actual=%s" testLabel expected actual
else
printfn "Test %s passed" testLabel
let ``when I input 1 + 2, I expect 3``() =
[Digit One; Op Add; Digit Two; Action Equals]
|> processInputs
|> assertResult "1+2=3" "3"
let ``when I input 1 + 2 + 3, I expect 6``() =
[Digit One; Op Add; Digit Two; Op Add; Digit Three; Action Equals]
|> processInputs
|> assertResult "1+2+3=6" "6"
// run tests
do
``when I input 1 + 2, I expect 3``()
``when I input 1 + 2 + 3, I expect 6``()

And of course, this would be easily adapted to using NUnit or similar.

How can I develop a better design?

1773

Calculator Walkthrough: Part 3

I messed up! As I said earlier, the implementation itself was not the problem. I think the typefirst design process worked. The real problem was that I was too hasty and just dived into
the design without really understanding the requirements.
How can I prevent this from happening again next time?
One obvious solution would be to switch to a proper TDD approach. But I'm going to be a bit
stubborn, and see if I can stay with a type-first design!
In the next post, I will stop being so ad-hoc and over-confident, and instead use a process
that is more thorough and much more likely to prevent these kinds of errors at the design
stage.
The code for this post is available on GitHub in this gist (unpatched) and this gist (patched).

1774

Calculator Walkthrough: Part 4

In this series of posts, I've been developing a simple pocket calculator app.
In the first post, we completed a first draft of the design, using type-first development. and in
the second post, we created an initial implemeentation.
In the previous post, we created the rest of the code, including the user interface, and
attempted to use it.
But the final result was unusable! The problem wasn't that the code was buggy, it was that I
didn't spend enough time thinking about the requirements before I started coding!
Oh well. As Fred Brooks famously said: "plan to throw one away; you will, anyhow"
(although that is a bit simplistic).
The good news is that I have learned from the previous bad implementation, and have a
plan to make the design better.

Reviewing the bad design

Looking at the design and implementation (see this gist), a few things stand out:
First, the event handling types such as UpdateDisplayFromDigit did not take into account the
context, the current state of the calculator. The allowAppend flag we added as a patch was
one way to take the context into account, but it smells awful bad.
Second there was a bit of special case code for certain inputs ( Zero and
DecimalSeparator ), as you can see from this code snippet:

1775

Calculator Walkthrough: Part 4

let appendCh=
match digit with
| Zero ->
// only allow one 0 at start of display
if display="0" then "" else "0"
| One -> "1"
| // snip
| DecimalSeparator ->
if display="" then
// handle empty display with special case
"0" + config.decimalSeparator
else if display.Contains(config.decimalSeparator) then
// don't allow two decimal separators
""
else
config.decimalSeparator

This makes me think that these inputs should be treated as different in the design itself and
not hidden in the implementation -- after all we want the design to also act as documentation
as much as possible.

Using a finite state machine as a design tool

So if the ad-hoc, make-it-up-as-you-go-along approach failed, what should I do instead?
Well, I am a big proponent of using finite state machines ("FSMs" -- not be confused with the
True FSM) where appropriate. It is amazing how often a program can be modelled as a state
machine.
What are the benefits of using state machines? I'm going to repeat what I said in another
post.
Each state can have different allowable behavior. In other words, a state machine forces
you to think about context, and what options are available in that context.
In this case, I forgot that the context changed after an Add was processed, and thus the
rules for accumulating digits changed too.
All the states are explicitly documented. It is all too easy to have important states that are
implicit but never documented.
For example, I have created special code to deal with zero and decimal separators.
Currently it is buried away in the implementation, but it should be part of the design.

1776

Calculator Walkthrough: Part 4

It is a design tool that forces you to think about every possibility that could occur. A
common cause of errors is that certain edge cases are not handled, but a state machine
forces all cases to be thought about.
In this case, in addition to the most obvious bug, there are still some edge cases that are not
dealt with properly, such as immediately following a math operation with another math
operation. What should happen then?

How to implement simple finite state machines

in F#
You are probably familiar with complex FSMs, such as those used in language parsers and
regular expressions. Those kinds of state machines are generated from rule sets or
grammars, and are quite complicated.
The kinds of state machines that I'm talking about are much, much simpler. Just a few cases
at the most, with a small number of transitions, so we don't need to use complex generators.
Here's an example of what I am talking about:

So what is the best way to implement these simple state machines in F#?
Now, designing and implementing FSMs is a complex topic in in own right, with its own
terminology (NFAs and DFAs, Moore vs. Mealy, etc), and whole businesses built around it.
In F#, there are a number of possible approaches, such as table driven, or mutually
recursive functions, or agents, or OO-style subclasses, etc.
But my preferred approach (for an ad-hoc manual implementation) makes extensive use of
union types and pattern matching.

1777

Calculator Walkthrough: Part 4

First, create a union type that represents all the states. For example, if there are three states
called "A", "B" and "C", the type would look like this:
type State =
| AState
| BState
| CState

In many cases, each state will need to store some data that is relevant to that state. So we
will need to create types to hold that data as well.
type State =
| AState of AStateData
| BState of BStateData
| CState
and AStateData =
{something:int}
and BStateData =
{somethingElse:int}

Next, all possible events that can happen are defined in another union type. If events have
data associated with them, add that as well.
type InputEvent =
| XEvent
| YEvent of YEventData
| ZEvent
and YEventData =
{eventData:string}

Finally, we can create a "transition" function that, given a current state and input event,
returns a new state.
let transition (currentState,inputEvent) =
match currentState,inputEvent with
| AState, XEvent -> // new state
| AState, YEvent -> // new state
| AState, ZEvent -> // new state
| BState, XEvent -> // new state
| BState, YEvent -> // new state
| CState, XEvent -> // new state
| CState, ZEvent -> // new state

1778

Calculator Walkthrough: Part 4

What I like about this approach in a language with pattern matching, like F#, is that if we
forget to handle a particular combination of state and event, we get a compiler
warning. How awesome is that?
It's true that, for systems with many states and input events, it may be unreasonable to
expect every possible combination to be explicitly handled. But in my experience, many
nasty bugs are caused by processing an event when you shouldn't, exactly as we saw with
the original design accumulating digits when it shouldn't have.
Forcing yourself to consider every possible combination is thus a helpful design practice.
Now, even with a small number of states and events, the number of possible combinations
gets large very quickly. To make it more manageable in practice, I typically create a series of
helper functions, one for each state, like this:
let aStateHandler stateData inputEvent =
match inputEvent with
| XEvent -> // new state
| YEvent _ -> // new state
| ZEvent -> // new state
let bStateHandler stateData inputEvent =
match inputEvent with
| XEvent -> // new state
| YEvent _ -> // new state
| ZEvent -> // new state
let cStateHandler inputEvent =
match inputEvent with
| XEvent -> // new state
| YEvent _ -> // new state
| ZEvent -> // new state
let transition (currentState,inputEvent) =
match currentState with
| AState stateData ->
// new state
aStateHandler stateData inputEvent
| BState stateData ->
// new state
bStateHandler stateData inputEvent
| CState ->
// new state
cStateHandler inputEvent

So let's try this approach and attempt to implement the state diagram above:

1779

Calculator Walkthrough: Part 4

let aStateHandler stateData inputEvent =

match inputEvent with
| XEvent ->
// transition to B state
BState {somethingElse=stateData.something}
| YEvent _ ->
// stay in A state
AState stateData
| ZEvent ->
// transition to C state
CState
let bStateHandler stateData inputEvent =
match inputEvent with
| XEvent ->
// stay in B state
BState stateData
| YEvent _ ->
// transition to C state
CState
let cStateHandler inputEvent =
match inputEvent with
| XEvent ->
// stay in C state
CState
| ZEvent ->
// transition to B state
BState {somethingElse=42}
let transition (currentState,inputEvent) =
match currentState with
| AState stateData ->
aStateHandler stateData inputEvent
| BState stateData ->
bStateHandler stateData inputEvent
| CState ->
cStateHandler inputEvent

If we try to compile this, we immediately get some warnings:

(near bStateHandler) Incomplete pattern matches on this expression. For example, the
value 'ZEvent' may indicate a case not covered by the pattern(s).

(near cStateHandler) Incomplete pattern matches on this expression. For example, the
value 'YEvent (_)' may indicate a case not covered by the pattern(s).

This is really helpful. It means we have missed some edge cases and we should change our
code to handle these events.

1780

Calculator Walkthrough: Part 4

By the way, please do not fix the code with a wildcard match (underscore)! That defeats the
purpose. If you want to ignore an event, do it explicitly.
Here's the fixed up code, which compiles without warnings:
let bStateHandler stateData inputEvent =
match inputEvent with
| XEvent
| ZEvent ->
// stay in B state
BState stateData
| YEvent _ ->
// transition to C state
CState
let cStateHandler inputEvent =
match inputEvent with
| XEvent
| YEvent _ ->
// stay in C state
CState
| ZEvent ->
// transition to B state
BState {somethingElse=42}

You can see the code for this example in this gist.

Designing the state machine for the calculator

Let's sketch out a state machine for the calculator now. Here's a first attempt:

1781

Calculator Walkthrough: Part 4

Each state is a box, and the events that trigger transitions (such as a digit or math operation
or Equals ) are in red.
If we follow through a sequence of events for something like 1 Add 2 Equals , you can
see that we'll end up at the "Show result" state at the bottom.
But remember that we wanted to raise the handling of zero and decimal separators up to the
design level?
So let's create special events for those inputs, and a new state "accumulate with decimal"
that ignores subsequent decimal separators.
Here's version 2:

1782

Calculator Walkthrough: Part 4

Finalizing the state machine

"Good artists copy. Great artists steal." -- Pablo Picasso (but not really)
At this point, I'm thinking that surely I can't be only person to have thought of using a state
machine to model a calculator? Perhaps I can do some research and steal borrow someone
else's design?
Sure enough, googling for "calculator state machine" brings up all sorts of results, including
this one which has a detailed spec and state transition diagram.
Looking at that diagram, and doing some more thinking, leads to the following insights:
The "clear" state and zero state are the same. Sometimes there is a pending op,
sometimes not.
A math operation and Equals are very similar in that they update the display with any
pending calculation. The only difference is whether a pending op is added to the state or
not.
The error message case definitely needs to be a distinct state. It ignores all input other
than Clear .
With these insights in mind then, here's version 3 of our state transition diagram:

1783

Calculator Walkthrough: Part 4

I'm only showing the key transitions -- it would be too overwhelming to show all of them. But
it does give us enough information to get started on the detailed requirements.
As we can see, there are five states:
ZeroState
AccumulatorState
AccumulatorDecimalState
ComputedState
ErrorState
And there are six possible inputs:
Zero
NonZeroDigit
DecimalSeparator
MathOp
Equals
Clear
Let's document each state, and what data it needs to store, if any.

1784

Calculator Walkthrough: Part 4

Data associated with

state

State

Special behavior?

ZeroState

(optional) pending op

Ignores all Zero input

AccumulatorState

buffer and (optional)

pending op

Accumulates digits in buffer

AccumulatorDecimalState

buffer and (optional)

pending op

Accumulates digits in buffer,

but ignores decimal separators

ComputedState

Calculated number and

(optional) pending op

ErrorState

Error message

Ignores all input other than

Clear

Documenting each state and event

combination
Next we should think about what happens for each state and event combination. As with the
sample code above, we'll group them so that we only have to deal with the events for one
state at a time.
Let's start with the ZeroState state. Here are the transitions for each type of input:
Input

Action

New State

Zero

(ignore)

ZeroState

NonZeroDigit

Start a new accumulator with the

digit.

AccumulatorState

DecimalSeparator

Start a new accumulator with "0."

AccumulatorDecimalState

MathOp

Go to Computed or ErrorState state.

If there is a pending op, update the
display based on the result of the
calculation (or error).
Also, if calculation was successful,
push a new pending op, built from
the event, using a current number of
"0".

ComputedState

Equals

As with MathOp, but without any

pending op

ComputedState

Clear

(ignore)

ZeroState

We can repeat the process with the AccumulatorState state. Here are the transitions for
each type of input:

1785

Calculator Walkthrough: Part 4

Input

Action

New State

Zero

Append "0" to the buffer.

AccumulatorState

NonZeroDigit

Append the digit to the buffer.

AccumulatorState

DecimalSeparator

Append the separator to the buffer,

and transition to new state.

AccumulatorDecimalState

MathOp

Go to Computed or ErrorState state.

ComputedState

Equals

As with MathOp, but without any

pending op

ComputedState

Clear

Go to Zero state. Clear any pending

op.

ZeroState

The event handling for AccumulatorDecimalState state is the same, except that
DecimalSeparator is ignored.

What about the ComputedState state. Here are the transitions for each type of input:
Input

Action

New State

Zero

Go to ZeroState state, but preserve

any pending op

ZeroState

NonZeroDigit

Start a new accumulator, preserving

any pending op

AccumulatorState

DecimalSeparator

Start a new decimal accumulator,

preserving any pending op

AccumulatorDecimalState

MathOp

Stay in Computed state. Replace

any pending op with a new one built
from the input event

ComputedState

Equals

Stay in Computed state. Clear any

pending op

ComputedState

Clear

Go to Zero state. Clear any pending

op.

ZeroState

Finally, the ErrorState state is very easy. :

1786

Calculator Walkthrough: Part 4

Input

Action

New
State

Zero, NonZeroDigit,
DecimalSeparator
MathOp, Equals

(ignore)

ErrorState

Clear

Go to Zero state. Clear any

pending op.

ZeroState

Converting the states into F# code

Now that we've done all this work, the conversion into types is straightforward.
Here are the main types:
type Calculate = CalculatorInput * CalculatorState -> CalculatorState
// five states
and CalculatorState =
| ZeroState of ZeroStateData
| AccumulatorState of AccumulatorStateData
| AccumulatorWithDecimalState of AccumulatorStateData
| ComputedState of ComputedStateData
| ErrorState of ErrorStateData
// six inputs
and CalculatorInput =
| Zero
| Digit of NonZeroDigit
| DecimalSeparator
| MathOp of CalculatorMathOp
| Equals
| Clear
// data associated with each state
and ZeroStateData =
PendingOp option
and AccumulatorStateData =
{digits:DigitAccumulator; pendingOp:PendingOp option}
and ComputedStateData =
{displayNumber:Number; pendingOp:PendingOp option}
and ErrorStateData =
MathOperationError

If we compare these types to the first design (below), we have now made it clear that there
is something special about Zero and DecimalSeparator , as they have been promoted to
first class citizens of the input type.

1787

Calculator Walkthrough: Part 4

// from the old design

Also, in the old design, we had a single state type (below) that stored data for all contexts,
while in the new design, the state is explicitly different for each context. The types
ZeroStateData , AccumulatorStateData , ComputedStateData , and ErrorStateData make this

obvious.
// from the old design
type CalculatorState = {
display: CalculatorDisplay
pendingOp: (CalculatorMathOp * Number) option
}
// from the new design
type CalculatorState =
| ZeroState of ZeroStateData
| AccumulatorState of AccumulatorStateData
| AccumulatorWithDecimalState of AccumulatorStateData
| ComputedState of ComputedStateData
| ErrorState of ErrorStateData

Now that we have the basics of the new design, we need to define the other types
referenced by it:

1788

Calculator Walkthrough: Part 4

and DigitAccumulator = string

And finally, we can define the services:

// services used by the calculator itself
type AccumulateNonZeroDigit = NonZeroDigit * DigitAccumulator -> DigitAccumulator
type AccumulateZero = DigitAccumulator -> DigitAccumulator
type AccumulateSeparator = DigitAccumulator -> DigitAccumulator
type DoMathOperation = CalculatorMathOp * Number * Number -> MathOperationResult
type GetNumberFromAccumulator = AccumulatorStateData -> Number
// services used by the UI or testing
type GetDisplayFromState = CalculatorState -> string
type GetPendingOpFromState = CalculatorState -> string
type CalculatorServices = {
accumulateNonZeroDigit :AccumulateNonZeroDigit
accumulateZero :AccumulateZero
accumulateSeparator :AccumulateSeparator
doMathOperation :DoMathOperation
getNumberFromAccumulator :GetNumberFromAccumulator
getDisplayFromState :GetDisplayFromState
getPendingOpFromState :GetPendingOpFromState
}

Note that because the state is much more complicated, I've added helper function
getDisplayFromState that extracts the display text from the state. This helper function will be

used the UI or other clients (such as tests) that need to get the text to display.
I've also added a getPendingOpFromState , so that we can show the pending state in the UI
as well.

Creating a state-based implementation

Now we can create a state-based implementation, using the pattern described earlier.
1789

Calculator Walkthrough: Part 4

(The complete code is available in this gist.)

Let's start with the main function that does the state transitions:
let createCalculate (services:CalculatorServices) :Calculate =
// create some local functions with partially applied services
let handleZeroState = handleZeroState services
let handleAccumulator = handleAccumulatorState services
let handleAccumulatorWithDecimal = handleAccumulatorWithDecimalState services
let handleComputed = handleComputedState services
let handleError = handleErrorState
fun (input,state) ->
match state with
| ZeroState stateData ->
handleZeroState stateData input
| AccumulatorState stateData ->
handleAccumulator stateData input
| AccumulatorWithDecimalState stateData ->
handleAccumulatorWithDecimal stateData input
| ComputedState stateData ->
handleComputed stateData input
| ErrorState stateData ->
handleError stateData input

As you can see, it passes the responsibility to a number of handlers, one for each state,
which will be discussed below.
But before we do that, I thought it might be instructive to compare the new state-machine
based design with the (buggy!) one I did previously.
Here is the code from the previous one:
let createCalculate (services:CalculatorServices) :Calculate =
fun (input,state) ->
match input with
| Digit d ->
let newState = updateDisplayFromDigit services d state
newState //return
| Op op ->
let newState1 = updateDisplayFromPendingOp services state
let newState2 = addPendingMathOp services op newState1
newState2 //return
| Action Clear ->
let newState = services.initState()
newState //return
| Action Equals ->
let newState = updateDisplayFromPendingOp services state
newState //return

1790

Calculator Walkthrough: Part 4

If we compare the two implementations, we can see that there has been a shift of emphasis
from events to state. You can see this by comparing how main pattern matching is done in
the two implementations:
In the original version, the focus was on the input, and the state was secondary.
In the new version, the focus is on the state, and the input is secondary.
The focus on input over state, ignoring the context, is why the old version was such a bad
design.
To repeat what I said above, many nasty bugs are caused by processing an event when you
shouldn't (as we saw with the original design). I feel much more confident in the new design
because of the explicit emphasis on state and context from the very beginning.
In fact, I'm not alone in noticing these kinds of issues. Many people think that classic "eventdriven programming" is flawed and recommend a more "state driven approach" (e.g. here
and here), just as I have done here.

Creating the handlers

We have already documented the requirements for each state transition, so writing the code
is straightforward. We'll start with the code for the ZeroState handler:

1791

Calculator Walkthrough: Part 4

let handleZeroState services pendingOp input =

// create a new accumulatorStateData object that is used when transitioning to oth
er states
let accumulatorStateData = {digits=""; pendingOp=pendingOp}
match input with
| Zero ->
ZeroState pendingOp // stay in ZeroState
| Digit digit ->
accumulatorStateData
|> accumulateNonZeroDigit services digit
|> AccumulatorState // transition to AccumulatorState
| DecimalSeparator ->
accumulatorStateData
|> accumulateSeparator services
|> AccumulatorWithDecimalState // transition to AccumulatorWithDecimalState
| MathOp op ->
let nextOp = Some op
let newState = getComputationState services accumulatorStateData nextOp
newState // transition to ComputedState or ErrorState
| Equals ->
let nextOp = None
let newState = getComputationState services accumulatorStateData nextOp
newState // transition to ComputedState or ErrorState
| Clear ->
ZeroState None // transition to ZeroState and throw away any pending ops

Again, the real work is done in helper functions such as accumulateNonZeroDigit and
getComputationState . We'll look at those in a minute.

Here is the code for the AccumulatorState handler:

1792

Calculator Walkthrough: Part 4

let handleAccumulatorState services stateData input =

match input with
| Zero ->
stateData
|> accumulateZero services
|> AccumulatorState // stay in AccumulatorState
| Digit digit ->
stateData
|> accumulateNonZeroDigit services digit
|> AccumulatorState // stay in AccumulatorState
| DecimalSeparator ->
stateData
|> accumulateSeparator services
|> AccumulatorWithDecimalState // transition to AccumulatorWithDecimalState
| MathOp op ->
let nextOp = Some op
let newState = getComputationState services stateData nextOp
newState // transition to ComputedState or ErrorState
| Equals ->
let nextOp = None
let newState = getComputationState services stateData nextOp
newState // transition to ComputedState or ErrorState
| Clear ->
ZeroState None // transition to ZeroState and throw away any pending op

Here is the code for the ComputedState handler:

let handleComputedState services stateData input =
let emptyAccumulatorStateData = {digits=""; pendingOp=stateData.pendingOp}
match input with
| Zero ->
ZeroState stateData.pendingOp // transition to ZeroState with any pending op
| Digit digit ->
emptyAccumulatorStateData
|> accumulateNonZeroDigit services digit
|> AccumulatorState // transition to AccumulatorState
| DecimalSeparator ->
emptyAccumulatorStateData
|> accumulateSeparator services
|> AccumulatorWithDecimalState // transition to AccumulatorWithDecimalState
| MathOp op ->
// replace the pending op, if any
let nextOp = Some op
replacePendingOp stateData nextOp
| Equals ->
// replace the pending op, if any
let nextOp = None
replacePendingOp stateData nextOp
| Clear ->
ZeroState None // transition to ZeroState and throw away any pending op

1793

Calculator Walkthrough: Part 4

The helper functions

Finally, let's look at the helper functions:
The accumulator helpers are trivial -- they just call the appropriate service and wrap the
result in an AccumulatorData record.
let accumulateNonZeroDigit services digit accumulatorData =
let digits = accumulatorData.digits
let newDigits = services.accumulateNonZeroDigit (digit,digits)
let newAccumulatorData = {accumulatorData with digits=newDigits}
newAccumulatorData // return

The getComputationState helper is much more complex -- the most complex function in the
entire code base, I should think.
It's very similar to the updateDisplayFromPendingOp that we implemented before, but there
are a couple of changes:
The services.getNumberFromAccumulator code can never fail, because of the statebased approach. That makes life simpler!
The match result with Success/Failure code now returns two possible states:
ComputedState or ErrorState .

If there is no pending op, we still need to return a valid ComputedState , which is what
computeStateWithNoPendingOp does.

1794

Calculator Walkthrough: Part 4

let getComputationState services accumulatorStateData nextOp =

// helper to create a new ComputedState from a given displayNumber
// and the nextOp parameter
let getNewState displayNumber =
let newPendingOp =
nextOp |> Option.map (fun op -> op,displayNumber )
{displayNumber=displayNumber; pendingOp = newPendingOp }
|> ComputedState
let currentNumber =
services.getNumberFromAccumulator accumulatorStateData
// If there is no pending op, create a new ComputedState using the currentNumber
let computeStateWithNoPendingOp =
getNewState currentNumber
maybe {
let! (op,previousNumber) = accumulatorStateData.pendingOp
let result = services.doMathOperation(op,previousNumber,currentNumber)
let newState =
match result with
| Success resultNumber ->
// If there was a pending op, create a new ComputedState using the res
ult
getNewState resultNumber
| Failure error ->
error |> ErrorState
return newState
} |> ifNone computeStateWithNoPendingOp

Finally, we have a new piece of code that wasn't in the previous implementation at all!
What do you do when you get two math ops in a row? We just replace the old pending op (if
any) with the new one (if any).
let replacePendingOp (computedStateData:ComputedStateData) nextOp =
let newPending = maybe {
let! existing,displayNumber = computedStateData.pendingOp
let! next = nextOp
return next,displayNumber
}
{computedStateData with pendingOp=newPending}
|> ComputedState

Completing the calculator

1795

Calculator Walkthrough: Part 4

To complete the application, we just need to implement the services and the UI, in the same
way as we did before.
As it happens, we can reuse almost all of the previous code. The only thing that has really
changed is the way that the input events are structured, which affects how the button
handlers are created.
You can get the code for the state machine version of the calculator here.
If you try it out the new code, I think that you will find that it works first time, and feels much
more robust. Another win for state-machine driven design!

Exercises
If you liked this design, and want to work on something similar, here are some exercises that
you could do:
First, you could add some other operations. What would you have to change to
implement unary ops such as 1/x and sqrt ?
Some calculators have a back button. What would you have to do to implement this?
Luckily all the data structures are immutable, so it should be easy!
Most calculators have a one-slot memory with store and recall. What would you have to
change to implement this?
The logic that says that there are only 10 chars allowed on the display is still hidden
from the design. How would you make this visible?

Summary
I hope you found this little experiment useful. I certainly learned something, namely: don't
shortcut requirements gathering, and consider using a state based approach from the
beginning -- it might save you time in the long run!

1796

Enterprise Tic-Tac-Toe

Enterprise Tic-Tac-Toe
UPDATE: Slides and video from my talk on this topic
This post is one of series in I which I hope to close the gap between theory and practice in
functional programming. I pick a small project and show you my thought processes as I go
about designing and implementing it from beginning to end.
For the next project in this series of posts, I'm going to do a walkthrough of a Tic-Tac-Toe
(aka Noughts and Crosses) implementation, written in a functional style.

Now, to be clear, I'm not a games developer in any shape or form, so I won't be focused on
performance or UX at all, just on the design process -- taking some requirements that we all
know (I hope) and translating them to functional code.
In fact, to be very clear, I'll deliberately be going a bit overboard on the design just to
demonstrate what you can do. There will be no objects. Everything will be immutable,
Everything will be typed. There will be capability based security, and more. Performance will
definitely be taking a back seat. Luckily, Tic-Tac-Toe does not need to support a high frame
rate!
In fact, I'm going to call this version "Enterprise Tic-Tac-Toe"!
Why? Well let's look at what you need for "Enterprise":
We need separation of concerns so that specialist teams can work on different parts
of the code at the same time.
We need a documented API so that the different teams can work effectively in parallel.
We need a security model to prevent unauthorized actions from occurring.
We need well-documented code so that the architect can ensure that the
implementation matches the UML diagrams.
We need auditing and logging to ensure that the system is SOX compliant.
We need scalability to ensure that the system is ready for the challenges of rapid
customer acquisition.

1797

Enterprise Tic-Tac-Toe

Actually, those are the stated reasons, but we all know that this is not the whole story. The
real reasons for an "enterprise design" become apparent when you talk to the people
involved:
Development Manager: "We need separation of concerns because the front-end team
and back-end team hate each other and refuse to work in the same room."
Front-end team: "We need a documented API so that those dummies building the backend won't keep breaking our code on every commit."
Back-end team: "We need a security model because those idiots building the front-end
will always find a way to do something stupid unless we constrain them."
Maintenance team: "We need well-documented code because we're fed up of having to
reverse engineer the hacked-up spaghetti being thrown at us."
Testers and Operations: "We need auditing and logging so that we can see what the
effing system is doing inside."
Everyone: "We don't really need scalability at all, but the CTO wants to us to be
buzzword compliant."
It's true that there are already some wonderful "enterprise" projects out there, such as Easy
Plus in PHP and Fizz Buzz Enterprise Edition in Java, but I hope that my own small
contribution to this genre will be considered worthy.
Seriously, I hope that the code won't be quite as bad amusing as those other enterprise
projects. In fact, I hope to demonstrate that you can have "enterprise" ready functional code
which is still readable!

Designing the domain

"Write the game such that someone unfamiliar with it could learn the rules by looking at
the source code" -- Raganwald
As always, let's use a type-first design. If you recall, this approach means that:
We start with types only -- no implementation code.
Every use-case or scenario corresponds to a function type, with one input and one
output (which means I'll use tuples when multiple parameters are needed).
We work mostly top-down and outside-in, but occasionally bottom up as well.
We ignore the UI for now. And there will be no events or observables in the design of
the core domain. It will be purely functional.
In fact, an alternative title for this post might be growing functional software, guided by
types.

1798

Enterprise Tic-Tac-Toe

As I have said before, I like to drive the design by working from the events that can happen,
rather than the objects involved. I'm old school, so I call them use-cases, but I also like the
event-storming approach.
Either way, for the Tic-Tac-Toe "domain", we have three different "event-driven use-cases"
(in this case, just various mouse clicks!) to think about:
Initialize a game
Player X moves
Player O moves
Let's start with the first: initialization. This is equivalent to a new -style constructor in an OO
program.
For Tic-Tac-Toe, there are no configuration parameters needed, so the input would be "null"
(aka unit ) and the output would be a game ready to play, like this:
type InitGame = unit -> Game

Now, what is this Game ? Since everything is immutable, the other scenarios are going to
have to take an existing game as input, and return a slightly changed version of the game.
So Game is not quite appropriate. How about GameState instead? A "player X moves"
function will thus look something like this:
type PlayerXMoves = GameState * SomeOtherStuff -> GameState

You'll see that I added SomeOtherStuff to the input parameters because there's always
some other stuff! We'll worry about what the "other stuff" is later.
Ok, What should we do next? Should we look more deeply into the internals of GameState ?
No. Let's stay high-level and do more "outside-in" style design. I like this approach in general
because it allows me to focus on what's important and not get side-tracked by
implementation details.

Designing the move functions

I said originally that we should have a function for each scenario. Which means we would
have functions like this:
type PlayerXMoves = GameState * SomeOtherStuff -> GameState
type PlayerOMoves = GameState * SomeOtherStuff -> GameState

1799

Enterprise Tic-Tac-Toe

For each player's move, we start with the current game state, plus some other input created
by the player, and end up with a new game state.
The problem is that both functions look exactly the same and could be easily substituted for
each other. To be honest, I don't trust the user interface to always call the right one -- or at
least, it could be a potential issue.
One approach is to have only one function, rather than two. That way there's nothing to go
wrong.
But now we need to handle the two different input cases. How to do that? Easy! A
discriminated union type:
type UserAction =
| PlayerXMoves of SomeStuff
| PlayerOMoves of SomeStuff

And now, to process a move, we just pass the user action along with the state, like this:
type Move = UserAction * GameState -> GameState

So now there is only one function for the UI to call rather than two, and less to get wrong.
This approach is great where there is one user, because it documents all the things that they
can do. For example, in other games, you might have a type like this:
type UserAction =
| MoveLeft
| MoveRight
| Jump
| Fire

However in this situation, this way doesn't feel quite right. Since there are two players, what I
want to do is give each player their own distinct function to call and not allow them to use the
other player's function. This not only stops the user interface component from messing up,
but also gives me my capability-based security!
But now we are back to the original problem: how can we tell the two functions apart?
What I'll do is to use types to distinguish them. We'll make the SomeOtherStuff be owned by
each player, like this:
type PlayerXMoves = GameState * PlayerX's Stuff -> GameState
type PlayerOMoves = GameState * PlayerO's Stuff -> GameState

1800

Enterprise Tic-Tac-Toe

This way the two functions are distinct, and also PlayerO cannot call PlayerX's function
without having some of PlayerX's Stuff as well. If this sound's complicated, stay tuned -it's easier than it looks!

What is SomeOtherStuff?
What is this mysterious SomeOtherStuff ? In other words, what information do we need to
make a move?
For most domains, there might quite a lot of stuff that needs to be passed in, and the stuff
might vary based on the context and the state of the system.
But for Tic-Tac-Toe, it's easy, it's just the location on the grid where the player makes their
mark. "Top Left", "Bottom Center", and so on.
How should we define this position using a type?
The most obvious approach would be to use a 2-dimensional grid indexed by integers:
(1,1) (1,2) (1,3) , etc. But I have to admit that I'm too lazy to write unit tests that deal with

bounds-checking, nor can I ever remember which integer in the pair is the row and which the
column. I want to write code that I don't have to test!
Instead, let's define a type explicitly listing each position of horizontally and vertically:
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom

And then the position of a square in the grid (which I'm going to call a "cell") is just a pair of
these:
type CellPosition = HorizPosition * VertPosition

If we go back to the "move function" definitions, we now have:

type PlayerXMoves = GameState * CellPosition -> GameState
type PlayerOMoves = GameState * CellPosition -> GameState

which means: "to play a move, the input is a game state and a selected cell position, and the
output is an updated game state".
Both player X and player O can play the same cell position, so, as we said earlier, we need
to make them distinct.

1801

Enterprise Tic-Tac-Toe

I'm going to do that by wrapping them in a single case union:

type PlayerXPos = PlayerXPos of CellPosition
type PlayerOPos = PlayerOPos of CellPosition

And with that, our move functions now have different types and can't be mixed up:
type PlayerXMoves = GameState * PlayerXPos -> GameState
type PlayerOMoves = GameState * PlayerOPos -> GameState

What is the GameState?

Now let's focus on the game state. What information do we need to represent the game
completely between moves?
I think it is obvious that the only thing we need is a list of the cells, so we can define a game
state like this:
type GameState = {
cells : Cell list
}

But now, what do we need to define a Cell ?

First, the cell's position. Second, whether the cell has an "X" or an "O" on it. We can
therefore define a cell like this:
type CellState =
| X
| O
| Empty
type Cell = {
pos : CellPosition
state : CellState
}

Designing the output

What about the output? What does the UI need to know in order to update itself?

1802

Enterprise Tic-Tac-Toe

One approach is just to pass the entire game state to the UI and let the UI redisplay the
whole thing from scratch. Or perhaps, to be more efficient, the UI could cache the previous
state and do a diff to decide what needs to be updated.
In more complicated applications, with thousands of cells, we can be more efficient and
make the UI's life easier by explicitly returning the cells that changed with each move, like
this:
// added "ChangedCells"
type PlayerXMoves = GameState * PlayerXPos -> GameState * ChangedCells
type PlayerOMoves = GameState * PlayerOPos -> GameState * ChangedCells

Since Tic-Tac-Toe is a tiny game, I'm going to keep it simple and just return the game state
and not anything like ChangedCells as well.
But as I said at the beginning, I want the UI to be as dumb as possible! The UI should not
have to "think" -- it should be given everything it needs to know by the backend, and to just
follow instructions.
As it stands, the cells can be fetched directly from the GameState , but I'd rather that the UI
did not know how GameState is defined. So let's give the UI a function ( GetCells , say) that
can extract the cells from the GameState :
type GetCells = GameState -> Cell list

Another approach would be for GetCells to return all the cells pre-organized into a 2D grid - that would make life even easier for the UI.
type GetCells = GameState -> Cell[,]

But now the game engine is assuming the UI is using a indexed grid. Just as the UI
shouldn't know about the internals of the backend, the backend shouldn't make assumptions
about how the UI works.
It's fair enough to allow the UI to share the same definition of Cell as the backend, so we
can just give the UI a list of Cell s and let it display them in its own way.
Ok, the UI should have everything it needs to display the game now.

Review of the first version of the design

Great! Let's look at what we've got so far:

1803

Enterprise Tic-Tac-Toe

module TicTacToeDomain =
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom
type CellPosition = HorizPosition * VertPosition
type CellState =
| X
| O
| Empty
type Cell = {
pos : CellPosition
state : CellState
}
type PlayerXPos = PlayerXPos of CellPosition
type PlayerOPos = PlayerOPos of CellPosition
// the private game state
type GameState = exn // use a placeholder
// the "use-cases"
type InitGame = unit -> GameState
type PlayerXMoves = GameState * PlayerXPos -> GameState
type PlayerOMoves = GameState * PlayerOPos -> GameState
// helper function
type GetCells = GameState -> Cell list

Note that in order to make this code compile while hiding the implementation of GameState ,
I've used a generic exception class ( exn ) as a placeholder for the actual implementation of
GameState . I could also have used unit or string instead, but exn is not likely to get

mixed up with anything else, and will prevent it being accidentally overlooked later!

A note on tuples
Just a reminder that in this design phase, I'm going to combine all the input parameters into
a single tuple rather than treat them as separate parameters.
This means that I'll write:
InputParam1 * InputParam2 * InputParam3 -> Result

rather than the more standard:

1804

Enterprise Tic-Tac-Toe

InputParam1 -> InputParam2 -> InputParam3 -> Result

I'm doing this just to make the input and output obvious. When it comes to the
implementation, it's more than likely that we'll switch to the standard way, so that we can
take advantage of the techniques in our functional toolbox such as partial application.

Doing a design walkthrough

At this point, with a rough design in place, I like to do a walkthrough as if it were being used
for real. In a larger design, I might develop a small throwaway prototype, but in this case, the
design is small enough that I can do it in my head.
So, let's pretend that we are the UI and we are given the design above. We start by calling
the initialization function to get a new game:
type InitGame = unit -> GameState

Ok, so now we have a GameState and we are ready to display the initial grid.
At this point, the UI would create, say, a grid of empty buttons, associate a cell to each
button, and then draw the cell in the "empty" state.
This is fine, because the UI doesn't have to think. We are explicitly giving the UI a list of all
cells, and also making the initial cell state Empty , so the UI doesn't have to know which is
the default state -- it just displays what it is given.
One thing though. Since there is no input needed to set up the game, and the game state is
immutable, we will have exactly the same initial state for every game.
Therefore we don't need a function to create the initial game state, just a "constant" that gets
reused for each game.
type InitialGameState = GameState

When does the game stop?

Next in our walkthrough, let's play a move.
A player, "X" or "O", clicks on a cell
We combine the player and CellPosition into the appropriate type, such as a
PlayerXPos

1805

Enterprise Tic-Tac-Toe

We then pass that and the GameState into the appropriate Move function
type PlayerXMoves =
GameState * PlayerXPos -> GameState

The output is a new GameState . The UI then calls GetCells to get the new cells. We loop
through this list, update the display, and now we're ready to try again.
Excellent!
Umm... except for the bit about knowing when to stop.
As designed, This game will go on forever. We need to include something in the output of
the move to let us know whether the game is over!
So let's create a GameStatus type to keep track of that.
type GameStatus =
| InProcess
| PlayerXWon
| PlayerOWon
| Tie

And we need to add it to the output of the move as well, so now we have:
type PlayerXMoves =
GameState * PlayerXPos -> GameState * GameStatus

So now we can keep playing moves repeatedly while GameStatus is InProcess and then
stop.
The pseudocode for the UI would look like

1806

Enterprise Tic-Tac-Toe

// loop while game not over

let rec playMove gameState =
let pos = // get position from user input
let newGameState,status =
playerXMoves (gameState,pos) // process move
match status with
| InProcess ->
// play another move
playMove newGameState
| PlayerXWon ->
// show that player X won
| etc
// start the game with the initial state
let startGame() =
playMove initialGameState

I think we've got everything we need to play a game now, so let's move on to error handling.

What kind of errors can happen?

Before we starting thinking about the internals of the game, let's think about what kinds of
errors the UI team could make when using this design:
Could the UI create an invalid GameState and corrupt the game?
No, because we are going to keep the internals of the game state hidden from the UI.
Could the UI pass in an invalid CellPosition ?
No, because the horizontal and vertical components of CellPosition are restricted and
therefore it cannot be created in an invalid state. No validation is needed.
Could the UI pass in a valid CellPosition but at the wrong time?
Ah, now you're talking! Yes -- that is totally possible. In the design we have so far, there is
nothing stopping a player playing the same square twice!
Could the UI allow player X to play twice in a row?
Again, yes. Nothing in our design prevents this.
What about when the game has ended but the dumb UI forgets to check the
GameStatus and doesn't notice. Should the game logic still accept moves?

Of course not, but yet again our design fails to do this.

1807

Enterprise Tic-Tac-Toe

The big question is: can we fix these three issues in our design without having to rely on
special validation code in the implementation? That is, can we encode these rules into types.
At this point you might be thinking "why bother with all these types?"
The advantage of using types over validation code is that the types are part of the design,
which means that business rules like these are self-documenting. On the other hand,
validation code tends to be scattered around and buried in obscure classes, so it is hard to
get a big picture of all the constraints.
In general then, I prefer to use types rather than code if I can.

Enforcing the rules through types

So, can we encode these rules using types? The answer is yes!
To stop someone playing the same square twice we can change the game engine so that it
outputs a list of valid moves. And then we can require that only items in this list are allowed
to be played in the next turn.
If we do this, our move type will look like this:
type ValidPositionsForNextMove = CellPosition list
// a move returns the list of available positions for the next move
type PlayerXMoves =
GameState * PlayerXPos -> // input
GameState * GameStatus * ValidPositionsForNextMove // output

And we can extend this approach to stop player X playing twice in a row too. Simply make
the ValidPositionsForNextMove be a list of PlayerOPos rather than generic positions. Player
X will not be able to play them!
type ValidMovesForPlayerX = PlayerXPos list
type ValidMovesForPlayerO = PlayerOPos list
type PlayerXMoves =
GameState * PlayerXPos -> // input
GameState * GameStatus * ValidMovesForPlayerO // output
type PlayerOMoves =
GameState * PlayerOPos -> // input
GameState * GameStatus * ValidMovesForPlayerX // output

1808

Enterprise Tic-Tac-Toe

This approach also means that when the game is over, there are no valid moves available.
So the UI cannot just loop forever, it will be forced to stop and deal with the situation.
So now we have encoded all three rules into the type system -- no manual validation
needed.

The second thing we can do is to note that we only need the valid moves in the InProcess
case, not the Won or Tie cases, so let's merge GameStatus and ValidMovesForPlayer into
a single type called MoveResult , say:

1809

Enterprise Tic-Tac-Toe

type ValidMovesForPlayerX = PlayerXPos list

type ValidMovesForPlayerO = PlayerOPos list
type MoveResult =
| PlayerXToMove of GameState * ValidMovesForPlayerX
| PlayerOToMove of GameState * ValidMovesForPlayerO
| GameWon of GameState * Player
| GameTied of GameState

We've replaced the InProcess case with two new cases PlayerXToMove and
PlayerOToMove , which I think is actually clearer.

The move functions now look like:

type PlayerXMoves =
GameState * PlayerXPos ->
GameState * MoveResult
type PlayerOMoves =
GameState * PlayerOPos ->
GameState * MoveResult

I could have had the new GameState returned as part of MoveResult as well, but I left it
"outside" to make it clear that is not to be used by the UI.
Also, leaving it outside will give us the option of writing helper code that will thread a game
state through a series of calls for us. This is a more advanced technique, so I'm not going to
discuss it in this post.
Finally, the InitialGameState should also take advantage of the MoveResult to return the
available moves for the first player. Since it has both a game state and a initial set of moves,
let's just call it NewGame instead.
type NewGame = GameState * MoveResult

If the initial MoveResult is the PlayerXToMove case, then we have also constrained the UI so
that only player X can move first. Again, this allows the UI to be ignorant of the rules.

Second recap
So now here's the tweaked design we've got after doing the walkthrough.

1810

Enterprise Tic-Tac-Toe

We're not quite done with the outside-in design yet. One question is yet to be resolved: how
can we hide the implementation of GameState from the UI?

Decoupling shared and private types

1811

Enterprise Tic-Tac-Toe

In any design, we want to decouple the "interface" from the "implementation". In this case,
we have:
A set of shared data structures and functions that are used by both the UI and the game
engine. ( CellState , MoveResult , PlayerXPos , etc.)
A set of private data structures and functions that should only be accessed by the game
logic. (just GameState so far)
It's obviously a good idea to keep these types separate. How should we do this?
In F#, the easiest way is to put them into separate modules, like this:
/// Types shared by the UI and the game logic
module TicTacToeDomain =
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom
type CellPosition = HorizPosition * VertPosition
type Player = PlayerO | PlayerX
type CellState =
| Played of Player
| Empty
type PlayerXMoves =
GameState * PlayerXPos -> GameState * MoveResult
// etc
/// Private types used by the internal game logic
module TicTacToeImplementation =
open TicTacToeDomain
// private implementation detail
type GameState = {
cells : Cell list
}
// etc

But if we want to keep the internals of the game logic private, what do we do with
GameState ? It's used by public functions such as PlayerXMoves , but we want to keep its

structure secret. How can we do that?

Option 1 - put the public and private types in the same

module

1812

Enterprise Tic-Tac-Toe

The first choice might be to put the public and private types in the same module, and have
this module be the "core" domain module that all other modules depend on.
Here's some code that demonstrates what this approach would look like:
module TicTacToeImplementation =
// public types
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom
type CellPosition = HorizPosition * VertPosition
type CellState =
| Played of Player
| Empty
type PlayerXMoves =
GameState * PlayerXPos -> GameState * MoveResult
// etc
// --------------------
// private types
type private InternalType = // to do
// ------------------- // public types with private constructor
type GameState = private {
cells : Cell list
}
// etc

All the types are in one module.

Many of the types, such as CellState , will be public by default. That's fine.
But you can see that some of the types, such as InternalType , have been marked private.
That means that they cannot be used outside the module at all.
Finally, GameState is not private, but its constructor is, which means that it can be used
outside the module, but because its constructor is private, new ones can't be created, which
sounds like what we need.
We might have appeared to solve the issue, but this approach often causes problems of its
own. For starters, trying to keep the public and private qualifiers straight can cause
annoying compiler errors, such as this one:

1813

Enterprise Tic-Tac-Toe

The type 'XXX' is less accessible than the value, member or type 'YYY' it is used in

And even if this weren't a problem, putting the "interface" and the "implementation" in the
same file will generally end up creating extra complexity as the implementation gets larger.

Option 2 - representing GameState with an abstract base

class
The object-oriented way of approaching this would be to represent GameState as an
abstract base class or interface, and then have a particular implementation inherit from the
abstract class.
This allows all the shared types to reference the abstract class or interface safely, while any
particular implementation is hidden.
Here's how you might do this in F#:
/// Types shared by the UI and the game logic
module TicTacToeDomain =
// abstract base class
type GameState() = class end
/// Private types used by the internal game logic
module TicTacToeImplementation =
open TicTacToeDomain
type GameStateImpl() =
inherit GameState()
// etc

But alas, there are problems with this approach too.

First, it's not very functional, is it? F# does support classes and interfaces for those
situations when we need them, but we should really be able to find a more idiomatic
functional solution than this!
Second, it's potentially not safe. The actual implementation would have to downcast
GameState into the type it expects in order to get at the internal data. But if I had two

implementations that inherited GameState , what's to stop me passing a game state from
implementation B into a function that is expecting a game state from implementation A?
Nothing! Chaos would ensue!

1814

Enterprise Tic-Tac-Toe

Note that in a pure OO model this situation could not happen because the GameState itself
would have stateful methods instead of the pure functional API that we have here.

Option 3 - parameterize the implementation

Let's think about the requirements again: "The GameState is public but we don't know what
the implementation will be."
When you rephrase it like this, the functional way of modeling this becomes clear, which is to
use generic parameters (aka "parametric polymorphism").
In other words, we make GameState a generic type which represents a particular
implementation.
This means that the UI can work with the GameState type, but because the actual
implementation type used is not known, the UI cannot accidentally "look inside" and extract
any information, even if the implementation type is public.
This last point is important, so I'm going to say it again with another example. If I give you a
object of type List<T> in C#, you can work with the list in many ways, but you cannot know
what the T is, and so you can never accidentally write code that assumes that T is an
int or a string or a bool . And this "hidden-ness" has got nothing to do with whether T

is a public type or not.

If we do take this approach then we can allow the internals of the game state to be
completely public, safe in the knowledge that the UI cannot use that information even if it
wanted to!
So here's some code demonstrating this approach.
First the shared types, with GameState<'T> being the parameterized version.
/// Types shared by the UI and the game logic
module TicTacToeDomain =
// unparameterized types
type PlayerXPos = PlayerXPos of CellPosition
type PlayerOPos = PlayerOPos of CellPosition
// parameterized types
type PlayerXMoves<'GameState> =
'GameState * PlayerXPos -> 'GameState * MoveResult
type PlayerOMoves<'GameState> =
'GameState * PlayerOPos -> 'GameState * MoveResult
// etc

1815

Enterprise Tic-Tac-Toe

The types that don't use the game state are unchanged, but you can see that
PlayerXMoves<'T> has been parameterized with the game state.

Adding generics like this can often cause cascading changes to many types, forcing them all
to be parameterized. Dealing with all these generics is one reason why type inference is so
helpful in practice!
Now for the types internal to the game logic. They can all be public now, because the UI
won't be able to know about them.
module TicTacToeImplementation =
open TicTacToeDomain
// can be public
type GameState = {
cells : Cell list
}

Finally, here's what the implementation of a playerXMoves function might look like:
let playerXMoves : PlayerXMoves<GameState> =
fun (gameState,move) ->
// logic

This function references a particular implementation, but can be passed into the UI code
because it conforms to the PlayerXMoves<'T> type.
Furthermore, by using generic parameters, we naturally enforce that the same
implementation, say "GameStateA", is used throughout.
In other words, the game state created by InitGame<GameStateA> can only be passed to a
PlayerXMoves<GameStateA> function which is parameterized on the same implementation

type.

Glueing it all together with "dependency

injection"
Finally, let's talk about how everything can be glued together.
The UI code will be designed to work with a generic implementation of GameState , and thus
generic versions of the newGame and move functions.
But of course, at some point we need to get access to the newGame and move functions for
a specific implementation. What's the best way to glue all this together?

1816

Enterprise Tic-Tac-Toe

The answer is the functional equivalent of dependency injection. We will have an

"application" or "program" component as a top-level layer that will construct an
implementation and pass it to the UI.
Here's an example of what such code would look like:
The GameImplementation module exports specific implementations of newGame and the
move functions.

The UserInterface module exports a TicTacToeForm class that accepts these

implementations in its constructor.
The Application module glues everything together. It creates a TicTacToeForm and
passes it the implementations exported from the GameImplementation module.
Here's some code to demonstrate this approach:

1817

Enterprise Tic-Tac-Toe

module TicTacToeImplementation =
open TicTacToeDomain
/// create the state of a new game
let newGame : NewGame<GameState> =
// return new game and current available moves
let validMoves = // to do
gameState, PlayerXToMove validMoves
let playerXMoves : PlayerXMoves<GameState> =
fun (gameState,move) ->
// implementation
module WinFormUI =
open TicTacToeDomain
open System.Windows.Forms
type TicTacToeForm<'T>
(
// pass in the required functions
// as parameters to the constructor
newGame:NewGame<'T>,
playerXMoves:PlayerXMoves<'T>,
playerOMoves:PlayerOMoves<'T>,
getCells:GetCells<'T>
) =
inherit Form()
// implementation to do
module WinFormApplication =
open WinFormUI
// get functions from implementation
let newGame = TicTacToeImplementation.newGame
let playerXMoves = TicTacToeImplementation.playerXMoves
let playerOMoves = TicTacToeImplementation.playerOMoves
let getCells = TicTacToeImplementation.getCells
// create form and start game
let form =
new TicTacToeForm<_>(newGame,playerXMoves,playerOMoves,getCells)
form.Show()

A few notes on this code:

First, I'm using WinForms rather than WPF because it has Mono support and because it
works without NuGet dependencies. If you want to use something better, check out
ETO.Forms.

1818

Enterprise Tic-Tac-Toe

Next, you can see that I've explicitly added the type parameters to TicTacToeForm<'T> like
this.
TicTacToeForm<'T>(newGame:NewGame<'T>, playerXMoves:PlayerXMoves<'T>, etc)

I could have eliminated the type parameter for the form by doing something like this instead:
TicTacToeForm(newGame:NewGame<_>, playerXMoves:PlayerXMoves<_>, etc)

or even:
TicTacToeForm(newGame, playerXMoves, etc)

and let the compiler infer the types, but this often causes a "less generic" warning like this:
warning FS0064: This construct causes code to be less generic than indicated by the ty
pe annotations.
The type variable 'T has been constrained to be type 'XXX'.

By explicitly writing TicTacToeForm<'T> , this can be avoided, although it is ugly for sure.

Some more refactoring

We've got four different functions to export. That's getting a bit much so let's create a record
to store them in:
// the functions exported from the implementation
// for the UI to use.
type TicTacToeAPI<'GameState> =
{
newGame : NewGame<'GameState>
playerXMoves : PlayerXMoves<'GameState>
playerOMoves : PlayerOMoves<'GameState>
getCells : GetCells<'GameState>
}

This acts both as a container to pass around functions, and as nice documentation of what
functions are available in the API.
The implementation now has to create an "api" object:

1819

Enterprise Tic-Tac-Toe

module TicTacToeImplementation =
open TicTacToeDomain
/// create the functions to export
let newGame : NewGame<GameState> = // etc
let playerXMoves : PlayerXMoves<GameState> = // etc
// etc
// export the functions
let api = {
newGame = newGame
playerOMoves = playerOMoves
playerXMoves = playerXMoves
getCells = getCells
}

But the UI code simplifies as a result:

module WinFormUI =
open TicTacToeDomain
open System.Windows.Forms
type TicTacToeForm<'T>(api:TicTacToeAPI<'T>) =
inherit Form()
// implementation to do
module WinFormApplication =
open WinFormUI
// get functions from implementation
let api = TicTacToeImplementation.api
// create form and start game
let form = new TicTacToeForm<_>(api)
form.Show()

Prototyping a minimal implementation

It seems like we're getting close to a final version. But let's do one more walkthrough to
exercise the "dependency injection" design, this time writing some minimal code to test the
interactions.
For example, here is some minimal code to implement the newGame and playerXMoves
functions.
The newGame is just an game with no cells and no available moves
The minimal implementation of move is easy -- just return game over!
1820

Enterprise Tic-Tac-Toe

let newGame : NewGame<GameState> =

// create initial game state with empty everything
let gameState = { cells=[]}
let validMoves = []
gameState, PlayerXToMove validMoves
let playerXMoves : PlayerXMoves<GameState> =
// dummy implementation
fun gameState move -> gameState,GameTied
let playerOMoves : PlayerOMoves<GameState> =
// dummy implementation
fun gameState move -> gameState,GameTied
let getCells gameState =
gameState.cells
let api = {
newGame = newGame
playerOMoves = playerOMoves
playerXMoves = playerXMoves
getCells = getCells
}

Now let's create a minimal implementation of the UI. We won't draw anything or respond to
clicks, just mock up some functions so that we can test the logic.
Here's my first attempt:
type TicTacToeForm<'GameState>(api:TicTacToeAPI<'GameState>) =
inherit Form()
let mutable gameState : 'GameState = ???
let mutable lastMoveResult : MoveResult = ???
let displayCells gameState =
let cells = api.getCells gameState
for cell in cells do
// update display
let startGame()=
let initialGameState,initialResult = api.newGame
gameState <- initialGameState
lastMoveResult <- initialResult
// create cell grid from gameState
let handleMoveResult moveResult =
match moveResult with
| PlayerXToMove availableMoves ->
// show available moves

1821

Enterprise Tic-Tac-Toe

| PlayerOToMove availableMoves ->

// show available moves
| GameWon player ->
let msg = sprintf "%A Won" player
MessageBox.Show(msg) |> ignore
| GameTied ->
MessageBox.Show("Tied") |> ignore
// handle a click
let handleClick() =
let gridIndex = 0,0 // dummy for now
let cellPos = createCellPosition gridIndex
match lastMoveResult with
| PlayerXToMove availableMoves ->
let playerXmove = PlayerXPos cellPos
// if move is in available moves then send it
// to the api
let newGameState,newResult =
api.playerXMoves gameState playerXmove
handleMoveResult newResult
//update the globals
gameState <- newGameState
lastMoveResult <- newResult
| PlayerOToMove availableMoves ->
let playerOmove = PlayerOPos cellPos
// if move is in available moves then send it
// to the api
// etc
| GameWon player ->
?? // we aleady showed after the last move

As you can see, I'm planning to use the standard Form event handling approach -- each cell
will have a "clicked" event handler associated with it. How the control or pixel location is
converted to a CellPosition is something I'm not going to worry about right now, so I've just
hard-coded some dummy data.
I'm not going to be pure here and have a recursive loop. Instead, I'll keep the current
gameState as a mutable which gets updated after each move.

But now we have got a tricky situation... What is the gameState when the game hasn't
started yet? What should we initialize it to? Similarly, when the game is over, what should it
be set to?
let mutable gameState : 'GameState = ???

One choice might be to use a GameState option but that seems like a hack, and it makes
me think that we are failing to think of something.

1822

Enterprise Tic-Tac-Toe

Similarly, we have a field to hold the result of the last move ( lastMoveResult ) so we can
keep track of whose turn it is, or whether the game is over.
But again, what should it be set to when the game hasn't started?
Let's take a step back and look at all the states the user interface can be in -- not the state of
the game itself, but the state of the user interface.
We start off in an "idle" state, with no game being played.
Then the user starts the game, and we are "playing".
While each move is played, we stay in the "playing" state.
When the game is over, we show the win or lose message.
We wait for the user to acknowledge the end-of-game message, then go back to idle
again.
Again, this is for the UI only, it has nothing to do with the internal game state.
So -- our solution to all problems! -- let's create a type to represent these states.
type UiState =
| Idle
| Playing
| Won
| Lost

But do we really need the Won and Lost states? Why don't we just go straight back to
Idle when the game is over?

So now the type looks like this:

type UiState =
| Idle
| Playing

The nice thing about using a type like this is that we can easily store the data that we need
for each state.
What data do we need to store in the Idle state? Nothing that I can think of.
What data do we need to store in the Playing state? Well, this would be a perfect
place to keep track of the gameState and lastMoveResult that we were having
problems with earlier. They're only needed when the game is being played, but not
otherwise.
So our final version looks like this. We've had to add the <'GameState> to UiState because
we don't know what the actual game state is.

1823

Enterprise Tic-Tac-Toe

type UiState<'GameState> =
| Idle
| Playing of 'GameState * MoveResult

With this type now available, we no longer need to store the game state directly as a field in
the class. Instead we store a mutable UiState , which is initialized to Idle .
type TicTacToeForm<'GameState>(api:TicTacToeAPI<'GameState>) =
inherit Form()
let mutable uiState = Idle

When we start the game, we change the UI state to be Playing :

let startGame()=
uiState <- Playing api.newGame
// create cell grid from gameState

And when we handle a click, we only do something if the uiState is in Playing mode, and
then we have no trouble getting the gameState and lastMoveResult that we need, because
it is stored as part of the data for that case.

1824

Enterprise Tic-Tac-Toe

let handleClick() =
match uiState with
| Idle -> ()
// do nothing
| Playing (gameState,lastMoveResult) ->
let gridIndex = 0,0 // dummy for now
let cellPos = createCellPosition gridIndex
match lastMoveResult with
| PlayerXToMove availableMoves ->
let playerXmove = PlayerXPos cellPos
// if move is in available moves then send it
// to the api
let newGameState,newResult =
api.playerXMoves gameState playerXmove
// handle the result
// e.g. if the game is over
handleMoveResult newResult
// update the uiState with newGameState
uiState <- Playing (newGameState,newResult)
| PlayerOToMove availableMoves ->
// etc
| _ ->
// ignore other states

If you look at the last line of the PlayerXToMove case, you can see the global uiState field
being updated with the new game state:
| PlayerXToMove availableMoves ->
// snipped
let newGameState,newResult = // get new state
// update the uiState with newGameState
uiState <- Playing (newGameState,newResult)

So where have we got to with this bit of prototyping?

It's pretty ugly, but it has served its purpose.
The goal was to quickly implement the UI to see if the design held up in use, and I think we
can say that it did, because the design of the domain types and api has remained
unchanged.
We also understand the UI requirements a bit better, which is a bonus. I think we can stop
now!
1825

Enterprise Tic-Tac-Toe

The complete game, part 1: The design

To finish off, I'll show the code for the complete game, including implementation and user
interface.
If you don't want to read this code, you can skip to the questions and summary below.
All the code shown is available on GitHub in this gist.
We'll start with our final domain design:

1826

Enterprise Tic-Tac-Toe

module TicTacToeDomain =
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom
type CellPosition = HorizPosition * VertPosition
type Player = PlayerO | PlayerX
type CellState =
| Played of Player
| Empty
type Cell = {
pos : CellPosition
state : CellState
}
type PlayerXPos = PlayerXPos of CellPosition
type PlayerOPos = PlayerOPos of CellPosition
type ValidMovesForPlayerX = PlayerXPos list
type ValidMovesForPlayerO = PlayerOPos list
type MoveResult =
| PlayerXToMove of ValidMovesForPlayerX
| PlayerOToMove of ValidMovesForPlayerO
| GameWon of Player
| GameTied
// the "use-cases"
type NewGame<'GameState> =
'GameState * MoveResult
type PlayerXMoves<'GameState> =
'GameState -> PlayerXPos -> 'GameState * MoveResult
type PlayerOMoves<'GameState> =
'GameState -> PlayerOPos -> 'GameState * MoveResult
// helper function
type GetCells<'GameState> =
'GameState -> Cell list
// the functions exported from the implementation
// for the UI to use.
type TicTacToeAPI<'GameState> =
{
newGame : NewGame<'GameState>
playerXMoves : PlayerXMoves<'GameState>
playerOMoves : PlayerOMoves<'GameState>
getCells : GetCells<'GameState>
}

1827

Enterprise Tic-Tac-Toe

The complete game, part 2: The game logic

implementation
Next, here's a complete implementation of the design which I will not discuss in detail. I hope
that the comments are self-explanatory.
module TicTacToeImplementation =
open TicTacToeDomain
/// private implementation of game state
type GameState = {
cells : Cell list
}
/// the list of all horizontal positions
let allHorizPositions = [Left; HCenter; Right]
/// the list of all horizontal positions
let allVertPositions = [Top; VCenter; Bottom]
/// A type to store the list of cell positions in a line
type Line = Line of CellPosition list
/// a list of the eight lines to check for 3 in a row
let linesToCheck =
let makeHLine v = Line [for h in allHorizPositions do yield (h,v)]
let hLines= [for v in allVertPositions do yield makeHLine v]
let makeVLine h = Line [for v in allVertPositions do yield (h,v)]
let vLines = [for h in allHorizPositions do yield makeVLine h]
let diagonalLine1 = Line [Left,Top; HCenter,VCenter; Right,Bottom]
let diagonalLine2 = Line [Left,Bottom; HCenter,VCenter; Right,Top]
// return all the lines to check
[
yield! hLines
yield! vLines
yield diagonalLine1
yield diagonalLine2
]
/// get the cells from the gameState
let getCells gameState =
gameState.cells
/// get the cell corresponding to the cell position
let getCell gameState posToFind =
gameState.cells
|> List.find (fun cell -> cell.pos = posToFind)

1828

Enterprise Tic-Tac-Toe

/// update a particular cell in the GameState

/// and return a new GameState
let private updateCell newCell gameState =
// create a helper function
let substituteNewCell oldCell =
if oldCell.pos = newCell.pos then
newCell
else
oldCell
// get a copy of the cells, with the new cell swapped in
let newCells = gameState.cells |> List.map substituteNewCell
// return a new game state with the new cells
{gameState with cells = newCells }
/// Return true if the game was won by the specified player
let private isGameWonBy player gameState =
// helper to check if a cell was played by a particular player
let cellWasPlayedBy playerToCompare cell =
match cell.state with
| Played player -> player = playerToCompare
| Empty -> false
// helper to see if every cell in the Line has been played by the same player
let lineIsAllSamePlayer player (Line cellPosList) =
cellPosList
|> List.map (getCell gameState)
|> List.forall (cellWasPlayedBy player)
linesToCheck
|> List.exists (lineIsAllSamePlayer player)

/// Return true if all cells have been played

let private isGameTied gameState =
// helper to check if a cell was played by any player
let cellWasPlayed cell =
match cell.state with
| Played _ -> true
| Empty -> false
gameState.cells
|> List.forall cellWasPlayed
/// determine the remaining moves for a player
let private remainingMovesForPlayer playerMove gameState =
// helper to return Some if a cell is playable
let playableCell cell =

1829

Enterprise Tic-Tac-Toe

match cell.state with

| Played player -> None
| Empty -> Some (playerMove cell.pos)
gameState.cells
|> List.choose playableCell

/// create the state of a new game

1830

Enterprise Tic-Tac-Toe

let newGameState = gameState |> updateCell newCell

if newGameState |> isGameWonBy PlayerO then
// return the new state and the move result
newGameState, GameWon PlayerO
elif newGameState |> isGameTied then
// return the new state and the move result
newGameState, GameTied
else
let remainingMoves =
newGameState |> remainingMovesForPlayer PlayerXPos
newGameState, PlayerXToMove remainingMoves
// Exercise - refactor to remove the duplicate code from
// playerXMoves and playerOMoves

/// export the API to the application

let api = {
newGame = newGame
playerOMoves = playerOMoves
playerXMoves = playerXMoves
getCells = getCells
}

The complete game, part 3: A console based

user interface
And to complete the implementation, here's the code for a console based user interface.
Obviously this part of the implementation is not pure! I'm writing to and reading from the
console, duh. If you want to be extra good, it would be easy enough to convert this to a pure
implementation using IO or similar.
Personally, I like to focus on the core domain logic being pure and I generally don't bother
about the UI too much, but that's just me.
/// Console based user interface
module ConsoleUi =
open TicTacToeDomain
/// Track the UI state
type UserAction<'a> =
| ContinuePlay of 'a
| ExitGame
/// Print each available move on the console
let displayAvailableMoves moves =

1831

Enterprise Tic-Tac-Toe

moves
|> List.iteri (fun i move ->
printfn "%i) %A" i move )
/// Get the move corresponding to the
/// index selected by the user
let getMove moveIndex moves =
if moveIndex < List.length moves then
let move = List.nth moves moveIndex
Some move
else
None
/// Given that the user has not quit, attempt to parse
/// the input text into a index and then find the move
/// corresponding to that index
let processMoveIndex inputStr gameState availableMoves makeMove processInputAgain
=
match Int32.TryParse inputStr with
// TryParse will output a tuple (parsed?,int)
| true,inputIndex ->
// parsed ok, now try to find the corresponding move
match getMove inputIndex availableMoves with
| Some move ->
// corresponding move found, so make a move
let moveResult = makeMove gameState move
ContinuePlay moveResult // return it
| None ->
// no corresponding move found
printfn "...No move found for inputIndex %i. Try again" inputIndex
// try again
processInputAgain()
| false, _ ->
// int was not parsed
printfn "...Please enter an int corresponding to a displayed move."

// try again
processInputAgain()
/// Ask the user for input. Process the string entered as
/// a move index or a "quit" command
let rec processInput gameState availableMoves makeMove =
// helper that calls this function again with exactly
// the same parameters
let processInputAgain() =
processInput gameState availableMoves makeMove
printfn "Enter an int corresponding to a displayed move or q to quit:"
let inputStr = Console.ReadLine()
if inputStr = "q" then
ExitGame
else

1832

Enterprise Tic-Tac-Toe

processMoveIndex inputStr gameState availableMoves makeMove processInputAg

ain
/// Display the cells on the console in a grid
let displayCells cells =
let cellToStr cell =
match cell.state with
| Empty -> "-"
| Played player ->
match player with
| PlayerO -> "O"
| PlayerX -> "X"
let printCells cells =
cells
|> List.map cellToStr
|> List.reduce (fun s1 s2 -> s1 + "|" + s2)
|> printfn "|%s|"
let topCells =
cells |> List.filter (fun cell -> snd cell.pos = Top)
let centerCells =
cells |> List.filter (fun cell -> snd cell.pos = VCenter)
let bottomCells =
cells |> List.filter (fun cell -> snd cell.pos = Bottom)
printCells topCells
printCells centerCells
printCells bottomCells
printfn "" // add some space
/// After each game is finished,
/// ask whether to play again.
let rec askToPlayAgain api =
printfn "Would you like to play again (y/n)?"
match Console.ReadLine() with
| "y" ->
ContinuePlay api.newGame
| "n" ->
ExitGame
| _ -> askToPlayAgain api
/// The main game loop, repeated
/// for each user input
let rec gameLoop api userAction =
printfn "\n------------------------------\n" // a separator between moves
match userAction with
| ExitGame ->
printfn "Exiting game."
| ContinuePlay (state,moveResult) ->
// first, update the display
state |> api.getCells |> displayCells

1833

Enterprise Tic-Tac-Toe

// then handle each case of the result

match moveResult with
| GameTied ->
printfn "GAME OVER - Tie"
printfn ""
let nextUserAction = askToPlayAgain api
gameLoop api nextUserAction
| GameWon player ->
printfn "GAME WON by %A" player
printfn ""
let nextUserAction = askToPlayAgain api
gameLoop api nextUserAction
| PlayerOToMove availableMoves ->
printfn "Player O to move"
displayAvailableMoves availableMoves
let newResult = processInput state availableMoves api.playerOMoves
gameLoop api newResult
| PlayerXToMove availableMoves ->
printfn "Player X to move"
displayAvailableMoves availableMoves
let newResult = processInput state availableMoves api.playerXMoves
gameLoop api newResult
/// start the game with the given API
let startGame api =
let userAction = ContinuePlay api.newGame
gameLoop api userAction

And finally, the application code that connects all the components together and launches the
UI:
module ConsoleApplication =
let startGame() =
let api = TicTacToeImplementation.api
ConsoleUi.startGame api

Example game
Here's what the output of this game looks like:
|-|X|-|
|X|-|-|
|O|-|-|
Player O to move
0) PlayerOPos (Left, Top)

1834

Enterprise Tic-Tac-Toe

1) PlayerOPos (HCenter, VCenter)

2) PlayerOPos (HCenter, Bottom)
3) PlayerOPos (Right, Top)
4) PlayerOPos (Right, VCenter)
5) PlayerOPos (Right, Bottom)
Enter an int corresponding to a displayed move or q to quit:
1
-----------------------------|-|X|-|
|X|O|-|
|O|-|-|
Player X to move
0) PlayerXPos (Left, Top)
1) PlayerXPos (HCenter, Bottom)
2) PlayerXPos (Right, Top)
3) PlayerXPos (Right, VCenter)
4) PlayerXPos (Right, Bottom)
Enter an int corresponding to a displayed move or q to quit:
1
-----------------------------|-|X|-|
|X|O|-|
|O|X|-|
Player O to move
0) PlayerOPos (Left, Top)
1) PlayerOPos (Right, Top)
2) PlayerOPos (Right, VCenter)
3) PlayerOPos (Right, Bottom)
Enter an int corresponding to a displayed move or q to quit:
1
-----------------------------|-|X|O|
|X|O|-|
|O|X|-|
GAME WON by PlayerO
Would you like to play again (y/n)?

Logging
Oops! We promised we would add logging to make it enterprise-ready!

1835

Enterprise Tic-Tac-Toe

That's easy -- all we have to do is replace the api functions with equivalent functions that log
the data we're interested in
module Logger =
open TicTacToeDomain
let logXMove (PlayerXPos cellPos)=
printfn "X played %A" cellPos
let logOMove (PlayerOPos cellPos)=
printfn "O played %A" cellPos
/// inject logging into the API
let injectLogging api =
// make a logged version of the game function
let playerXMoves state move =
logXMove move
api.playerXMoves state move
// make a logged version of the game function
let playerOMoves state move =
logOMove move
api.playerOMoves state move
// create a new API with
// the move functions replaced
// with logged versions
{ api with
playerXMoves = playerXMoves
playerOMoves = playerOMoves
}

Obviously, in a real system you'd replace it with a proper logging tool such as log4net and
generate better output, but I think this demonstrates the idea.
Now to use this, all we have to do is change the top level application to transform the original
api to a logged version of the api:
module ConsoleApplication =
let startGame() =
let api = TicTacToeImplementation.api
let loggedApi = Logger.injectLogging api
ConsoleUi.startGame loggedApi

And that's it. Logging done!

1836

Enterprise Tic-Tac-Toe

Oh, and remember that I originally had the initial state created as a function rather than as a
constant?
type InitGame = unit -> GameState

I changed to a constant early on in the design. But I'm regretting that now, because it means
that I can't hook into the "init game" event and log it. If I do want to log the start of each
game, I should really change it back to a function again.

Questions
Question: You went to the trouble of hiding the internal structure of GameState , yet
the PlayerXPos and PlayerOPos types are public. Why?
I forgot! And then laziness kept me from updating the code, since this is really just an
exercise in design.
It's true that in the current design, a malicious user interface could construct a PlayerXPos
and then play X when it is not player X's turn, or to play a position that has already been
played.
You could prevent this by hiding the implementation of PlayerXPos in the same way as we
did for game state, using a type parameter. And of course you'd have to tweak all the related
classes too.
Here's a snippet of what that would look like:
type MoveResult<'PlayerXPos,'PlayerOPos> =
| PlayerXToMove of 'PlayerXPos list
| PlayerOToMove of 'PlayerOPos list
| GameWon of Player
| GameTied
type NewGame<'GameState,'PlayerXPos,'PlayerOPos> =
'GameState * MoveResult<'PlayerXPos,'PlayerOPos>
type PlayerXMoves<'GameState,'PlayerXPos,'PlayerOPos> =
'GameState -> 'PlayerXPos ->
'GameState * MoveResult<'PlayerXPos,'PlayerOPos>
type PlayerOMoves<'GameState,'PlayerXPos,'PlayerOPos> =
'GameState -> 'PlayerOPos ->
'GameState * MoveResult<'PlayerXPos,'PlayerOPos>

1837

Enterprise Tic-Tac-Toe

We'd also need a way for the UI to see if the CellPosition a user selected was valid. That
is, given a MoveResult and the desired CellPosition , if the position is valid, return Some
move, otherwise return None .
type GetValidXPos<'PlayerXPos,'PlayerOPos> =
CellPosition * MoveResult<'PlayerXPos,'PlayerOPos> -> 'PlayerXPos option

It's getting kind of ugly now, though. That's one problem with type-first design: the type
parameters can get complicated!
So it's a trade-off. How much do you use types to prevent accidental bugs without
overwhelming the design?
In this case, I do think the GameState should be secret, as it is likely to change in the future
and we want to ensure that the UI is not accidentally coupled to implementation details.
For the move types though, (a) I don't see the implementation changing and (b) the
consequence of a malicious UI action is not very high, so overall I don't mind having the
implementation be public.
UPDATE 2015-02-16: In the next post I solve this problem in a more elegant way, and get rid
of GameState as well!
Question: Why are you using that strange syntax for defining the initGame and move
functions?
You mean, why I am defining the functions like this:
/// create the state of a new game
let newGame : NewGame<GameState> =
// implementation
let playerXMoves : PlayerXMoves<GameState> =
fun (gameState,move) ->
// implementation

rather than in the "normal" way like this:

/// create the state of a new game
let newGame =
// implementation
let playerXMoves (gameState,move) =
// implementation

1838

Enterprise Tic-Tac-Toe

I'm doing this when I want to treat functions as values. Just as we might say "x is a value of
type int" like this x :int = ... , I'm saying that "playerXMoves is a value of type
PlayerXMoves" like this: playerXMoves : PlayerXMoves = ... . It's just that in this case, the
value is a function rather than a simple value.
Doing it this way follows from the type-first approach: create a type, then implement things
that conform to that type.
Would I recommend doing this for normal code? Probably not!
I'm only doing this as part of an exploratory design process. Once the design stabilizes, I
would tend to switch back to the normal way of doing things.
Question: This seems like a lot of work. Isn't this just BDUF under another guise?
This might seem like quite a long winded way of doing design, but in practice, it would
probably not take very long. Certainly no longer than mocking up an exploratory prototype in
another language.
We've gone through a number of quick iterations, using types to document the design, and
using the REPL as a "executable spec checker" to make sure that it all works together
properly.
And at the end of all this, we now have a decent design with some nice properties:
There is a "API" that separates the UI from the core logic, so that work on each part can
proceed in parallel if needed.
The types act as documentation and will constrain the implementation in a way that
UML diagrams could never do!
The design is encoded in types, so that any the inevitable changes that occur during
development can be made with confidence.
I think this whole process is actually pretty agile, once you get used to working this way.
Question: Come on, would you really write Tic-Tac-Toe this way?
It depends. If it was just me, maybe not. :-)
But if it was a more complex system with different teams for the front-end and back-end,
then I would certainly use a design-first approach like this. In cases like that, things like datahiding and abstract interfaces are critical, and I think this approach delivers that.
Question: Why is the design so specific? It seems like none of it will be reusable at
all. Why not?
Yes, this code is full of very specific types: Cell , GameState , etc. And it's true that none of
it will be reusable.

1839

Enterprise Tic-Tac-Toe

There is always a tension between a very domain-specific and non-reusable design, like this
one, and an abstract and reusable library of things like lists and trees.
Ideally, you would start with low-level, reusable components and then compose them into
larger more-specific ones (e.g. a DSL), from which you can build a application. (Tomas has a
good post on exactly this).
The reasons why I did not do that here is that, first, I always like to start with very concrete
designs. You can't even know what a good abstraction looks like until you have built
something a few times.
We have separated the UI from the core logic, but going any further than that does not make
sense to me right now. If I was going to build lots of other kinds of games that were similar to
Tic-Tac-Toe, then some useful abstractions might become apparent.
Second, designs with concrete types are easier for non-experts to understand. I'd like to
think that I could show these domain types to a non-programmer (e.g. a domain expert) and
have them understand and comment sensibly on them. If they were more abstract, that
would not be possible.

Exercises
If you want a challenge, here are some exercises for you:
The playerXMoves and playerOMoves functions have very similar code. How would you
refactor them to reduce that?
Do a security audit and think of all the ways that a malicious user or UI could corrupt the
game using the current design. Then fix them!

Summary
In this post, we've seen how to design a system using mostly types, with the occasional
code fragments to help us clarify issues.
It was definitely an exercise in design overkill but I hope that there are some ideas in there
that might be applicable to real non-toy projects.
At the start, I claimed that this design would be "enterprise" ready. Is it?
We do have separation of concerns via the functions that are exported to the UI.
We do have a well documented API. There are no magic numbers, the names of the
types are self-documenting, and the list of functions exported is in one place.
We do have a security model to prevent unauthorized actions from occurring. As it
1840

Enterprise Tic-Tac-Toe

stands, it would be hard to accidentally mess up. And if we go the extra distance by
parameterizing the move types as well, then it becomes really quite hard for the game
to be corrupted.
We do have well-documented code, I think. Even though this is "enterprise", the code is
quite explicit in what it does. There are no wasted abstractions -- no
AbstractSingletonProxyFactoryBean to make fun of.

We did add auditing and logging easily, and in an elegant way after the fact, without
interfering with the core design.
We get scalability for free because there is no global session data. All we have to do is
persist the game state in the browser (Or we could use MongoDb and be web scale).
This is not a perfect design -- I can think of a number of ways to improve it -- but overall I'm
quite happy with it, considering it was a straight brain-to-code dump.
What do you think? Let me know in the comments.
UPDATE 2015-02-16: I ended up being unhappy with this design after all. In the next
post I tell you why, and present a better design.
NOTE: The code for this post is available on GitHub in this gist.

1841

Enterprise Tic-Tac-Toe, part 2

UPDATE: Slides and video from my talk on this topic
This post is one of series in I which I hope to close the gap between theory and practice in
functional programming. I pick a small project and show you my thought processes as I go
about designing and implementing it from beginning to end.
In the previous post, I did a design for a Tic-Tac-Toe (aka Noughts and Crosses) game.
It wasn't bad for a direct-to-code brain dump, but there were a couple of things that I wasn't
happy with.
Unfortunately, the more I thought about it, the more those little niggles became full-fledged
annoyances, and I got unhappier and unhappier.
In this post, I'll explain why I was so unhappy, and how I arrived at a design that I am much
more satisfied with.

The old design

To recap the previous post briefly, here is the old design:
There is a hidden GameState known only to the implementation.
There are some functions that allow the players to move ( PlayerXMoves and
PlayerOMoves ).

The UI (or other client) passes the game state into each move, and gets a updated
game state back.
Each move also returns a MoveResult which contains the game status (in process,
won, tied), and if the game is still in process, whose turn it is, and what the available
moves are.
Here's the code:

1842

Enterprise Tic-Tac-Toe, part 2

What's wrong with the old design?

So what's wrong with this design? Why was I so unhappy?
First, I was unhappy about the use of the PlayerXPos and PlayerOPos types. The idea was
to wrap a CellPosition in a type so that it would be "owned" by a particular player. By doing
this, and then having the valid moves be one of these types, I could prevent player X from
playing twice, say. That is, after player X has moved, the valid moves for the next run would
be wrapped in a PlayerOPos type so that only player O could use them.

1843

Enterprise Tic-Tac-Toe, part 2

The problem was that the PlayerXPos and PlayerOPos types are public, so that a malicious
user could have forged one and played twice anyway!
Yes, these types could have been made private by parameterizing them like the game state,
but the design would have become very ugly very quickly.
Second, even if the moves were made unforgeable, there's that game state floating about.
It's true that the game state internals are private, but a malicious user could have still caused
problems by reusing a game state. For example, they could attempt to play one of the valid
moves with a game state from a previous turn, or vice versa.
In this particular case, it would not be dangerous, but in general it might be a problem.
So, as you can see, this design was becoming a bit smelly to me, which was why I was
becoming unhappy.

What's up with this malicious user?

Why I am assuming that the user of the API will be so malicious -- forging fake moves and
all that?
The reason is that I use this as a design guideline. If a malicious user can do something I
don't want, then the design is probably not good enough.
In my series on capability based security I point out that by designing for the Principle Of
Least Authority ("POLA"), you end up with a good design as a side-effect.
That is, if you design the most minimal interface that the caller needs, then you will both
avoid accidental complexity (good design) and increase security (POLA).
I had a little tip in that post: design for malicious callers and you will probably end up
with more modular code.
I think I will follow my own advice and see where I end up!

Designing for POLA

So, let's design for POLA -- let's give the user the minimal "capability" to do something and
no more.
In this case, I want to give the user the capability to mark a specific position with an "X" or
"O".
Here's what I had before:
1844

Enterprise Tic-Tac-Toe, part 2

type PlayerXMoves =
GameState * PlayerXPos -> // input
GameState * MoveResult // output

The user is passing in the location ( PlayerXPos ) that they want to play.
But let's now take away the user's ability to choose the position. Why don't I give the user a
function, a MoveCapability say, that has the position baked in?
type MoveCapability =
GameState -> // input
GameState * MoveResult // output

In fact, why not bake the game state into the function too? That way a malicious user can't
pass the wrong game state to me.
This means that there is no "input" at all now -- everything is baked in!
type MoveCapability =
unit -> // no input
GameState * MoveResult // output

But now we have to give the user a whole set of capabilities, one for each possible move
they can make. Where do these capabilities come from?
Answer, the MoveResult of course! We'll change the MoveResult to return a list of
capabilities rather than a list of positions.
type MoveResult =
| PlayerXToMove of MoveCapability list
| PlayerOToMove of MoveCapability list
| GameWon of Player
| GameTied

Excellent! I'm much happier with this approach.

And now that the MoveCapability contains the game state baked in, we don't need the
game state to be in the output either!
So our move function has simplified dramatically and now looks like this:
type MoveCapability =
unit -> MoveResult

1845

Enterprise Tic-Tac-Toe, part 2

Look ma! No 'GameState parameter! It's gone!

A quick walkthrough from the UI's point of view

So now let's pretend we are the UI, and let's attempt to use the new design.
First, assume that we have a list of available capabilities from the previous move.
Next, the user must pick one of the capabilities (e.g. squares) to play -- they can't just
create any old cell position and play it, which is good. But how will the user know which
capability corresponds to which square? The capabilities are completely opaque. We
can't tell from the outside what they do!
Then, given that the user has picked a capability somehow, we run it (with no
parameters).
Next we update the display to show the result of the move. But again, how are we going
to know what to display? There is no game state to extract the cells from any longer.
Here's some pseudo-code for the UI game loop:
// loop while game not over
let rec playMove moveResult =
let availableCapabilities = // from moveResult
// get capability from user input somehow
let capability = ??
// use the capability
let newMoveResult = capability()
// display updated grid
let cells = ?? // from where
// play again
match newMoveResult with
| PlayerXToMove capabilities ->
// play another move
playMove newMoveResult
| etc

Let's deal with the first issue: how does the user know which capability is associated with
which square?
The answer is just to create a new structure that "labels" the capability. In this case, with the
cell position.

1846

Enterprise Tic-Tac-Toe, part 2

type NextMoveInfo = {
posToPlay : CellPosition
capability : MoveCapability }

And now we must change the MoveResult to return a list of these labelled capabilities,
rather than the unlabelled ones:
type MoveResult =
| PlayerXToMove of NextMoveInfo list
| PlayerOToMove of NextMoveInfo list
| GameWon of Player
| GameTied

Note that the cell position is for the user's information only -- the actual position is still baked
into the capability and cannot be forged.
Now for the second issue: how does the UI know what to display as a result of the move?
Let's just return that information to it directly in a new structure:
/// Everything the UI needs to know to display the board
type DisplayInfo = {
cells : Cell list
}

And once again, the MoveResult must be changed, this time to return the DisplayInfo for
each case:
type MoveResult =
| PlayerXToMove of DisplayInfo * NextMoveInfo list
| PlayerOToMove of DisplayInfo * NextMoveInfo list
| GameWon of DisplayInfo * Player
| GameTied of DisplayInfo

Dealing with circular dependencies

Here's our final design:

1847

Enterprise Tic-Tac-Toe, part 2

/// The capability to make a move at a particular location.

/// The gamestate, player and position are already "baked" into the function.
type MoveCapability =
unit -> MoveResult
/// A capability along with the position the capability is associated with.
/// This allows the UI to show information so that the user
/// can pick a particular capability to exercise.
type NextMoveInfo = {
// the pos is for UI information only
// the actual pos is baked into the cap.
posToPlay : CellPosition
capability : MoveCapability }
/// The result of a move. It includes:
/// * The information on the current board state.
/// * The capabilities for the next move, if any.
type MoveResult =
| PlayerXToMove of DisplayInfo * NextMoveInfo list
| PlayerOToMove of DisplayInfo * NextMoveInfo list
| GameWon of DisplayInfo * Player
| GameTied of DisplayInfo

But oops! This won't compile!

MoveCapability depends on MoveResult which depends on NextMoveInfo which in turn

depends on MoveCapability again. But the F# compiler does not allow forward references
in general.
Circular dependencies like this are generally frowned upon (I even have a post called "cyclic
dependencies are evil"!) and there are normally work-arounds which you can use to remove
them.
In this case though, I will link them together using the and keyword, which replaces the
type keyword and is useful for just these kinds of cases.

type MoveCapability =
// etc
and NextMoveInfo = {
// etc
and MoveResult =
// etc

Revisiting the API

What does the API look like now?

1848

Enterprise Tic-Tac-Toe, part 2

Originally, we had an API with slots for the three use-cases and also a helper function
getCells :

type TicTacToeAPI<'GameState> =
{
newGame : NewGame<'GameState>
playerXMoves : PlayerXMoves<'GameState>
playerOMoves : PlayerOMoves<'GameState>
getCells : GetCells<'GameState>
}

But now, we don't need the playerXMoves or playerOMoves , because they are returned to us
in the MoveResult of a previous move.
And getCells is no longer needed either, because we are returning the DisplayInfo
directly now.
So after all these changes, the new API just has a single slot in it and looks like this:
type NewGame = unit -> MoveResult
type TicTacToeAPI =
{
newGame : NewGame
}

I've changed NewGame from a constant to a parameterless function, which is in fact, just a
MoveCapability in disguise.

The new design in full

Here's the new design in full:

1849

Enterprise Tic-Tac-Toe, part 2

module TicTacToeDomain =
type HorizPosition = Left | HCenter | Right
type VertPosition = Top | VCenter | Bottom
type CellPosition = HorizPosition * VertPosition
type Player = PlayerO | PlayerX
type CellState =
| Played of Player
| Empty
type Cell = {
pos : CellPosition
state : CellState
}
/// Everything the UI needs to know to display the board
type DisplayInfo = {
cells : Cell list
}
/// The capability to make a move at a particular location.
/// The gamestate, player and position are already "baked" into the function.
type MoveCapability =
unit -> MoveResult
/// A capability along with the position the capability is associated with.
/// This allows the UI to show information so that the user
/// can pick a particular capability to exercise.
and NextMoveInfo = {
// the pos is for UI information only
// the actual pos is baked into the cap.
posToPlay : CellPosition
capability : MoveCapability }
/// The result of a move. It includes:
/// * The information on the current board state.
/// * The capabilities for the next move, if any.
and MoveResult =
| PlayerXToMove of DisplayInfo * NextMoveInfo list
| PlayerOToMove of DisplayInfo * NextMoveInfo list
| GameWon of DisplayInfo * Player
| GameTied of DisplayInfo
// Only the newGame function is exported from the implementation
// all other functions come from the results of the previous move
type TicTacToeAPI =
{
newGame : MoveCapability
}

1850

Enterprise Tic-Tac-Toe, part 2

I'm much happier with this design than with the previous one:
There is no game state for the UI to worry about.
There are no type parameters to make it look ugly.
The api is even more encapsulated -- a malicious UI can do very little now.
It's shorter -- always a good sign!

The complete application

I have updated the implementation and console application to use this new design.
The complete application is available on GitHub in this gist if you want to play with it.
Surprisingly, the implementation also has become slightly simpler, because all the state is
now hidden and there is no need to deal with types like PlayerXPos any more.

Logging revisited
In the previous post, I demonstrated how logging could be injected into the API.
But in this design, the capabilities are opaque and have no parameters, so how are we
supposed to log that a particular player chose a particular location?
Well, we can't log the capabilities, but we can log their context, which we have via the
NextMoveInfo . Let's see how this works in practice.

First, given a MoveCapability , we want to transform it into another MoveCapability that also
logs the player and cell position used.
Here's the code for that:
/// Transform a MoveCapability into a logged version
let transformCapability transformMR player cellPos (cap:MoveCapability) :MoveCapabilit
y =
// create a new capability that logs the player & cellPos when run
let newCap() =
printfn "LOGINFO: %A played %A" player cellPos
let moveResult = cap()
transformMR moveResult
newCap

This code works as follows:

Create a new capability newCap function that is parameterless and returns a

1851

Enterprise Tic-Tac-Toe, part 2

MoveResult just like the original one.

When it is called, log the player and cell position. These are not available from the
MoveCapability that was passed in, so we have to pass them in explicitly.

Next, call the original capability and get the result.

The result itself contains the capabilities for the next move, so we need to recursively
transform each capability in the MoveResult and return a new MoveResult . This is done
by the transformMR function that is passed in.
Now that we can transform a MoveCapability , we can go up a level and transform a
NextMoveInfo .

/// Transform a NextMove into a logged version

let transformNextMove transformMR player (move:NextMoveInfo) :NextMoveInfo =
let cellPos = move.posToPlay
let cap = move.capability
{move with capability = transformCapability transformMR player cellPos cap}

This code works as follows:

Given a NextMoveInfo , replace its capability with a transformed one. The output of
transformNextMove is a new NextMoveInfo .

The cellPos comes from the original move.

The player and transformMR function are not available from the move, so must be
passed in explicitly again.
Finally, we need to implement the function that will transform a MoveResult :
/// Transform a MoveResult into a logged version
let rec transformMoveResult (moveResult:MoveResult) :MoveResult =
let tmr = transformMoveResult // abbreviate!
match moveResult with
| PlayerXToMove (display,nextMoves) ->
let nextMoves' = nextMoves |> List.map (transformNextMove tmr PlayerX)
PlayerXToMove (display,nextMoves')
| PlayerOToMove (display,nextMoves) ->
let nextMoves' = nextMoves |> List.map (transformNextMove tmr PlayerO)
PlayerOToMove (display,nextMoves')
| GameWon (display,player) ->
printfn "LOGINFO: Game won by %A" player
moveResult
| GameTied display ->
printfn "LOGINFO: Game tied"
moveResult

1852

Enterprise Tic-Tac-Toe, part 2

This code works as follows:

Given a MoveResult , handle each case. The output is a new MoveResult .
For the GameWon and GameTied cases, log the result and return the original
moveResult.
For the PlayerXToMove case, take each of the NextMoveInfo s and transform them,
passing in the required player ( PlayerX ) and transformMR function. Note that the
transformMR function is a reference to this very function! This means that
transformMoveResult must be marked with rec to allow this self-reference.

For the PlayerOToMove case, do the same as the PlayerXToMove case, except change
the player to PlayerO .
Finally, we can inject logging into the API as a whole by transforming the MoveResult
returned by newGame :
/// inject logging into the API
let injectLogging api =
// create a new API with the functions
// replaced with logged versions
{ api with
newGame = fun () -> api.newGame() |> transformMoveResult
}

So there you go. Logging is a bit trickier than before, but still possible.

A warning on recursion
In this code, I've been passing around functions that call each other recursively. When you
do this, you have to be careful that you don't unwittingly cause a stack overflow.
In a game like this, when the number of nested calls is guaranteed to be small, then there is
no issue. But if you are doing tens of thousands of nested calls, then you should worry about
potential problems.
In some cases, the F# compiler will do tail-call optimization, but I suggest that you stress test
your code to be sure!

Data-centric vs capability-centric designs

There is an interesting difference between the original design and the new design.

1853

Enterprise Tic-Tac-Toe, part 2

The original design was data-centric. Yes, we gave each player a function to use, but it was
the same function used over and over, with different data passed in each time.
The new design is function-centric (or as I prefer, capability-centric). There is very little data
now. Instead, the result of each function call is another set of functions than can be used for
the next step, and so on, ad infinitum.
In fact, it reminds me somewhat of a continuation-based approach, except that rather than
passing in a continuation, the function itself returns a list of continuations, and then you pick
one to use.

Capabilities and RESTful designs -- a match

made in heaven
If for some crazy reason you wanted to turn this design into a web service, how would you
go about doing that?
In a data-centric design, we have a function to call (an endpoint URI in the web API) and
then we pass data to it (as JSON or XML). The result of the call is more data that we use to
update the display (e.g. the DOM).
But in a capability-centric design, where's the data? And how do we pass functions around?
It seems like this approach would not work for web services at all.
It might surprise you to know that there is a way to do this, and what's more it is exactly the
same approach used by a RESTful design using HATEOAS.
What happens is that each capability is mapped to a URI by the server, and then visiting that
URI is the same as exercising that capability (e.g. calling the function).
For example, in a web-based application based on this Tic-Tac-Toe design, the server would
initially return nine URIs, one for each square. Then, when one of those squares was
clicked, and the associated URI visited, the server would return eight new URIs, one for
each remaining unplayed square. The URI for the just-played square would not be in this list,
which means that it could not be clicked on again.
And of course when you click on one of the eight unplayed squares, the server would now
return seven new URIs, and so on.
This model is exactly what REST is supposed to be; you decide what to do next based on
the contents of the returned page rather than hard-code the endpoints into your app.
One possible downside of this approach is that it is does not appear to be stateless.

1854

Enterprise Tic-Tac-Toe, part 2

In the data-centric version, all the data needed for a move was passed in each time,
which means that scaling the backend services would be trivial.
In capability-centric approach though, the state has to be stored somewhere. If the
complete game state can be encoded into the URI, then this approach will allow
stateless servers as well, but otherwise, some sort of state-storage will be needed.
Some web frameworks have made this function-centred approach a key part of their design,
most notably Seaside.
The excellent WebSharper framework for F# also uses something similar, I think (I don't
know WebSharper as well as I want to, alas, so correct me if I'm wrong).

Summary
In this post, I tore up my original design and replaced it with an even more function-centric
one, which I like better.
But of course it still has all those qualities we love: separation of concerns, an API, a
watertight security model, self-documenting code, and logging.
I'm going to stop with Tic-Tac-Toe now -- I think I've wrung it dry! I hope you found these two
walkthoughs interesting; I learned a lot myself.
NOTE: The code for this post is available on GitHub in this gist.

1855

Ten reasons not to use a statically typed functional programming language

Ten reasons not to use a statically typed

functional programming language
Are you fed up with all the hype about functional programming? Me too! I thought I'd rant
about some reasons why sensible people like us should stay away from it.
Just to be clear, when I say "statically typed functional programming language", I mean
languages that also include things such as type inference, immutability by default, and so
on. In practice, this means Haskell and the ML-family (including OCaml and F#).

Reason 1: I don't want to follow the latest fad

Like most programmers, I'm naturally conservative and I dislike learning new things. That's
why I picked a career in IT.
I don't jump on the latest bandwagon just because all the "cool kids" are doing it -- I wait until
things have matured and I can get some perspective.
To me, functional programming just hasn't been around long enough to convince me that it is
here to stay.
Yes, I suppose some pedants will claim that ML) and Haskell) have been around almost as
long as old favorites like Java and PHP, but I only heard of Haskell recently, so that
argument doesn't wash with me.
And look at the baby of the bunch, F#. It's only seven years old, for Pete's sake! Sure, that
may be a long time to a geologist, but in internet time, seven years is just the blink of an eye.
So, all told, I would definitely take the cautious approach and wait a few decades to see if
this functional programming thing sticks around or whether it is just a flash in the pan.

Reason 2: I get paid by the line

I don't know about you, but the more lines of code I write, the more productive I feel. If I can
churn out 500 lines of code in a day, that's a job well done. My commits are big, and my
boss can see that I've been busy.
But when I compare code written in a functional language with a good old C-like language,
there's so much less code that it scares me.
I mean, just look at this code written in a familiar language:

1856

Ten reasons not to use a statically typed functional programming language

public static class SumOfSquaresHelper

{
public static int Square(int i)
{
return i * i;
}
public static int SumOfSquares(int n)
{
int sum = 0;
for (int i = 1; i <= n; i++)
{
sum += Square(i);
}
return sum;
}
}

and compare it with this:

let square x = x * x
let sumOfSquares n = [1..n] |> List.map square |> List.sum

That's 17 lines vs. only 2 lines. Imagine that difference multiplied over a whole project!
If I did use this approach, my productivity would drop drastically. I'm sorry -- I just can't afford
it.

Reason 3: I love me some curly braces

And that's another thing. What's up with all these languages that get rid of curly braces. How
can they call themselves real programming languages?
I'll show you what I mean. Here's a code sample with familiar curly braces.

1857

Ten reasons not to use a statically typed functional programming language

public class Squarer

{
public int Square(int input)
{
var result = input * input;
return result;
}
public void PrintSquare(int input)
{
var result = this.Square(input);
Console.WriteLine("Input={0}. Result={1}", input, result);
}
}

And here's some similar code, but without curly braces.

type Squarer() =
let Square input =
let result = input * input
result
let PrintSquare input =
let result = Square input
printf "Input=%i. Result=%i" input result

Look at the difference! I don't know about you, but I find the second example a bit disturbing,
as if something important is missing.
To be honest, I feel a bit lost without the guidance that curly braces give me.

Reason 4: I like to see explicit types

Proponents of functional languages claim that type inference makes the code cleaner
because you don't have to clutter your code with type declarations all the time.
Well, as it happens, I like to see type declarations. I feel uncomfortable if I don't know the
exact type of every parameter. That's why Java is my favorite language.
Here's a function signature for some ML-ish code. There are no type declarations needed
and all types are inferred automatically.
let GroupBy source keySelector =
...

1858

Ten reasons not to use a statically typed functional programming language

And here's the function signature for similar code in C#, with explicit type declarations.
public IEnumerable<IGrouping<TKey, TSource>> GroupBy<TSource, TKey>(
IEnumerable<TSource> source,
Func<TSource, TKey> keySelector
)
...

I may be in the minority here, but I like the second version much better. It's important to me
to know that the return is of type IEnumerable<IGrouping<TKey, TSource>> .
Sure, the compiler will type check this for you and warn you if there is a type mismatch. But
why let the compiler do the work when your brain can do it instead?
Ok, I admit that if you do use generics, and lambdas, and functions that return functions, and
all the other newfangled stuff, then yes, your type declarations can get really hairy and
complex. And it gets really hard to type them properly.
But I have an easy fix for that -- don't use generics and don't pass around functions. Your
signatures will be much simpler.

Reason 5: I like to fix bugs

To me, there's nothing quite like the thrill of the hunt -- finding and killing a nasty bug. And if
the bug is in a production system, even better, because I'll be a hero as well.
But I've read that in statically typed functional languages, it is much harder to introduce
bugs.
That's a bummer.

Reason 6: I live in the debugger

And talking of bug fixing, I spend most of my day in the debugger, stepping through code.
Yes, I know I should be using unit tests, but easier said than done, OK?
Anyway, apparently with these statically typed functional languages, if your code compiles, it
usually works.
I'm told that you do have to spend a lot of time up front getting the types to match up, but
once that is done and it compiles successfully, there is nothing to debug. Where's the fun in
that?

1859

Ten reasons not to use a statically typed functional programming language

Which brings me to...

Reason 7: I don't want to think about every

little detail
All this matching up types and making sure everything is perfect sounds tiring to me.
In fact, I hear that you are forced to think about all the possible edge cases, and all the
possible error conditions, and every other thing that could go wrong. And you have to do this
at the beginning -- you can't be lazy and postpone it till later.
I'd much rather get everything (mostly) working for the happy path, and then fix bugs as they
come up.

Reason 8: I like to check for nulls

I'm very conscientious about checking for nulls on every method. It gives me great
satisfaction to know that my code is completely bulletproof as a result.
void someMethod(SomeClass x)
{
if (x == null) { throw new NullArgumentException(); }
x.doSomething();
}

Haha! Just kidding! Of course I can't be bothered to put null-checking code everywhere. I'd
never get any real work done.
But I've only ever had to deal with one bad crash caused by a NPE. And the business didn't
lose too much money during the few weeks I spent looking for the problem. So I'm not sure
why this is such a big deal.

Reason 9: I like to use design patterns

everywhere
I first read about design patterns in the Design Patterns book (for some reason it's referred
to as the Gang of Four book, but I'm not sure why), and since then I have been diligent in
using them at all times for all problems. It certainly makes my code look serious and
"enterprise-y", and it impresses my boss.

1860

Ten reasons not to use a statically typed functional programming language

But I don't see any mention of patterns in functional design. How can you get useful stuff
done without Strategy, AbstractFactory, Decorator, Proxy, and so on?
Perhaps the functional programmers are not aware of them?

Reason 10: It's too mathematical

Here's some more code for calculating the sum of squares. This is way too hard to
understand because of all the weird symbols in it.
ss=: +/ @: *:

Oops, sorry! My mistake. That was J code).

But I do hear that functional programs use strange symbols like <*> and >>= and obscure
concepts called "monads" and "functors".
I don't know why the functional people couldn't stick with things I already know -- obvious
symbols like ++ and != and easy concepts such as "inheritance" and "polymorphism".

Summary: I don't get it

You know what. I don't get it. I don't get why functional programming is useful.
What I'd really like is for someone to just show me some real benefits on a single page,
instead of giving me too much information.
UPDATE: So now I've read the "everything you need to know on one page" page. But it's too
short and simplistic for me.
I'm really looking for something with a bit more depth -- something I can get my teeth into.
And no, don't say that I should read tutorials, and play with examples, and write my own
code. I just want to grok it without doing all of that work.
I don't want to have to change the way I think just to learn a new paradigm.

1861

Why I won't be writing a monad tutorial

"A ?newbie', in Haskell, is someone who hasn't yet implemented a compiler. They've only
written a monad tutorial" - Pseudonymn
Let's start with a story...

Alice learns to count

Young Alice and her father (who happens to be a mathematician) are visiting a petting zoo...
Alice: Look at those kitties.

Daddy: Aren't they cute. There are two of them.

Alice: Look at those doggies.

1862

Why I won't be writing a monad tutorial

Daddy: That's right. Can you count? There are two doggies.
Alice: Look at those horsies.

Daddy: Yes darling. Do you know what the kitties and doggies and horsies all have in
common?
Alice: No. Nothing in common!
Daddy: Well, actually they do have something in common. Can you see what it is?

1863

Why I won't be writing a monad tutorial

Alice: No! A doggy is not a kitty. A horsie is not a kitty.

Daddy: How about I explain for you? First, let us consider a set S which is strictly wellordered with respect to set membership and where every element of S is also a subset of S.
Does that give you a clue?
Alice: [Bursts into tears]

How not to win friends and influence people

No (sensible) parent would ever try to explain how to count by starting with a formal
definition of ordinal numbers.
So why is it that many people feel compelled to explain a concept like monads by
emphasizing their formal definition?
That might be fine for a college-level math class, but it plainly does not work for regular
programmers, who just want to create something useful.
As an unfortunate result of this approach, though, there is now a whole mystique around the
concept of monads. It has become a bridge you must cross on the way to true
enlightenment. And there are, of course, a plethora of monad tutorials to help you cross it.
Here's the truth: You don't need to understand monads to write useful functional code. This
is especially true for F# compared to say, Haskell.
Monads are not a golden hammer. They won't make you any more productive. They won't
make your code less buggy.
So really, don't worry about them.

Why I won't be writing a monad tutorial

So this is why I won't be writing a monad tutorial. I don't think it will help people learn about
functional programming. If anything, it just creates confusion and anxiety.
Yes, I will use examples of monads in many different posts, but, other than right here, I will
try to avoid using the word "monad" anywhere on this site. In fact, it has pride of place on my
list of banned words!

Why you should write a monad tutorial

On the other hand, I do think that you should write a monad tutorial. When you try to explain
something to somebody else, you end up understanding it better yourself.

1864

Why I won't be writing a monad tutorial

Here's the process I think you should follow:

1. First, write lots of practical code involving lists, sequences, options, async workflows,
computation expressions, etc.
2. As you become more experienced, you will start to use more abstractions, focusing on
the shapes of things rather than the details.
3. At some point, you will have an aha! moment -- a sudden insight that all the abstractions
have something in common.
4. Bingo! Time to write your monad tutorial!
The key point is that you have to do it in this order -- you cannot jump straight to the last step
and then work backwards. It is the very act of working your way through the details that
enables you to understand the abstraction when you see it.
Good luck with your tutorial -- I'm off to eat a burrito.

1865

Is your programming language unreasonable?

Is your programming language

unreasonable?
As should be obvious, one of the goals of this site is to persuade people to take F# seriously
as a general purpose development language.
But as functional idioms have become more mainstream, and C# has added functional
capabilities such as lambdas and LINQ, it seems like C# is "catching up" with F# more and
more.
So, ironically, I've now started to hear people say things like this:
"C# already has most of the features of F#, so why should I bother to switch?"*
"There is no need to change. All we have do is wait a couple of years and C# will get
many of the F# features that provide the most benefits."
"F# is slightly better than C#, but not so much that it's really worth the effort to move
towards it."
"F# seems really nice, even if it's a bit intimidating. But I can't see a practical purpose to
use it over C#."
No doubt, the same comments are being made in the JVM ecosystem about Scala and
Clojure vs. Java, now that Java has lambdas too.
So for this post, I'm going to stray away from F#, and focus on C# (and by proxy, other
mainstream languages), and try to demonstrate that, even with all the functional features in
the world, programming in C# will never be the same as programming in F#.
Before I start, I want to make it clear that I am not hating on C#. As it happens I like C# very
much; it is one of my favorite mainstream languages, and it has evolved to be very powerful
while being consistent and backwards compatible, which is a hard thing to pull off.
But C# is not perfect. Like most mainstream OO languages, it contains some design
decisions which no amount of LINQ or lambda goodness can compensate for.
In this post, I'll show you some of the issues that these design decisions cause, and suggest
some ways to improve the language to avoid them.
(I'm now going to don my flameproof suit. I think I might need it!)

UPDATE: Many people have seriously misread this post, it seems. So let me be clear:

1866

Is your programming language unreasonable?

I am not saying that statically typed languages are "better" than dynamic languages.
I am not saying that FP languages are "better" than OO languages.
I am not saying that being able to reason about code is the most important aspect of a
language.
What I am saying is:
Not being able to reason about code has costs that many developers might not be
aware of.
Therefore, being "reasonable" should be one of the (many) factors under consideration
when choosing a programming language, not just ignored due to lack of awareness.
IF you want to be able to reason about your code, THEN it will be much easier if your
language supports the features that I mention.
The fundamental paradigm of OO (object-identity, behavior-based) is not compatible
with "reasonability", and so it will be hard to retrofit existing OO languages to add this
quality.
That's it. Thank you!

What is a "reasonable" programming language,

anyway?
If you hang around functional programmers, you will often hear the phrase "reason about",
as in "we want to reason about our programs".
What does that mean? Why use the word "reason" rather than just "understand"?
The use of "reasoning" goes back to mathematics and logic, but I'm going to use a simple
and pragmatic definition:
"reasoning about the code" means that you can draw conclusions using only the
information that you have right in front of you, rather than having to delve into other
parts of the codebase.
In other words, you can predict the behavior of some code just by looking at it. You may
need to understand the interfaces to other components, but you shouldn't need to look
inside them to see what they do.
Since, as developers, we spend most of our time looking at code, this is a pretty important
aspect of programming!

1867

Is your programming language unreasonable?

Of course, there is a huge amount of advice out there on how to do just this: naming
guidelines, formatting rules, design patterns, etc., etc.
But can your programming language by itself help your code to be more reasonable, more
predictable? I think the answer is yes, but I'll let you judge for yourself.
Below, I'll present a series of code fragments. After each snippet, I'm going to ask you what
you think the code does. I've deliberately not shown my own comments so that you can think
about it and do your own reasoning. After you have thought about it, scroll down to read my
opinion.

Example 1
Let's start off by looking at the following code.
We start with a variable x that is assigned the integer 2 .
Then DoSomething is called with x as a parameter.
Then y is assigned to x - 1 .
The question I would ask you is simple: What is the value of y ?
var x = 2;
DoSomething(x);
// What value is y?
var y = x - 1;

(scroll down for answer)

1868

Is your programming language unreasonable?

The answer is -1 . Did you get that answer? No? If you can't figure it out, scroll down again.
1869

Is your programming language unreasonable?

1870

Is your programming language unreasonable?

Trick question! This code is actually JavaScript!

Here's the whole thing:
function DoSomething (foo) { x = false}
var x = 2;
DoSomething(x);
var y = x - 1;

Yes, it's horrible! DoSomething accesses x directly rather than through the parameter, and
then turns it into a boolean of all things! Then, subtracting 1 from x casts it from false to
0 , so that y is -1 .

Don't you totally hate this? Sorry to mislead you about the language, but I just wanted to
demonstrate how annoying it is when the language behaves in unpredictable ways.
JavaScript is a very useful and important language. But no one would claim that
reasonableness was one of its strengths. In fact, most dynamically-typed languages have
quirks that make them hard to reason about in this way.
Thanks to static typing and sensible scoping rules, this kind of thing could never happen in
C# (unless you tried really hard!) In C#, if you don't match up the types properly, you get a
compile-time error rather than a run-time error.
In other words, C# is much more predictable than JavaScript. Score one for static typing!
So now we have our first requirement for making a language predictable:
How to make your language predictable:
1. Variables should not be allowed to change their type.
C# is looking good compared to JavaScript. But we're not done yet...

1871

Is your programming language unreasonable?

UPDATE: This is an admittedly silly example. In retrospect, I could have picked a better one.
Yes, I know that no one sensible would ever do this. The point still stands: the JavaScript
language does not prevent you from doing stupid things with implicit typecasts.

Example 2
In this next example, we're going to create two instances of the same Customer class, with
exactly the same data in them.
The question is: Are they equal?
// create two customers
var cust1 = new Customer(99, "J Smith");
var cust2 = new Customer(99, "J Smith");
// true or false?
cust1.Equals(cust2);

(scroll down for answer)

1872

Is your programming language unreasonable?

// true or false?
cust1.Equals(cust2);

Who knows? It depends on how the Customer class has been implemented. This code is
not predictable.

1873

Is your programming language unreasonable?

You'll have to look at whether the class implements IEquatable at least, and you'll probably
have to look at the internals of the class as well to see exactly what is going on.
But why is this even an issue?
Let me ask you this:
How often would you NOT want the instances to be equal?
How often have you had to override the Equals method?
How often have you had a bug caused by forgetting to override the Equals method?
How often have you had a bug caused by mis-implementing GetHashCode (such as
forgetting to change it when the fields that you compare on change)?
Why not make the objects equal by default, and make reference equality testing the special
case?
So let's add another item to our list.
How to make your language predictable:
1. Variables should not be allowed to change their type.
2. Objects containing the same values should be equal by default.

Example 3
In this next example, I've got two objects containing exactly the same data, but which are
instances of different classes.
The question again is: Are they equal?
// create a customer and an order
var cust = new Customer(99, "J Smith");
var order = new Order(99, "J Smith");
// true or false?
cust.Equals(order);

(scroll down for answer)

1874

Is your programming language unreasonable?

1875

Is your programming language unreasonable?

// true or false?
cust.Equals(order);

Who cares! This is almost certainly a bug! Why are you even comparing two different
classes like this in the first place?
Compare their names or ids, certainly, but not the objects themselves. This should be a
compiler error.
If it isn't, why not? You probably just used the wrong variable name by mistake but now you
have a subtle bug in your code. Why does your language let you do this?
So let's add another item to our list.
How to make your language predictable:
1. Variables should not be allowed to change their type.
2. Objects containing the same values should be equal by default.
3. Comparing objects of different types is a compile-time error.

UPDATE: Many people have pointed out that you need this when comparing classes related
by inheritance. This is true, of course. But what is the cost of this feature? You get the ability
to compare subclasses, but you lose the ability to detect accidental errors.
Which is more important in practice? That's for you to decide, I just wanted to make it clear
that there are costs associated with the status quo, not just benefits.

Example 4
In this snippet, we're just going to create a Customer instance. That's all. Can't get much
more basic than that.

1876

Is your programming language unreasonable?

// create a customer
var cust = new Customer();
// what is the expected output?
Console.WriteLine(cust.Address.Country);

Now the question is: what is the expected output of WriteLine ?

(scroll down for answer)

1877

Is your programming language unreasonable?

// what is the expected output?

Console.WriteLine(cust.Address.Country);

Who knows?
It depends on whether the Address property is null or not. And that is something you can't
tell without looking at the internals of the Customer class again.
Yes, we know that it is a best practice that constructors should initialize all fields at
construction time, but why doesn't the language enforce it?
If the address is required, then make it be required in the constructor. And if the address is
not always required, then make it clear that the Address property is optional and might be
missing.
So let's add another item to our list of improvements.
How to make your language predictable:
1. Variables should not be allowed to change their type.
2. Objects containing the same values should be equal by default.
3. Comparing objects of different types is a compile-time error.

1878

Is your programming language unreasonable?

4. Objects must always be initialized to a valid state. Not doing so is a compile-time

error.

Example 5
In this next example, we're going to:
Create a customer.
Add it to a set that uses hashing.
Do something with the customer object.
See if the customer is still in the set.
What could possibly go wrong?
// create a customer
var cust = new Customer(99, "J Smith");
// add it to a set
var processedCustomers = new HashSet<Customer>();
processedCustomers.Add(cust);
// process it
ProcessCustomer(cust);
// Does the set contain the customer? true or false?
processedCustomers.Contains(cust);

So, does the set still contain the customer at the end of this code?
(scroll down for answer)

1879

Is your programming language unreasonable?

1880

Is your programming language unreasonable?

// Does the set contain the customer?

processedCustomers.Contains(cust);

Maybe. Maybe not.

It depends on two things:
First, does the hash code of the customer depend on a mutable field, such as an id.
Second, does ProcessCustomer change this field?
If both are true, then the hash will have been changed, and the customer will not longer
appear to exist in the set (even though it is still in there somewhere!).
This might well cause subtle performance and memory problems (e.g. if the set is a cache).
How could the language prevent this?
One way would be to say that any field or property used in GetHashCode must be immutable,
while allowing other properties to be mutable. But that is really impractical.
Better to just make the entire Customer class immutable instead!
Now if the Customer class was immutable, and ProcessCustomer wanted to make changes,
it would have to return a new version of the customer, and the code would look like this:
// create a customer
var cust = new ImmutableCustomer(99, "J Smith");
// add it to a set
var processedCustomers = new HashSet<ImmutableCustomer>();
processedCustomers.Add(cust);
// process it and return the changes
var changedCustomer = ProcessCustomer(cust);
// true or false?
processedCustomers.Contains(cust);

Notice that the ProcessCustomer line has changed to:

var changedCustomer = ProcessCustomer(cust);

It's clear that ProcessCustomer has changed something just by looking at this code. If
ProcessCustomer hadn't changed anything, it wouldn't have needed to return an object at all.

Going back to the question, it's clear that in this implementation the original version of the
customer is guaranteed to still be in the set, no matter what ProcessCustomer does.
1881

Is your programming language unreasonable?

Of course, that doesn't solve the issue of whether the new one or the old one (or both)
should be in the set. But unlike the implementation using the mutable customer, this issue is
now staring you in the face and won't go unnoticed accidentally.
So immutability FTW!
So that's another item for our list.
How to make your language predictable:
1. Variables should not be allowed to change their type.
2. Objects containing the same values should be equal by default.
3. Comparing objects of different types is a compile-time error.
4. Objects must always be initialized to a valid state. Not doing so is a compile-time error.
5. Once created, objects and collections must be immutable.
Time for a quick joke about immutability:
"How many Haskell programmers does it take to change a lightbulb?"
"Haskell programmers don't "change" lightbulbs, they "replace" them. And you must
also replace the whole house at the same time."
Almost done now -- just one more!

Example 6
In this final example, we'll try to fetch a customer from a CustomerRepository .
// create a repository
var repo = new CustomerRepository();
// find a customer by id
var customer = repo.GetById(42);
// what is the expected output?
Console.WriteLine(customer.Id);

The question is: after we do customer = repo.GetById(42) , what is the value of

customer.Id ?

(scroll down for answer)

1882

Is your programming language unreasonable?

1883

Is your programming language unreasonable?

var customer = repo.GetById(42);

// what is the expected output?
Console.WriteLine(customer.Id);

It all depends, of course.

If I look at the method signature of GetById , it tells me it always returns a Customer . But
does it really?
What happens if the customer is missing? Does repo.GetById return null ? Does it throw
an exception? You can't tell just by looking at the code that we've got.
In particular, null is a terrible thing to return. It's a turncoat that pretends to be a Customer
and can be assigned to Customer variables with nary a complaint from the compiler, but
when you actually ask it to do something, it blows up in your face with an evil cackle.
Unfortunately, I can't tell by looking at this code whether a null is returned or not.
Exceptions are a little better, because at least they are typed and contain information about
the context. But it's not apparent from the method signature which exceptions might be
thrown. The only way that you can know for sure is by looking at the internal source code
(and maybe the documentation, if you're lucky and it is up to date).
But now imagine that your language did not allow null and did not allow exceptions. What
could you do instead?
The answer is, you would be forced to return a special class that might contain either a
customer or an error, like this:

1884

Is your programming language unreasonable?

// create a repository
var repo = new CustomerRepository();
// find a customer by id and
// return a CustomerOrError result
var customerOrError = repo.GetById(42);

The code that processed this "customerOrError" result would then have to test what kind of
result it was, and handle each case separately, like this:
// handle both cases
if (customerOrError.IsCustomer)
Console.WriteLine(customerOrError.Customer.Id);
if (customerOrError.IsError)
Console.WriteLine(customerOrError.ErrorMessage);

This is exactly the approach taken by most functional languages. It does help if the language
provides conveniences to make this technique easier, such as sum types, but even without
that, this approach is still the only way to go if you want to make it obvious what your code is
doing. (You can read more about this technique here.)
So that's the last two items to add to our list, at least for now.
How to make your language predictable:
1. Variables should not be allowed to change their type.
2. Objects containing the same values should be equal by default.
3. Comparing objects of different types is a compile-time error.
4. Objects must always be initialized to a valid state. Not doing so is a compile-time error.
5. Once created, objects and collections must be immutable.
6. No nulls allowed.
7. Missing data or errors must be made explicit in the function signature.
I could go on, with snippets demonstrating the misuse of globals, side-effects, casting, and
so on. But I think I'll stop here -- you've probably got the idea by now!

Can your programming language do this?

I hope that it is obvious that making these additions to a programming language will help to
make it more reasonable.
Unfortunately, mainstream OO languages like C# are very unlikely to add these features.

1885

Is your programming language unreasonable?

First of all, it would be a major breaking change to all existing code.

Second, many of these changes go deeply against the grain of the object-oriented
programming model itself.
For example, in the OO model, object identity is paramount, so of course equality by
reference is the default.
Also, from an OO point of view, how two objects are compared is entirely up to the objects
themselves -- OO is all about polymorphic behavior and the compiler needs to stay out of it!
Similarly, how objects are constructed and initialized is again entirely up to the object itself.
There are no rules to say what should or should not be allowed.
Finally, it is very hard to add non-nullable reference types to a statically typed OO language
without also implementing the initialization constraints in point 4. As Eric Lippert himself has
said "Non-nullability is the sort of thing you want baked into a type system from day one, not
something you want to retrofit 12 years later".
In contrast, most functional programming languages have these "high-predictability" features
as a core part of the language.
For example, in F#, all but one of the items on that list are built into the language:
1. Values are not allowed to change their type. (And this even includes implicit casts from
int to float, say).
2. Records with the same internal data ARE equal by default.
3. Comparing values of different types IS a compile-time error.
4. Values MUST be initialized to a valid state. Not doing so is a compile-time error.
5. Once created, values ARE immutable by default.
6. Nulls are NOT allowed, in general.
Item #7 is not enforced by the compiler, but discriminated unions (sum types) are generally
used to return errors rather than using exceptions, so that the function signature indicates
exactly what the possible errors are.
It's true that when working with F# there are still many caveats. You can have mutable
values, you can create and throw exceptions, and you may indeed have to deal with nulls
that come from non-F# code.
But these things are considered code smells and are unusual, rather than being the general
default.
Other languages such as Haskell are even purer (and hence even more reasonable) than
F#, but even Haskell programs will not be perfect.

1886

Is your programming language unreasonable?

In fact, no language can be reasoned about perfectly and still be practical. But still, some
languages are certainly more reasonable than others.
I think that one of the reasons why many people have become so enthusiastic about
functional-style code (and call it "simple" even though it's full of strange symbols!) is exactly
this: immutability, and lack of side effects, and all the other functional principles, act together
to enforce this reasonability and predictability, which in turn helps to reduce your cognitive
burden so that you need only focus on the code in front of you.

Lambdas aren't the solution

So now it should be clear that this list of proposed improvements has nothing to do with
language enhancements such as lambdas or clever functional libraries.
In other words, when I focus on reasonability, I don't care what my language will let me
do, I care more about what my language won't let me do. I want a language that stops
me doing stupid things by mistake.
That is, if I had to choose between language A that didn't allow nulls, or language B that had
higher-kinded types but still allowed objects to be null easily, I would pick language A without
hesitation.

Questions
Let me see if I can prempt some questions...
Question: These examples are very contrived! If you code carefully and follow good
practices, you can write safe code without these features!
Yes, you can. I'm not claiming you can't. But this post is not about writing safe code, it's
about reasoning about the code. There is a difference.
And it's not about what you can do if you are careful. It's about what can happen if you are
not careful!
That is, does your programming language (not your coding guidelines, or tests, or IDE, or
development practices) give you support for reasoning about your code?
Question: You're telling me that a language should have these features. Isn't that very
arrogant of you?
Please read carefully. I am not saying that at all. What I am saying is that:
IF you want to be able to reason about your code, THEN it will be much easier if your

1887

Is your programming language unreasonable?

language supports the features that I mention.

If reasoning about your code is not that important to you, then please do feel free to ignore
everything I've said!
Question: Focusing on just one aspect of a programming language is too limiting.
Surely other qualities are just as important?
Yes, or course they are. I am not a absolutist on this topic. I think that factors such as
comprehensive libraries, good tooling, a welcoming community, and the strength of the
ecosystem are very important too.
But the purpose of this post was to address the specific comments I mentioned at the
beginning, such as: "C# already has most of the features of F#, so why should I bother to
switch?".
Question: Why are you dismissing dynamic languages so quickly?
First, my apologies to JavaScript developers for the dig earlier!
I like dynamic languages a lot, and one of my favorite languages, Smalltalk, is completely
unreasonable by the standards I've talked about. Luckily, this post is not trying to persuade
you which languages are "best" in general, but rather just discussing one aspect of that
choice.
Question: Immutable data structures are slow, and there will be lots of extra allocation
going on. Won't this affect performance?
This post is not attempting to address the performance impact (or any other aspect) of these
features.
But it is indeed a valid question to ask which should have a higher priority: code quality or
performance? That's for you to decide, and it depends on the context.
Personally, I would go for safety and quality first, unless there was a compelling reason not
to. Here's a sign I like:

1888

Is your programming language unreasonable?

Summary
I said just above that this post is not trying to persuade you to pick a language based on
"reasonability" alone. But that's not quite true.
If you have already picked a statically typed, high-level language such as C# or Java, then
it's clear that reasonability or something like it was an important criterion in your language
decision.
In that case, I hope that the examples in this post might have made you more willing to
consider using an even more "reasonable" language on your platform of choice (.NET or
JVM).
The argument for staying put -- that your current language will eventually "catch up" -- may
be true purely in terms of features, but no amount of future enhancements can really change
the core design decisions in an OO language. You'll never get rid of nulls, or mutability, or
having to override equality all the time.
What's nice about F#, or Scala/Clojure, is that these functional alternatives don't require you
to change your ecosystem, but they do immediately improve your code quality.
In my opinion, it's quite a low risk compared with the cost of business as usual.
(I'll leave the issue of finding skilled people, training, support, etc, for another post. But see
this, this, this, and this if you're worried about hiring)

1889

We don't need no stinking UML diagrams

In my talk on functional DDD, I often use this slide (in context):

Which is of course is a misquote of this famous scene. Oops, I mean this one.
Ok, I might be exaggerating a bit. Some UML diagrams are useful (I like sequence diagrams
for example) and in general, I do think a good picture or diagram can be worth 1000 words.
But I believe that, in many cases, using UML for class diagrams is not necessary.
Instead, a concise language like F# (or OCaml or Haskell) can convey the same meaning in
a way that is easier to read, easier to write, and most important, easier to turn into working
code!
With UML diagrams, you need to translate them to code, with the possibility of losing
something in translation. But if the design is documented in your programming language
itself, there is no translation phase, and so the design must always be in sync with the
implementation.
To demonstrate this in practice, I decided to scour the internet for some good (and not-sogood) UML class diagrams, and convert them into F# code. You can compare them for
yourselves.

Regular expressions
1890

We don't need no stinking UML diagrams

Let's start with a classic one: regular expressions (source)

Here's the UML diagram:

And here's the F# equivalent:

type RegularExpression =
| Literal of string
| Sequence of RegularExpression list
| Alternation of RegularExpression * RegularExpression
| Repetition of RegularExpression
// An interpreter takes a string input and a RegularExpression
// and returns a value of some kind
type Interpret<'a> = string -> RegularExpression -> 'a

That's quite straightforward.

Student enrollment
Here's another classic one: enrollment (source).
Here's the UML diagram:

1891

We don't need no stinking UML diagrams

And here's the F# equivalent:

1892

We don't need no stinking UML diagrams

type Student = {
Name: string
Address: string
PhoneNumber: string
EmailAddress: string
AverageMark: float
}
type Professor= {
Name: string
Address: string
PhoneNumber: string
EmailAddress: string
Salary: int
}
type Seminar = {
Name: string
Number: string
Fees: float
TaughtBy: Professor option
WaitingList: Student list
}
type Enrollment = {
Student : Student
Seminar : Seminar
Marks: float list
}
type EnrollmentRepository = Enrollment list
// ==================================
// activities / use-cases / scenarios
// ==================================
type IsElegibleToEnroll = Student -> Seminar -> bool
type GetSeminarsTaken = Student -> EnrollmentRepository -> Seminar list
type AddStudentToWaitingList = Student -> Seminar -> Seminar

The F# mirrors the UML diagram, but I find that by writing functions for all the activities
rather than drawing pictures, holes in the original requirements are revealed.
For example, in the GetSeminarsTaken method in the UML diagram, where is the list of
seminars stored? If it is in the Student class (as implied by the diagram) then we have a
mutual recursion between Student and Seminar and the whole tree of every student and
seminar is interconnected and must be loaded at the same time unless hacks are used.
Instead, for the functional version, I created an EnrollmentRepository to decouple the two
classes.
1893

We don't need no stinking UML diagrams

Similarly, it's not clear how enrollment actually works, so I created an EnrollStudent
function to make it clear what inputs are needed.
type EnrollStudent = Student -> Seminar -> Enrollment option

Because the function returns an option , it is immediately clear that enrollment might fail
(e.g student is not eligible to enroll, or is enrolling twice by mistake).

Order and customer

Here's another one (source).

And here's the F# equivalent:

1894

We don't need no stinking UML diagrams

type Customer = {name:string; location:string}

type NormalOrder = {date: DateTime; number: string; customer: Customer}
type SpecialOrder = {date: DateTime; number: string; customer: Customer}
type Order =
| Normal of NormalOrder
| Special of SpecialOrder
// these three operations work on all orders
type Confirm = Order -> Order
type Close = Order -> Order
type Dispatch = Order -> Order
// this operation only works on Special orders
type Receive = SpecialOrder -> SpecialOrder

I'm just copying the UML diagram, but I have to say that I hate this design. It's crying out to
have more fine grained states.
In particular, the Confirm and Dispatch functions are horrible -- they give no idea of what
else is needed as input or what the effects will be. This is where writing real code can force
you to think a bit more deeply about the requirements.

Order and customer, version 2

Here's a much better version of orders and customers (source).

And here's the F# equivalent:

type Date = System.DateTime

1895

We don't need no stinking UML diagrams

// == Customer related ==
type Customer = {
name:string
address:string
}
// == Item related ==
type [<Measure>] grams
type Item = {
shippingWeight: int<grams>
description: string
}
type Qty = int
type Price = decimal

// == Payment related ==
type PaymentMethod =
| Cash
| Credit of number:string * cardType:string * expDate:Date
| Check of name:string * bankID: string
type Payment = {
amount: decimal
paymentMethod : PaymentMethod
}
// == Order related ==
type TaxStatus = Taxable | NonTaxable
type Tax = decimal
type OrderDetail = {
item: Item
qty: int
taxStatus : TaxStatus
}
type OrderStatus = Open | Completed
type Order = {
date: DateTime;
customer: Customer
status: OrderStatus
lines: OrderDetail list
payments: Payment list
}

1896

We don't need no stinking UML diagrams

// ==================================
// activities / use-cases / scenarios
// ==================================
type GetPriceForQuantity = Item -> Qty -> Price
type CalcTax = Order -> Tax
type CalcTotal = Order -> Price
type CalcTotalWeight = Order -> int<grams>

I've done some minor tweaking, adding units of measure for the weight, creating types to
represent Qty and Price .
Again, this design might be improved with more fine grained states, such as creating a
separate AuthorizedPayment type (to ensure that an order can only be paid with authorized
payments) and a separate PaidOrder type (e.g. to stop you paying for the same order
twice).
Here's the kind of thing I mean:
// Try to authorize a payment. Note that it might fail
type Authorize = UnauthorizedPayment -> AuthorizedPayment option
// Pay an unpaid order with an authorized payment.
type PayOrder = UnpaidOrder -> AuthorizedPayment -> PaidOrder

Hotel Booking
Here's one from the JetBrains IntelliJ documentation (source).

1897

We don't need no stinking UML diagrams

Here's the F# equivalent:

type Date = System.DateTime
type User = {
username: string
password: string
name: string
}
type Hotel = {
id: int
name: string
address: string
city: string
state: string
zip: string
country: string

1898

We don't need no stinking UML diagrams

price: decimal
}
type CreditCardInfo = {
card: string
name: string
expiryMonth: int
expiryYear: int
}
type Booking = {
id: int
user: User
hotel: Hotel
checkinDate: Date
checkoutDate: Date
creditCardInfo: CreditCardInfo
smoking: bool
beds: int
}
// What are these?!? And why are they in the domain?
type EntityManager = unit
type FacesMessages = unit
type Events = unit
type Log = unit
type BookingAction = {
em: EntityManager
user: User
hotel: Booking
booking: Booking
facesMessages : FacesMessages
events: Events
log: Log
bookingValid: bool
}
type ChangePasswordAction = {
user: User
em: EntityManager
verify: string
booking: Booking
changed: bool
facesMessages : FacesMessages
}
type RegisterAction = {
user: User
em: EntityManager
facesMessages : FacesMessages
verify: string
registered: bool

1899

We don't need no stinking UML diagrams

I have to stop there, sorry. The design is driving me crazy. I can't even.
What are these EntityManager and FacesMessages fields? And logging is important of
course, but why is Log a field in the domain object?
By the way, in case you think that I am deliberately picking bad examples of UML design, all
these diagrams come from the top results in an image search for "uml class diagram".

Library
This one is better, a library domain (source).

Here's the F# equivalent. Note that because it is code, I can add comments to specific types
and fields, which is doable but awkward with UML.

1900

We don't need no stinking UML diagrams

Note also that I can say ISBN: string option to indicate an optional ISBN rather that the
awkward [0..1] syntax.
type Author = {
name: string
biography: string
}
type Book = {
ISBN: string option
title: string
author: Author
summary: string
publisher: string
publicationDate: Date
numberOfPages: int
language: string
}
type Library = {
name: string
address: string
}
// Each physical library item - book, tape cassette, CD, DVD, etc. could have its own
item number.
// To support it, the items may be barcoded. The purpose of barcoding is
// to provide a unique and scannable identifier that links the barcoded physical item
// to the electronic record in the catalog.
// Barcode must be physically attached to the item, and barcode number is entered into
// the corresponding field in the electronic item record.
// Barcodes on library items could be replaced by RFID tags.
// The RFID tag can contain item's identifier, title, material type, etc.
// It is read by an RFID reader, without the need to open a book cover or CD/DVD case
// to scan it with barcode reader.
type BookItem = {
barcode: string option
RFID: string option
book: Book
/// Library has some rules on what could be borrowed and what is for reference onl
y.
isReferenceOnly: bool
belongsTo: Library
}
type Catalogue = {
belongsTo: Library
records : BookItem list
}
type Patron = {

1901

We don't need no stinking UML diagrams

name: string
address: string
}
type AccountState = Active | Frozen | Closed
type Account = {
patron: Patron
library: Library
number: int
opened: Date
/// Rules are also defined on how many books could be borrowed
/// by patrons and how many could be reserved.
history: History list
state: AccountState
}
and History = {
book : BookItem
account: Account
borrowedOn: Date
returnedOn: Date option
}

Since the Search and Manage interfaces are undefined, we can just use placeholders
( unit ) for the inputs and outputs.
type Librarian = {
name: string
address: string
position: string
}
/// Either a patron or a librarian can do a search
type SearchInterfaceOperator =
| Patron of Patron
| Librarian of Librarian
type SearchRequest = unit // to do
type SearchResult = unit // to do
type SearchInterface = SearchInterfaceOperator -> Catalogue -> SearchRequest -> Search
Result
type ManageRequest = unit // to do
type ManageResult = unit // to do
/// Only librarians can do management
type ManageInterface = Librarian -> Catalogue -> ManageRequest -> ManageResult

1902

We don't need no stinking UML diagrams

Again, this might not be the perfect design. For example, it's not clear that only Active
accounts could borrow a book, which I might represent in F# as:
type Account =
| Active of ActiveAccount
| Closed of ClosedAccount
/// Only ActiveAccounts can borrow
type Borrow = ActiveAccount -> BookItem -> History

If you want to see a more modern approach to modelling this domain using CQRS and event
sourcing, see this post.

Software licensing
The final example is from a software licensing domain (source).

Here's the F# equivalent.

open System
type Date = System.DateTime
type String50 = string

1903

We don't need no stinking UML diagrams

type String5 = string

// ==========================
// Customer-related
// ==========================
type AddressDetails = {
street : string option
city : string option
postalCode : string option
state : string option
country : string option
}
type CustomerIdDescription = {
CRM_ID : string
description : string
}
type IndividualCustomer = {
idAndDescription : CustomerIdDescription
firstName : string
lastName : string
middleName : string option
email : string
phone : string option
locale : string option // default : "English"
billing : AddressDetails
shipping : AddressDetails
}
type Contact = {
firstName : string
lastName : string
middleName : string option
email : string
locale : string option // default : "English"
}
type Company = {
idAndDescription : CustomerIdDescription
name : string
phone : string option
fax : string option
contact: Contact
billing : AddressDetails
shipping : AddressDetails
}
type Customer =
| Individual of IndividualCustomer
| Company of Company

1904

We don't need no stinking UML diagrams

// ==========================
// Product-related
// ==========================
/// Flags can be ORed together
[<Flags>]
type LockingType =
| HL
| SL_AdminMode
| SL_UserMode
type Rehost =
| Enable
| Disable
| LeaveAsIs
| SpecifyAtEntitlementTime
type BatchCode = {
id : String5
}
type Feature = {
id : int
name : String50
description : string option
}
type ProductInfo = {
id : int
name : String50
lockingType : LockingType
rehost : Rehost
description : string option
features: Feature list
bactchCode: BatchCode
}
type Product =
| BaseProduct of ProductInfo
| ProvisionalProduct of ProductInfo * baseProduct:Product
// ==========================
// Entitlement-related
// ==========================
type EntitlementType =
| HardwareKey
| ProductKey
| ProtectionKeyUpdate
type Entitlement = {
EID : string
entitlementType : EntitlementType

1905

We don't need no stinking UML diagrams

startDate : Date
endDate : Date option
neverExpires: bool
comments: string option
customer: Customer
products: Product list
}

This diagram is just pure data and no methods, so there are no function types. I have a
feeling that there are some important business rules that have not been captured.
For example, if you read the comments in the source, you'll see that there are some
interesting constraints around EntitlementType and LockingType . Only certain locking
types can be used with certain entitlement types.
That might be something that we could consider modelling in the type system, but I haven't
bothered. I've just tried to reproduct the UML as is.

Summary
I think that's enough to get the idea.
My general feeling about UML class diagrams is that they are OK for a sketch, if a bit
heavyweight compared to a few lines of code.
For detailed designs, though, they are not nearly detailed enough. Critical things like context
and dependencies are not at all obvious. In my opinion, none of the UML diagrams I've
shown have been good enough to write code from, even as a basic design.
Even more seriously, a UML diagram can be very misleading to non-developers. It looks
"official" and can give the impression that the design has been thought about deeply, when
in fact the design is actually shallow and unusable in practice.
Disagree? Let me know in the comments!

1906

Introvert and extrovert programming languages

Introvert and extrovert programming

languages
What's the difference between an introvert and extrovert mathematician?
An introvert mathematician looks at his shoes while talking to you; an extrovert
mathematician looks at your shoes.
For a long time, I've been aware of differences in how programming languages present
themselves to the outside world.
If programming languages had personalities, I would be tempted to call some of them
"introvert", and some of them "extrovert".
An extrovert programming language is all about the outside world, never happier than when
partying with IO and external data sources.
On the other hand, an introvert programming language is quite happy to be alone and would
prefer not to deal with the outside world, if possible. Sure, it can be social and work with IO
when it needs to, but it finds the activity quite tiring, and is relieved when IO has gone home
and it can go back to reading a good book.
What's interesting is that you can tell a lot about the personality type of a language by
looking at what is considered important in a user guide or tutorial.
For example, the classic "C Programming Language" book famously has printf("hello,
world\n") at the very beginning, and most other C books follow the same pattern.

And indeed, C is a very extrovert language. Coding examples are littered with files and
console IO. Similarly, you can tell that PHP, Python and Perl are equally extrovert with just
one glance at their manuals.
In fact, I would say that all of the most popular languages are extrovert, and the reasons are
obvious. They ooze confidence, they make friends easily, and they get things done.
On the other hand, I would say that Haskell is a great example of an introvert language.
For example, in the book "Learn You A Haskell" the "hello world" example doesn't appear
until chapter 9! And in "Real World Haskell", IO is not invited to dinner until chapter 7.
If you don't have the full manual handy, another telling clue that a language is introverted is if
it early on introduces you to its close friend, the Fibonacci function. Introvert languages love
recursion!

1907

Introvert and extrovert programming languages

Now, just as in the real world, introvert languages are misunderstood by extroverts. They are
accused of being too arrogant, too serious, too risk-averse.
But that's not fair -- introvert languages are really just more reflective and thoughtful, and
thus more likely to have deep insights than the shallow, yapping extroverts.

But...
"All generalisations are false including this one"
You might think that imperative and OO languages would be extrovert, while languages with
more declarative paradigms (functional, logic) would be introvert, but that is not always the
case.
For example, SQL is a declarative language, but its whole purpose in life is data-processing,
which makes it extrovert in my book.
And nearer to home, F# is a functional-first language, but is very happy to do IO, and in fact
has excellent support for real-world data processing via type providers and Deedle.
Just as people are not all one or the other, so with programming languages. There is a
range. Some languages are extremely extrovert, some extremely introvert, and some inbetween.

A lot of people say I am egocentric -- but enough about

them
"Reality is merely an illusion, albeit a very persistent one."
Some languages do not fall so neatly into this personality type spectrum.
Take Smalltalk for example.
In many ways, Smalltalk is extremely extrovert. It has lots of support for user interface
interaction, and itself was one of the first graphical development environments.
But there's a problem. Sure, it's friendly and chatty and great at intense one-to-one
conversations, but it has a dark side -- it doesn't play well with others. It only reluctantly
acknowledges the operating system, and rather than dealing with messiness of external
libraries, prefers to implement things in its own perfect way.
Most Lisps have the same failing as well. Idealistic and elegant, but rather isolated. I'm going
to call this personality type solipsistic.

1908

Introvert and extrovert programming languages

In their defence, many important programming concepts were first developed in solipsistic
languages. But alas, despite their ardent followers, they never gain the widespread
recognition they deserve.

Where Do We Come From? What Are We? Where Are We

Going?
And that brings me, inevitably, to the all-devouring black hole that is JavaScript. Where does
that fit in this silly scheme?
Obviously, its original purpose was to to aid communication with the user (and also, for
animating monkeys), so at first glance it would seem to be extrovert.
But the fact that it runs in a sandbox (and until somewhat recently didn't do many kinds of
IO) makes me think otherwise. The clincher for me is node.js. Need a server language?
Let's create one in our own image! Let's write all the libraries from scratch!
But it seems to have worked, for now at least. So, solipsism for the win!

Concluding Unscientific Postscript

"Nature has made us frivolous to console us for our miseries"
At this point, I should wrap up with some profound observations about different programming
communities and what their language preference reveals about them.
But really, this is a just a bit of whimsy, so I won't.

1909

Swapping type-safety for high performance using compiler directives

Swapping type-safety for high

performance using compiler directives
TL;DR; An experiment: you can use lots of domain modelling types at development time and
swap them out for a more performant implementation later using compiler directives.

Domain Modelling vs. Performance

I am a big fan of using types for domain modelling -- lots and lots and lots of types!
These types act both as documentation and as a compile time constraint to ensure that only
valid data is used.
For example, if I have two types CustomerId and OrderId , I can represent them as
separate types:
type CustomerId = CustomerId of int
type OrderId = OrderId of int

and by doing this, I guarantee that I can't use an OrderId where I need an CustomerId .
The problem is that adding a layer of indirection like this can affect performance:
the extra indirection can cause data access to be much slower.
the wrapper class needs extra memory, creating memory pressure.
this in turn triggers the garbage collector more often, which can often be the cause of
performance problems in managed code.
In general, I don't generally worry about micro-performance like this at design-time. Many
many things will have a much bigger impact on performance, including any kind of I/O, and
the algorithms you choose.
As a result, I am very much against doing micro-benchmarks out of context. You should
always profile a real app in a real context, rather than worrying too much over things that
might not be important.
Having said that, I am now going to do some micro-benchmarks!

Micro-benchmarking a wrapper type

1910

Swapping type-safety for high performance using compiler directives

Let's see how a wrapper type fares when used in large numbers. Let's say we want to:
create ten million customer ids
then, map over them twice
then, filter them
Admittedly, it's a bit silly adding 1 to a customer id -- we'll look at a better example later.
Anyway, here's the code:
// type is an wrapper for a primitive type
type CustomerId = CustomerId of int
// create two silly functions for mapping and filtering
let add1ToCustomerId (CustomerId i) =
CustomerId (i+1)
let isCustomerIdSmall (CustomerId i) =
i < 100000
// --------------------------------// timed with a 1 million element array
// --------------------------------#time
Array.init 1000000 CustomerId
// map it
|> Array.map add1ToCustomerId
// map it again
|> Array.map add1ToCustomerId
// filter it
|> Array.filter isCustomerIdSmall
|> ignore
#time

The code sample above is available on GitHub.

(Again, let me stress that this is a terrible way to profile code!)
A typical timed result looks like this:
Real: 00:00:00.296, CPU: 00:00:00.296, GC gen0: 6, gen1: 4, gen2: 0

That is, it takes about 0.3 seconds to do those steps, and it creates quite a bit of garbage,
triggering four gen1 collections. If you are not sure what "gen0", "gen1", and "gen2" mean,
then this is a good place to start.

1911

Swapping type-safety for high performance using compiler directives

DISCLAIMER: I'm going to be doing all my benchmarking in F# interactive. Compiled code

with optimizations might have a completely different performance profile. Past performance
is no guarantee of future results. Draw conclusions at your own risk. Etc., etc.
If we increase the array size to 10 million, we get a more than 10x slower result:
Real: 00:00:03.489, CPU: 00:00:03.541, GC gen0: 68, gen1: 46, gen2: 2

That is, it takes about 3.5 seconds to do those steps, and it creates a lot of garbage,
including a few gen2 GC's, which are really bad. In fact, you might even get an "out of
memory" exception, in which case, you'll have to restart F# Interactive!
So what are the alternatives to using a wrapper? There are two common approaches:
Using type aliases
Using units of measure
Let's start with type aliases.

Using type aliases

In the type alias approach, I would simply dispense with the wrapper, but keep the type
around as documentation.
type CustomerId = int
type OrderId = int

If I want to use the type as documentation, I must then annotate the functions appropriately.
For example, in the add1ToCustomerId below both the parameter and the return value have
been annotated so that it has the type CustomerId -> CustomerId rather than int -> int .
let add1ToCustomerId (id:CustomerId) :CustomerId =
id+1

Micro-benchmarking a type alias

Let's create another micro-benchmark:

1912

Swapping type-safety for high performance using compiler directives

type CustomerId = int

// create two silly functions for mapping and filtering
let add1ToCustomerId (id:CustomerId) :CustomerId =
id+1
// val add1ToCustomerId : id:CustomerId -> CustomerId
let isCustomerIdSmall (id:CustomerId) =
id < 100000
// val isCustomerIdSmall : id:CustomerId -> bool
// --------------------------------// timed with a 1 million element array
// --------------------------------#time
Array.init 1000000 (fun i -> i)
// map it
|> Array.map add1ToCustomerId
// map it again
|> Array.map add1ToCustomerId
// filter it
|> Array.filter isCustomerIdSmall
|> Array.length
#time

The code sample above is available on GitHub.

The results are spectacularly better!
Real: 00:00:00.017, CPU: 00:00:00.015, GC gen0: 0, gen1: 0, gen2: 0

It takes about 17 milliseconds to do those steps, and more importantly, very little garbage
was generated.
If we increase the array size to 10 million, we get a 10x slower result, but still no garbage:
Real: 00:00:00.166, CPU: 00:00:00.156, GC gen0: 0, gen1: 0, gen2: 0

Compared with the earlier version at over three seconds, that's excellent.

Problems with type aliases

Alas, the problem with type aliases is that we have completely lost type safety now!
To demonstrate, here's some code that creates a CustomerId and an OrderId :

1913

Swapping type-safety for high performance using compiler directives

type CustomerId = int

type OrderId = int
// create two
let cid : CustomerId = 12
let oid : OrderId = 12

And sadly, the two ids compare equal, and we can pass an OrderId to function expecting a
CustomerId without any complaint from the compiler.

cid = oid // true

// pass OrderId to function expecting a CustomerId
add1ToCustomerId oid // CustomerId = 13

Ok, so that doesn't look promising! What next?

Using units of measure

The other common option is to use units of measure to distinguish the two types, like this:
type [<Measure>] CustomerIdUOM
type [<Measure>] OrderIdUOM
type CustomerId = int<CustomerIdUOM>
type OrderId = int<OrderIdUOM>

CustomerId and OrderId are still two different types, but the unit of measure is erased, so

by the time the JIT sees it the type looks like an primitive int.
We can see that this is true when we time the same steps as before:

1914

Swapping type-safety for high performance using compiler directives

// create two silly functions for mapping and filtering

let add1ToCustomerId id =
id+1<CustomerIdUOM>
let isCustomerIdSmall i =
i < 100000<CustomerIdUOM>
// --------------------------------// timed with a 1 million element array
// --------------------------------#time
Array.init 1000000 (fun i -> LanguagePrimitives.Int32WithMeasure<CustomerIdUOM> i)
// map it
|> Array.map add1ToCustomerId
// map it again
|> Array.map add1ToCustomerId
// filter it
|> Array.filter isCustomerIdSmall
|> ignore
#time

The code sample above is available on GitHub.

A typical timed result looks like this:
Real: 00:00:00.022, CPU: 00:00:00.031, GC gen0: 0, gen1: 0, gen2: 0

Again, the code is very fast (22 milliseconds), and just as importantly, very little garbage was
generated again.
If we increase the array size to 10 million, we maintain the high performance (just as with the
type alias approach) and still no garbage:
Real: 00:00:00.157, CPU: 00:00:00.156, GC gen0: 0, gen1: 0, gen2: 0

Problems with units of measure

The advantage of units of measure is that the CustomerId and OrderId types are
incompatible, so we get the type safety that we want.
But I find them unsatisfactory from an esthetic point of view. I like my wrapper types!
And also, units of measure are really meant to be used with numeric values. For example, I
can create a customer id and order id:

1915

Swapping type-safety for high performance using compiler directives

let cid = 12<CustomerIdUOM>

let oid = 4<OrderIdUOM>

and then I can divide CustomerId(12) by OrderId(4) to get three...

let ratio = cid / oid
// val ratio : int<CustomerIdUOM/OrderIdUOM> = 3

Three what though? Three customer ids per order id? What does that even mean?
Yes, surely this will never happen in practice, but still it bothers me!

Using compiler directives to get the best of

both worlds
Did I mention that I really like wrapper types? I really like them up until I get a call saying that
production systems are having performance hiccups because of too many big GCs.
So, can we get the best of both worlds? Type-safe wrapper types AND fast performance?
I think so, if you are willing to put up with some extra work during development and build.
The trick is to have both the "wrapper type" implemention and the "type alias"
implementation available to you, and then switch between them based on a compiler
directive.
For this to work:
you will need to tweak your code to not access the type directly, but only via functions
and pattern matching.
you will need to create a "type alias" implementation that implements a "constructor",
various "getters" and for pattern matching, active patterns.
Here's an example, using the COMPILED and INTERACTIVE directives so that you can play
with it interactively. Obviously, in real code, you would use your own directive such as
FASTTYPES or similar.

1916

Swapping type-safety for high performance using compiler directives

#if COMPILED // uncomment to use aliased version

//#if INTERACTIVE // uncomment to use wrapped version
// type is an wrapper for a primitive type
type CustomerId = CustomerId of int
// constructor
let createCustomerId i = CustomerId i
// get data
let customerIdValue (CustomerId i) = i
// pattern matching
// not needed
#else
// type is an alias for a primitive type
type CustomerId = int
// constructor
let inline createCustomerId i :CustomerId = i
// get data
let inline customerIdValue (id:CustomerId) = id
// pattern matching
let inline (|CustomerId|) (id:CustomerId) :int = id
#endif

You can see that for both versions I've created a constructor createCustomerId and a getter
customerIdValue and, for the type alias version, an active pattern that looks just like
CustomerId .

With this code in place, we can use CustomerId without caring about the implementation:

1917

Swapping type-safety for high performance using compiler directives

// test the getter

let testGetter c1 c2 =
let i1 = customerIdValue c1
let i2 = customerIdValue c2
printfn "Get inner value from customers %i %i" i1 i2
// Note that the signature is as expected:
// c1:CustomerId -> c2:CustomerId -> unit
// test pattern matching
let testPatternMatching c1 =
let (CustomerId i) = c1
printfn "Get inner value from Customers via pattern match: %i" i
match c1 with
| CustomerId i2 -> printfn "match/with %i" i
// Note that the signature is as expected:
// c1:CustomerId -> unit
let test() =
// create two ids
let c1 = createCustomerId 1
let c2 = createCustomerId 2
let custArray : CustomerId [] = [| c1; c2 |]
// test them
testGetter c1 c2
testPatternMatching c1

And now we can run the same micro-benchmark with both implementations:
// create two silly functions for mapping and filtering
let add1ToCustomerId (CustomerId i) =
createCustomerId (i+1)
let isCustomerIdSmall (CustomerId i) =
i < 100000
// --------------------------------// timed with a 1 million element array
// --------------------------------#time
Array.init 1000000 createCustomerId
// map it
|> Array.map add1ToCustomerId
// map it again
|> Array.map add1ToCustomerId
// filter it
|> Array.filter isCustomerIdSmall
|> Array.length
#time

1918

Swapping type-safety for high performance using compiler directives

The code sample above is available on GitHub.

The results are similar to the previous examples. The aliased version is much faster and
does not create GC pressure:
// results using wrapped version
Real: 00:00:00.408, CPU: 00:00:00.405, GC gen0: 7, gen1: 4, gen2: 1
// results using aliased version
Real: 00:00:00.022, CPU: 00:00:00.031, GC gen0: 0, gen1: 0, gen2: 0

and for the 10 million element version:

// results using wrapped version
Real: 00:00:03.199, CPU: 00:00:03.354, GC gen0: 67, gen1: 45, gen2: 2
// results using aliased version
Real: 00:00:00.239, CPU: 00:00:00.202, GC gen0: 0, gen1: 0, gen2: 0

A more complex example

In practice, we might want something more complex than a simple wrapper.
For example, here is an EmailAddress (a simple wrapper type, but constrained to be nonempty and containing a "@") and some sort of Activity record that stores an email and the
number of visits, say.

1919

Swapping type-safety for high performance using compiler directives

module EmailAddress =
// type with private constructor
type EmailAddress = private EmailAddress of string
// safe constructor
let create s =
if System.String.IsNullOrWhiteSpace(s) then
None
else if s.Contains("@") then
Some (EmailAddress s)
else
None
// get data
let value (EmailAddress s) = s
module ActivityHistory =
open EmailAddress
// type with private constructor
type ActivityHistory = private {
emailAddress : EmailAddress
visits : int
}
// safe constructor
let create email visits =
{emailAddress = email; visits = visits }
// get data
let email {emailAddress=e} = e
let visits {visits=a} = a

As before, for each type there is a constructor and a getter for each field.
NOTE: Normally I would define a type outside a module, but because the real constructor
needs to be private, I've put the type inside the module and given the module and the type
the same name. If this is too awkward, you can rename the module to be different from the
type, or use the OCaml convention of calling the main type in a module just "T", so you get
EmailAddress.T as the type name.

To make a more performant version, we replace EmailAddress with a type alias, and
Activity with a struct, like this:

1920

Swapping type-safety for high performance using compiler directives

module EmailAddress =
// aliased type
type EmailAddress = string
// safe constructor
let inline create s :EmailAddress option =
if System.String.IsNullOrWhiteSpace(s) then
None
else if s.Contains("@") then
Some s
else
None
// get data
let inline value (e:EmailAddress) :string = e
module ActivityHistory =
open EmailAddress
[<Struct>]
type ActivityHistory(emailAddress : EmailAddress, visits : int) =
member this.EmailAddress = emailAddress
member this.Visits = visits
// safe constructor
let create email visits =
ActivityHistory(email,visits)
// get data
let email (act:ActivityHistory) = act.EmailAddress
let visits (act:ActivityHistory) = act.Visits

This version reimplements the constructor and a getter for each field. I could have made the
field names for ActivityHistory be the same in both cases too, but. in the struct case, type
inference would not work. By making them different, the user is forced to use the getter
functions rather than dotting in.
Both implementations have the same "API", so we can create code that works with both:

1921

Swapping type-safety for high performance using compiler directives

let rand = new System.Random()

let createCustomerWithRandomActivityHistory() =
let emailOpt = EmailAddress.create "abc@example.com"
match emailOpt with
| Some email ->
let visits = rand.Next(0,100)
ActivityHistory.create email visits
| None ->
failwith "should not happen"
let add1ToVisits activity =
let email = ActivityHistory.email activity
let visits = ActivityHistory.visits activity
ActivityHistory.create email (visits+1)
let isCustomerInactive activity =
let visits = ActivityHistory.visits activity
visits < 3

// execute creation and iteration for a large number of records

let mapAndFilter noOfRecords =
Array.init noOfRecords (fun _ -> createCustomerWithRandomActivity() )
// map it
|> Array.map add1ToVisits
// map it again
|> Array.map add1ToVisits
// filter it
|> Array.filter isCustomerInactive
|> ignore // we don't actually care!

Pros and cons of this approach

One nice thing about this approach is that it is self-correcting -- it forces you to use the "API"
properly.
For example, if I started accessing fields directly by dotting into the ActivityHistory record,
then that code would break when the compiler directive was turned on and the struct
implementation was used.
Of course, you could also create a signature file to enforce the API.
On the negative side, we do lose some of the nice syntax such as {rec with ...} . But you
should really only be using this technique with small records (2-3 fields), so not having
with is not a big burden.

Timing the two implementations

1922

Swapping type-safety for high performance using compiler directives

Rather than using #time , this time I wrote a custom timer that runs a function 10 times and
prints out the GC and memory used on each run.
/// Do countN repetitions of the function f and print the
/// time elapsed, number of GCs and change in total memory
let time countN label f =
let stopwatch = System.Diagnostics.Stopwatch()
// do a full GC at the start but NOT thereafter
// allow garbage to collect for each iteration
System.GC.Collect()
printfn "Started"
let getGcStats() =
let gen0 = System.GC.CollectionCount(0)
let gen1 = System.GC.CollectionCount(1)
let gen2 = System.GC.CollectionCount(2)
let mem = System.GC.GetTotalMemory(false)
gen0,gen1,gen2,mem

printfn "======================="
printfn "%s (%s)" label WrappedOrAliased
printfn "======================="
for iteration in [1..countN] do
let gen0,gen1,gen2,mem = getGcStats()
stopwatch.Restart()
f()
stopwatch.Stop()
let gen0',gen1',gen2',mem' = getGcStats()
// convert memory used to K
let changeInMem = (mem'-mem) / 1000L
printfn "#%2i elapsed:%6ims gen0:%3i gen1:%3i gen2:%3i mem:%6iK" iteration sto
pwatch.ElapsedMilliseconds (gen0'-gen0) (gen1'-gen1) (gen2'-gen2) changeInMem

The code sample above is available on GitHub.

Let's now run mapAndFilter with a million records in the array:
let size = 1000000
let label = sprintf "mapAndFilter: %i records" size
time 10 label (fun () -> mapAndFilter size)

The results are shown below:

1923

Swapping type-safety for high performance using compiler directives

=======================
mapAndFilter: 1000000 records (Wrapped)
=======================
# 1 elapsed: 820ms gen0: 13 gen1: 8 gen2: 1 mem: 72159K
# 2 elapsed: 878ms gen0: 12 gen1: 7 gen2: 0 mem: 71997K
# 3 elapsed: 850ms gen0: 12 gen1: 6 gen2: 0 mem: 72005K
# 4 elapsed: 885ms gen0: 12 gen1: 7 gen2: 0 mem: 72000K
# 5 elapsed: 6690ms gen0: 16 gen1: 10 gen2: 1 mem:-216005K
# 6 elapsed: 714ms gen0: 12 gen1: 7 gen2: 0 mem: 72003K
# 7 elapsed: 668ms gen0: 12 gen1: 7 gen2: 0 mem: 71995K
# 8 elapsed: 670ms gen0: 12 gen1: 7 gen2: 0 mem: 72001K
# 9 elapsed: 6676ms gen0: 16 gen1: 11 gen2: 2 mem:-215998K
#10 elapsed: 712ms gen0: 13 gen1: 7 gen2: 0 mem: 71998K
=======================
mapAndFilter: 1000000 records (Aliased)
=======================
# 1 elapsed: 193ms gen0: 7 gen1: 0 gen2: 0 mem: 25325K
# 2 elapsed: 142ms gen0: 8 gen1: 0 gen2: 0 mem: 23779K
# 3 elapsed: 143ms gen0: 8 gen1: 0 gen2: 0 mem: 23761K
# 4 elapsed: 138ms gen0: 8 gen1: 0 gen2: 0 mem: 23745K
# 5 elapsed: 135ms gen0: 7 gen1: 0 gen2: 0 mem: 25327K
# 6 elapsed: 135ms gen0: 8 gen1: 0 gen2: 0 mem: 23762K
# 7 elapsed: 137ms gen0: 8 gen1: 0 gen2: 0 mem: 23755K
# 8 elapsed: 140ms gen0: 8 gen1: 0 gen2: 0 mem: 23777K
# 9 elapsed: 174ms gen0: 7 gen1: 0 gen2: 0 mem: 25327K
#10 elapsed: 180ms gen0: 8 gen1: 0 gen2: 0 mem: 23762K

Now this code no longer consists of only value types, so the profiling is getting muddier now!
The mapAndFilter function uses createCustomerWithRandomActivity which in turn uses
Option , a reference type, so there will be a large number of reference types being

allocated. Just as in real life, it's hard to keep things pure!

Even so, you can see that the wrapped version is slower than the aliased version (approx
800ms vs. 150ms) and creates more garbage on each iteration (approx 72Mb vs 24Mb) and
most importantly has two big GC pauses (in the 5th and 9th iterations), while the aliased
version never even does a gen1 GC, let alone a gen2.
NOTE: The fact that aliased version is using up memory and yet there are no gen1s makes
me a bit suspicious of these figures. I think they might be different if run outside of F#
interactive.

What about non-record types?

What if the type we want to optimise is a discriminated union rather than a record?

1924

Swapping type-safety for high performance using compiler directives

My suggestion is to turn the DU into a struct with a tag for each case, and fields for all
possible data.
For example, let's say that we have DU that classifies an Activity into Active and
Inactive , and for the Active case we store the email and visits and for the inactive case

we only store the email:

module Classification =
open EmailAddress
open ActivityHistory
type Classification =
| Active of EmailAddress * int
| Inactive of EmailAddress
// constructor
let createActive email visits =
Active (email,visits)
let createInactive email =
Inactive email
// pattern matching
// not needed

To turn this into a struct, I would do something like this:

module Classification =
open EmailAddress
open ActivityHistory
open System
[<Struct>]
type Classification(isActive : bool, email: EmailAddress, visits: int) =
member this.IsActive = isActive
member this.Email = email
member this.Visits = visits
// constructor
let inline createActive email visits =
Classification(true,email,visits)
let inline createInactive email =
Classification(false,email,0)
// pattern matching
let inline (|Active|Inactive|) (c:Classification) =
if c.IsActive then
Active (c.Email,c.Visits)
else
Inactive (c.Email)

1925

Swapping type-safety for high performance using compiler directives

Note that Visits is not used in the Inactive case, so is set to a default value.
Now let's create a function that classifies the activity history, creates a Classification and
then filters and extracts the email only for active customers.
open Classification
let createClassifiedCustomer activity =
let email = ActivityHistory.email activity
let visits = ActivityHistory.visits activity
if isCustomerInactive activity then
Classification.createInactive email
else
Classification.createActive email visits
// execute creation and iteration for a large number of records
let extractActiveEmails noOfRecords =
Array.init noOfRecords (fun _ -> createCustomerWithRandomActivityHistory() )
// map to a classification
|> Array.map createClassifiedCustomer
// extract emails for active customers
|> Array.choose (function
| Active (email,visits) -> email |> Some
| Inactive _ -> None )
|> ignore

The code sample above is available on GitHub.

The results of profiling this function with the two different implementations are shown below:

1926

Swapping type-safety for high performance using compiler directives

=======================
extractActiveEmails: 1000000 records (Wrapped)
=======================
# 1 elapsed: 664ms gen0: 12 gen1: 6 gen2: 0 mem: 64542K
# 2 elapsed: 584ms gen0: 14 gen1: 7 gen2: 0 mem: 64590K
# 3 elapsed: 589ms gen0: 13 gen1: 7 gen2: 0 mem: 63616K
# 4 elapsed: 573ms gen0: 11 gen1: 5 gen2: 0 mem: 69438K
# 5 elapsed: 640ms gen0: 15 gen1: 7 gen2: 0 mem: 58464K
# 6 elapsed: 4297ms gen0: 13 gen1: 7 gen2: 1 mem:-256192K
# 7 elapsed: 593ms gen0: 14 gen1: 7 gen2: 0 mem: 64623K
# 8 elapsed: 621ms gen0: 13 gen1: 7 gen2: 0 mem: 63689K
# 9 elapsed: 577ms gen0: 11 gen1: 5 gen2: 0 mem: 69415K
#10 elapsed: 609ms gen0: 15 gen1: 7 gen2: 0 mem: 58480K
=======================
extractActiveEmails: 1000000 records (Aliased)
=======================
# 1 elapsed: 254ms gen0: 32 gen1: 1 gen2: 0 mem: 33162K
# 2 elapsed: 221ms gen0: 33 gen1: 0 gen2: 0 mem: 31532K
# 3 elapsed: 196ms gen0: 32 gen1: 0 gen2: 0 mem: 33113K
# 4 elapsed: 185ms gen0: 33 gen1: 0 gen2: 0 mem: 31523K
# 5 elapsed: 187ms gen0: 33 gen1: 0 gen2: 0 mem: 31532K
# 6 elapsed: 186ms gen0: 32 gen1: 0 gen2: 0 mem: 33095K
# 7 elapsed: 191ms gen0: 33 gen1: 0 gen2: 0 mem: 31514K
# 8 elapsed: 200ms gen0: 32 gen1: 0 gen2: 0 mem: 33096K
# 9 elapsed: 189ms gen0: 33 gen1: 0 gen2: 0 mem: 31531K
#10 elapsed: 3732ms gen0: 33 gen1: 1 gen2: 1 mem:-256432K

As before, the aliased/struct version is more performant, being faster and generating less
garbage (although there was a GC pause at the end, oh dear).

Questions
Isn't this a lot of work, creating two implementations?
Yes! I don't think you should do this in general. This is just an experiment on my part.
I suggest that turning records and DUs into structs is a last resort, only done after you have
eliminated all other bottlenecks first.
However, there may be a few special cases where speed and memory are critical, and then,
perhaps, it might be worth doing something like this.

What are the downsides?

In addition to all the extra work and maintenance, you mean?

1927

Swapping type-safety for high performance using compiler directives

Well, because the types are essentially private, we do lose some of the nice syntax available
when you have access to the internals of the type, such as {rec with ...} , but as I said,
you should really only be using this technique with small records anyway.
More importantly, value types like structs are not a silver bullet. They have their own
problems.
For example, they can be slower when passed as arguments (because of copy-by-value)
and you must be careful not to box them implicitly, otherwise you end up doing allocations
and creating garbage. Microsoft has guidelines on using classes vs structs, but see also this
commentary on breaking these guidelines and these rules.

What about using shadowing?

Shadowing is used when the client wants to use a different implementation. For example,
you can switch from unchecked to checked arithmetic by opening the Checked module.
More details here.
But that would not work here -- I don't want each client to decide which version of the type
they will use. That would lead to all sorts of incompatibility problems. Also, it's not a permodule decision, it's a decision based on deployment context.

What about more performant collection types?

I am using array everywhere as the collection type. If you want other high performing
collections, check out FSharpx.Collections or Funq collections.

You've mixed up allocations, mapping, filtering. What about

a more fine-grained analysis?
I'm trying to keep some semblage of dignity after I said that micro-benchmarking was bad!
So, yes, I deliberately created a case with mixed usage and measured it as a whole rather
than benchmarking each part separately. Your usage scenarios will obviously be different, so
I don't think there's any need to go deeper.
Also, I'm doing all my benchmarking in F# interactive. Compiled code with optimizations
might have a completely different performance profile.

What other ways are there to increase performance?

Since F# is a .NET language, the performance tips for C# work for F# as well, standard stuff
like:

1928

Swapping type-safety for high performance using compiler directives

Make all I/O async. Use streaming IO over random access IO if possible. Batch up your
requests.
Check your algorithms. Anything worse than O(n log(n)) should be looked at.
Don't do things twice. Cache as needed.
Keep things in the CPU cache by keeping objects in contiguous memory and avoiding
too many deep reference (pointer) chains. Things that help with this are using arrays
instead of lists, value types instead of reference types, etc.
Avoid pressure on the garbage collector by minimizing allocations. Avoid creating longlived objects that survive gen0 collections.
To be clear, I don't claim to be an expert on .NET performance and garbage collection. In
fact, if you see something wrong with this analysis, please let me know!
Here are some sources that helped me:
The book Writing High-Performance .NET Code by Ben Watson.
Martin Thompson has a great blog on performance and some excellent videos, such as
Top 10 Performance Folklore. (Good summary here.)
Understanding Latency, a video by Gil Tene.
Essential Truths Everyone Should Know about Performance in a Large Managed
Codebase, a video by Dustin Cambell at Microsoft.
For F# in particular:
Yan Cui has some blog posts on records vs structs and memory layout.
Jon Harrop has a number of good articles such as this one but some of it is behind
a paywall.
Video: High Performance F# in .NET and on the GPU with Jack Pappas. The
sound is bad, but the slides and discussion are good!
Resources for Math and Statistics on fsharp.org

Summary
"Keep it clean; keep it simple; aim to be elegant." -- Martin Thompson
This was a little experiment to see if I could have my cake and eat it too. Domain modelling
using lots of types, but with the ability to get performance when needed in an elegant way.
I think that this is quite a nice solution, but as I said earlier, this optimization (and uglification)
should only ever be needed for a small number of heavily used core types that are allocated
many millions of times.
Finally, I have not used this approach myself in a large production system (I've never needed
to), so I would be interested in getting feedback from people in the trenches on what they do.

1929

Swapping type-safety for high performance using compiler directives

The code samples used in this post are available on GitHub.

1930

Viz Api Dev Guide PDF
No ratings yet
Viz Api Dev Guide PDF
1,230 pages
What Is Fsharp Microsoft Document
No ratings yet
What Is Fsharp Microsoft Document
582 pages
F# Bog
No ratings yet
F# Bog
332 pages
F# Path To Relaxation
100% (1)
F# Path To Relaxation
88 pages
Fs Applied II
No ratings yet
Fs Applied II
276 pages
FSharpSpec 4.1 Latest
No ratings yet
FSharpSpec 4.1 Latest
333 pages
Fp Simplified 1.0
No ratings yet
Fp Simplified 1.0
1,020 pages
F# High Performance
From Everand
F# High Performance
Eriawan Kusumawardhono
No ratings yet
Functional Programming in C#: How to write better C# code 1st Edition Enrico Buonanno - Discover the ebook with all chapters in just a few seconds
No ratings yet
Functional Programming in C#: How to write better C# code 1st Edition Enrico Buonanno - Discover the ebook with all chapters in just a few seconds
62 pages
WikiBooks - FSharp
No ratings yet
WikiBooks - FSharp
126 pages
Gruop 2 (F#) .CSE425.5
No ratings yet
Gruop 2 (F#) .CSE425.5
12 pages
深度学习实战手册（R语言版）: Chinese Edition
From Everand
深度学习实战手册（R语言版）: Chinese Edition
Posts & Telecom Press
No ratings yet
Functional Programming
No ratings yet
Functional Programming
1,020 pages
Full Fsharp Blog
No ratings yet
Full Fsharp Blog
11 pages
R Fundamentals (Hadley Wickham - Rice Univ)
No ratings yet
R Fundamentals (Hadley Wickham - Rice Univ)
66 pages
The F Survival Guide PDF
No ratings yet
The F Survival Guide PDF
136 pages
Unit 2a
No ratings yet
Unit 2a
51 pages
Metacognitive Aspects of Moral Reasoning and Behavior
No ratings yet
Metacognitive Aspects of Moral Reasoning and Behavior
20 pages
سلسلة المحاضرات الإليكترونية فى علم المساحة المحاضرة 3 المساحات وتقسيم الألأراضى Areas & Land subdivision
67% (3)
سلسلة المحاضرات الإليكترونية فى علم المساحة المحاضرة 3 المساحات وتقسيم الألأراضى Areas & Land subdivision
48 pages
Fpsimplified Free Preview
100% (1)
Fpsimplified Free Preview
611 pages
Fizica Yale
No ratings yet
Fizica Yale
53 pages
Fundamentals of F# Programming by Theophilus Edet
100% (1)
Fundamentals of F# Programming by Theophilus Edet
242 pages
Mill, Index Machine & Tool 6459870
No ratings yet
Mill, Index Machine & Tool 6459870
1 page
8th Lesson Stats
No ratings yet
8th Lesson Stats
42 pages
Fsharp
No ratings yet
Fsharp
14 pages
Research Paper
No ratings yet
Research Paper
19 pages
Introduction To Software Architecture
No ratings yet
Introduction To Software Architecture
89 pages
4th sem EC 401,02,03,ES CS 401,BSm401,BSB401 ilovepdf_merged (1)
No ratings yet
4th sem EC 401,02,03,ES CS 401,BSm401,BSB401 ilovepdf_merged (1)
11 pages
Fomrhi Q138 Supplementlowres
100% (1)
Fomrhi Q138 Supplementlowres
32 pages
Dokumen - Tips Mathematics Kumar by 1 M S Kumarswamy Tgtmaths Mathematics Question Bank
No ratings yet
Dokumen - Tips Mathematics Kumar by 1 M S Kumarswamy Tgtmaths Mathematics Question Bank
61 pages
c++final revision fall 2024
No ratings yet
c++final revision fall 2024
8 pages
Mastering Python: A Journey Through Programming and Beyond
From Everand
Mastering Python: A Journey Through Programming and Beyond
THE NORTHERN HIMALAYAS
No ratings yet
Programming F# 3.0 - Chris Smith
100% (2)
Programming F# 3.0 - Chris Smith
724 pages
How to learn F (sharp)
No ratings yet
How to learn F (sharp)
5 pages
Programming F# (Chris Smith)
No ratings yet
Programming F# (Chris Smith)
408 pages
Rust Design Patterns
No ratings yet
Rust Design Patterns
91 pages
Professor Frisby's Mostly Adequate Guide to Functional Programming
No ratings yet
Professor Frisby's Mostly Adequate Guide to Functional Programming
183 pages
C# Basics
No ratings yet
C# Basics
9 pages
Weebly
No ratings yet
Weebly
6 pages
Algorithms - Assignment - UNIT 4
No ratings yet
Algorithms - Assignment - UNIT 4
28 pages
F# Documentation
No ratings yet
F# Documentation
626 pages
Measuring and Modeling Vertical Crosshead Vibration in Api-618 Reciprocating Compressors
No ratings yet
Measuring and Modeling Vertical Crosshead Vibration in Api-618 Reciprocating Compressors
18 pages
C# 101: Intro To C# Programming
No ratings yet
C# 101: Intro To C# Programming
71 pages
DBMS MCQs Gr8AmbitionZ
No ratings yet
DBMS MCQs Gr8AmbitionZ
20 pages
Iemh 1 An
No ratings yet
Iemh 1 An
27 pages
F#
No ratings yet
F#
142 pages
Ramanujan - the mathamatics genius
No ratings yet
Ramanujan - the mathamatics genius
9 pages
What Is Functional Programming?
No ratings yet
What Is Functional Programming?
16 pages
Basic C# Cheat Sheet
No ratings yet
Basic C# Cheat Sheet
13 pages
Course Overview: Nate Foster and Michael George Cornell University Spring 2014
No ratings yet
Course Overview: Nate Foster and Michael George Cornell University Spring 2014
34 pages
F# Cheat Sheet
No ratings yet
F# Cheat Sheet
3 pages
Functional Programming for the Object-Orie - Brian_Marick
No ratings yet
Functional Programming for the Object-Orie - Brian_Marick
382 pages
Functional Programming Using F
100% (2)
Functional Programming Using F
536 pages
Poetry
No ratings yet
Poetry
100 pages
FDM FFF 3D Printing Quick Start Guide Nov21 Release Makersmuse
No ratings yet
FDM FFF 3D Printing Quick Start Guide Nov21 Release Makersmuse
15 pages
The Ball-Pivoting Algorithm For Surface Reconstruction
No ratings yet
The Ball-Pivoting Algorithm For Surface Reconstruction
11 pages
07 TLG17 MarceloLlano-Serna Final
No ratings yet
07 TLG17 MarceloLlano-Serna Final
9 pages
Ignou Mca Solved Assignments
No ratings yet
Ignou Mca Solved Assignments
19 pages
The F# Survival Guide
100% (1)
The F# Survival Guide
136 pages
Functional Programming 2.2
No ratings yet
Functional Programming 2.2
114 pages
Formic Acid
No ratings yet
Formic Acid
20 pages
Microsoft Word with NVDA
From Everand
Microsoft Word with NVDA
NV Access
3/5 (1)
Haskell - The Craft of Functional Programming, 2ed (Addison-Wesley, 1999) by Tantanoid
No ratings yet
Haskell - The Craft of Functional Programming, 2ed (Addison-Wesley, 1999) by Tantanoid
505 pages
Class 9 - SCT
No ratings yet
Class 9 - SCT
9 pages
Fsharp Cheatsheet
No ratings yet
Fsharp Cheatsheet
17 pages
Expressions Dari QGIS USER DOCUMENT
No ratings yet
Expressions Dari QGIS USER DOCUMENT
9 pages
Fsharp Cheatsheet
No ratings yet
Fsharp Cheatsheet
3 pages
2f30025f-382e-488d-8351-ca72dec5a1b8_C (2)
No ratings yet
2f30025f-382e-488d-8351-ca72dec5a1b8_C (2)
33 pages
Functional Programming Using F# PDF
No ratings yet
Functional Programming Using F# PDF
376 pages
CITS 3242 Programming Paradigms: Introduction To Functional Programming
No ratings yet
CITS 3242 Programming Paradigms: Introduction To Functional Programming
17 pages
Predefined C# Value Types: Boolean Types Are Declared Using The Keyword, Bool. They Have Two Values: True or False
No ratings yet
Predefined C# Value Types: Boolean Types Are Declared Using The Keyword, Bool. They Have Two Values: True or False
25 pages
Ultraviolet: Ultraviolet (UV) Light Is
No ratings yet
Ultraviolet: Ultraviolet (UV) Light Is
17 pages
Wintering Bees in Cold Climates - Compressed
No ratings yet
Wintering Bees in Cold Climates - Compressed
21 pages
311 Fall 2017 Final Exam
No ratings yet
311 Fall 2017 Final Exam
10 pages
Process Design in Impression Die Forging
No ratings yet
Process Design in Impression Die Forging
12 pages
Mediasite System Overview
No ratings yet
Mediasite System Overview
14 pages
FL Handbook 13
100% (2)
FL Handbook 13
36 pages
CSPC41 Es 2022
No ratings yet
CSPC41 Es 2022
2 pages
Fsharp Cheatsheet
No ratings yet
Fsharp Cheatsheet
3 pages
Fsharpcheatsheet
No ratings yet
Fsharpcheatsheet
7 pages
C# C Programming Unlocked - Andrew Sullivan
No ratings yet
C# C Programming Unlocked - Andrew Sullivan
338 pages
Coltman Boehm 1.35 Rotated
No ratings yet
Coltman Boehm 1.35 Rotated
9 pages
Vipassana For Hackers
No ratings yet
Vipassana For Hackers
7 pages
Mastering Fortran Programming: From Basics to Expert Proficiency
From Everand
Mastering Fortran Programming: From Basics to Expert Proficiency
William Smith
No ratings yet
Build Your Own Professional-Grade Audio Amp On TH
No ratings yet
Build Your Own Professional-Grade Audio Amp On TH
3 pages
Bar Whole cm1
No ratings yet
Bar Whole cm1
2 pages
Introduction to bada: A Developer's Guide
From Everand
Introduction to bada: A Developer's Guide
Ben Morris
No ratings yet
Fajardo Wedge Dimensions
100% (1)
Fajardo Wedge Dimensions
1 page
GETTING STARTED WITH OPENOFFICE WRITER
From Everand
GETTING STARTED WITH OPENOFFICE WRITER
Remy Lentzner
No ratings yet
Getting started with Sparkle
From Everand
Getting started with Sparkle
Remy Lentzner
No ratings yet
Learning F# Functional Data Structures and Algorithms - Sample Chapter
No ratings yet
Learning F# Functional Data Structures and Algorithms - Sample Chapter
26 pages
Project ON Risk and Return Analysis of Mutual Funds
No ratings yet
Project ON Risk and Return Analysis of Mutual Funds
4 pages
Learn WPF MVVM - XAML, C# and the MVVM pattern
From Everand
Learn WPF MVVM - XAML, C# and the MVVM pattern
Arnaud Weil
4/5 (6)
iOS 8 App Development Essentials
From Everand
iOS 8 App Development Essentials
Neil Smyth
No ratings yet
Metal Spinning Handout
No ratings yet
Metal Spinning Handout
4 pages
Fennel Explained: A Lisp for Lua in Game Development and Embedding
From Everand
Fennel Explained: A Lisp for Lua in Game Development and Embedding
Robert Johnson
No ratings yet
Mostly Adequate Guide
100% (2)
Mostly Adequate Guide
115 pages
Quality Software: Volume 1.1: How Software Is Built
From Everand
Quality Software: Volume 1.1: How Software Is Built
Gerald M. Weinberg
No ratings yet
Mediasite - SAML 2.0 Configuration
No ratings yet
Mediasite - SAML 2.0 Configuration
2 pages
Modeling Reference - Valve With Linear Area Change
No ratings yet
Modeling Reference - Valve With Linear Area Change
2 pages
Getting started with Adobe Acrobat Pro
From Everand
Getting started with Adobe Acrobat Pro
Rémy Lentzner
5/5 (1)
CFD Analysis
No ratings yet
CFD Analysis
7 pages
Iso 25862-2019
No ratings yet
Iso 25862-2019
56 pages
The Swift Codebook: A Beginner's Guide from Basics to Best Practices
From Everand
The Swift Codebook: A Beginner's Guide from Basics to Best Practices
Grace Huang
No ratings yet
Functional Programming Fundamentals
No ratings yet
Functional Programming Fundamentals
66 pages
Mastering Visual Studio Code: Navigating the Future of Development
From Everand
Mastering Visual Studio Code: Navigating the Future of Development
Kameron Hussain
No ratings yet