xpressive:

Library Design on the Edge

or, How I Learned to

Stop Worrying and Love
Template Meta-
Programming

Copyright 2005 Eric Niebler

“Why program by hand in five days
what you can spend five years of your
life automating?”

-- Terrence Parr, author ANTLR/PCCTS

Copyright 2005 Eric Niebler

Overview

 Domain-Specific (Embedded) Languages

 xpressive and Dual-Mode DSEL Design
 Programming at the Compile-time / Runtime
boundary

Copyright 2005 Eric Niebler

Domain-Specific Languages

 Mini-languages, everywhere!
– GNU Make
– Backus Naur Form
– Regular expressions
 Syntax, constructs and abstractions for
working efficiently in some narrow domain
 Not general purpose!

Copyright 2005 Eric Niebler

Why DSLs?

 General purpose languages are low-level,

procedural
 DSLs are high-level and declarative
 Solution space shares concepts with problem
space
 DSL code is easier to write, read, reason
about and maintain.

Copyright 2005 Eric Niebler

Embedded DSLs

 A DSL in a library
 All the wholesome goodness of a DSL in the
host language of your choice!
 C++ is a good host language
– operator overloading
– low abstraction penalty

Copyright 2005 Eric Niebler

Expression Templates

 (Ab)uses C++ operator overloading to

approximate the syntax of the embedded
language
 Must map embedded language syntax and
constructs into C++
 Used by:
– Blitz++ : high-performance scientific computing
– Spirit : parser generator (EBNF)

Copyright 2005 Eric Niebler

DSEL example: EBNF Calculator

rule<> group, fact, term, expr;

group =
::=
'('
'('>>expr
expr')'
>> ')';
fact =::=
integer
integer| group;
| group
term =::=
fact
fact>>(('*'
*(('*'fact)
>> fact)
| ('/'
| fact))*
('/' >>
fact));
expr ::= term (('+' term) | ('-' term))*
expr = term >> *(('+' >> term) | ('-' >>
term));

Copyright 2005 Eric Niebler

Types of DSELs: Dynamic

 Example:
SQLCommand c = "SELECT * from Employees";
 Advantages:
– Unconstrained syntax
– Statements can be specified at runtime
 Disadvantages:
– Syntax errors discovered at runtime
– Performance costs of interpretation

Copyright 2005 Eric Niebler

Types of DSELs: Static

 Example:
double d = (matrix * vector)(3, 4);
 Advantages:
– Syntax errors checked at compile-time
– Aggressive inlining, domain-specific codegen
 Disadvantages:
– Constrained by rules for legal C++ expressions
– Cannot accept new statements at runtime

Copyright 2005 Eric Niebler

Hrrmmm ...

Static
DSELs...
Can’t I have it
both ways?

Dynamic
DSELs...

Copyright 2005 Eric Niebler

Dual-Mode DSEL Interface

 Provide both a static and dynamic interface!

 Sounds good but ...
– Can we really get the advantages of both?
– Can we share the implementation to avoid code
duplication?

Copyright 2005 Eric Niebler

Regular Expression
Expression Template

"\\w" _w
"\\w+" +_w
"a\\w" 'a' >> _w
"a|b" as_xpr('a') | 'b'
"(\\w)\\1" (s1= _w) >> s1
"[^a-z]" ~range('a', 'z')
"(?=foo)" before("foo")

Copyright 2005 Eric Niebler

Get a Date

// Match a date of the form 10-03-2005

#
regex date = regex::compile(
_d >> !_d >> '-'
/\d\d?-\d\d?-\d\d(?:\d\d)?/ // match month
"\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?");
"\\d\\d?-\\d\\d?-\\d\\d(?:\\d\\d)?";
>> _d >> !_d >> '-' // match day
>> _d >> _d >> !(_d >> _d); // match year

Copyright 2005 Eric Niebler

Regex aliases, anyone?

regex date = /* ... */;

// A line in a log file is a date followed by a

// space, and everything up to the newline.
regex log = date >> ' ' >> +~set['\n'];

Copyright 2005 Eric Niebler

Semantic Constraints

// Only match valid dates

regex date =
(_d >> !_d)[if_is_month()] >> '-'
>> (_d >> !_d)[if_is_day()] >> '-'
>> (_d >> _d >> !(_d >> _d))[if_is_year()];

Copyright 2005 Eric Niebler

Two great tastes ...
... that taste great together

// A line in a log file is a date followed by a

// space, and everything up to the newline.
regex date = regex::compile(get_date_pattern());

regex log = date >> ' ' >> +~set['\n'];

Copyright 2005 Eric Niebler

“Some people, when confronted with
a problem, think, ‘I know, I’ll use
regular expressions.’ Now they have
two problems.”

--Jamie Zawinski, in comp.lang.emacs

Copyright 2005 Eric Niebler

Recursive regexen!
regex parens;
parens // A balanced set of parens ...
= '(' // is an opening paren ...
>> // followed by ...
*( // zero or more ...
keep( +~(set='(',')') ) // of a bunch of things that are
// not parens ...
| // or ...
by_ref(parens) // a balanced set of parens
) // (ooh, recursion!) ...
>> // followed by ...
')' // a closing paren
;

Copyright 2005 Eric Niebler

A Regex Calculator?!

regex group, fact, term, expr;

group = '(' >> by_ref(expr) >> ')';

fact = +_d | group;
term = fact >> *(('*' >> fact) | ('/' >>
fact));
expr = term >> *(('+' >> term) | ('-' >>
term));

Copyright 2005 Eric Niebler

C++ library design at the edge

Wife: It's a floor wax!

Husband: No, it's a dessert topping!
Announcer: Stop! You're both right. It's a
floor wax and a dessert
topping!

-- Saturday Night Live

Copyright 2005 Eric Niebler

STL, MPL and Fusion, Oh My!

// Just the data, ma’am

std::list<int> integers;

// Just the types, ma’am

typedef
mpl::list<int, double, std::string>
types;

// Types and data, please!

fusion::tuple<int, double, std::string> data
= fusion::make_tuple(1, 3.14, "hello");

Copyright 2005 Eric Niebler

The Fusion Library

 Heterogeneous data structures

 STL-influenced
– containers, iterators, algorithms
 MPL-compatible
 by Joel de Guzman, part of Boost.Spirit
– http://spirit.sourceforge.net

Copyright 2005 Eric Niebler

A Simple Fusion-esque List

struct nil = {};

template<class Car, class Cdr = nil>

struct cons {
Car car; Cdr cdr;
cons(Car const & a, Cdr const & d = Cdr())
: car(a), cdr(d) {}
};

inline cons<Car,Cdr>
make_cons(Car const & a, Cdr const & d)
{ return cons(a, d); }
Copyright 2005 Eric Niebler
Simple Fusion-esque List, cont.

cons<int,cons<double,cons<std::string> > > data

=
make_cons(1,
make_cons(3.14,
make_cons(std::string("hello"))));

Copyright 2005 Eric Niebler

Fusion-esque algorithms

template<typename F>
void for_each(nil, F) {}

template<class Car, class Cdr, class F>

void for_each(cons<Car, Cdr> const & l, F f)
{
f(l.car);
for_each(l.cdr, f);
}

Copyright 2005 Eric Niebler

Programming challenge!!!

Write the type of a std::pair<First, Second>

where Second is a pointer to the whole
std::pair ...

std::pair<int, std::pair<int,
??? *> ???*> *>
std::pair<int, ???*> *> *>
std::pair<int,
std::pair<int,
std::pair<int,
std::pair<int,
std::pair<int, Copyright 2005 Eric Niebler
Dual-Mode DSEL Design Strategy

 recursive algorithms
 one modular core
 two binding policies: static and dynamic
 acyclic data structures

Copyright 2005 Eric Niebler

xpressive Matchers

// Match any single character

struct any_matcher
{
template< class Iter, class Next >
bool match(Iter i1, Iter i2, Next const &n)
const
{
if ( i1 == i2 ) { return false; }
return n.match( ++i1, i2 );
}
};

Copyright 2005 Eric Niebler

xpressive static Scaffold

template< class Matcher, class Next >

struct static_xpression {
Matcher matcher;
Next next;

template< class Iter >

bool match( Iter i1, Iter i2 ) const {
return matcher.match( i1, i2, next );
}
};

Copyright 2005 Eric Niebler

xpressive dynamic Scaffold

template< class Iter >

struct matchable {
virtual bool match( Iter, Iter ) const = 0;
};

Copyright 2005 Eric Niebler

xpr: dynamic Scaffold, cont.

template< class Matcher, class Iter >

struct dynamic_xpression : matchable< Iter > {
Matcher matcher;
matchable< Iter > * pnext;

bool match( Iter i1, Iter i2 ) const {

return matcher.match( i1, i2, *pnext );
}
};

Copyright 2005 Eric Niebler

Separation of Concerns

 Matchers implement core functionality

 Scaffolds implement binding policy (static or
dynamic)
 Matchers are binding-neutral, reusable
 Best of both worlds
– perf of static binding
– flexibility of dynamic dispatch

Copyright 2005 Eric Niebler

References

 xpressive:
– http://boost-sandbox.sf.net/libs/xpressive
 Spirit and Fusion
– by Joel de Guzman
– http://spirit.sf.net

Copyright 2005 Eric Niebler

Questions?

Copyright 2005 Eric Niebler