Visual Programming:

Cooking the Spaghetti

GOTO Chicago '13

Rick Lane
rick.lane@tradingtechnologies.com
@R1ck_L4n3 @trading_tech @GOTOChgo
First things first
● Visual Programming < Regular Programming (usually)
● But, there's a time and a place
● How to make it work, when it makes sense to work
Who I am
● Chief technology officer at Trading Technologies
○ TT makes software for professional traders around
the world
● Background in wargaming: terrorist
network modeling and analysis, agent-
based modeling, etc.
● Many similarities between these
industries: a knowledge gap between the
idea generators and the technologists
 e,
Lakin, Fred, Visua cienc
P Who I am not
Seattle, Washing
Key
Eco
a
mnv
nd weor
pirer
y,ds Raan
to
l Grammars for V

jedevPahrnases: visual gr
d
n,
M
Ju ly 19 87 .
am
isual Languages,

m
P ro ce ed in gs of the American A,ss
iv e r
Dep t. o f Com
sity ociation for Artificial Intellige
put e r S

icsa ar no nce,
KAe.7 yw
al tioStu na dyl ,gr1ap 9 9 hicsa,rgviasu real t Blis u rp,n e
tation, declarativ
e programSm t a te Un
in
o
: Lordgiscala 3 IEEE tt
vi ,su IsalItem E acs, vmacs, VP o n g, vi sual ruele
uBsuerrn-e ndngPua hrge asses: F Symposium o asier to W , L-OrIV eg.C : Parsinga, nVgPuL- ag s s, formal visual languages, info
intte t, rM fa o rms/3, V n r it
m es /
M 3 l VI.A: Formal defin rmal
acregadreesig V r a ed
1992
Meyer Pr,oBcer t M n . is
a s u e PL-II.A.4 isouuarl oLfaFnoguag trix Meatn-bipausla tion Pro
ition of VPLs, VP
L-II.
eend ,g BeyondnIc d on Aslls: ed: F Torm-b es, dBsehreg
Kaenyg
Y
IIo
L
Prowce
Iby
.gEic :SE
,oS
apr
● I am not a visual programming academic
lvR
redd. sainn
einnege
ags
din
ndMof
d P. hBth
s o f
e Ate
ruarsneIn
dvaat
estt: , rn ncio
e n
euna
ToL
drnVleiW
. w ar
tAt,m
ds
A ler,
orbks
G u
N
hoG
id
ew
p eon
M
n
etaphorsndfosran
s erfada
a s e d p r e
Vdissua
a
prleQaue
e n
dsry
,
hee
N
La
o rway, A
ng ua ge
grams V
ugust 2
4
isually o
r Textua
t lah r,
ti
a onLo
n nd on , 19 F Fo
M r o
. mB C s u a l I m e
- In
bra tel i z ce s to D at t- b a s e d s fo r S pa - 2tia7,l 1In9fo rm lly? An
KeKyey wowror d s h ip
lingrsg,a1re99 t 93 r ).
m o
s/3, Vcrete n n t eFroface r i n g Eve ab as e Systems, Glasg la nguage 9 3 . ation Sys
● There is a sea of research on the various forms,
dsds anan dnP dd P
hM r hr
a a as es: VPL-EII. 4 IE EP S -PI:m A
IL.Lo . 4 :
F o r m
-II.A.4 s toInGteernna s n t Detec ow , 11
su
s ,3-V13
al PL5,
Pr og
- V ra
Ju mly m19 ing Late ngmua s, ges,
a s e s : A L.7 y p gi : e t i t i o an dli ng in Vi I. C .1 92 . (p ub
d
.
ith G e 1995. orm
Forsm/3s,/3V
, V P
osciula mng
A
Foonua
De rm V
cla ge
-isbra as,stiv VreP aoliL-
Apnza e ldB
II.pr W oa
A .1ob:ch
rs Dtr
to nEv
ia
an en d t-HR e p o
: sCiuom gnoitnivelis hed
anVP
s , J u u n : F d Al len L. AmL - II ble
.A .4r, u a e
l Ld a an n k s ah c
ogr
pt ti
p.am
o n
15 s m
-1 at
8, s
ic 19p o
la 92n
ng s.ua a l S y m n, d L-
y
Ha Bu J
6,t, Marhga set
re sM . an : FoSe A, 34 g duas-4p 0, Se , th
Romreough P intV e ge s, V
aestPio,L-n .A :ab Abstr ac tio
TR 919
w
5 -
o
6rn
92
r
0 -et
dsIE
families, perils and merits of visual programming
anEE
r a
d PWorkshop on Visual Lang
Fo rm s/3 , VP
ua
L-
ge
II. A.
s, r m at
-
4: Form-baseddan
b tlea s, W
ed an d spresad
g e
r e s
spreads heLsh
a, dS
P
t.
ha ep
eotuee
ee t-
ist-btas
r-)b, a1, s1M e
,
d
I
hOtI,ed
e
lan I n
gu e r
ag
i
aElyE,EMrspectivael Prog
l a6-14,aO
n
s u
y 27-2 -Orien
c 9
VP V L-
.B :IIID
ratemm
d
at
A i n
as
g
e languages
Key Keywords and Phrases: oste r n
based la guag tober, 41992 eveal ly
ming (P nguageepu s,t,VJo ur -na
7,igl1h .99LV4isuaLl a
of sLangng
is oufaua ges
ges,
III.E: Even ○ You won’t find any of that here
t ha nd
Laki
lin g
r C oGnra strha aint Pr VL:
m ,
o g
Gr 5-9,V1is9ua
r a m
95l.Specification of Gra m Proc. on
ph
iu
ic al O ut s V P e Lr y
- I IHI .A: Ab .
stracutle-based
.N n
e ic t ho f o la s m b e e s p o
X Sym terface generatio
s
I.Age . s: R ion, VPL-
1 1
Cordy, JamV es RKa.elan n irFor n
ndv, T. e
Cm y , Septe sed languag E NIU nVla Png L-Iua
Hltardin ri,, A
u
is 3, w
ting,
yE
19 osrt92 dd
a
d , r m a n
.t, GCeomp nstraint-b l langua a
e ge Rs, uleVsPL-V.E:, U S se r- in
lanDgis agbuted Implementation of Multi-
utri e s ,
an
a r v
C ie
e om
y pu F D u a
n r m s a n.A : C o
u t:
2II.A.5 ng w raph Ri Fu nc tio na e w r it r A ut om
d ig mat ic nto, October
M. POcto ngu N
ag.an easnd,dPJhr .
castioVes nrP :LV -IIP dL-.P hras wites Gith l Views as a M odelultfo i-para pe rative Work, Toro 1 Workshop
K leyL aw b or
e ds s e M
s : o ra l l a rn h , R el
T at io na9 4 . . 8 : M up po rte d C oo
Vis uKa
Boursier, rP.19and 9 h.r.a M. Mainguenaud, lasis , Nrnd
g aLgm
To mgre gUes: c Query
inSpatial 9G
1p0th0e,xC1tLanguages .AomputeSQL
L: -Extended
IIC r-S vs. Visual Languages vs. WG'9
Hypermaps, 5th
IB
y G
w eora
yrw
International dha s rm
o adn, dT.
s
P3 CStaNnicpho
Symposium id P oan
ron :u
Spatial
Aa Gee
of s
t io
,nthV
Data e
n e,ra8Fo o3m
l
-ur
Handling,
P u t g
-on
u a rfe ehsnc
agpre
Charleston,
, VePon
i USA, August 3-7, 1992. t e m s , Proc. asl Report A
e a ad ed in gs c ia u i
nn c s
y ration laTng ge
ic
K Koe
Uese lm ,r aA A,npp d.rlicenaO dsio
at Z,PCns
Rhra,inPgro
sE
ce
e,sNVPL-II:
:CIVX.P Asso PLt-iv
Impera Classifications Ve.A rplaosge w
:sGe Viis
F
thutael D orms, P .E: User-hinRteerfa wrce iting geSne also:
ecua hn
Keywords r r Dand , R Phrases: , HU S anLId.-A Language
. 6 : x a P L- V r p 9 1 . sed
th u
Sc e19192 9 g 9 .
e 2 s A ( VH L
Keyw . L v )
an Ba: leI ,
s
M en
V lPb L
o
I
-I GA. .RI .
A 3.5 :
o D
la
: n
a
Fu gau
t nc g e
ational la nng eraua l -
ge t s,
-g t V
a f
raphrw lo
e s L o
faoc regG
n e ud
r a
i n , 4 8
cation - 6 1 9
2, Language, Proc. WG'89
. 1 1 : R ule-ba
a s, Introdu CMP/S Poshr e , aurm dL- II. t ic y - flowtelaGraph p urtpruoctar cBasedu i fo mr Vegrgla Specifi
a g -II.A
aneg
K
Schuerr, LAndrea yu
K w
ey orw rddsor sads ndan rIG
hdction a ArdPassPDaes
to
:
PROG
S
:V
ndygm
ia ranP n emA, a
RES, .ASum an, A Attribu
euldFeuScienc ncttiic ngounatgro Gramm
elsS seS(Nagl p rningreM., s, ed.),
es, o Tfetchhnic
e er alVerlag g s, VPL 5, 1989.
,e151-16
y w o a n d I I .B . 1 pts P ph in
o Compu s t r a ter r s ,
is o n C
e, LNCS , V411,
7 P 0 L, la g v is u Spring L I S R
n e
g u
p a
o
WorkshopKe on Grapht heoreti LP-h c sConce rassiuem onteliarm in SCIaLl-,LVV 5 -V.A: G uaege al padigm laP Co rt, UnivRule-b
Glinert, agEphraimes, VP P.raand es:Charles VPImpera -IrIf.,AD. osNorton, eP Ap, p2Novis:
n: dVlangua 0 9d-n2C eA, Visual NPiC S
s: uA.8:
aam Laboratory nesuExploring
for ara tges,
langua tern DesignVPL-II. nfeA.11: ersity oased
Keywordslaand nguPhrase s: VPL-II.A.6:
u e n
Lo
d N. tive
3 : D a ut tLe -
r I IS.
li
A
ges,
c eie c1VPL-II.
4
o m, Np u
l P u Multi-p
rlto i-
g p u r
I p
.
aradigm
A o . 8
s e: M r,altVi-
l- pu
p
P L r-pIVosthe e la m a n
of
renProcessor
c e
fArrays,
d
TItaesramtio n,
ania,
Journal of Visual Languages
A . Z and o m
Computing, p a - f lo w . Vol.
5 : 3,
lanFguunacg age2s8, 1V o
Number t
v in
. g , 2, P Lr - I
135-159,
a m v June
is u 1992. . D : n g u ai p u , S n t t a n
languages, rnetet,rr,B.1:
BuVPL-II. reas, mmatic
ndDiagra langua
epts inVPL-II.A.1:
C ges -1D8e8-1VPL-II.B.2: 1m in,g1I al programTm rans gAelssastig ion nme anford
Keywords
S c h u M Aand
a r g Phrases:
r e tic C oncNovis, a n y , 1991. Concurrent a t iv e la ngtuie sn,aVPL
olanguages,l - I I I c . 1
98
, 9n9te2r.fIconicelanguages, n S ining g laetnor VPL-IV.A:
le
v s
lang Computational
uage ro Univw
Langu h-theaoret M , G erm representation m p er lang .A.2: 1P99 aectewe irGraph-
o(innm t Gramm t arsl floersi
approaches,
ein
Schuerr,KAndreaywG
aa g p
r s,ePROG , JoTuH VPL-VI.C.6:
r n RES:
A achaA . e n
ndVHL-La Visual
A Vll P
e L - I.A
nguage
I . 6 : I Based n g u a g issues
on s
e Graph Gramm gees Confe u aars,
t h
Proc. r o 4
c li 4th
c t
. dural B Int.
a
t
b
Worksh
o Geo
g
op on
P L - I
e
I
er n t
I.pBr:etP , C
e
o rroceeisdual l ty, Augu
n , v
W al ofraVses: n L. Amatic 532, Springer Verlag la d 989. also:latTechni s racutio arga V
sh,i Report AIB 90-16, s an ingsiso st
and Their s 9
abApplica 1
o
-1rd7s,tion R andto Pn Compu
d P h ter isuaScienc l ia g r a e,
m m
LNCS b le r , a n g u a ges a,n641-65 , O c t9,
. 11991. d n g ncal
ep
, V c
P aL l- dVcis
f om
p
ual f, VPL-II 1980.
t r a c r d s
on, V ny,D.L a h r a s . 1 : D L a n g I n t e u a l L 8 - 1 4 3 -b a s e V
I nI .
f C
o . 4 n a l o p i lers) .A.5:
RWTH Aachen Key,tw
Norton, io Charles
Germa e P
s , L V- I PI
-and
1990.IeI.sB: FEphraim orms/3 u P.agGlinert,
. B uersne a tnt, VAisrVisual
d
active Environment
m e , V aly
Itis , 13
s
for Designing
p r e adshee
t and Simulating Execution
n m e r,m
ntRule-b Jo :auofrcoProcessor
S
t i o n pe
ased
a s:g I . EWorkshop M
t , VVisual C o
oRmpuVPL-II. u a d
anMulti-p aradigm langua ges, VPL-II. s ig A.11: Syst
CaArrays, ngu1990 IEEE VPL-II. : EA.6: veanrg areon
t Impera PL langua
tive aLanguages,
es,ges,
.g4:Concurrent
Skokie, edl DaIllinois,
sA.8: ta Abs227-232, Octoberin1990. gle As losw
Keywords and
rls olan,Phrase
Keywords P a and e n L .
Phrases: a nd M Novis, V
hanudaVPL-II.A.1:
is l
li nLgan-gIIu.A A F
. 4o :r F
m o rm tin-b ga ,languages,
2 9 - 6 0 t r a
VPL-II.B.2: c t io P r e s e rve Slanguages, VPL-IV.A:
Iconic . B : C o ntroe
Computational
l fm , Pro
langua(ges,
posteVPL-II. r , uA l
B.1: ll
a Diagra mmatic o n langua ges L - I I . - b a sed an Iteraatrio , M t h a t n in L - I I I ceed
Am rp
approaches,ba le
p
ndoM r
VPL-VI.C.6:
W k asrhgoap Visual r m s / 2 , VP
representation issues d f chn 199 a Dec
la u a g e s , VP ings
Keywo 89 IEeE r)E , r e s
t : F o r m s o s p r e.adsh 4 . r
la ant g
iv e Visu of
19rds an s 1a9n9d5PIE hrEas Burnett, I is u a l Fo 9 9 0 e e t - b a s ed a
r dd E n t e n e t t , V u n e 1 ed ts- h
b e
a l P r o
Keywo Phrases: F Symrgpaorseiu tm M. Bgurrating mA
s, J
iclgPorreits -based and s
prea sed la
nguag
gramm
ing
n o
d r Mm as / o
, n
A c Va d e h r m e s , V
L . a 3 , V 1 ( 2 ) is u a 4 : F o A n P L -III.A.1
, Allen uting, PL-I -II.lAL. an imat
Ambler es and Comp Forms/2I.,AV.4P:LForm- guages, Da ion into a D : Data
g u a g s e s : b a s ed and r m s e c la ra
Lan Phra ta
spread dt, German tive Visual P
rds and
Keywo sheet- y,
based September ogramming
r
langua 1995. Langu
ges age
Low-latency trading & VPL
Communication Barriers

Safety
Why I decided to build a VPL
● I didn't! - I stumbled upon it

● I was still learning the business

● I had insanely aggressive deadlines (see: minutes)

for writing and deploying production code

It was built out of necessity

I got tired of these guys yelling at me
 Modules made my life easier
int qty = currOrderQty - _officialQty;
if (qty > 0)
{
OrderProfileBase orderProfile = null;
AutospreaderSyntheticOrderProfile aop = null;
if (currInstrument.IsASEInstrument)
{
i med d
T g e g e
Hed - L eg d
//aop = new AutospreaderSyntheticOrderProfile(currInstrument.OrderFeed,

r 2 r e a
aop = TopMostAlgorithm.CreateAutospreaderOrderProfile(currInstrument, r
Syntheti
c}
orderProfile d
r = eaop;
O ter Sp
Stop o
else Qu
{
orderProfile = TopMostAlgorithm.CreateAlgoExchangeChildOrderProfile(cur

//orderProfile = new AlgoExchangeChildOrderProfile(currInstrument.Order

Arb
AlgoExchangeChildOrderProfile algoExchangeChildOrderProfile = (AlgoExch
Routi=neleaveOnPause;
algoExchangeChildOrderProfile.LeaveWhenPaused
algoExchangeChildOrderProfile.LeaveWhenCanceled = leaveOnCancel;
if (IsFlagColorSet)
M
algoExchangeChildOrderProfile.Color
arke = flagColor;
Mak t
}

orderProfile.Action =
er
OrderAction.Add;
orderProfile.QuantityToWork = Quantity.FromInt(m_CurrInstrument.Instrument,
orderProfile.BuySell = m_Buy ? BuySell.Buy : BuySell.Sell;
orderProfile.OrderTag = m_Flag;
orderProfile.OrderType = m_OrderType;
But modules weren't enough
● Bugs could still spell disaster

● I wanted the traders to be in control

○ Just piecing them together took work and

programming know-how

○ This "work" wasn't going to add any business

value
The VPL was born
● I built a GUI that let them visually link these modules
together
● I didn't know I was building a VPL
● Algo Design Lab (ADL)
The "Deutsch Limit"

“The problem with visual programming languages is

that you can't have more than 50 visual primitives on
the screen at the same time.”
-Peter Deutsch
The "Deutsch Limit"
● The limit of 50 primitives is arbitrary

● We rely on our IDE to help us solve this problem

○ "Find References/Usages"
○ "Go to Definition/Declaration"
○ "Find in Files"
○ Refactoring

● Mental Map
The solution is threefold

1. Domain specificity

2. Hiding the "dirty stuff"

3. Mitigating the "Deutsch Limit"

1. Domain specificity
"The ability for the general public to understand a VPL is
inversely proportional to the likelihood that it will actually
do something useful."
-me

● This is really no different from a well-made API

○ Abstracts away unnecessary granularity
● Unspecific: Scratch, AppInventor, etc.
● Specific: Lego Mindstorms
2. Hiding the "dirty stuff"
● All VPLs hide some of the generic dirty stuff

● Goes hand-in-hand with domain specificity

○ If not domain-specific, it's hard to identify the
dirty stuff

● Examples of dirty stuff in automated trading

○ In-flight Orders
○ Fairness
○ "It just works"
3. Mitigating the "Deutsch Limit"
● Browser-based navigation
● Show Up/Downstream Blocks
● Block searching
● Componentization
● Breakpoints
● "Jump Blocks"
We want our users to eat fruit
Kathy Sierra - http://goo.gl/hKd95

● Cake vs. Fruit

● We want them to focus on idea generation, not
implementation
● It's also about safety
Let's do an experiment
● Build a simple trading strategy:
○ Place an order to buy 100 shares of stock (e.g.
GOOG) at the last-traded-price (LTP)
○ Whenever the LTP of GOOG changes, we should
change our order's price to match
○ If the LTP of GOOG *ever* falls below $770, we
should cancel our order

● Scratch-->Domain Specific Language-->ADL

stock=GOOG;volume=5433728;LTP=782.03;
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Domain Specific
Algo Design Lab (ADL)

(that's it.)
Domain Specific
Domain Specific
Wrap-up*

● VPLs can work for a very narrow set of

industries and applications
● Domain specificity
● Hiding the dirty stuff
● Mitigating the "Deutsch Limit"
● Are our users fruit or cake eaters?

* Please exit through the back for processing

Questions?
Appendix
Flowcharts are evil
“The flowchart is a very poor abstraction of software
structure.”
-Fred Brooks
Flowcharts are evil
● Poor representation of the actual effort needed to
realize the intended goal
● Flowcharts tend to be written after you write code,
not before
● Too many assumptions; too abstract