Dynamic of Information

as Natural Computation
Cordana uodlg Crnkovlc
rofessor of CompuLer Sclence
Malardalen unlverslLy,
School of lnnovauon, ueslgn and
Lnglneerlng

gordana.dodlg-crnkovlc[mdh.se


http://www.infotec.com.mx/ 09 April2014
2
M|arda|en Un|vers|ty
M|arda|en Un|vers|ty Sweden
12,000 sLudenLs and around 900 employees, of whlch 67 are professors

- 8esearch MeLhods ln naLural Sclences and Lnglneerlng

- Compuung and hllosophy

- CompuLauonal 1hlnklng and Wrlung 1oolbox

- lormal Languages, AuLomaLa and 1heory of CompuLauon

- rofesslonal LLhlcs

My background: teach|ng
p. 4
hu ln 1heoreucal hyslcs from Zagreb unlverslLy (1988)
hu ln Compuung from Malardalen unlverslLy (2006)

hup://www.ldL.mdh.se/personal/gdc/work/courses.hLml

- Compuung aradlgms, lnfo-compuLauonallsm
- lnfo-compuLauonal knowledge generauon
- lnfo-compuLauonal aspecLs of lnLelllgence and Cognluon
- 1heory of Sclence/ hllosophy of Sclence,
- lnformauon sclence (generauon of lnformauon ln cognluve sysLems)
- Compuung and hllosophy and
- LLhlcs (LLhlcs of Compuung, lnformauon LLhlcs, 8oboeLhlcs and Lnglneerlng LLhlcs).

hup://www.ldL.mdh.se/personal/gdc/work/publlcauons.hLml
My background: research
p. 5
Informanon - sc|ence |n mak|ng
!" $%&$'( )(*+(), %-' -% * &$,$' '- *% *)./*(-0-1$.*0 23,(324
53' )*'/() -% *% *+&(%'3)( $% ,.$(%.( $% 2*6$%1!

ll[a rlgoglne ln 1he Lnd of CerLalnLy: 1lme, Chaos and new
Laws of naLure, 1997
p. 6
7
lnformauon ls everywhere - wheLher lL's dlglLal lnformauon ln our devlces,
analogue conLenL ln books, or blologlcal daLa encoded ln our cells. 8uL how do Lhey
compare ln Lerms of sLorage amounL? Pow have our means of sLorlng lnformauon
grown over Lhe decades? And do you know whaL a zeuabyLe or a youabyLe ls?

88C - luLure - 1echnology - 8yLe-slzed graphlc gulde Lo daLa sLorage
hup://www.bbc.com/fuLure/sLory/20130621-byLe-slzed-gulde-Lo-daLa-sLorage
Increas|ng amounts of |nformanon
8
Million 10
6
M (mega)
9
Billion (Milliard) 10
9
G (giga)
10
Billion (Milliard) 10
9
G (giga)
11
Trillion (Billion) 10
12
T (tera)
12
Quadrillion 10
15
P (peta)
13
Quintillion (Trillion) 10
18
E (exa)
14
Sextillion 10
21
Z (zetta)
15
Septillion (Quadrillion) 10
24
Y (yotta)
1he word lnformauon" has been glven dlerenL meanlngs
by varlous wrlLers ln Lhe general eld of lnformauon Lheory. lL
ls llkely LhaL aL leasL a number of Lhese wlll prove sumclenLly
useful ln cerLaln appllcauons Lo deserve furLher sLudy and
permanenL recognluon.
lL ls hardly Lo be expecLed LhaL a slngle concepL of lnformauon
would sausfacLorlly accounL for Lhe numerous posslble
appllcauons of Lhls general eld.

C Shannon, 1993, 1he Lamce 1heory of
lnformauon"
Informanon concept
- C|aude Shannon
A speclal lssue of Lhe !ournal of Loglc, Language and
lnformauon (volume 12 no 4 2003) !ohan van 8enLhem,
8oberL van 8ooy, LdLs. is dedicated to the different facets
of information.

leLer Adrlaans, !ohan van 8enLhem, LdLs. (2008)
hllosophy of lnformauon (Pandbook of Lhe hllosophy of
Sclence) Llsevler.






hup://www.lllc.uva.nl/Pl/
What |s |nformanon? - |eter
Adr|aans, Iohan van 8enthem
lf lnformauon ls Lo replace mauer/energy as Lhe prlmary sLu
of Lhe unlverse, as P von 8aeyer (2003) suggesLs, lL wlll
provlde a new baslc unlfylng framework for modellng of
reallLy ln Lhe LwenLy-rsL cenLury.

L llorldl proposes lnformauonal SLrucLural 8eallsm - a vlew of
Lhe world as Lhe LoLallLy of lnformauonal ob[ecLs dynamlcally
lnLeracung wlLh each oLher.
Informanon as the pr|mary stu of
the un|verse - |nformanona||sm
(|nformanona| structura| rea||sm) -
Luc|ano I|or|d|
Von Baeyer, H. (2004) Information: The New Language of Science. Cambridge:
Harvard University Press.
Floridi, L. (2008). A defence of informational structural realism. Synthese, 161(2),
219-253
Luciano Floridi (2011) The philosophy of information, Oxford Univesity Press
Informanon ow - Ired Dretske,
Ion 8arw|se and Ierry Se||gman
! 1radluonally, lnformauon was consldered Lo be a bulldlng
block of knowledge and Lhus supposed Lo always be Lrue.
! 1he early developmenLs of Lhe eld of dynamlc of
lnformauon such as semlnal work of (ureLske, 1999)
(lnformauon ow as llngulsuc regularlues) and (8arwlse &
Sellgman, 1997) (lnformauonal relauon beLween slLuauons)
! uescrlbed ln Lhe hllosophy of lnformauon handbook
(8enLhem van & Adrlaans, 2008), as well as (8urgln, 2010)
or (llorldl, 2011).
p. 19
Dretske, F. (1999). Knowledge and the flow of information (pp. 1-288). N. Y.: Cambridge Univ. Press.
Barwise, J., & Seligman, J. (1997). Information flow: the logic of distributed systems. Cambridge:
Cambridge University Press.
! ureLske formulaLed '/( ,')-%107 ,(2*%8. &$(9 -:
$%:-)2*8-%, LhaL lnformauon musL be Lrue: 'lalse
lnformauon and mls-lnformauon are noL klnds of
lnformauon-any more Lhan decoy ducks and rubber ducks
are klnds of ducks.' (ureLske, 1981, p. 33). *
! Powever, 8urgln argues LhaL false lnformauon ls
lnformauon Loo,
ln Lhe same way as false sLaLemenL
ls a sLaLemenL and a fall lmpresslon ls lmpresslon.
p. 20
1rue and fa|se |nformanon
Mark 8urg|n
*aLrlck Allo Ld. (2011) umng lnformauon llrsL: Luclano llorldl and Lhe hllosophy of lnformauon,
Wlley-8lackwell
ureLske, l. (1999). ;%-90(+1( *%+ '/( <-9 -: $%:-)2*8-% (pp. 1-288). n. ?.: Cambrldge unlv. ress.
Mark 8urgln (2011) 1heory of lnformauon: lundamenLallLy, ulverslLy and unlcauon, World Sclenuc
! lnLeracuons wlLhln lnformauon envlronmenLs and Lhelr
lnLernal developmenLs, eLc.,
! lnformauon llfe cycles, from lnlual occurrence Lo nal
uullzauon and posslble dlsappearance, and
! CompuLauon, boLh ln Lhe 1urlng-machlne sense of
algorlLhmlc processlng, and ln Lhe wlder sense of
lnformauon processlng. (llorldl, 2002)
llorldl, L. (2002). WhaL ls Lhe hllosophy of lnformauon? =('*>/$0-,->/7, ??(1&2),
123-143.
p. 21
Dynam|cs of |nformanon
Luc|ano I|or|d|
Log|ca| dynam|cs of |nformanon
Iohan van 8enthem
! A recenL sLudy of lnformauon dynamlcs wlLhln a framework
of loglc ls presenLed ln Van 8enthem's book Log|ca|
Dynam|cs of Informanon and Interacnon, developed as a
Lheory of |nformanon-dr|ven ranona| agency and
|nte|||gent |nteracnon between |nformanon-process|ng
agents.
! van 8enLhem connecLs loglc, phllosophy, compuLer
sclence, llngulsucs and game Lheory ln a unled
maLhemaucal Lheory whlch provldes dynamlc loglcs for
lnference, observauon and communlcauon, wlLh updaLe of
knowledge and revlslon of bellefs, changlng of preferences
and goals, group acuon and sLraLeglc lnLeracuon ln games.
p. 22
Benthem van, J. (2011). Logical dynamics of information and interaction (pp. 1-384).
Cambridge: Cambridge University Press. Retrieved from http://ai.stanford.edu/~epacuit/lograt/
Dyn2010.Ch1.pdf (preprint).
Lmergent |nformanon
Wo|fgang no|rchner
! lnformauonal dynamlcs may be characLerlzed as processes
of self-organlzauon. Whenever self-organlzlng sysLems
relaLe Lo Lhe envlronmenL, Lhey creaLe/generaLe
lnformauon, called emergenL lnformauon".
! 1he lnformauon generauon ls conslsung of cognluon
(lnformauon generauon of a self-organlzlng sysLem vls--vls
lLs envlronmenL), communlcauon (Lhe coupllng of cognluve
processes of aL leasL Lwo self-organlzlng sysLems) and
cooperauon. Cooperauon feeds back Lo communlcauon as
communlcauon does Lo cognluon. 1haL's Lhe baslc
dynamlcs of emergenL lnformauon.
p. 23
Wolfgang Polrchner (2011) LmergenL lnformauon: A unled 1heory
of lnformauon lramework, World Sclenuc
1hree forms of |nformanon
1errence Deacon
! ueacon (2011) dlsungulshes beLween Lhe followlng Lhree
forms of lnformauon:
! "%:-)2*8-% @ (Shannon) (daLa, pauern, slgnal) (daLa
communlcauon) [synLax]
! "%:-)2*8-% A (Shannon + 8olLzmann) (lnLenuonallLy,
abouLness, reference, represenLauon, relauon Lo ob[ecL or
referenL) [semanucs]
! "%:-)2*8-% ? ((Shannon + 8olLzmann) + uarwln) (funcuon,
lnLerpreLauon, use, pragmauc consequence) [pragmaucs]
p. 24
ueacon's Lhree Lypes of lnformauon parallel hls Lhree
:-)2*8&( 2(./*%$,2,:
! [Mass-energeuc [Self-organlzauon [Self-preservauon
(semlouc)]]]
! B2()1(%' +7%*2$.,: [1hermo- [Morpho- [1eleo-dynamlcs]]]
! Correspondlng C)$,'-'0(D, .*3,(,: [LmclenL cause [formal
cause [nal cause]]].
p. 25
1hree forms of |nformanon
1errence Deacon
Levels of Description Levels of
Abstraction Levels of Organization
p. 26
A complex sysLem ls a sysLem composed of lnLerconnecLed
parLs LhaL as a whole exhlblL one or more properues
(behavlor among Lhe posslble properues) noL obvlous from
Lhe properues of Lhe lndlvldual parLs.
E$,-)1*%$F(+ complexlLy ls a resulL of a very large number
of parLs, and -)1*%$F(+ complexlLy ls a mauer of Lhe
sub[ecL sysLem exhlblung (2()1(%' properues.

Complex sysLems are found beLween orderly sysLems wlLh
hlgh lnformauon compresslblllLy and low lnformauon
conLenL and random sysLems wlLh low compresslblllLy and
hlgh lnformauon conLenL.

Information and complexity
p. 27
http://en.wikipedia.org/wiki/Complex_system
Comp|ex|ty
ln a complex sysLem, whaL we see ls
dependenL on where we are and whaL
sorL of lnLeracuon ls used Lo sLudy Lhe
sysLem. (observer dependence)
SLudy of complex sysLems:
Generanve Mode|s
Pow does Lhe complexlLy arlse?
Lvoluuon ls Lhe mosL well known
generauve mechanlsm.
ComplexlLy generaLed from slmple
elemenLs wlLh slmple rules (CA)
p. 28
http://www.morphwize.com/company/index.php?option=com_k2&view=itemlist&task=tag&tag=complex+system+solution
29
Protein network in yeast cells
Social network
Human protein interaction network
Human connectome
Network mode||s of comp|ex systems
Computanon as |nformanon
process|ng
With information as the primary stuff of the universe (informational
structural realism), the most general view of computation is as
dynamics of information (information processes).

This results in a dual-aspect universe: informational structure with
computational dynamics. (Info-Computationalism, Dodig Crnkovic)

Information and computation are closely related no computation
without information, and no information without computation
(dynamics).

8eyond Convennona| Compunng
Mach|nery: Natura| Compunng
! Accordlng Lo Lhe Pandbook of naLural Compuung naLural
compuung ls "the he|d of research that |nvesngates both
human-des|gned compunng |nsp|red by nature and
compunng tak|ng p|ace |n nature."
! lL lncludes among oLhers areas of cellular auLomaLa and
neural compuLauon, evoluuonary compuLauon, molecular
compuLauon, quanLum compuLauon, naLure-lnsplred
algorlLhms and alLernauve models of compuLauon.
p. 31
Grzegorz Rozenberg, Joost N. Kok, Thomas Bck (2102) Handbook of Natural Computing, Springer.

Cognition as processing of
information
Biophysics of Computation: Information Processing in Single Neurons
Christof Koch, 1999. http://www.klab.caltech.edu/~koch/biophysics-book/
100 billions of neurons connected
with tiny "wires" in total longer
more than two times the earth
circumference. This intricate and
apparently messy neural circuit
t hat i s responsi bl e f or our
cognition and behavior.
http://www.istc.cnr.it/group/locen

! rocess of percelvlng, reasonlng, declslon
maklng and Lhlnklng.
! 8odys & braln s way of processlng
lnformauon Lo creaLe meanlng / make sense
for an agenL.
! Cognluon = Llfe (MaLurana & varela, 1980)
What |s Cogn|non?
33
Maturana, H. & Varela, F., 1980. Autopoiesis and cognition: the realization of the living, Dordrecht
Holland: D. Reidel Pub. Co.
1echno|og|ca| App||canon:
Cogn|nve Network 1echno|ogy
p. 34 http://iptechwiki.cttc.es/Cognitive_Network_Optimization_Techniques
now does nature compute?
Se|f-organ|zanon + evo|unon
Crlucs of Lhe evoluuonary approach menuon Lhe
lmposslblllLy of bllnd chance" Lo produce such hlghly
complex sLrucLures as lnLelllgenL llvlng organlsms.
roverblal monkeys Lyplng Shakespeare are oen used
as lllusLrauon (an lnLeresung accounL ls glven by Cell-
Man ln hls G3*)6 *%+ '/( H*13*).)
Chalun and 8enneL: 1yplng monkeys' argumenL does
noL Lake lnLo accounL physlcal laws of Lhe unlverse,
whlch dramaucally llmlL whaL can be Lyped. Moreover,
Lhe unlverse ls noL a LypewrlLer, buL a compuLer, so a
monkey Lypes random lnpuL lnLo a compuLer. 1he
compuLer lnLerpreLs Lhe sLrlngs as programs.
Natura||z|ng ep|stemo|ogy
! naLurallzed eplsLemology (leldman, kornbllLh, Such) ls, ln
general, an |dea that know|edge may be stud|ed as a
natura| phenomenon -- LhaL Lhe sub[ecL mauer of
eplsLemology ls noL our concepL of knowledge, buL Lhe
knowledge lLself.
! I/( ,8230*8-% -: /$, ,(%,-)7 )(.(>'-), $, *00 '/( (&$+(%.(
*%75-+7 /*, /*+ '- 1- -%4 3082*'(074 $% *))$&$%1 *' /$,
>$.'3)( -: '/( 9-)0+J !"# %&' ()*' *++ "&, '"-*
.&%*'/).0&% /+122# 3/&.++4*5 K/7 %-' ,(L0( :-)
>,7./-0-17M ("LplsLemology naLurallzed", ulne 1969,
emphasls mlne)
! l wlll re-phrase Lhe quesuon Lo be: Why not seu|e for
compunng?
Why not seu|e for compunng when
mode||ng know|edge?

In this framework knowledge is seen as a result of the structuring of
input data:

data ! information ! knowledge

by an interactive computational process going on in the nervous
system during the adaptive interplay of an agent with the
environment, which clearly increases its ability to cope with the
dynamical changing of the world.

Hebbian learning
http://en.wikipedia.org/wiki/Hebbian_theory

! A greaL concepLual advanLage of cognluon as a focus of
sLudy ls LhaL all llvlng organlsms possess some cognluon, ln
some degree. MaLurana and varela (1980)
! MaLurana's and varelas' undersLandlng of cognluon ls mosL
sulLable as Lhe basls for a compuLauonallsL accounL of Lhe
naLurallzed evoluuonary eplsLemology.
Natura||st understand|ng of
cogn|non
See also: Dodig-Crnkovic, G. Where do New Ideas Come From? How do they Emerge?
Epistemology as Computation (Information Processing) in Randomness & Complexity, from Leibniz
to Chaitin, C. Calude ed. 2007
lnfo-compuLauonallsL pro[ecL of naLurallzlng eplsLemology by
denlng cognluon as lnformauon processlng phenomenon ls
closely relaLed Lo Lhe developmenL of mululevel dynamlcal
compuLauonal models and slmulauons of cognlzlng sysLems,
and has lmporLanL consequences for Lhe developmenL of
aruclal lnLelllgence and aruclal llfe.

naLural compuLauon opens posslblllues Lo lmplemenL
embodled cognluon lnLo aruclal agenLs, and perform
experlmenLs on slmulaLed eco-sysLems.

Cogn|non as re-structur|ng an agent
through |nteracnon w|th the
env|ronment

! Symbolic simulation is thus a two-stage affair: first
the mapping of inference structure of the theory onto
hardware states which defines symbolic computation;
second, the mapping of inference structure of the
theory onto hardware states which (under appropriate
conditions) qualifies the processing as a symbolic
simulation.
! Analog simulation, in contrast, is defined by a single
mapping from causal relations among elements of the
simulation to causal relations among elements of the
simulated phenomenon.

R. Trenholme, Analog Simulation, Philosophy of Science, vol. 61,
no. 1, pp. 115131, 1994.
D|g|ta| vs. ana|og, d|screte vs. connnuous
and symbo||c vs. sub symbo||c |nformanon
p. 40
Symbo||c vs. sub-symbo||c
computanon (|nformanon process|ng)
p. 41
Douglas Hofstadter in his dialogue Prelude!Ant fugue in Godel, Escher, Bach.
Info-computationalism,
in a nutshell
! Nature can be described as a complex informational
structure for a cognizing agent.
! Computation is information dynamics (information
processing)
! Computation is constrained and governed by the laws
of physics on the fundamental level (morphology).

p. 42
! Turi ng proposed di ff usi on-react i on model of
morphogenesis as the explanation of the development
of biological patterns (spots and stripes on animal skin).
! Morphogenesis is a process of morphological
comput i ng. Physi cal pr ocess t hough not
computational in the traditional sense, presents natural
(unconventional), morphological computation. Essential
element in this process is the interplay between the
informational structure and the computational process -
information self-structuring and information integration,
both synchronic and diachronic, going on in different
time and space scales in physical bodies.
p. 43
Morphogenenc]morpho|og|ca|
compunng. Irom raw data to
semannc |nformanon
Morphogenenc]morpho|og|ca|
compunng. Irom raw data to
semannc |nformanon
! lnfo-compuLauonal naLurallsm descrlbes naLure as
lnformauonal sLrucLure - a successlon of levels of
organlzauon of lnformauon.
! Morphologlcal compuung on LhaL lnformauonal sLrucLure
leads Lo new lnformauonal sLrucLures vla processes of self-
organlzauon of lnformauon.
! Lvoluuon lLself ls a process of morphologlcal compuLauon
on a long-Lerm scale.
p. 44
A Mathematical Model for Info-
computationalism- A. C. Lhresmann
p. 45
pen peer commenLary on Lhe arucle
lnfo-compuLauonal ConsLrucuvlsm and
Cognluon" by Cordana uodlg-Crnkovlc.

Lhresmann proposes a maLhemaucal
approach Lo Lhe framework developed by
uodlg-Crnkovlc. 8ased on Lhe roperLy of
naLural compuLauon, called Lhe
mulupllclLy prlnclple developmenL of
lncreaslngly complex cognluve processes
and knowledge ls descrlbed.
Local dynamlcs are classlcally
compuLable, a consequence of Lhe M ls
LhaL Lhe global dynamlcs ls noL, Lhus
ralslng Lhe problem of developlng more
elaboraLe compuLauon models."
An Info-Computanona| Mode| for
(Neuro-)cogn|nve Systems Capab|e of
Creanv|ty - Andre C. Lhresmann

Andre C. Lhresmann hup://www.mdpl.com/1099-4300/14/9/1703
p. 46
The model, based on a dynamic
Category Theory, accounting for
the functioning of the neural,
cognitive and mental systems at
different levels of description and
across different timescales.
Topics of importance for the development of new
understanding of computation as the dynamic of
information and its role in the physical world:
! Natural computation as generalized model of
computation (natural information processing)
! Interactivity as fundamental for computational
modelling of concurrent distributed information
processing systems such as living organisms and their
networks
! It will be instructive to study processes of self
organization of information in physical agents (and
their networks) able to re-structure themselves through
interactions with the environment as a result of
morphological/morphogenetic computation.
Conc|us|ons
p. 47
(O)ur task is nothing less than to discover a new,
broader, notion of computation, and to understand the
world around us in terms of information processing.
G. Rozenberg and L. Kari, The many facets of natural computing,
Communications of the ACM, vol. 51, pp. 7283, 2008.
p. 48
In preparation: Information Journal, Special Issue "Physics of
Information" , Hector Zenil and Gordana Dodig-Crnkovic, guest editors
p. 49
A Computable Universe
Computation, Information,
Cognition
Editor(s): Gordana Dodig Crnkovic and
Susan Stuart, Cambridge Scholars
Publishing, 2007


Computation, Information, Cognition
p. 50
Information and Computation
Editor(s): Gordana Dodig Crnkovic and
Mark Burgin, World Scientific, 2011
Computing Nature
Editor(s): Gordana Dodig Crnkovic and
Raffaela Giovagnoli, Springer, 2013
http://dx.doi.org/10.1007/978-3-642-37225-4
Based on the Articles

! uodlg-Crnkovlc C., uynamlcs of lnformauon as naLural CompuLauon, lnformauon 2011,
2(3), 460-477, dol:10.3390/lnfo2030460 Speclal lssue: SelecLed apers from llS 2010
8el[lng Conference, 2011.
hup://www.mdpl.com/[ournal/lnformauon/speclallssues/selecLedpapbel[lng
hup://www.mdpl.com/2078-2489/2/3/460/ See also:
hup://llvlngbooksabouLllfe.org/books/LnergyConnecuons
! uodlg-Crnkovlc, C. lnformauon, CompuLauon, Cognluon. Agency-based Plerarchles of
Levels. ln v. C. Mller (Ld.), lundamenLal lssues of Aruclal lnLelllgence (SynLhese
Llbrary). 8erlln: Sprlnger. (forLhcomlng)

p. 51