Raptor (Symbian Build System v2)

Daniel Jacobs
Software Engineer
Nokia

1
Why a new build system?

  Need better performance – faster builds

  Developer build times reduced by half or more depending on
hardware
  Better utilisation of hardware e.g. multi-CPU machines
  Incremental builds – full dependency trees can be built

  Need to run on Linux

  Open source, fast file system, more secure

  Better and easier configurability

  XML configuration

  Easier clustering and scalability

  E.g. Electric Accelerator, PVMGMake

2
Features 1

  Integrates with Carbide 2.0.4 onwards

  Multi-platform – Linux and Windows

  Written in Python and Gnu Make

  POSIX-like command-line interface

  E.g. sbs -v / sbs --version and sbs -h / sbs --help

  Drive builds using system definition or build

information files (bld.inf’s)
  E.g. sbs -s system_definition.xml or sbs -b ~/
componentA/bld.inf -b ~/componentB/bld.inf

3
Features 2

  Fewer Windows limitations

  EPOCROOT can have a drive letter!
  EPOCROOT, source code, Raptor can all be installed on
different drives!!

  XML logging provides standard format

  Plug-in “filters” allow analysis of logs during build
  Classify and report build errors to component owners e.g. by
email
  Gather build statistics e.g. compilation time
  Generate RSS feed of build results

4
Eclipse Public License 1

  Raptor is available under the EPL

  Open source license based largely on IBM’s

Common Public License

  Approved by the Open Source Initiative

5
Eclipse Public License 2

  Allows developers of changes and additions to

make their own separate license (any type of
license) for the changes and additions.
  EPL-licensed parts must stay licensed under EPL

  No requirement to provide source code

6
Using Raptor with Carbide 1

  Use file import wizard

7
Using Raptor with Carbide 2

  Using the new project wizard

8
Command line usage 1

  sbs
  Look in current directory for a bld.inf and build for default
configurations

  sbs –b ~/a/path/to/a/bld.inf –c tools2

  Build ~/a/path/to/a/bld.inf for the TOOLS2 “platform”

  sbs clean
  Perform a clean on the current directory’s bld.inf (remove
released files)
  Also can do sbs CLEAN (case-insensitive)

9
Command line usage 2

  sbs -c armv5_urel -c armv7_udeb -b C:\a\path

\to\a\bld.inf –j 32
  Build
C:\a\path\to\a\bld.inf
for
armv5
urel
and
armv7

udeb
using
32
parallel
jobs
using
Gnu
Make
(default)


  sbs -c armv5_urel -c armv7_udeb -b C:\a\path

\to\a\bld.inf –j 32 –e emake

  Same
as
above,
but
using
emake
as
the
build
engine


10
Using Raptor in the Foundation
Andrew Simpson
Senior Integration Engineer
Symbian Foundation

11
Historical Structure
ex SymbianOS Model
Developers & <SystemDefinition> Platform built
Teams <layer> from xml
<logicalset> system model
build using Module <component>
individual Module <component>
bld.inf or lists component <logicalset>
of bld.inf <component>
component

ex S60 Model
<SystemDefinition>
Module <layer>

Module <logicalset>
component <component>
<component>
component <logicalset>
<component>

12
Package Based Foundation Structure

Package Symbian Foundation Model

<SystemDefinition>
<layer>
Package <package>
<SystemDefinition>
<collection>
<layer>
<component>
<package>
<component>
<collection>
.....
<component>
<package> n.b.
<component>
System Definition syntax is
being updated.
Developers & In v2.x or earlier <block> is
Teams typically used for
“package”
build using Full platform build In 3.x <package> is added
package
Uses system model as a type
definition xml
files which is created by
collating the package
definition xml files.
13
 Building a Foundation package
  Get a base environment and tools
Get   PDK + PDT (everything) or
  SDK + ADT (Just enough for app dev)
Env

  Get package source from Mercurial

e.g.
Get hg clone http://developer.symbian.org/oss/FCL/sf/mw/serviceapi sf/mw/serviceapi

Source
  Clean - Raptor Clean or True Clean
sbs -s sf/mw/serviceapi/package_definition.xml -k -c winscw CLEAN -f clean_log.txt
perl utilities/truclean.pl --packageexpr=mw/serviceapi --releasablesdir=\releaseables
Clean
  Build
sbs -s sf/mw/serviceapi/package_definition.xml -k -c winscw.whatlog -f build_log.txt

Check   Check what you built

sbs -s sf/mw/serviceapi/package_definition.xml -k -c winscw --check > check_log.txt 2>&1

14
 Raptor Logs
Context Information
  See help within Carbide for format of log files
Type of action
  Most items are in “recipe” statements layer / package / phase
<recipe name='msvctoolscompile' target='M:/epoc32/build/uitools/c_28e6bf596feae056/fontcomp_exe/tools/rel/
fontcomp.obj' host='c100726' layer='mw' component='uitools_build' bldinf='M:/sf/mw/uitools/group/bld.inf' mmp='M:/sf/mw/
uitools/gfxtools/gditools/group/fontcomp.mmp' config='tools_rel.whatlog' platform='tools' phase='ALL' source='M:/sf/mw/uitools/
gfxtools/gditools/fontcomp/fontcomp.cpp'>
<![CDATA[
+ cl.exe -MT -c -nologo -Zp4 -GF -O1 -Op -W4 -D __SYMBIAN32__ -D __VC32__ -D __TOOLS__ -D __MSVC6__ -D __MSVC600__ -D WIN32 -D WINDOWS -D
__SUPPORT_CPP_EXCEPTIONS__ -D NDEBUG -D __EXE__ -FI bldvariant.hrh -I M:/sf/mw/uitools/gfxtools/gditools/inc -I M:/sf/mw/uitools/inc -I M:/sf/mw/uitools/
gfxtools/inc -I M:/epoc32/include -I M:/epoc32/include -I M:/epoc32/include/mw -I M:/epoc32/include/platform/mw -I M:/epoc32/include/platform -I M:/epoc32/
include/platform/loc -I M:/epoc32/include/platform/mw/loc -I M:/epoc32/include/platform/loc/sc -I M:/epoc32/include/platform/mw/loc/sc '-FRM:\epoc32\build
\uitools\c_28e6bf596feae056\fontcomp_exe\tools\rel\fontcomp.sbr' '-FoM:\epoc32\build\uitools\c_28e6bf596feae056\fontcomp_exe\tools\rel\fontcomp.obj' M:/sf/
mw/uitools/gfxtools/gditools/fontcomp/fontcomp.cpp
fontcomp.cpp
]]><time start='1254941308.746203000' elapsed='0.171' />
+ RV=0
+ set +x
Full command line
<status exit='ok' attempt='1' />
</recipe>
and response

  Also check for <error> and <warning> tags and a few items outside of tags
</clean>
make.exe: Nothing to be done for `EXPORT'.
<info>Making M:/output/logs/symbian2_FCL.27.108__tools_rel.whatlog_single_thread_GT_tb91sf_Makefile.bitmap</info>

15
Raptor Logs

  Raptor logs can be long...

  Raptor supports filters,
  e.g. to hide successful recipes
--filter=FilterSquashMake
  Can add your own filters

  A number of post analysis tools available

  http://developer.symbian.org/oss/FCL/interim/fbf/configs/
default/file/5fac205a712c/common/tools/analysis

16
Final Thoughts
  Using multiple threads
  Typically choose value for –j of about 2x physical
CPUs
  Remember order of execution may vary

  Incremental builds
  Take advantage of the full system dependency
information
  Build only what has changed

17
Build & Test Tools in Symbian

18