Professional Documents
Culture Documents
Symbian OS Localization
Symbian OS Localization
Symbian OS Localization
LOCALIZATION
3
Localization
part of the SDN++ series
1st Edition, 07/08
Published by:
Symbian Software Limited
2-6 Boundary Row
Southwark
London SE1 8HP
UK
www.symbian.com
Compiled by:
Will Bamberg
Managing Editor:
Ashlee Godwin
Reviewed by:
Tim Band
Chris Cooper
Kai Duan
Antony Edwards
Freddie Gjertsen
James Heginbottom
Jasdeep Sawhney
Renota Schoepp
Jo Stichbury
An early draft of this booklet was reviewed by Chris Cooper, who passed away in early July
2008. He will be sadly missed for his contribution to the Text and Internationalization team in
Symbian, and to the character of the company itself.
4
Contents
INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
TEXT SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
SCRIPT SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
FONT SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
LOCALES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
WHAT IS A LOCALE? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
SUPPLIER INTERFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
CLIENT INTERFACE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
LOCALE LIFECYCLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
CHECKLIST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
GLOSSARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5
Introduction
Symbian smartphones are used in many different countries, currently selling through more than
250 major network operators worldwide. There is no single type of ‘smartphone user’ and it is
important for phone manufacturers to be able to tailor their devices for a number of different
languages, regions, and cultures. Besides the fundamental differences of language, scripts, and
reading direction, there are a number of additional characteristics that must be varied, such as
the way names, dates, numbers, and currencies are formatted.
Symbian OS, and the application layers above it, can be readily adapted for those variations.
For example, the Contacts application does not need to be changed when it ships in two
phones, each built to use a different sort order for the contact names.
Localization is analogous to base porting. Symbian OS does not make assumptions about the
specific hardware it runs on, but runs unchanged on top of an adaptation layer that needs to
be reimplemented as part of the base port to a different hardware platform. Similarly, Symbian
OS does not make assumptions about geography, culture, or language. It supplies a number of
adaptation mechanisms, and device creators supply the data and algorithms that vary from
one market to another.
It may help to think of them in terms of the development roles. The software developers inside
Symbian, who design and implement the features of Symbian OS for supporting different
locales, are internationalization engineers. Developers working inside device creation
companies, who create phones for different regions, are localization engineers.
The terms internationalization and localization are frequently abbreviated to i18n (where 18
stands for the number of letters between the i and the n), and L10n respectively. The capital L
on L10n helps to distinguish it from the lowercase i in i18n.
This booklet provides an overview of the tasks a device creator needs to consider when
localizing Symbian OS. It divides into two sections. The first section describes text processing
issues that may arise when creating a new language variant. The second describes the support
that the operating system provides for sets of user preferences, such as date format, that tend
to vary from one language or geographic region to another, which are usually referred to as
locales.
Like all specialized technical domains, this one includes a significant amount of domain-
specific terminology. Brief definitions of such terms are given where they are used, but the
definitive reference is the Unicode glossary, at www.unicode.org/glossary.
6
Text Support
This section considers issues of text support when creating a new language variant. The
following must be guaranteed:
• Symbian OS has support for rendering and editing the script
• all text input to Symbian OS is converted into the Unicode UTF-16 encoding used internally,
meaning that:
- the appropriate character convertors are included to convert text entering the
device from the outside world
- the appropriate Front-end Processors (FEPs) are included for scripts that require
specialized input methods (FEPs are explained in more detail in the section ‘Front-
end Processors,’ later in this booklet)
• suitable fonts are included in Symbian OS.
Figure 1 shows an extremely simplified view of the text processing components in Symbian OS.
As the section ‘Converting The Inputs’ describes, text enters the system either from direct user
input, or from the world outside the device (for example, in an email). Text entering the text
processing subsystem must be in UTF-16. UTF-16 is a standard for encoding Unicode, defined
in RFC 2781.
The Symbian OS text formatting code fetches the abstract character properties for the text
(such as whether we are allowed to break a line at a given character) from the base Unicode
support, and the physical dimensions of the text from the text rendering component. Then it
uses all this information to calculate how to lay out the text.
The text rendering component maps groups of characters onto glyphs for the purposes of
measuring and drawing the text (a glyph is the representation in a font of a piece of text such as
a character, including its dimensions and associated bitmap). The mapping between characters
and glyphs is not always straightforward, so the text rendering code needs to work out how a
group of characters resolves to a group of glyphs, and then retrieve them from the font.
7
Device creators must consider three aspects when localizing Symbian OS for a different writing
system (script):
• basic script support: ensuring that the version of Symbian OS you are taking supports the
script in question
• converting the inputs: ensuring that text entering the text processing components has been
converted into UTF-16 encoded Unicode
• font support: supplying the correct fonts.
Script Support
Some scripts are not supported by all releases of Symbian OS. For example, versions of
Symbian OS prior to v9.2 did not support any Indic writing systems.
The concept of ‘support’ for a script is really a kind of shorthand for supporting a number of
specific use cases such as correct rendering and intuitive editing of the script.
8
This section gives an overview of some of the fundamental elements in supporting such use
cases, but cannot pretend to be definitive. Even where one device creator has successfully
shipped devices localized to use a specific script, this is no guarantee that the support offered
by Symbian OS will be acceptable, as it is, to all device creators. Each may have subtly
different requirements, for example, how cursor movement in bidirectional text should work.
But it should give a good approximation of what is possible with the different releases.
Symbian OS stores the UCD internally in a table. The text formatting code retrieves the
character properties from the table and uses this information to determine, for example, the
directionality of the text or where line breaks are allowed.
So for the platform to support a script its internal copy of the UCD needs to contain code
points for all the script’s characters.
Symbian OS releases from v9.1 to v9.3 contain the version of the UCD from Unicode 3.0.0, so
any scripts added to the standard after 3.0.0 are not supported by those versions of the
platform. From v9.4, Symbian OS contains a few extra code points needed for the following
Indic scripts: Devanagari, Gujarati, Bengali, Gurmukhi, Tamil, and Kannada.
Most notably, Symbian OS does not yet support any code points outside the Basic Multilingual
Plane: that is, code points above FFFF, which therefore need to be encoded using a ‘surrogate
pair’ consisting of two 16-bit values. Scripts containing characters mapped to code points
above FFFF (also known as supplementary characters) are not currently supported.
Symbian OS
Script
release
v6.0 Japanese
v6.0 Chinese
v7.0s Arabic
v7.0s Hebrew
v8.0 Thai
v9.2 Vietnamese
v9.2 Devanagari
v9.4 Gujarati
v9.4 Kannada
Internally, all text processing in Symbian OS assumes UTF-16 encoded Unicode text. So, any
input to the text-processing subsystem must be in UTF-16 encoded Unicode. Input to the text-
processing subsystem comes either from the user (for example, in the form of keystrokes), in
which case it is converted using a Front-end Processor (FEP), or it comes from some other
computer (for example, a mail server), in which case it is converted using a Character
Convertor (Charconv) plug-in.
Charconv
The Character Conversion component, Charconv, is a plug-in framework for converting text
between some other character encoding (or charset), referred to as the ‘foreign’ encoding, and
the native UTF-16 encoding. Each convertor implements conversion between a single foreign
character encoding and UTF-16, and is identified by UID. The UIDs are listed in the header file
charconv.h.
Symbian OS supplies:
• A number of convertors built into the framework. These are convertors that the large
majority of language variants are expected to need.
• A group of additional convertors separately as ECOM plug-ins. These are convertors that are
only needed for certain variants.
Some convertors, such as those mapping Unicode, are universal, but most are intended to
encode specific scripts or groups of scripts. Part of localizing the device for a new script
involves ensuring that the device contains convertors for all the encodings in which the new
language is commonly encoded. This may mean simply ensuring that the correct Symbian-
supplied plug-ins are included, but in some cases the device creator may need to write
additional plug-ins.
Table 2 summarizes the convertors supplied by Symbian OS. A § symbol is used to indicate
10
the release in which each first appeared if it was not available in the first EKA2 release of
Symbian OS (v9.1). Each of the scripts shown is a plug-in, unless indicated by a ‡ symbol.
Japanese encoding
ISO 8859-6 Arabic J5 DoCoMo
for DoCoMo
Japanese encoding
ISO 8859-13 Baltic Shift-JIS DoCoMo
for DoCoMo
J5 KDDI Japanese encoding
ISO 8859-14 Celtic
(§ Symbian OS v9.3) for KDDI
Shift-JIS KDDI Japanese encoding
ISO 8859-2 Central European
(§ Symbian OS v9.3) for KDDI
Universal (Unicode)
ISO 8859-8 Hebrew IMAP UTF-7 ‡
for IMAP
Universal (Unicode)
eucJP Japanese Java UTF-8 ‡
for Java
Western European /
ISO 2022-JP Japanese CP-1252 ‡
US
Western European /
ISO 2022-JP1 Japanese ISO 8859-1 ‡
US
Western European /
JIS Japanese ASCII ‡
US
Western European /
JIS X 0201 Japanese SMS 7-bit ‡
US
Western European /
JIS X 0208 Japanese ISO 8859-15
US
CP 850 Western European /
JIS X 0212 Japanese
(§ Symbian OS v9.2) US
For example, a device to ship in Russia would need the ISO 8859-5 Cyrillic plug-in, but the
device creator would also need to write a Windows-1251 plug-in, as that encoding is more
widely used. Similarly, for a device to ship in Thailand the device creator would need to write
an ISO 8859-11 convertor, as this is also not supplied by Symbian.
Also note that the Shift-JIS convertor needs special attention. Different operators use different
versions of the standard, and so two variants are produced: one for NTT DoCoMo and another
for KDDI. Both versions of the standard use the same IANA name, so the convertors use the
same UID and are not allowed to coexist in the same device. Documentation explaining how to
build the correct Shift-JIS plug-in can be found in the Symbian OS codebase in the following
location: \syslibs\charconv\plugins\documentation\shift-jis_howto.doc.
Front-end Processors
Front-end Processors (FEPs) are plug-ins that convert user input into UTF-16 encoded input.
Typical FEPs implement handwriting recognition systems or predictive text systems. Their
relevance to script support is that certain scripts contain too many characters to fit on a
device’s keyboard, so other input techniques are needed. The most obvious example is
Chinese, in which the most common ways to implement text entry are Pinyin and handwriting
recognition.
When dealing with Arabic or Hebrew input, care should be taken to ensure the correct use of
Bidirectional Ordering Control characters in the resulting input to the text processing
subsystem (www.unicode.org/reports/tr9). This is particularly important when mixing Right-to-
Left and Left-to-Right text.
Font Support
The rendering code works out which sets of glyphs a set of Unicode characters maps to, and
fetches the glyphs from the font in use.
In order for a device to support a specific script, certain glyphs must be supported. Symbian
publishes a font specification that documents these and the glyph codes that must be used to
identify them. It can be found in the Symbian OS codebase in the following location:
\graphics\fonts\Documentation\Symbian_OS_Font_Specification.doc.
12
Locales
This section describes the support Symbian OS provides for locales, and includes a description
of:
• what a locale is
• how to provision a device with new locales
• which locale elements need special attention
• how the locale is initialized, and how changes to it propagate through the system.
Although the description here is specific to Symbian OS v9.4, it is largely valid for all versions
of Symbian OS from v9.1 onwards.
What is a Locale?
A locale is defined in Symbian OS as a set of data values that vary according to language and
geographic region, for example, the names of the days of the week, the currency symbol, and
the rules for sorting strings (collation).
The locale settings include the language ID itself, which is a TLanguage enumeration value
used to determine which localized resource file is loaded for an application when the
application (or the application framework acting on its behalf ) calls
BaflUtils::NearestLanguageFile().
Symbian OS also defines four locale ‘aspects,’ or groups of settings, and allows client code to
load individual aspects in isolation from each other. The defined aspects are:
• language settings, such as the names of the days of the week
• time and date settings, such as date format
• region settings, such as the currency symbol
• collation settings, which determine the sort order for strings containing text.
The reason for these distinctions is that it should be possible for certain subgroups of locale
settings to vary independently of each other. For example, the desired language should be
able to vary independently of the currently selected region.
Symbian’s locale support implements the mapping between two interfaces, as can be seen in
Figure 2:
• the client interface, which allows code on the device to retrieve and change locale settings
• the supplier interface: the binary interface that must be implemented by suppliers of locale
DLLs.
13
Application
Application
Settings
Application
Application
Client interface
Localization FW
Supplier interface
Licensee code
Localization Localization
DLL 1 DLL 2
Supplier Interface
Locale DLLs
Each locale is supplied by the device creator and is packaged as a polymorphic interface DLL.
Each DLL must implement the binary interface defined in eloclu.def. This is an
implementation of the static functions in the Locl class, declared in localise.h:
class Locl
{
public:
IMPORT_C static TLanguage Language();
IMPORT_C static TBool UniCode();
IMPORT_C static void LocaleData(SLocaleData *aLocale);
IMPORT_C static const TLocaleText* CurrencySymbol();
IMPORT_C static const TLocaleText* ShortDateFormatSpec();
IMPORT_C static const TLocaleText* LongDateFormatSpec();
IMPORT_C static const TLocaleText* TimeFormatSpec();
IMPORT_C static const TFatUtilityFunctions* FatUtilityFunctions();
IMPORT_C static const TLocaleText* const *DateSuffixTable();
IMPORT_C static const TLocaleText* const *DayTable();
IMPORT_C static const TLocaleText* const *DayAbbTable();
IMPORT_C static const TLocaleText* const *MonthTable();
IMPORT_C static const TLocaleText* const *MonthAbbTable();
IMPORT_C static const TLocaleText* const *AmPmTable();
IMPORT_C static const TLocaleText* const *MsgTable();
IMPORT_C static const LCharSet *CharSet();
IMPORT_C static const TUint8 *TypeTable();
IMPORT_C static const TLocaleText* UpperTable();
IMPORT_C static const TLocaleText* LowerTable();
IMPORT_C static const TLocaleText* FoldTable();
IMPORT_C static const TLocaleText* CollTable();
};
The DLL is conventionally named ‘elocl.XYZ’ where ‘XYZ’ is taken from the TLanguage
enumeration, defined in e32lang.h. Thus, the locale for UK English is named elocl.01
and that for Arabic is named elocl.37.
If TLanguage does not contain a value for the locale you want to create, post a request on
the SDN++ discussion forum for Symbian APIs
(developer.symbian.com/forum/forum.jspa?forumID=8), stating the locale that you need.
Symbian’s LOCE32 component contains some reference locale DLLs. The simplest way to create
a locale is to copy and adapt one of these. The documentation of LOCE32 dates from Symbian
OS v6, but is still quite useful, and may be found in Symbian’s codebase under
\base\loce32\ongoing\doc.
Device creators that wish to make use of initialiselocale need to supply a copy of the
central repository keyspace that contains the initial default values for the locale. The UID for
this keyspace is 0x1020E4D3, and a sample keyspace is provided along with the
initialiselocale component, which can be found in the
\syslibs\bafl\initlocale\data\ directory of the Symbian OS codebase.
Client Interface
TExtendedLocale locale;
locale.LoadSystemSettings();
TExtendedLocale locale;
locale.LoadSystemSettings();
locale.SetCurrencySymbol(currencySymbol);
locale.SaveSystemSettings();
TExtendedLocale locale;
locale.LoadSystemSettings();
locale.LoadLocale(dllName);
locale.SaveSystemSettings();
TExtendedLocale locale;
locale.LoadSystemSettings();
locale.LoadLocaleAspect(ELocaleLanguageSettings), dllName);
locale.SaveSystemSettings();
16
Note that not all locale settings belong to an aspect. FatUtilityFunctions is used only
by the file server, and the collection of settings in SLocaleData is not part of any aspect,
and can only be loaded by loading the entire locale.
Collation Settings
The supplier interface function Locl::CharSet() returns the collation settings to use for
the current locale.
Symbian OS supports the Unicode Collation Algorithm. The rules for collation defined by
version 2.1.9 of the standard are encoded as constant static data in collate.h, and these
are referenced by the default implementation of Locl::CharSet().
It is possible that a specific locale will require different sorting to that specified in the
standard. This is most likely to be where the same characters are used in different languages
and the different languages expect different sorting for them.
If you need different sorting to that specified in the standard, then you need to generate a
custom collation table using the COLTAB tool and build that into your locale. This process is
documented in Symbian’s codebase under \base\loce32\ongoing\coltab.
For example, a Japanese user will want to use Japanese names for files and directories on their
SD card, and this means the file system plug-in will need to convert file and directory names
from the UTF-16 encoding used internally into an encoding conventional in Japan.
So a command like MkDirL() will take a 16-bit name as an argument, and the FAT file
system plug-in will convert this to an 8-bit legal short name before writing the directory entry:
Correspondingly, commands to read directory entries convert the names read from the disk
into UTF-16 encoded Unicode, using ConvertToUnicodeL().
But the relationship is not straightforward: in particular, we do not necessarily want the
character encoding to change whenever the UI language or even the locale change, because
this could invalidate, or at least change the names of, existing files on FAT file systems.
• This pointer does not change until the next reboot, whether or not new locales containing
different pointers are loaded.
The process of packaging the functions inside locale DLLs is documented in detail in the
document ‘SGL.TS0019.219 – FatCharSetConv Integration HowTo-2.doc,’
which may be found in Symbian’s codebase under \base\documentation\. It is
anticipated that in future releases of the OS the file server will load FATCharsetConv plug-ins
directly, and the device creator will no longer have to go through this step.
Locale Lifecycle
The locale lifecycle involves the interaction of several different system components. Note that
the lifecycle described here is the default one provided by Symbian: several of the system
components, including initialiselocale and those involved in controlling the boot
sequence, may be replaced by a device creator.
Boot
Locale setup during boot is controlled by the EStart component and passes through three
distinct phases.
First phase
In the first phase EStart sets the locale values to some hard coded defaults:
Note that in this phase the locale properties are set directly, bypassing TExtendedLocale.
After this phase EStart can mount local drives.
19
Second phase
In the second phase, EStart initializes the HAL data, and then uses the ELanguageIndex
HAL setting to load the initial locale:
HalSettings uses the file HAL.DAT to initialize the HAL settings. Once they are initialized
EStart retrieves the language setting and uses that to construct the name of a locale DLL to
load (for instance, ‘ELOCL.37’ if the language index returned by HAL is 37, for Arabic). It then
loads the locale and sets it as the current system locale.
Third phase
In the third phase EStart runs SysStart, which runs initialiselocale to read the
persisted locale settings from the central repository keyspace and publish them to the system.
This is shown in Figure 6.
20
Persisting Changes
When a client changes any of the currently selected locale values either individually, or by
loading a locale aspect, or by loading an entire locale, the change notifier is triggered:
This causes initialiselocale to persist the new locale values to the central repository:
So at the next boot, these values will be installed as the system settings.
The device creator should take care to set the keyspace metadata controlling the reset
behavior, otherwise restoring the locale settings to the factory default will not function
correctly.
Checklist
This section summarizes the main questions you need to ask when preparing a new
localization of a Symbian OS-based device.
Localization Checklist
1 Does Symbian have basic support for the writing system (script)?
Will any of the locales need conversion functions for FAT file
5b
systems?
Will any of the locales need language identifiers not already defined
5c
in TLanguage?
Glossary
The most comprehensive resource available for this topic is the Unicode glossary, which can be
found at www.unicode.org/glossary.
24
References
Unicode Character Database
www.unicode.org/ucd
OpenType
partners.adobe.com/public/developer/opentype/index_spec.html
How to create a Charconv plug-in (in the Symbian OS v9.3 Symbian Developer Library)
www.symbian.com/developer/techlib/v9.3docs/doc_source/ToolsAndUtilities93
25
New from
from
Also from
Mobile Python
Mobile Python is a practical hands-on
book that introduces the popular open
source programming language Python
to the mobile space. It teaches how to
program your own powerful - and fun -
applications easily on Nokia
smartphones based on Symbian OS and
the S60 platform.
Also from
Symbian OS Explained
by Jo Stichbury
Symbian OS Internals
by Jane Sales
Also from
Published Booklets
Coding Standards
Coding Tips
Performance Tips
Essential UIQ - Getting Started
Getting to Market
Getting Started
Quick Recipes Taster
Java ME on Symbian OS
P.I.P.S
Carbide.c++ v1.3
Data Sharing Tips
Essential S60 - Developers’ Guide
Translated Booklets
Chinese Spanish
Japanese Russian
Korean Persian
30
Notes
31
Notes
SDN++
LOCALIZATION
SDN++ Why? What? Where? How?
Symbian Press
Symbian Press publishes books designed to communicate
authoritative, timely, relevant and practical information
about Symbian OS and related technologies. Information
about the Symbian Press series can be found at
developer.symbian.com/books