Professional Documents
Culture Documents
Cover Sheet
Cover Sheet
SPECIFICATION
Classification
Edition 01 02
Date 20031013 20050512
Change Note
Appraisor:
Comp.
Dept.
Name B. GAUGUET B. MEGZARI
Originator:
Comp. NM&S NM
Dept.
Name F.BUILLER I.BENTAHIR
ABSTRACT
A1300 OMC_CS
PM_A83XX
TRS
REFERENCED DOCUMENTS...........................................................................................................................7
RELATED DOCUMENTS...................................................................................................................................7
PREFACE...........................................................................................................................................................7
1 INTRODUCTION ..........................................................................................................................................8
2 RECALL OF REQUIREMENTS...................................................................................................................8
3 FUNCTIONAL DESCRIPTION.....................................................................................................................9
3.1 Recall on counters .....................................................................................................................10
3.1.1 Counters configuration..................................................................................................10
3.1.2 Counters description.....................................................................................................10
3.2 Impact on COLL_IM ...................................................................................................................12
3.2.1 initialization ...................................................................................................................12
3.2.2 data collection and storage...........................................................................................12
3.3 IMPACT on Q3ESMED ...............................................................................................................13
4 ARCHITECTURAL PRINCIPLES ..............................................................................................................14
4.1 PM_A83XX building blocks .......................................................................................................14
4.2 Communication principles ........................................................................................................14
4.2.1 Into the main thread......................................................................................................14
4.2.2 With the COMM_SERVER ...........................................................................................15
4.2.3 With the EML_IM ..........................................................................................................15
4.2.4 With the database.........................................................................................................15
4.3 Data Extraction ...........................................................................................................................15
4.4 Data storage organization .........................................................................................................15
4.4.1 Choice of the database.................................................................................................15
4.4.2 Database structure .......................................................................................................16
4.4.3 Tables description.........................................................................................................16
4.5 Snapshot Request script...........................................................................................................18
4.5.1 Definition .......................................................................................................................18
4.5.2 Counters selection ........................................................................................................18
4.5.3 Input ..............................................................................................................................19
4.5.4 Output ...........................................................................................................................20
4.5.5 Error cases....................................................................................................................21
4.5.6 Error file format .............................................................................................................21
5 PLATFORM INTEGRATION......................................................................................................................22
5.1 Installation ..................................................................................................................................22
5.2 Defense .......................................................................................................................................22
5.2.1 COLL_IM startup ..........................................................................................................22
5.2.2 COLL_IM crash.............................................................................................................22
5.2.3 MySQL crash ................................................................................................................22
5.2.4 INSERT problem...........................................................................................................22
5.2.5 Incomplete dispatch......................................................................................................22
5.2.6 Overload .......................................................................................................................23
5.2.7 Maximum table size exceeded .....................................................................................23
5.3 Access Rights Management .....................................................................................................23
5.4 Backup/Restore ..........................................................................................................................23
5.5 Report files..................................................................................................................................23
5.6 Error files ....................................................................................................................................23
5.7 Client/Server Architecture.........................................................................................................23
7 DIMENSIONING .........................................................................................................................................25
7.1 Dimensioning objectives...........................................................................................................25
not permitted without written authorisation from Alcatel.
All rights reserved. Passing on and copying of this
document, use and communication of its contents
TABLES
Table 1 TICKET HEADER ............................................................................................................................... 10
Table 2 TICKET BODY.................................................................................................................................... 11
Table 3 Integrity constraints on tables............................................................................................................. 17
Table 4 Possible combination of ranks............................................................................................................ 18
Table 5 tables dimensionning .......................................................................................................................... 25
Table 6 database dimensionning..................................................................................................................... 25
[3] OBSERVATION
AAC020023800DS
[4] MySql web site : http://www.mysql.org
RELATED DOCUMENTS
PREFACE
This documents describes the functional specification for the Performance Management relative to the
A83xx network equipments.
The PM_A83XX is the Performance Management application for all A83xx based NEs until R8. Its scope is
the OMC_CS project. No control on NE release will be made in the code.
2 RECALL OF REQUIREMENTS
The aim of the present application is to provide facilities to store in “real-time” mode RCP, HLR and COMB
counters in a local OMC-CS database. The Database structure is open and documented for operator needs.
The final user interface is based on SQL scripts that retrieve counters data from the database. This scripts
should be available from the operator workplace:
− Using simple SQL SELECT commands and predefined SQL scripts, the user should be able to visualize
the value of any counter in the database one by one or using multiple selection criteria.
− SQL scripts can be posted to the “crontab” (unix calendar) to be run periodically every day over a given
period of time - results being stored in files.
As mentioned in the previous chapter, the PM_A83XX application is in charge of storing RCP, HLR and
COMB counters in a database.
As the COLL_IM component already processes bulk data in order to put it in files and send it to the Q3ES
component, PM_A83XX application will be integrated to the COLL_IM.
Another advantage of using COLL_IM is that this component is already scalable, there is no need to create a
new multi-instanciable component.
The counters are obtained through Observation Messages provided by the NEs. They are then extracted by
the COLL_IM and put in a MySql database. An operator has then the possibility to use SQL based scripts in
order to visualize the value of counters.
The next picture sums up the general processing of the application and introduces its main building blocks :
NE
NE
NE
(A83xx)
OMC_CS / NMC2 1
PM_A83xx COLL_IM
database
SQL scripts
1. The Observation Messages containing the counter data are sent by the NEs to the OMC_CS. These
messages are first processed by the Communication Server component of NMC2. Then the counters
are processed by the COLL_IM component..
2. The counters are stored by the COLL_IM component in a database. Just the last received value for each
counter is stored, no history is kept
3. SQL scripts provide a raw display of counters where information is displayed in a formatted way.
Each of the components of the PM_A83XX will now be detailed in a dedicated chapter but prior to this, a
recall on counters will be provided.
Prior to the processing in the PM_A83XX application, some configuration must be done on the concerned
NEs. Indeed, there are two categories of counters :
− the counters which are automatically raised by the NEs without any configuration
− the counters which must be configured on the NEs to raise information to the OMC_CS. This
configuration is done through MMC (Man Machine Command) mechanism and requires the intervention
of an authorized operator.
Note The counter configuration is out of the scope of the PM_A83XX application and is mentioned here
only for clarity purpose.
The different types of services offered in A83xx NEs are listed here:
− For RCF observations in RCP (family numbers between 30000 and 39999):
• fast report: type/service=”OBRC”
• slow report: type/service=”OBR2”
− For VLR observations in RCP (family numbers between 20000 and 29999):
• fast report: type/service=”OBVL”
• slow report: type/service=”OBV2”
− For AK obs in RCP: type/service=”OBAK” (family numbers between 0 and 9999)
− For HLR observations: type/service=”OBHL” (family numbers between 10000 and 19999)
− For AK obs in HLR: type/service=”OBAK” (family numbers between 0 and 9999)
Observation counters emitted by the observation application of the A83XX are sent in a dispatch format. The
observation dispatch may be split in several tickets sent one after the other. A dispatch is composed of many
tickets.
The order in which families of a dispatch, or counters of a family, are sent is not guaranteed.
N.B. (1) The time is the same for all tickets pertaining to the same dispatch
• the parameter is set to an undefined value ’FF’, except for the last ticket of the dispatch (example
of ticket counting: 1/FF, 2/FF, 3/FF, 4/4)
not permitted without written authorisation from Alcatel.
All rights reserved. Passing on and copying of this
document, use and communication of its contents
• the exact value is given in each ticket (example of ticket counting: 1/4, 2/4, 3/4, 4/4)
TICKET BODY is composed of a succession of counters with the following format (note 2):
Table 2 TICKET BODY
N.B. (2) All Counters of the same family are grouped together in a dispatch (not true for RTOS counters).
N.B. (4) A global validity information, common to all application and RTOS observation counters, is
increased by one on each switchover. Load type counters have no validity information.
order to define whether the operator wants to have the Performance Management functionality or not. This
variable should be read by the COLL_IM process at startup.
3.2.1 initialization
The start-up sequence of COLL_IM should be modified in order to add the subscription to COMM_SERVER
to receive bulk data by means of an EFD.
Each COLL_IM process has to make this subscription for all NEs according to the NEgroup used in the
different EML IM instances.
The maximum number of NEs one COLL_IM instance can manage is 8. Coll_IM should create at startup 5
tables for each managed RCP, 2 tables for each managed HLR and 6 tables for each managed COMB to
store their data:
When subscribed data arrive in message format, the COLL_IM should make the following operations :
• the process does not maintain any internal data which requires resynchronization
• if a counter data is lost, it will be automatically recovered at the next update coming from the NE
• there is no history management for counters in the database, if a counter is missing, it will be
inserted in the following period
All error messages related to this new functionnality should be logged in the SMF log.
The architecture aspects will be discussed in a next chapter (< See 4>).
To resume, subscription and acknowledgment of bulk data shouls be made in the Q3ESMED if the
Performance Monitoring is activated on the OMC-CS.
The PM_A83XX application will be integrated in the NMC2/OMC_CS environment in the NMC2 COLL_IM
component.
NMC2
NE
(A83XX)
COMM
SERVER
Q3IC
EMLIM
COLL_IM
SQL
Database
SQL
SQL script
As indicated on the general architecture picture, the COLL_IM component interfaces with the
Communication Server of the NMC2. Communication principles used will be:
− Q3 protocol
− EFD mechanism : at start-up of the process, if the COLLIM_PM_AVAILABILITY variable is set to true,
an EFD will be set by the COLL_IM process in order to receive all the Observation messages
notifications.
If a NE is not supervised, its data is not received in the COMM_SERVER and nothing should be made in the
database,
If a NE starts being supervised, it starts sending its bulk data which is received by the COMM_SERVER and
sent then to the COLL_IM to store it in the database
When a COLL_NE create ACTION is received, COLL_IM should store the NE_Label and NE_Type, create
its corresponding tables and then subscribe to the corresponding data in COMM_SERVER by means of an
EFD.
When a COLL_NE delete ACTION is received, COLL_IM should unsubscribe from to the corresponding data
in COMM_SERVER, the database should then be updated in order to delete all tables related to that NE.
COLL_IM should be connected to the MySql database and should use the MySql API to store/read records
(< See Ref [2]> for details).
The management of counters in the database, as made in the PM_A83XX, does not require advanced
features – it needs just an efficient storage and simple access to the data. All the databases propose such
basic services and as the OMC_CS already uses MySql for other features. A new MySQL database will be
created for PM needs, the database name will be PM_DATABASE. Another important reason is the license
cost : MySql is an open source database and therefore is an interesting economical choice.
MySql proposes several data organization. For the counters storage,as the amount of data to be stored is
very important, the most appropriate table type to use is HEAP tables, this because it uses hashed indexes
not permitted without written authorisation from Alcatel.
To store all necessary information on counters, and knowing that global INSERTs are much more faster
than individual INSERTs or UPDATEs, knowing also that all counters of a given type/service is sent in a
single dispatch, 5 tables per RCP, 2 tables for HLR and 6 tables for COMB will be created (see 3.2.1):
This table will contain all counter values received for a given NE, no historical data is kept in this table. New
received values of counters will overwrite the old ones.
NENAME used in 4.4.2 should be replaced by the real name of the NE.
AUTO_INCREMENT: columns shouldn’t use this parameter as the table type is HEAP
AVG_ROW_LENGTH: it is not necessary set this as rows length has fixed size
CHECKSUM: should be set to 0 as it is not necessary to maintain a checksum for all rows
MAX_ROWS: should be set to maximum number of counters in this table (example: 3500 for HLR tables,
see Table 6 database dimensionning)
PACK_KEYS: should be set to 0 as we don’t need to have a smaller index. This usually makes updates
faster and reads slower
DELAY_KEY_WRITE: should be set to 0 as we don’t need to delay key table updates until the table is
closed.
Columns of the table should be declared to be NOT NULL. It makes everything faster and it saves one bit
per column.
4.5.1 Definition
This function gives the possibility for an operator to get the last received value of a counter in a raw display
mode.
The Snapshot Request function is called through a script where the operator can define the counters he
wants to read. The results can be stored in a file.
As there are many tables per NE (according to its type: 5 for RCP, 2 for HLR and 6 for COMB), the script
should search for counters in the corresponding table according to the family number, see 3.1.1 to have
family mapping according to observation type (RCF, VLR, …). If a counter is present in both fast and slow
tables, only values found in fast tables should be displayed, so the script should first search in fast tables, if
the counter is found no searching is made in the slow tables.
The configuration of counters on the NEs make the configured counters available for the PM_A83XX
application. Nevertheless, whatever the processing mode is, the operator must also define (among the
configured ones), the counter(s) he wants to read. As mentioned before, a counter can be a single value
(family identifier) or a composed value whose family identifier is completed with one or two additional fields
called rank identifiers.
The naming of the family and rank identifiers can be done explicitly (by the name or the numerical value of
the id) or by using the two following special characters:
− the asterisk character “*” : that means the display of sum of all ranks for a family. This only makes
sense for peg counters
− the sharp character “#” : that means the display of all ranks for a family. This makes sense for peg and
load counters
* * global report, sum of all the values for all the ranks
The operator will have to enter the list of counters he wants to get. For each counter, the following
arguments are required (parameters are separated with “,” character):
not permitted without written authorisation from Alcatel.
All rights reserved. Passing on and copying of this
document, use and communication of its contents
“Parameter reading method” indicates whether the user will give the list of parameters in the command line
(case1), in this case value should be 0, or if the list of counters will be read from a file (case2), in this case
value should be 1
The “export option” indicates if the user wants to export the result to a file (the entered value is 1) or if the
results will be displayed in the current Unix terminal (the entered value is 0).
“Report file” indicates the name of the result file. It is mandatory if the “export option” is set to 1. The user
may enter just a space character, in this case the default naming rules for result files will be applied, see 5.5.
The “file to read” argument means that the list of counters is specified in a file, this is interesting if the user
requests for two many counters, the file would contain lines each one having the information:
<NE_Name>,<Family_id>,<rank1>,<rank2>. A space character should be given for this parameter if the user
gives directly the counter name in the command line (case2).
The “check option” is a mandatory argument. Its possible values are check or nocheck: checking errors for
entered parameters can take too much time, especially if the input parameters are read from a file, that’s why
it is necessary to always check the coherence of the input parameters before calling the script to generate a
result:
− If entered value is “check”, the script checks the coherence of input parameters given in the command
line or present in the « file to read ». It displays then in the terminal all encountered problems (if they
exist). The check option doesn’t generate the result file neither displays the result in the terminal.
− If entered value is “nocheck”, the script generates a result without checking the coherence of the input
parameters, if errors are encountered they are ignored: no result is generated for the concerned line or
command line.
Concerning the rank ids, you may use the “*” and “#” special characters as explained in < See 4.5.2>. If the
request concerns a rank id containing space characters, and the input parameters are read from the
command line, this rank id should be delimited by a double quote character “.
4.5.4 Output
The result will be displayed in the current terminal or saved in a file in CSV format. The first line should
contain the field names: <NEID><FAMID> <RANKID1>< RANKID2><VALUE> <TIMESTAMP>. All text fields
should be enclosed in quotation marks, and all numeric fields should not, each field should be separated by
a comma.
If the user uses '*' for rank fields, this character is re-displayed. This means that the result is a SUM of
values.
The returned values correspond to the last stored values of counters in the database, it may be different from
the current values in the NE.
Example : Only columns <NEID> <FAMID> <RANKID1> < RANKID2> <VALUE> from the counters table will
be given
NEID FAMID RANKID1 RANKID2 VALUE
NE01 30747 BSC1 LAC1 1
NE01 30747 BSC1 LAC2 10
NE01 30747 BSC3 LAC2 100
NE01 30747 BSC3 LAC3 4
NE01 30747 BSC4 LAC8 11
*, * (BSC1, LAC1) + (BSC1, LAC2) + (BSC3, LAC2) + (BSC3, LAC3) + (BSC4, LAC8) = 126
*, # (BSC1, LAC1) = 1
(BSC1,LAC2) + (BSC3,LAC2) = 110
(BSC3,LAC3) = 4
(BSC4,LAC8) = 11
#, # (BSC1, LAC1) = 1
(BSC1, LAC2) = 10
(BSC3, LAC2) = 100
(BSC3, LAC3) = 4
(BSC4, LAC8) = 11
If the script execution encounters errors when using the « check » option, these errors should be stored in a
file having the same name as the report file but with the extension .err and under the directory
/home_dir/PM/logs. If there is no report file, default naming rules should be applied for the error file, but
using extension .err.
− Unknow NE : No table is found for the requested NE
− Counter not received:
• no value is found in the database for the selected counter
• when ‘*’ or ‘#’ characters are used in the request and no record is found
− incorrect rank:
• ‘*’ character should not be used for load counters
• the number of ranks doesn’t correspond to the expected one for this counter
− Incorrect parameters: the keying of parameters is incorrect
− SQL error: the connection to the database fails
5 PLATFORM INTEGRATION
All rights reserved. Passing on and copying of this
document, use and communication of its contents
5.1 Installation
As said in 3.2, a global environment variable is set at installation time. This variable specifies whether the
Performance Management functionality will be used by the operator or not.
The snapshot request script is placed under directory /alcatel/nmc/omccs/PM/script. This directory is added
in the PATH.
Configuration files are placed under /alcatel/nmc/omccs/PM/conf and log data is located under
/alcatel/nmc/omccs/PM/conf/logs
5.2 Defense
COLL_IM process is already supervised by DSM.
A new EFD is created to receive bulk data from COMM_SERVER (if the variable
COLLIM_PM_AVAILABILITY is set to true)
COLL_IM also connects to MySQL server and create the corresponding tables for each NE it manages.
Subscription to receive bulk data from each NE is made when the process restarts.
At COLL_IM restart, if some NEs have been deleted from the network, their corresponding tables are not
deleted from the database, a cleanup is made at mysql restart.
New tables are created if some new NEs have been introduced in the network during the crash.
As COLL_IM has no way to detect MySQL crash (except polling it which is not the most suitable solution),
the connection to the database is made when trying to insert a new entry in a table (a complete dispatch),
this operation can fail until the database is available.
Nothing is made to reactivate the MySQL daemon. The administrator should make the necessary operations
to investigate why MySQL crashed and to restart it correctly.
If a problem is encountered when trying to insert counters in the database, COLLIM_IM traces it in SMF
LOGS.
5.2.6 Overload
Tables sizes given in Table 5 should not be exceeded, if the total number of received counters exceeds this
maximum value (which should be defined in a configuration file), an error message should be logged in SMF
LOGS, only the maximum authorised number of counters should be stored, all other counters should be
ignored.
root profile will be used in order to create database, tables and store counters values
Mysql_operator account will be used in the scripts in order to access the database to read counters values.
All users logged to OMC CS should have read-only access to the database via the mysql_operator profile.
5.4 Backup/Restore
No backup/restore is offered for the Performance Management tables, as they contain always the last
received values. Tables are created and filled according to the last received counters from NEs.
With :
YY : year
MM : month
DD : day
HH : hour
MM : minutes
Pid : process id of the script
No purge on report files will be made, each user should delete his own report and error files.
To speed up the duration of counters storage in the database, HEAP tables are used. Tables of this type are
stored in memory and not on disk, they are much faster then ordinary tables stored on disk but if the
COLL_IM process crashes or the server is rebooted, all data contained in those tables is lost. As counter
data is updated periodically, this is not very weighty.
7 DIMENSIONING
NENAME_RCF_SLOW 30000
NENAME_VLR_FAST 2500
NENAME_VLR_SLOW 2500
The following database estimation is made taking into account an OMC CS configuration with :
• 90 RCP, each RCP receiving 30000 counters
• 60 HLR, each HLR receiving 3000 counters
As it is not possible to have RCP, HLR and COMB in the same network, estimations is made taking into
account only RCPs and HLRs. COMB Nes are a mixture of RCP and HLR.
Some performance measures have been made in a C3000 server to estimate the expected time of treatment
of some SQL queries in the database, here are some statments :
− There is no way to use Multiple value lists UPDATE statements, this functionnality is not offered by SQL
language
− The speed of Update queries is greater when the query is done in many table than if it is made in the
same table
− The use of INDEX on the table doesn’t make UPDATE requests faster, results obtained are the same
with or without indexes
It takes 32s to update 256 rows of 8 different tables containing 256 rows
It takes 56s to update 256 rows of 8 different tables containing 15000 rows
It takes 71s to update 2048 rows of 8 different tables containing 2048 rows
It takes 76s to update 2048 rows of one table containing 2048 rows
APPENDIX A
All rights reserved. Passing on and copying of this
document, use and communication of its contents
Mail exchanged on Thu, 20 Nov 2003 15:26:41 after performing some tests on a B180 server :
Suite aux différents tests effectués depuis lundi afin de définir quelles seraient les performances de
l'application suite à l'ajout de la fonctionnalité Performance Management, voici un résumé de l'analyse
effectuée:
Environnement de test:
serveur B180
Base de données:
MySQL version 3.23.39
tables créées de type HEAP (en mémoire)
Moyen de test:
Script en C réalisant une connexion à la BD afin de faire des INSERT ou UPDATE et affichant la durée
d'execution des opérations dans la BD
Rappels sur les flux de données et leur stockage
un RCP peut générer jusqu'à 15000 compteurs en 5 minutes
un HLR peut générer jusqu'à 3000 compteurs en 5 minutes
les flux de données sont envoyés sous forme de dispatch, l'un après l'autre, chaque dispatch peut
contenir jusqu'à 255 compteurs
les données seront stockées dans des tables en mémoire (type HEAP pour MySQL), une table par NE
sera créée
le COLL_IM qui intégrera l'extraction et stockage des données et multi-instanciable, chaque instance peut
gérer jusqu'à 8 NE
Elements mesurés:
la première campagne de test simulait la mise en base des données correspondant à 1 RCP et 1 HLR, une
seule table est mise à jour
Résultats
INSERT global de 18000 compteurs: 2s
INSERT de 18000 compteurs un par un: 12s
UPDATE de 18000 compteurs un par un: 11min (on ne peut pas faire de UPDATE groupés avec le
language SQL)
Ces résultats confirment le fait que les INSERT globaux sont beaucoup plus rapides que les INSERT
unitaires
-----------------------------------------------------------------------------------------------
la deuxième campagne simule la mise en base de 2048 compteurs dans une même table, càd 8 dispatch de
256 compteurs provenant de 8 differents NE gérés par le même COLL_IM
Résultats
INSERT global de 2048 compteurs: 1s
UPDATE de 2048 compteurs un par un: 76s
-----------------------------------------------------------------------------------------------
la troisième campagne simule la mise en base de 256 compteurs dans 8 différentes tables, ce qui
correspond au stockage d'1 dispatch par NE et ce pour 8 différents NE d'un COLL_IM
Résultats
INSERT global de 256 compteurs dans 8 tables: 2s
UPDATE de 256 compteurs un par un dans 8 tables contenant 256 lignes: 32s
UPDATE de 256 compteurs un par un dans 8 tables contenant 15000 lignes: 56s
Conclusion
La mise en base des compteurs tel qu'il avait été prévu de faire en utilisant des update compteur par
compteur risque de poser des problèmes dans la mesure ou il faut traiter (au pire des cas) 58 dispatch par
NE en 5 minutes, sachant qu'un COLL_IM peut gérer jusqu'à 8 NE, ce qui correspond au traitement de 464
dispatch en 5 minutes, donc on doit mettre en moyenne 1.54 secondes pour traiter un dispatch d'un NE dans
le COLL_IM, chose qui n'est pas réalisable vu les essais effectués
PACKAGES
allomorphicPackage,
collNePkg,
csBulkDataPackage,
packagesPackage,
topPackage,
neUserLabelPackage,
neCharacteristicsPackage ;
ATTRIBUTES
allomorphs GET,
nameBinding GET,
networkElementId GET,
objectClass GET,
packages GET,
type GET,
userLabel GET REPLACE;;
NOTIFICATIONS
perfBulkData10,
perfBulkData11,
perfBulkData12,
perfBulkData13,
perfBulkData14,
perfBulkData15,
perfBulkData16,
perfBulkData20,
perfBulkData21;
ENMS Element and Network Management Server hosting EML layer and CS
IM Information Manager
NE Network Element
PM Performance Management
End of document