NORMA

Norma Introduction and Troubleshooting

Privileged and confidential. The information contained in this material is privileged and confidential, and is intended only for the use of the individual to whom it is addressed and others who have been specifically authorized to receive it.
If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this material is strictly prohibited. If you have received this material in error, please destroy it immediately.
 Lesson Objective

• By the end of this lesson you will be able to

Norma Introduction and
Troubleshooting

2
 Below topics we will cover during this presentation.

 Norma rule.
 Clients for managing Norma rule (KPI generator + PMM Admin)
 Norma Chain Flow
 Norma DB tables
 Norma environment variable
 Norma log reading
 Late Counter Handling Mechanism
 Norma Useful queries

3
 Norma rule.
• KPI Generator (also known as Norma) is a PMM utility for calculating KPIs from
performance data.

• In general, the KPI Generator process reads data from raw data tables. It
calculates for each instance per measurement time, and loads the calculated
KPIs into the counter table of the base resolution (or any other resolution).

The Norma process is for 2 main tasks:

• NE aggregation – It aggregate counters from lower NE(entity) level to higher

level in its hierarchy.
• Join tables – It do calculation between counters that are from 2 different tables
– this can not be done via mediation.

4
 Clients for managing Norma rule
(KPI generator + PMM Admin)

• KPI Generator:- This tool can be accessed from Teoco Studio. This tool is
used to create Norma rule.

• Lets Open this tool from Portal and understand the flow of creating
Norma rule. (This will done in PP3)

• PMM Admin: This client can be accessed from Teoco portal under PMM
tab. This tool is used to mange aggregation , SPC and KPI (Norma rule.)
(This will done in PP3)

• Lets open this tool from portal and understand its options. (This will done in
PP3)

5
Norma Chain Flow

6
 Norma Working Flow
The actions made by Norma process:

• Check that the data we have in the source table is newer (datestamp)
than last handled datestamp.
• Check that the data (datetime) is in the range of the gaps define for this
rule resolution.
• Create Keys table from the source table of the data that needs to be
insert to the target table, like: datetime, name of entity… ( this
mechanism is for late counters) by datestamp filter.
• Do join between the source table and the keys table and insert into the
target table. In case some of the data was already there (late
counters) , it puts the data into a temp table and then do delete in the
target table and later do the insert again, this is called update by delete.

7
 Norma Working Flow
There are 3 time parameters that the Norma consider:

• datestamp – The Norma will run on data that the datestamp >
pmm_norma_inputs.last_handled_datestamp column of that rule.

• datetime – The Norma will run on data that its datetime is in the gap defined in the
kpi_generator_dt_limit.connect per its resolution – how much to look back in gaps:

To change the gaps for handling the old data(pmm_datetime): $PMM2_PROJ_DIR/scripts/kpi_generator_dt_limit.connect

• ;;(define <rule name>_LIMIT_DT <num of gaps>)

• ;;(define TEST_RULE1_LIMIT_DT 5)                ;;5 gaps for for specific rule
•  
• ;;(define ALL_5M_LIMIT_DT 3)                    ;;3 gaps for all rules of the 5M resolution
• ;;(define ALL_15M_LIMIT_DT 3)                   ;;3 gaps for for all rules of the 15M resolution
• ;;(define ALL_30M_LIMIT_DT 3)                   ;;3 gaps for for all rules of the 30M resolution
• ;;(define ALL_HR_LIMIT_DT 7)                    ;;7 days for for all rules of the HR resolution
• ;;(define ALL_DY_LIMIT_DT 30)                   ;;30 days for for all rules of the DY resolution
• ;;(define ALL_WK_LIMIT_DT 30)                   ;;30 days for for all rules of the WK resolution
• ;;(define ALL_MO_LIMIT_DT 90)                   ;;90 days for for all rules of the MO resolution
• (define ALL_LIMIT_DT 5)                 ;;for all rules
• (echo "kpi_generator_dt_limit.connect is loaded")

• MaxQuery-  The range that the Norma will run the query in each iteration. It is define in table:
pmmconf_db.pmm_parameter  NORMA_COMP. The default value is 5 gaps. So for QH rule it will run a
query for 1 hr. and 15m = 5 gaps each query to db.

8
 Norma Server Process
Norma product Processes :
• connect -daemon dvx2_norma_for_father.connect

Norma connect Executable:

• $PMM2_IMP_DIR/scripts/<norma_name>

9
 Norma DB tables
• PMMCONF_DB schema: Related Norma rules:

• select * from pmmconf_db.pmm_norma;

• select * from pmmconf_db.pmm_norma_inputs;
• select * from pmmconf_db.pmm_norma_input_columns;
• select * from pmmconf_db.pmm_norma_input_keys;
• select * from pmmconf_db.pmm_norma_output_fields;
• select * from pmmconf_db.table_id_gen;
• select * from pmmconf_db.pmm_parameter where
PARAMETER_NAME=‘MaxQuery’;
• select * from comm_db.rc_records;

• CONFIG_DB schema: Related scheduler:

• Select * from config_db.n2_scheduler_task;

• Select * from config_db.n2_timer;
• Select * from config_db.N2_SCHEDULE_AUDIT;

10
 Norma Useful queries
For checking maximum DATETIME arrived to target table:

select max(DATETIME)
from [TARGET_TABLE]

For checking last successful run of Norma rule.

select NORMA_NAME, LAST_HANDLED_DATETIME, ACTIVE_FLAG

from PMMCONF_DB.PMM_NORMA where NORMA_NAME = ‘<Norma rule name>' ;

11
To see the LAST_HANDLED_DATESTAMP that were retrieved
from input tables
Select INPUT_TABLE , LAST_HANDLED_DATESTAMP
from PMMCONF_DB.PMM_NORMA_INPUTS
where NORMA_NAME=<norma name>

Reset last handled datestamp

update PMMCONF_DB.PMM_NORMA_INPUTS
set LAST_HANDLED_DATESTAMP = ' <DATE>'
where NORMA_NAME=<norma name>

12
 Norma environment variable

 $PMM2_DIR  This directory contain the product files. Lets go to this

directory and have a look on files.

 $PMM2_IMP_DIR  This directory is having below important Sub folders:

• config/  Contains the TrolGlobalFunctions.tpt file of each Norma rule.

• data/  Contains the trs and trl file for each Norma rule.
• Troll:
• PMM2_IMP_DIR/data /<NORMA NAME>.trs
• $KPI_GENERATOR_GLOBAL_TPT
• $KPI_GENERATOR_LOCAL_TPT
• $PMM2_IMP_DIR/data /<NORMA NAME>.trl

• scripts/  Contains the connect script for each Norma rule.

13
 Norma environment variable
 $PMM2_PROJ_DIR/  This directory is used to over right any
configuration of product. In this directory we copy some product file
and make some changes as per the project.

 For example: For changing DT limit and adding hint we edit below
two files under ‘scripts’ subdirectory
lets have a look on files.
/kpi_generator_dt_limit.connect
/kpi_generator_hint.connect

 $PMM2_LOG_DIR/  All the logs are present under this directory.

 Logs of Norma rule can be found under kpi_generator/ directory.
 You can either use env veriable $KPI_GENERATOR_LOG_DIR. It
will also take you to same path.

14
 Norma log reading
 Cycle beginning: search for "Constructor"

 Keys select: search for "keys select"

 Number of instances: search for "populated with"

 Data select: search for "Data select"

 Number of rows to insert: search for “ins="

 Data loading search for "sqlldr"

 Translate the query using n2dblayer

• Create a temporary file and put into a select
• Run the utility as
• n2dblayer –f <file_name>
15
 Late Counter Handling Mechanism

• When defining raw data tables that will be input to the KPI Generator
process, you must map a PMM_DATESTAMP column for each raw
data table. The PMM_DATESTAMP column stores the time that the
record was inserted in the table. The KPI Generator retains in its
internal table the last PMM_DATESTAMP handled for each raw data
table used for a specific KPI Generator rule. When the tool starts a
new cycle, it reads all the data that arrived after the last handled date
stamp. The PMM_DATESTAMP column on the raw data table is
mandatory. If its value is NULL, the record will not be retrieved by the
KPI Generator process.

16
Example:

KPI Generator runs every 5 minutes with an activation delay of 1 minute. It runs at 10:06:00. This
is the content of the table at run time:

Only data for SW1 was read and passed to the Transform.
At 10:10:29 the data with a PMM_DATETIME of 10:10:00 arrived for both SW1 and SW2.
In addition, the raw data for 10:05:00 for SW2 arrived late. This is how the table looks:

17
The PMM metadata tables retain the last handled PMM_DATESTAMP for this raw data table,
which is 10:05:23. In this cycle all the records that have a PMM_DATESTAMP larger than
10:00:05:23 will be retrieved. In the cycle that runs at 10:11:00, the following data will be passed
to the Transform:

18
• Late Counters Handling (cont.)

The KPI Generator supports 2 modes for handling the arriving counters:

 Insert case
 No database record has created before.
 This case is relevant to regular counter arrivals and time zone handling.

 Update case
 It includes joined Norma cases with Mandatory and Optional counters.
 The mandatory data arrived first and the optional counters later.
 In order to handle the late counters there is a need to update the database
record output.

19
• Late Counters Example

 KPI Generator rule is defined as a Join between a mandatory table and an

optional table.
Example:
 Suppose a Daily KPI rule was activated on 1/2/1019 – only the counters from the
mandatory table arrived.

 On a cycle performed on 1/3/2019 the KPI Generator discovered that the missing
optional data arrived.

 KPI Generator identifies late counters and it performs an update.

 Note: When the Norma tries to insert to the target table it uses simple insert  - so for
regular case that the data is new, it is loaded. But if it is late counters and the record is
already in the target table ( with less counters) then the first try will fail ( unique
constraint for duplicate) and then it will act as update by delete in order to delete exist
record and insert the new record.

20
Lesson Summary – Q&A
 Where can you find the connect of Norma?
 What is the log directory of Norma rules?

21