Power Advisor – System Health Client

Introduction
The System Health Client package is designed to monitor system health of typically PME and PSO
systems and notify if any device or system related issues occur. Typical checks include data log
health, device communication, system resources as well as service states. The system generally
serves three major use cases:
1. Ad-hoc System Diagnostics Checks;
2. Notify by email on exception when issues are detected;
3. Feed diagnostic information to a central sit e monitoring system such as Mission Control.
 

Installation
 
Installation of the System Health Client package is seamless via the packaged installer. Simply run
the ‘Power Advisor - System Health Client’ executable, follow the prompts and the required
components are installed for you.
 

1
Source Interval Calculation Frequency
During installation, you will be prompted to enter settings for source interval calculation frequency.
For some PME checks, to calculate log health efficiently, the system needs to know what the regular
logging intervals are for each source measurement pair in the system. As this can be a lengthy
process to determine, it is done in the background either daily or weekly. Your selection here should
correlate to how often devices are added to your system and how important it is for new devices log
health to be monitored immediately.

For example, if your system is largely complete and new devices to be added would be rare, then a
weekly re-calculation would be sufficient. If your system is constantly changing with new devices,
then daily would most likely be more appropriate.

Another option is available ‘Calculate Intervals during Install’ which allows you to toggle whether this
interval calculation happens during the install. On a very large system the interval calculation may
take some time so having the intervals calculated in the evening or weekend during the scheduled
time is preferred. Also, if the system is being upgraded from a previous version, then a recalculation
of intervals may not be necessary.

SMTP Server
The System Health Client requires an SMTP server to send diagnostics emails. The installer does not
install or configure a SMTP server and it is assumed that a local SMTP server has been configured,
setup and made available for use.

2
User Interface Overview

Launching the System Health Client configuration tool

Once installed, the System Health Client configuration interface can be launched by the shortcut
created in the start menu under the “Schneider Electric” folder to bring up the default screen.

Alternatively, the tool can be launched by navigating to the install directory and running
“Xpert.DataConnector.Manager.exe”. Note the installation directory differs for a PSO or PME install

PME: \Program Files (x86)\Schneider Electric\Power Monitoring Expert\Xpert System Diagnostics\

PSO: \Program Files (x86)\Xpert System Diagnostics\

3
New Profile Screen
The new profile screen is shown when the application is first launched. To begin, click the “New”
button to create a profile.

Exploring the areas of the initial interface include the following areas:

1. Profile List
This area contains a list of all profiles setup for a system, whether they are configured or not. This list
is empty by default. To begin, click the “New” button to create a profile from a pre-defined template
or import an existing profile.

2. New Profile
Allows the creation of new profiles selection from all available task sets.

3. Profile Configuration Area

Once a profile has been selected, scheduling details and additional functions are presented here.

4. Import
When selecting the new button, the resulting dialog allows the importing of a previously saved or
exported configuration profile (.json format).

4
Profile control screen
On selection of a profile in the profile list, the schedule information is shown and available for
configuration. Additionally, the profile can be renamed (6), duplicated (7), deleted (8) or exported to
a file (9).

1. Valid profile indicator

The colour indicator to the left of the profile list identifies if the profile has been configured with all
required fields filled in - red meaning unconfigured, green meaning configured. If the indicator is red,
the profile will fail to run. To fill in the required information, click the profile in the profile list and
click “Edit Tasks” button.

5
2. Profile schedule indicator
When a profile is scheduled, this is represented in the profile list with a bell icon. Additionally, the
scheduled frequency and recent/next run details are displayed. The frequency of schedule can range
from one minute to monthly.

3. Profile name
Each profile is enforced to be uniquely named. The name of the profile is used to create a scheduled
task that handles the scheduling. The profile can be renamed by clicking the “Rename” button. This
will also rename the scheduled task.

4. Run Profile
The run button runs the diagnostics tasks as configured and presents the results in a browser
window.

Figure 1 Preview of the diagnostics task run

6
5. Edit tasks button
The edit tasks button opens the profile configuration screen which allows the profile to be
configured and tasks to be added or removed from to customise the diagnostics report.

7
Profile configuration
The profile configuration screen is displayed when “Edit Tasks” button is clicked on the Profile
Control screen.

Profile configuration screen

1. Profile Task List

The list of tasks that will be executed in the profile. Click on each task to configure its parameters. A
red border around a task shows that it has an invalid configuration. These red tasks are what causes
the red indicator on the profile control screen in the profile list.

2. Edit Profile Task List

Clicking the edit profile task list button will display the list of tasks that will be executed in the
profile. These can be enabled and disabled by clicking the check box next to them.

3. Save Button
Changes to the profile are not saved until this button is clicked. When saved, the changes will be
written to disk and the scheduled task updated.

4. Back Button
Click to get back to the profile control screen. If the profile has been modified but not saved, a
warning will be displayed.

8
General Configuration Task

Site Name: This is the name of the site to be monitored.

Latitude & Longitude: Optionally specify location coordinates for use in a central site monitoring
system.

Remote Monitoring Link: Optionally specify a link to any accessible resource for use in a central
monitoring System like System Health Server or directly from the exception email. This can be real
time diagrams, websites, documents, images etc.

Priority: Specifies the priority of this system when feeding a central monitoring system like Mission
Control – to help with its order in relation to other sites of similar status. There are eight options
from Very Low up to Critical.

9
PME Alarms Task

This task queries the PME alarm log based on the source, priority, period and includes them in the
diagnostics report output.

Sources: Select any number of Sources to retrieve alarms for from PME.

Alarm Priorities: Filter which alarms are retrieved based on their priority in PME. Enter the alarm
priority range e.g. '60,80,120-255'. The parameter can include multiple values or ranges separated
by comma. Maximum alarm priority in PME is 255. Any alarms meeting the criteria and within the
specific range will be returned.

Alarm Period (Last n days): Filter the time period within which to retrieve alarms for. The number
entered is the number of days to query back relative to now.

Alarm Filter: Enter comma separated text strings to filter alarms. Where any text entered matches
part or all of an alarm name, it will be included in the results.

10
PME Log Health Task

Log Health (%): The percentage of logs that are present vs what is expected in the specified log
health period. If the health percentage value is less than is what is specified here, the report will flag
this as a failure.

Log Health Period: The length of time to look at the devices historical logs and compare against what
is expected. Note that if no data has been logged for a device then it will be excluded from the Log
Health calculation. If there are no logs in the selected period for a device but it has logged in the
past, then it will result in a Log Health % of 0 and cause a failure.

Calculate Log Health % per Measurement:

- When this is set to ‘No’, log health is calculated per source (i.e. the sum of all record for all
measurement divided by the expected number of records for all measurements * 100)
rather than at the measurement level.
- When this is set to ‘Yes’, log health is calculated per source/measurement pair resulting in a
calculation for each individual measurement being logged by each source, resulting in a
more granular level check.

Evaluate Only Active Sources: Set to ‘Yes’, this will look at all sources, even those that do not appear
in Management Console. Setting this to ‘Yes’ (default), will only look at active sources, i.e. those
which are considered active (not just historical) and are listed by default in Management Console.

Source Filter: Select any number of Sources you want to exclude/include from the diagnostic check.
You can select entire branches by selecting the parent in the selection dialog.

Measurement Filter: Select any number of Measurements you want to exclude/include from the
diagnostic check. You can select entire branches by selecting the parent in the selection dialog.

11
PME Device Communications Task

Device Comms Threshold (mins): If a site/device’s status becomes offline/timed out/disconnected,

this is the number of minutes that can elapse before the report will raise an exception for the
site/device as not communicating. Set this to 0 for the exception to be raised regardless of how long
the site has been offline.

Source Filter: Select any number of Sources you want to exclude/include from the diagnostic check.
You can select entire branches by selecting the parent in the selection dialog.

12
PME System Logs Task

Send System logs with Priority at or above: The minimum priority level for a log to reach to be
included in the diagnostics details. Note this number corresponds with the system log levels as
described below the entry field from diagnostic up to critical.

System Log Period (Hrs): The period to look-back in the system log for entries above the set
threshold.

System Log Failure Categories: Pops open a dialog to select the desired system log categories to
include in the diagnostic details. A default set is pre-selected by default. Note that these categories
correspond with those in the System Log view from Management console.

13
PSO Diagnostics Task
The PSO Diagnostics task connects to the PSO Server using ctAPI. It is important to ensure that the
PSO Project is running as well as the client processes:

It is additionally important that the ctAPI.Remote parameter is set to enable remote ctAPI
connections as below:

A valid PSO user must be configured with highest privileges and provided to PSO Diagnostics task
described as above. Any changes in user credentials must also be updated in the task to ensure
continual running of the diagnostics.

The PSO Diagnostics task will be unable to connect to the PSO Server when any of the above criteria
are not met. This could result into Diagnostics note getting captured. In this case, the user will be
notified of the situation with following:

14
PSO Server Address: IP Address of the PSO Server
PSO User: Name of an active PSO User. The user should have highest PSO Privilege (8).
PSO Password: PSO Password of the user configured above.
Filter Devices: List of devices to be excluded from PSO Diagnostics. Multiple device names are
separated by character ‘;’.

Based on this configuration, the PSO server will be polled periodically (as per the setup schedule) for
the following:

 Ensure all system processes are running:

15
 Notify (based on notification settings) of any devices in the system (except for those in the
exclude list) when communications are lost:

16
Disk Space Task

Drives to monitor: Enter the system drives to monitor for low disk space (comma separated).

Low Disk Space Threshold (%): If the free disk space on either the installation drive or the database
drive drops below this value then the report will raise an exception with the details of the disk space
shortage. If the databases are on the same drive that PME is installed on, then just one check is
made. If the architecture is distributed, then the disk space of the database server is not checked.

Regardless of problems being discovered or not, detailed information is always returned:

17
Resource Usage Task

CPU Usage Threshold (%): CPU utilization is captured via a perfmon task (created at install time) and
aggregated at hourly averages. If the threshold set here is exceeded by any of these hourly averages,
then the report will flag this as a failure. For example, if the threshold is set to 95% and the CPU
averages 98% utilization for that hour then a failure will be raised.

Free Memory Threshold (MB): Memory utilization is captured via a perfmon task (created at install
time) and aggregated at hourly averages. If the systems available memory is below the threshold set
here in any of these hourly averages, then the report will flag this as a failure. For example, if the
threshold is set to 1,000MB and the available memory averages 500MB for that hour then a failure
will be raised.

18
Windows Services Task

Services to Monitor: Click this button to be presented with a list of services currently registered on
the system. By default, selections have been made to monitor critical PME, PSO and Windows
services, such as all automatic PME services, Task Scheduler and IIS.

To add or remove services, simply check the box next to the service in the monitor column.

Additionally, you can filter the list by entering text in the filter box at the top to search for specific
services or service groups. Entering ‘ION’ will show all the services starting with ION.

Note that all services are shown regardless of start-up type. Care should be taken when selecting
services that are not set to automatic start-up, as if they are not found to be running by the
diagnostics checks they will flag an issue with the system.

19
Notification Task

Notify on: Select one of the following five options, depending on the desired behaviour:
- Do not Notify: Do not send a notification, regardless of the result.
- Every Time: A notification is always send/saved regardless of the result.
- Every State Change: A notification is sent whenever the State of the diagnostic run changes.
This includes:
o Success to Failure change;
o Failure to Success change;
- Every State Change or Increase In Number of Errors: A notification is sent whenever the -
State of the diagnostics run changes. This includes:
o Success to Failure change;
o Failure to Success change;
o Failure to Failure where the number of errors has increased;
- Every State Change and Every Error: A notification is sent whenever the State of the
diagnostics run changes as well as every time the diagnostic result remains in the failure
state. This includes:
o Success to Failure change;
o Failure to Success change;
o Every Report run resulting in a Failure;

20
Drop Files Task

  
Specify the location and file name to drop the diagnostic details as a json formatted file.

File Name: Specify the name of the resulting diagnostics file.

 
Output folder: The local file folder or valid, writable file share to save the output files to.
 
Encrypt Contents: Encrypt the contents of the resulting file (recommended).

21
Email Task

Configure this page with the SMTP and email details for email diagnostics data delivery.

SMTP Server: The SMTP Server address;

SMTP Port: The SMTP Port to connect to SMTP Server;
Use SSL: Use SSL while sending diagnostics Email;
SMTP User: The SMTP User to connect to SMTP Server;
SMTP Password: The password of the SMTP user;
Email To: Comma separated list of recipients for the diagnostics results;
Email Subject: Subject for the emails sent to recipients;
From Name: Name of the sender;
From Email Address: Senders Email address to be used while sending diagnostics Email;
Encrypt Contents: Encrypt the contents of the resulting email (recommended).

22
PME License Check

Although there are no configuration requirements, a task called ‘PME License Check’ is available:

When this is run (recommended to always be run on PME systems) it checks the validity of the PME
license, including trial. If a trial license is found or an expired license or no license at all then an error
is raised. If a valid license is found, then this information is returned:

General Information
With each execution of the diagnostic tool, general information about the system is returned at the
bottom of each report consisting of product versions, device counts, local settings and generation
times:

Default profile information

Additional information is included by default for each execution, consisting of the following:

 Product version information

 Diagnostics Report Version
 Generation date, time and location

23
Usage Scenarios
Ad-Hoc Usage
To run the diagnostics tool in an ad-hoc manner, simply open and configure as desired, including
email options. Then select ‘run’ to run through the tasks and see the results.

The results will appear in a separate browser window:

24
Email on Exception Usage
 To configure the diagnostics tool to notify on exception, perform the following steps:
 Select an appropriate 'Device Comms Threshold' and select any sources/measurements to be
excluded/included;
 Update as appropriate the Log health, disk space, CPU and memory thresholds;
 Set the email notification setting as desired, including recipients;
 Set ‘Notify On’ to the desired setting, either ‘Every State Change’ or ‘Every State Change and
Increase In Number of Errors’;
 Save the profile definition;
 Setup the desired schedule for the profile to run and click the Schedule button to save and
activate the schedule. Note the schedule icon appears next to the profile in the profile list to
indicate an active schedule exists.
 

 
For each profile scheduled, a corresponding Windows task is created to handle the actual
scheduling. These can be viewed via the Windows Scheduled tasks under the System Health Client
folder. Note these are added, removed and updated via the user interface so there is no need to
modify or maintain these directly in the Task Scheduler interface.

The results will be emailed to the specified recipients in the same format as when previewing.

25
Feeding a Central Monitoring System
 
To configure the report to feed a central monitoring system, perform the following steps:
 Select an appropriate 'Device Comms Threshold' and select any sources/measurements to be
excluded/included;
 Update as appropriate the Log health, disk space, CPU and memory thresholds;
 Set the Notification E-mail or File Share tasks depending on how your monitoring system is
setup to receive input;
 Set 'Notify On’ to ‘Every Time’, since we always want a notification sent;
 Set the 'Location' (latitude, longitude) to valid coordinates that Google maps will recognise (if
customer location on the map view is desired);
 Set the appropriate Priority and include any diagnostics link if desired.
 Save the profile, set the subscription to the desired interval (Hourly/Daily/Weekly/Monthly)
and click schedule.

Multi-Purpose Usage
 If so desired, you can duplicate any profile and adjust the necessary settings to email on exception
rather than to a central monitoring system. Additionally, by duplicating and scheduling multiple
profiles, these can be easily and quickly setup to be personalised for an individual.

Monitoring non PME/PSO systems

The diagnostics tool can be installed independently of any PSO or PME system to monitor general
windows system services and resources. The same installation and configuration procedures apply,
simply create a new profile based on the “Generic Windows Platform” profile template, or remove
the PME/PSO specific tasks from other profiles.

Licensing
System Health Client is initially trial licensed for 30 days. A full license can be obtained by contacting
AZZO Support and provided Windows Product ID of the system where the diagnostics will be
deployed.

Invalid or missing license will trigger appropriate notification to the user.

26
Advanced Configuration Notes

PME Background Tasks

To limit the use of system resources and provide timely report generation, scheduled tasks are
created at install time in the Windows Task Scheduler.

These apply to PME monitoring tasks only.

Load Source Quantity Intervals task

The task is labelled ‘[ION_Data_Custom] Load Source Quantity Intervals’ and is scheduled to run
either once a day at midnight or once a week on weekends (depending on the selection at install
time) to calculate device recording intervals from existing data logs. It needs to run regularly to pick
up any new devices or changes to logging intervals of existing devices. The job can be safely edited
to increase/decrease the frequency or change the run time, if one of the defaults is not suitable.

Reset Source Quantity Intervals task

The task is labelled ‘[ION_Data_Custom] Reset Source Quantity Intervals’ and it can be manually
triggered to ‘reset’ the calculated intervals. The primary use of this task is to remove any calculated
intervals for devices that may have been removed from the system or measurements that have been
intentionally disabled and can be excluded from the calculations.

27
Default exclusion of event-based measurements

In addition to the custom exclusions of Sources & Measurements via parameters configured in the
profile UI, an attempt is made to identify and automatically exclude event-based measurements
from the Log Health % checks since event based measurements are typically not applicable to this
type of check. In addition to this, some measurement groups are excluded, and this can be
configured if additional groups need to be added to the exclusion list. Two PowerShell scripts are
created in the following path ‘\config\cfg\DbScheduledTasks’, one for each of the scheduled tasks
previously mentioned. In these scripts, measurement groups are excluded by referencing the group
ID’s as follows:

$updateIntervalsQuery = "DECLARE @GroupIds IntegerIdList; INSERT INTO

@GroupIds(IntegerId) VALUES (1),(11),(12),(16),(17),(18),(26); EXEC
[Custom_LoadSourceQuantityIntervals] @DAYS = {0}, @GROUPSTOEXCLUDE = @GroupIds"
-f $numDays

As a convenience, a complete list of default Measurement groups is included in here to match the
ID’s with (see below).
These ID’s exist in the ‘ION_Network.vwRPT_MeasurementHierarchyNodes_Admin’ view, and if
custom groups exist that need to be excluded, then this view should be checked to ascertain the
relevant ID’s.

#1 CDM.MdValue.BREAKER_STATUS
#2 CDM.MdValue.COST
#3 CDM.MdValue.CURRENT
#4 CDM.MdValue.CUSTOM
#5 CDM.MdValue.DEMAND
#6 CDM.MdValue.ENERGY
#7 CDM.MdValue.ENVIRONMENTAL
#8 CDM.MdValue.FREQUENCY
#9 CDM.MdValue.GENERIC_I/O
#10 CDM.MdValue.HARMONICS
#11 CDM.MdValue.MAXIMUM
#12 CDM.MdValue.MINIMUM
#13 CDM.MdValue.MISCELLANEOUS
#14 CDM.MdValue.POWER
#15 CDM.MdValue.POWER_FACTOR
#16 CDM.MdValue.POWER_QUALITY
#17 CDM.MdValue.PROTECTION
#18 CDM.MdValue.TIMESTAMP
#19 CDM.MdValue.UNBALANCE
#20 CDM.MdValue.VOLTAGE
#21 CDM.MdValue.WAGES_-_AIR
#22 CDM.MdValue.WAGES_-_NATURAL_GAS
#23 CDM.MdValue.WAGES_-_STEAM
#24 CDM.MdValue.WAGES_-_WASTEWATER
#25 CDM.MdValue.WAGES_-_WATER
#26 CDM.MdValue.ALARM

28
Performance Monitor task
A ‘perfmon’ task is created at install time to log the necessary system performance information for
memory & CPU usage to allow the diagnostics report to work with. The task can be viewed via the
Performance Monitor interface and any settings can be altered via here if required. The perfmon
results are saved to the install directory ‘Applications\Xpert PME Diagnostics\PerfmonLogs’ folder by
default.

Configuring a distributed system

In a distributed environment (i.e. where the database instance is remote) additional database tools
are required to be added to the Application server to allow necessary tasks to run. The following
steps outline the configuration to be manually completed in a distributed installation only – this is
not necessary in a standalone installation:

Install System CLR Types for Microsoft® SQL Server® 2012

Install SQL Server® 2012 Shared Management Objects

Install PowerShell Extensions for Microsoft SQL Server 2012

29