VisionHub High Availability

Architecture

Version 6.x
My 2021

PROPRIETARY AND CONFIDENTIAL INFORMATION

All information contained herein is confidential, proprietary and the exclusive property of Qognify Ltd and its affiliates (“Qognify”). This document and any
parts thereof must not be reproduced, copied, disclosed or distributed without Qognify's written approval and any content or information hereof shall not
be used for any unauthorized purpose. The software described herein and any other feature or tools are provided "AS IS" and without any warranty or
guarantee of any kind.

All contents of this document are: Copyright © 2021 Qognify Ltd. All rights reserved.
Contents
1 About This Guide .................................................................................................... 2
1.1 Who Should Read This Guide ................................................................... 2
2 Introduction ............................................................................................................. 3
3 VisionHub High Availability Performance Specification .................................... 4
4 VisionHub Middle-tier VMware High Availability ................................................. 6
4.1 VisionHub Middle-tier All-In-One VM High Availability .............................. 7
4.2 VisionHub Middle-tier Large Site VM High Availability .............................. 7
5 MS cluster HA and SQL always on ....................................................................... 9
5.1 VisionHub RSVR (N+1 cluster) ............................................................... 10
5.2 VisionHub Dual Recording ...................................................................... 11
5.3 VisionHub MSVR (mirrored SVR 1+1 cluster) ......................................... 12
6 PTZ High Availability ............................................................................................ 13
6.1 PTZ High Availability with Dual Recording .............................................. 13
7 Video Wall High Availability ................................................................................ 14
8 Mobile Viewer High Availability ........................................................................... 15
8.1 Streamer .................................................................................................. 15
8.2 Transcoder .............................................................................................. 15
9 Suspect Search High Availability ........................................................................ 16
10 Terms and Abbreviations ................................................................................... 17
VisionHub High Availability Architecture About This Guide

1 About This Guide

This document describes the High Availability architecture for VisionHub components.

1.1 Who Should Read This Guide

The information and procedures described in this document are for use by system integrators
and installers, Qognify customer support, sales engineers and end users.

Page 2 of 18
VisionHub High Availability Architecture Introduction

2 Introduction
This document describes Qognify VisionHub high-availability (HA) architecture to support
service continuation in case of local site failure.

Under this document scope, there is no recommendation for a specific virtualization or storage
vendor – you should consult your own virtualization and HA vendor on the best ways to
configure your environment. For Qognify certified virtualization spec and configuration
procedure, refer to the VisionHub Virtualization Guidelines on The Q.

The server’s hardware and software requirements do not change whether they are installed
on physical or virtual servers. See the VisionHub System Requirement Guide. Make sure the
selected HA solution complies with those requirements.

Page 3 of 18
VisionHub High Availability Architecture VisionHub High Availability Performance
Specification

3 VisionHub High Availability Performance

Specification
Recovery defines two parameters for system and data loss that defines the data backup plan:

▪ Recovery time objective (RTO) – Defines the amount of time the business can be
without the service.
Example: An RTO value of 2 minutes means that if a service fails, it would be
recovered and restarted within 2 minutes.

▪ Recovery Point Objective (RPO) – The maximum time in which data might be lost if
there is a Major Incident. (It does not define how much data might be lost).
Example: An RPO value of 5 minutes means that in case of failure, data collected 5
minutes prior to the failure might be lost.

VisionHub supports several layers:

▪ Middle-tier (AMS)
▪ Recording
▪ PTZ – control over the captured scene therefore affects recording
▪ Viewing
– Video wall via decoders
– Mobile viewer via streamer and transcoders
Each layer may affect the data loss of video and dynamic data. The following table describes
the high availability RTO and RPO parameters for each layer and its relation to video and
dynamic data. Depending on the components deployment, the maximum RTO and RPO
number between components shall be taken to reflect the general VisionHub system high
availability RTO and RPO specification.

Table 3-1: HA Definitions for RTO and RPO for Each Layer
Video Dynamic data
Module performance (Sensor events) Notes
spec.
RTO RPO RTO RPO
[Minutes] [Minutes] [Minutes] [Minutes]
Middle tier (AMS) 0 0 7 1 Video can be viewed for
already logged-in users
manually. Automatic actions
related to video are not
operative.
RSVR 7 7 7 7 Video RPO can be reduced to
+ zero data loss, by supporting
edge storage and catch-up the
T missing recording from the
seconds edge storage during the
2MB failover time.
caching Video is cached in memory
prior to storage write. Caching
size is 2MB, the camera bit-
rate defines the time loss per
channel on a server failure.

Page 4 of 18
VisionHub High Availability Architecture VisionHub High Availability Performance
Specification

MSVR 0 0 0 0 No recording loss

Dual Recording 0 0 0 0 No recording loss
PTZ HA with dual recording 1 1 Not Not Derived from the time it takes
on a “direct PTZ” relevant relevant the user to switch between
architecture primary and secondary
recorders from the Video
Configurator.
PTZ HA with dual recording Up to 2 Up to 2 Not Not Derived from the time it takes
on legacy PTZ architecture relevant relevant the user to switch between
Values are per PTZ primary and secondary
camera recorders, and manually
associate all the redundant
channels to the PTZ cameras.
Decoder 1 1 Not Not Derived from the time it takes
relevant relevant the user to manually initiate all
streams on a new decoder.
Mobile viewer – Streamer 5 5 Not Not Reinitiating all the opened
Values are per Operator relevant relevant streams per operator.
*Reinitiating an export
operation may take longer time
to complete.
Mobile viewer – Transcoder 5 5 Not Not Reinitiating the opened
Values are per Operator relevant relevant streams per operator.
*Reinitiating an export
operation may take longer time
to complete.

Page 5 of 18
VisionHub High Availability Architecture VisionHub Middle-tier VMware High Availability

4 VisionHub Middle-tier VMware High

Availability
The VMware High Availability (HA) is an active-passive high availability solution that gives
storage protection rather than memory protection(which is supported for active-active fault
tolerance (FT) solution). The VH middle-tier services are installed on virtualized machines
configured for HA. The virtualization framework monitors the physical server and OS of each
VM, and if any of them fails, it spawns a new VM with that server image (on the same or
another physical machine) and re-routes the requests to the new VM.

VH middle-tier has two deployment flavors:

▪ Small site as All in One (AIO)

▪ Large site

Figure 4-1: VisionHub Large Site Services Deployments

NMS
Server

VH
Web server

OPAS

Video
Gateway
SQL DB

Decoder Mobile Viewer Mobile Viewer Non Video

VDAS SVR
server Web + Streamer Transcoder Gateway

VH Middle Tier VH Mobile Viewer

Each “vertical rectangle” represents a physical server or a VM. For an AIO (All in one)
deployment, all the middle tier blocks are combined to one server or VM.

Sizing guidelines:

▪ Support VMWare v5.5 UP2 and above

▪ Each VM requires the same hardware and software as if the services were installed
on a physical machine, as defined in the VisionHub Hardware & Software
Requirements Guide.
▪ The ESX server specification hosting the VM(s) shall have sufficient computation,
memory and storage for hosting the relevant VM(s).
▪ When building a new virtual environment, you need to allocate physical server(s) for
the VMWare management tools.
▪ At least 30% free storage is required for rebuild operation of the VSAN.
▪ Additional memory of at least 50% is needed on the physical server to support HA.
▪ Additional CPU of at least 10% is needed on the physical server to support HA.
▪ There shall be no performance degradation after the recovery process.
▪ The VH clients shall access the system via same IP address.

Page 6 of 18
VisionHub High Availability Architecture VisionHub Middle-tier VMware High Availability

▪ To enable virtualization support, a RAMS license per VH middle tier is required.

4.1 VisionHub Middle-tier All-In-One VM High

Availability
VMWare HA is certified for both VH deployments. For all in one (AIO) use case, a single
physical server is required to run VH middle-tier, as well as DB, VGW, Web, NMS and an
additional server as the passive physical server backup for the HA.

Figure 4-2: VisionHub AIO HA Virtualization Scheme

VH Free
AIO VM

VSXI VSXI

VSAN
(Shared Storage)

Server I Server II

Figure 4-3: VisionHub AIO HA Server Failover

VH VH
AIO AIO

VSXI VSXI

VSAN
(Shared Storage)

Server I Server II

4.2 VisionHub Middle-tier Large Site VM High

Availability
For the large site use case, there is a minimum of two physical servers separating the middle
tier and the SQL DB server. At least one additional physical server is required as the passive
physical server backup for the HA. A shared storage is required to enable transparent failover
between VMs. It shall support redundant storage to prevent potential data loss.

Page 7 of 18
VisionHub High Availability Architecture VisionHub Middle-tier VMware High Availability

Figure 4-4: VisionHub Large Site HA Virtualization Scheme

VH AMS VH VH
+ VGW + DataBase Non video
Web + NMS SQL server GW

VSXI VSXI VSXI

VSAN
(Shared Storage)

Server I Server II Server III

Figure 4-5: VisionHub Large Site HA VM Failover

VH AMS VH VH AMS VH
+ VGW + DataBase + VGW + Non video
Web + NMS SQL server Web + NMS GW

VSXI VSXI VSXI

VSAN
(Shared Storage)

Server I Server II Server III

Figure 4-6: VisionHub Large Site HA Server Failover

VH AMS VH VH VH
+ VGW + Non video DataBase Non video
Web + NMS GW SQL server GW

VSXI VSXI VSXI

VSAN
(Shared Storage)

Server I Server II Server III

Page 8 of 18
VisionHub High Availability Architecture MS cluster HA and SQL always on

5 MS cluster HA and SQL always on

The Microsoft SQL Always on enables high availability for the VH site database. The
deployment includes a primary and a secondary databases supporting failover. The failover is
not triggered by DB issues such as a lost DB file or curropted transction log. There is a data
synchronization process on the DB level between the primary and secondary DB. The primary
DB sends transcation log records to the secondary DB.

Deploying Always On availability groups for HA requires a Windows Server Failover Cluster
(WSFC). A cluster role should be created for the availability group in the failover cluster. The
failover cluster then monitors this role to evaluate the health of the primary replica.

Figure 5-1: SQL Always on with MS Cluster

SQL Server instance SQL Server instance

Primary Secondary

Always On
Availability Group

Node 1 MS Cluster Node 2

VH site has several deployment options:

1. All in one (All in one) server including the middle tier services, web tier and the SQL
server

2. Medium or Large site having the middle tier and web tier services on one server and
the SQL server on a separate server.

3. Option to deploy additional server for non-video gateways if required

Each deployed node requires an availability group to monitor node health and failover
accordingly.

Page 9 of 18
VisionHub High Availability Architecture MS cluster HA and SQL always on

Figure 5-2: VH Large site with MS Cluster HA

Availability Group #3
Availability Group #2
Availability Group #1

SQL Server SQL Server

instance instance

Primary Secondary
Non-video Middle Tier Middle Tier Non-video
Gateway services services Gateway

MS Cluster
Node 5 Node 3 Node 1 Node 2 Node 4 Node 6
Video High Availability

VH recorders are certified to run in a virtualized environment as described in Qognify SVR

Virtualization. This enables the recorders to run on a virtualized environment, but not with
VMWare availability features (HA, FT, DRS, etc.).

To support HA for the video the following alternatives are proposed:

▪ Active-Passive redundant recorder (RSVR) N+1 clusterActive-Active Dual recording

▪ Active-Active mirrored recorder (MSVR)

5.1 VisionHub RSVR (N+1 cluster)

Figure 5-3: VisionHub RSVR scheme

SVR I SVR I

SVR II SVR II

SVR III SVR III

Monitoring Monitoring
RSVR RSVR
The RSVR monitors the cluster health. Once a recorder fails, the RSVR detects the failure and
initiates a failover process. The process includes gaining control over the edge devices
connected to the failed recorder. Both the failure detection and gaining control over the
cameras is time consuming, which mainly depends on the edge devices vendor(s). Some IP
cameras support fast configuration, some do not. The RSVR notifies the VDAS on the failover
process, so the clients can maintain live operation from the RSVR. Playback is tailored
automatically according to availability between the original SVR and the RSVR.

Sizing Guidelines:

Page 10 of 18
VisionHub High Availability Architecture MS cluster HA and SQL always on

▪ RSVR requires a dedicated license per video channel in the cluster

▪ Support for single-recorder failure in the cluster
▪ There are no additional network bandwidth requirements. The camera streams to the
RSVR during failover time.

5.2 VisionHub Dual Recording

Figure 5-4: VisionHub Dual Recording Scheme

A SVR I

B SVR II
The same edge device can be recorded on two different SVRs to achieve zero down-time. The
dual recording concept introduces primary and secondary SVRs. The primary recorder is the
master SVR that controls the streaming configuration of the edge device. The secondary
recorder can only request streams and listen to events. Once the same edge device is added
to a secondary SVR, the dual recording scheme is enabled. This scheme is implemented on
a channel basis. A recorder might be primary for one camera and secondary for another
camera and vice versa. In terms of deployment, one SVR server can act as primary and the
second SVR server as secondary, or they can both serve as primary and secondary on the
same SVR but not for the same edge device. The user has access to both channels/video
sensors (primary and secondary) and can select which to view via authorization configuration
that would be enforced on the Control application.

Sizing guidelines:

▪ Dual recording requires a dedicated license per video channel.

▪ Dual recording is a camera model dependent feature which requires a plug-in
commitment if not already supported.
▪ Streaming considerations
– Dual unicast streaming
o May affect the device capabilities
o Requires double network bandwidth between camera and the
recorders
– Multicast streaming
o Requires camera and network infrastructure support
o Enables continuous live video monitoring using direct streaming
o Both recorders join the same multicast group
▪ Recorders must be connected to the same VDAS (Video Application server).
▪ Upon failover, manual switching between primary and secondary recorders is
supported via the Video Configurator application.
▪ Dual recording doubles the channels/video sensors in the system.

Page 11 of 18
VisionHub High Availability Architecture MS cluster HA and SQL always on

5.3 VisionHub MSVR (mirrored SVR 1+1 cluster)

Figure 5-5: VisionHub MSVR Scheme

A SVR
Monitoring
B MSVR
Mirrored recorder (MSVR) is a combination of RSVR and dual recording redundancy solutions.
The MSVR monitors a single SVR for failure. Upon failure the MSVR signal to VisionHub that
it becomes the primary recorder for live monitoring. Playback is automatically tailored from
both SVR and MSVR and is transparent to the operator. The MSVR solution is per recorder
which means the entire recorder becomes MSVR (i.e. not on a channel basis).
Sizing guidelines:

▪ MSVR requires a dedicated license per video channel for all the recorder channels.
▪ MSVR, similar to Dual recording, is a camera model dependent feature which
requires a plug-in commitment if not already supported.
▪ Streaming considerations
– Dual unicast streaming
o May affect the device capabilities
o Requires double network bandwidth between camera and the
recorders
o Enables almost continuous live video monitoring up to the detection of
main recorder failure
– Multicast streaming
o Requires camera and network infrastructure support
o Enables continuous live video monitoring using direct streaming
o Both recorders join the same multicast group
▪ Recorders must be connected to the same VDAS (Video Application server).
▪ Automatic switch between SVR and MSVR upon failure and failback
▪ Single logical channel on VisionHub supporting both main SVR and its MSVR
channel

Page 12 of 18
VisionHub High Availability Architecture PTZ High Availability

6 PTZ High Availability

VisionHub supports two architectures for PTZ deployment:

1. “Direct PTZ” – Starting from the upcoming VisionHub 4.3 release, VisionHub client
connects directly to the recorder when requesting PTZ commands; this architecture
supports both distributed and centralized sites.
2. Legacy – An old centralized architecture, which is still supported for backward
compatibility and for migrated from NiceVision sites. The PTZ commands are routed
from the VisionHub clients to the Application Server, PTZ Server (ICM Engine) and
PTZ Agent.

6.1 PTZ High Availability with Dual Recording

6.1.1 Direct PTZ
Only the primary recorder controls PTZ cameras, like streaming configuration or any other
parameter affecting the edge device. Upon failure of the primary recorder, privileged user
needs to manually switch between the primary and secondary recorders, making the
secondary recorder to become a primary and to control the PTZ. With dual recording the
primary and secondary channels are separated on the operator tree which would enable PTZ
control only from the secondary channels. once the dual recording physical channels would
be hosted under single logical channel the PTZ control would be transparent to the operators.

6.1.2 Legacy ICM Architecture

The architecture requires installation of PTZ Agents on both primary and secondary recorders.
Upon failure of the primary recorder, a privileged user needs to manually switch between the
primary and secondary recorders turning the secondary recorder to become primary recorder
for controlling the PTZ. The Video configurator does not allow associating a second channel
to a PTZ. Upon failure needs to manually associate the redundant video channel(s) to the
relevant PTZ camera(s).

Page 13 of 18
VisionHub High Availability Architecture Video Wall High Availability

7 Video Wall High Availability

Video wall decoders are tightly coupled with the physical screen connected to the decoder
graphic board(s). HA cannot be implemented using VMware, but can be supported by
manually adding more decoders which will serve as redundant decoders. Upon failure of one
of the decoders, the operator needs to manually move the current viewing configuration to a
redundant decoder to maintain business continuity.

Page 14 of 18
VisionHub High Availability Architecture Mobile Viewer High Availability

8 Mobile Viewer High Availability

The Mobile Viewer architecture includes streamer and transcoder(s) components.

8.1 Streamer
There are two HA solutions:

1. VMware-based solution – Requires Wowza version 4.x or above and a VMware

infrastructure
2. Scale-out solution – Uses additional streamer, web server and a Network Load
Balancer (NLB) which will route users to one of the streamers and web servers.

8.1.1 VMware-based Solution

The Mobile Viewer web application and streamer will failover to a different VM instance. In this
case, operators need to manually re-initiate video streaming.

8.1.2 Scale-out Solution

The IT department needs to implement an NLB which will route users to one of the web and
streamer servers. Upon a failure of one of the web and streamer servers, users will need to
re-login and to re-initiate video streaming on the second server.

When sizing the solution, each user will be allocated to a web server. Need to size a margin
for additional users, so that upon a failure the other web server will be able to support more
users due to the failure.

8.2 Transcoder
The transcoding component is heavily dependent on Intel libraries for performance
optimization which is tightly coupled with the Intel CPU. When using VMware, the CPU
identification is not transparent, and this results in performance decreasing; so, this is not a
feasible solution.

HA solution is implemented using scale-out servers. The streams which require transcoding
are allocated to several transcoders reducing the load while all servers are functional. Upon
failure of one of the servers, the streams associated to the failed server are re-allocated
(manually) to a different server(s) to maintain business continuity. The operators need to
manually re-initiate video streaming that was closed due to the transcoder server failure.
These streams will be automatically allocated between the remaining functional transcoding
servers.

Page 15 of 18
VisionHub High Availability Architecture Suspect Search High Availability

9 Suspect Search High Availability

HA solution is not supported for Suspect Search. The MPC server, like a recorder service,
catalogs metadata to the storage. The database that catalogs the metadata files on the storage
cannot reside on a shared storage enabling high availability solution.

Page 16 of 18
VisionHub High Availability Architecture Terms and Abbreviations

10 Terms and Abbreviations

The acronyms and abbreviations used in the VisionHub documents are listed below.

Table 10-1: Acronyms and Abbreviations

Term Meaning
ACS Access Control System
AD Active Directory
AMS (Middle-tier) Application Management Server
API Application Programming Interface
AS Application Server
AVI Video format for exporting files
AVMD Advanced Video Motion Detection
CR Control Room
DB Database
DNS Domain Name System
FOV Field of View
FPS Frame per Second
GMT Greenwich Mean Time (also Coordinated Universal Time)
HA High Availability
IP Internet Protocol
LAN Local Area Network
LOS Level of Service
Mbps Megabits per second
MIB The Management Information Base is a text file that defines the interface
between the agents and the NMS
MKV Video container format for exporting files
NAF Audio container format (Qognify proprietary)
NIC Network Interface Card
NMS Network Monitoring System (NMS) is the monitoring service which monitors
VisionHub components via SNMP. The SNMP trap events are logged in its
database and enables ticketing and notification policies.
NTP Network Time Protocol - a protocol for distributing time information between
computers on a network
NVF Video container format (Qognify proprietary)

OPAS Operational Application Server

OS Operating System

OSD OnScreen Display

PRTG Qognify is able to integrate with PRTG, Paessler's network monitoring
software.
PTZ Pan Tilt Zoom
RAID Redundant Array of Independent Disks

Page 17 of 18
VisionHub High Availability Architecture Terms and Abbreviations

SNMP Simple Network Management Protocol. A network protocol used for

managing and reporting device status and events
SNTP Simple Network Time Protocol - a simplified version of NTP
SMTP Simple Mail Transfer Protocol is an Internet standard for electronic mail (e-
mail) transmission across IP networks
SVR Smart Video Recorder
SOP Standard Operating Procedure

SSO Single Sign-On

TCP Transfer Control Protocol
TTL Transistor–transistor logic
UDP User Datagram Protocol
VDAS Video Application Server
VMD Video Motion Detection
VPO Visual Parameters Optimizer
WAN Wide Area Network

Page 18 of 18