PTC 46

Ref.
Ares(2020)7198980 - 30/11/2020
www.spirit-tools.com
Deliverable N° 7.2
Title: Trial Run Integration and Performance Testing (Report) (a) (WP7)
Start date of the project: 1/8/2018 / Duration: 36 months
Planned delivery date: 31/3/2020
Submission date: 31/3/2020
Work package: WP7/Task: 7.3
Work package leader: AES Deliverable leader: NST

Version: 16
Dissemination level ( where

appropriate)
Public v
Classified, as referred to Commission Decision 2001/844/EC
Confidential, only for members of the consortium (including the Commission Services)
Version 1.0
Date: 03/03/2020
Author(s):
Maria Theologou
Niki Rovatsou
Project funded by H2020 Programme – Grant Agreement number: 786993

VERSION MODIFICATION(S) DATE AUTHOR(S)

01 Initial Document Creation 26/2/2020 Niki Rovatsou, Maria Theologou (NST)
Trial runs and performance
02 02/03/2020 Niki Rovatsou (NST)
testing
Overall review and update
03 04/03/2020 Maria Theologou (NST)
relevant sections
04 Review 18/03/2020 Geoffrey Robinson (TVP)
Final revision before
05 23/03/2020 Costas Davarakis (SLG)
upload
Document revision after
06 26/10/2020 Niki Rovatsou, Maria Theologou (NST)
review edition
07 Technical content review 05/11/2020 Costas Davarakis (SLG)
08 ESB 20/11/2020 see APPENDIX 4A

Table of Contents
1 Foreground ...................................................................................................................................... 5
2 Trial Runs and Performance Testing ............................................................................................... 7
2.1 Performance Testing Methodology ......................................................................................... 7
2.1.1 Types of Performance Testing .......................................................................................... 7
2.1.2 Common performance problems ..................................................................................... 8
2.1.3 Performance Testing Process ........................................................................................... 8
2.1.4 Load Testing...................................................................................................................... 9
2.1.4.1 Test Case 1: Verify response time when users access different number of active
investigations.............................................................................................................................. 9
2.1.4.2 Test Case 2: Verify response time when users access different number of jobs ... 10
2.1.4.3 Test Case 3: Verify response time when users access a graph with different number
of links and nodes..................................................................................................................... 11
2.1.5 Stress Testing.................................................................................................................. 11
2.1.5.1 Test Case 4: Check CPU and memory usage when 3 different jobs insert a large
volume of data to the database simultaneously...................................................................... 11
2.1.6 Scalability Testing ........................................................................................................... 12
2.1.6.1 Test Case 5: Determine how the system scales with increasing workload ............ 12
2.2 Trial Runs and Functional Validation ..................................................................................... 13
2.2.1 LEA’s Requirements and Universal Use Case Scenario .................................................. 13
2.2.2 Authentication & Authorisation Service ........................................................................ 18
2.2.2.1 Test Case 1: Successful login ................................................................................... 18
2.2.2.2 Test Case 2: Logout ................................................................................................. 18
2.2.3 General User Interface ................................................................................................... 19
2.2.3.1 Test Case 3: List of active investigations ................................................................. 19
2.2.3.2 Test Case 4: Hide investigation ............................................................................... 19
2.2.3.3 Test Case 5: List of inactive investigations .............................................................. 20
2.2.3.4 Test Case 6: Unhide investigation ........................................................................... 20
2.2.3.5 Test Case 7: Create an investigation ....................................................................... 21
2.2.3.6 Test Case 8: Edit an investigation ........................................................................... 21
2.2.3.7 Test Case 9: Authorise an investigation .................................................................. 22
2.2.3.8 Test Case 10: Edit authorisation form..................................................................... 22

2.2.3.9 Test Case 11: Manage investigation permissions ................................................... 23

2.2.3.10 Test Case 12: Read only permissions on investigation ........................................... 23
2.2.3.11 Test Case 13: View jobs ........................................................................................... 24
2.2.3.12 Test Case 14: Access the Graph .............................................................................. 24
2.2.4 Logging and Privacy Control Service .............................................................................. 25
2.2.5 Refined Search Service ................................................................................................... 25
2.2.5.1 Test Case 15: Level 1 refined search ....................................................................... 25
2.2.5.2 Test Case 16: Level 2 refined search ....................................................................... 26
2.2.5.3 Test Case 17: Select level 2 refined search results and proceed to further analysis
26
2.2.6 Crawler Service ............................................................................................................... 27
2.2.6.1 Test Case 18: Initiate a crawler job ......................................................................... 27
2.2.7 Third Party Service.......................................................................................................... 28
2.2.7.1 Test Case 19: Initiate a third party job (twitter search by keyword) ...................... 28
2.2.7.2 Test Case 20: Initiate a third party job (google search by keyword) ...................... 28
2.2.7.3 Test Case 21: Initiate a third party job (twitter search by twitter id) ..................... 29
3 Test Results ................................................................................................................................... 31
3.1 Non-functional validation (to be completed by technical consortium) ................................ 31
3.2 Functional validation - Questionnaire (completed by LEAs during the Y1 testing evaluation)
33
3.2.1 Evaluation from Hellenic Border Police (HPB) ............................................................... 38
3.2.2 Evaluation of Thames Valley Police (TVP) ...................................................................... 43
3.2.3 Evaluation from Serbian Police (SERBLEA) ..................................................................... 49
3.2.4 Evaluation from West Midland Police (WMP) ............................................................... 54
3.2.5 Evaluation from STAD Antwerp Police (LPA).................................................................. 59
4 APPPENDIX A – SPIRIT INTERNAL REVIEWING PROCESS REVISING DELIVERABLES ...................... 66
Table of Figures
Figure 1 - SPIRIT Performance testing process for Y1 prototype ........................................................... 8

1 Foreground
The objective of this Trial Runs and Performance Testing (Report) document, is to describe the
procedure followed in order to organise and conduct specific sub-system trial runs, involving user
practitioners to access the functionality and the various aspects of the performance of the SPIRIT
system and the privacy and security implications in the operational Y1 Prototype deployment.
According to Task 7.3, a series of trialling events were scheduled during year two of the project
involving subsystem testing and usability evaluation. These trial runs were arranged so that their
results could enable formative evaluation of the system and progressive refinements of the design
by ensuring that these trials were run in a timely manner so as to integrate formatively with the
design and development cycles and final integration and demonstration timelines for the whole
project. This document has been composed by taking into account the SPIRIT Y1 prototype and the
Deliverable 7.1 System Integration and the Deliverable 2.4 Requirements Analysis Document.
Furthermore, this Deliverable contributes, impacts, and complies with the following:
1. LEA Requirements
 D7.2 takes into account the LEAs requirements for the forthcoming development cycles of
Year 1 as well as the Universal Use Case Scenario (a detailed description of LEAs
requirements and Universal Use Case Scenario can be found on Deliverable D2.4). The trial
runs and functional validation performed fulfilling the LEAs requirements (Section 2.2)
2. Simplistic
 SPIRIT platform provides a simple User Interface ensuring that has elements that are easy
to access, understand and use to facilitate all actions from the user. This means that the
training process becomes easier by reducing costs and resources. Moreover, all SPIRIT
components, could easily maintained and enhanced to support new LEA’s requirements
(Section 3.2)
3. Ethical
 This deliverable has been reviewed opposite the stipulations of the SPIRIT advisory board
and catered for the directions provided by the EU ethics screening panel. All modules
employed in the integration and performance testing and evaluation process conform to
the aforementioned terms and procedures and therefore this document fully complies with
the set ethical norms.
4. Using existing intelligence sources
 SPIRIT platform utilizes intelligence search tools based on open source libraries which are
appropriately configured to support the overall system architecture as well as the LEAs
requirements (Deliverable 7.1, Sections 2.2, 2.3).

5. Reliable & Timely
 The SPIRIT platform is designed to be robust and reliable as there may not be any post-
delivery support. During Year 2 and based on the results of the user’s evaluation of Year 1
Prototype, the SPIRIT Platform will be upgraded providing more stability and greater
response to the end users (section 3.2)
6. Auditable
 D7.2 is based on the auditing trail secured in Deliverable 7.1 which provides an overview
over the practical security actions that are being taken during the development of the
SPIRIT platform.
7. Secure
 D7.2 is based on other ethical deliverables documentation regarding security.
8. Training & Evaluation
 D7.2 takes into account training and evaluation based on test results of trial runs and
functional validation which are totally documented in SPIRIT GitLab projects (Section 3)
D7.2 links to the exploitation plan, as follows:
No. Linking to exploitation area Comment Level of contribution

1(None) to 5(Very)
1 The consortium LEAs (as the D7.2 tests and evaluates the performance 5
primary focus of the project) of tools described in the exploitation plan
document (D10.2)
2 The other consortium D7.2 clarifies performance expectations, 5

members (as a secondary training demands and prospects for all
focus – the further research partners.
aspect)
3 Other LEAs and governmental D7.2 clarifies performance expectations, 4

partners training demands and prospects for all end
users beyond SPIRIT.
4 Further research opportunities See No. 2 3
5 Commercial opportunities See No. 3 4

2 Trial Runs and Performance Testing
2.1 Performance Testing Methodology

During trial runs, practitioners performed a complete functional system testing to ensure that the
Year 1 Spirit Prototype fully complies with the Universal Use Case Scenario specified by LEAs during
Year 1 (refer to D2.1 Requirements Analysis deliverable).
Features and functionality supported by SPIRIT system is not the only concern. SPIRIT application's
performance like its response time, reliability, resource usage and scalability do matter. The goal of
Performance Testing is not to find bugs but to eliminate performance bottlenecks. In addition, to
provide LEAs stakeholders with information about their application regarding speed, stability, and
scalability. More importantly, Performance Testing uncovers what needs to be improved before the
system deliver to LEAs. Without Performance Testing, SPIRIT software is likely to suffer from issues
such as: running slow while several Jobs are running simultaneously and poor usability.
Performance testing will determine whether this software meets speed, scalability and stability
requirements under expected workloads. Applications delivered with poor performance metrics due
to non existent or poor performance testing are likely to gain a bad reputation and fail to meet
expected end-users goals.
2.1.1 Types of Performance Testing

Load testing - checks the application's ability to perform under anticipated user loads. The objective
is to identify performance bottlenecks before the software application goes live.
Stress testing - involves testing an application under extreme workloads to see how it handles high
traffic or data processing. The objective is to identify the breaking point of an application.
Endurance testing - is done to make sure the software can handle the expected load over a long
period of time.
Spike testing - tests the software's reaction to sudden large spikes in the load generated by users.
Volume testing - Under Volume Testing large no. of. Data is populated in a database and the overall
software system's behaviour is monitored. The objective is to check software application's
performance under varying database volumes.
Scalability testing - The objective of scalability testing is to determine the software application's
effectiveness in "scaling up" to support an increase in user load. It helps plan capacity addition to
SPIRIT system.
During this phase of the project specific types of performance testing were performed, described in
the following paragraphs.

2.1.2 Common performance problems

Most performance problems revolve around speed, response time, load time and poor scalability.
Speed is often one of the most important attributes of an application. A slow running application will
lose potential users. Performance testing is done to make sure an app runs fast enough to keep a
user's attention and interest. Bellow at the following list are presented common performance
problems and notice how speed is a common factor in many of them:
 Long Load time - Load time is normally the initial time it takes an application to start. This
should generally be kept to a minimum. While some applications are impossible to make load
in under a minute, Load time should be kept under a few seconds if possible.
 Poor response time - Response time is the time it takes from when a user inputs data into the
application until the application outputs a response to that input. Generally, this should be
very quick. Again if a user has to wait too long, they lose interest.
 Poor scalability -
 Bottlenecking - Bottlenecks are obstructions in a system which degrade overall system
performance. Bottlenecking is when either coding errors or hardware issues cause a decrease
of throughput under certain loads. Bottlenecking is often caused by one faulty section of code.
The key to fixing a bottlenecking issue is to find the section of code that is causing the
slowdown and try to fix it there. Bottlenecking is generally fixed by either fixing poor running
processes or adding additional hardware. Some common performance bottlenecks are:
o CPU utilization
o Memory utilization
o Network utilization
o Operating System limitations
o Disk usage
2.1.3 Performance Testing Process
The methodology adopted for performance testing demonstrates that SPIRIT system meets certain
pre-defined performance criteria and helps identify parts of system which degrade its performance.
Below is a generic process on how performance testing performed:
Figure 1 - SPIRIT Performance testing process for Y1 prototype
Identify the testing environment - Knew the physical test environment, production environment and
what testing tools are available. Understood details of the hardware, software and network
configurations used during testing before begin the testing process. It will help testers create more
efficient tests. It will also help identify possible challenges that testers may encounter during the
performance testing procedures.

Identify the performance acceptance criteria - This includes goals and constraints for throughput,
response times and resource allocation. It was also necessary to identified project success criteria
outside of these goals and constraints. Testers should be empowered to set performance criteria and
goals because often the project specifications will not include a wide enough variety of performance
benchmarks. Sometimes there may be none at all. When possible finding a similar application to
compare to is a good way to set performance goals.
Plan & design performance tests - Determined how usage was likely to vary amongst end users and
identify key scenarios to test for all possible use cases. It was necessary to simulate a variety of end
users, plan performance test data and outline what metrics would be gathered. Consortium technical
testers are not allowed to run and test the Spirit system into real data due to ethical constraints. To
that end, the Y1 Spirit prototype performance tests planned & designed to run on artificial data
created within the project. The performance testing of the subsequent deployments (Y2 and Y3) will
be performed on real data and with the security and ethics preserving responsibility of SPIRIT LEAs,
remotely supported by SPIRIT technical partners. Therefore, the testing environment will be
accordingly adapted.
Configuring the test environment - Prepared the testing environment before execution. Also,
arranged tools and other resources.
Implement test design - Created the performance tests according to SPIRIT test design.
Run the tests - Executed and monitored the tests.
Analyze, tune and retest - Consolidated, analyzed and shared test results. Then fine tuned and tested
again to see if there was an improvement or decrease in performance. Since improvements generally
grow smaller with each retest, stopped when bottlenecking is caused by the CPU. Then considered
option of increasing CPU power.
During trial runs, practitioners performed a complete functional system testing to ensure that the
Year 2 Spirit Prototype fully complies with the Universal Use Case Scenario specified by LEAs during
Year 2.
2.1.4 Load Testing
The Load testing was planned when the SPIRIT system became functionally stable. A large number of
test data was prepared in order to support the test cases described bellow. The consistency of
response time over the elapsed period was logged & the same was compared with various test runs.
2.1.4.1 Test Case 1: Verify response time when users access different number of active investigations
Test ID: 1
Test Objective: Verify response time of the application under low, normal, moderate and
heavy load conditions.

Test Description : Verify response time when user access different number of active
investigations
Test Steps:
# Description Expected Result
1 Prepare Test Data for:
 100 Investigations
2 Navigate to Investigations panel Access the list of active investigations.
Accepted response time for 100 investigations: 1 sec
2.1.4.2 Test Case 2: Verify response time when users access different number of jobs
Test ID: 2
Test Description : Verify response time when user access different number of jobs
Test Steps:
 100 Jobs
 1000 Jobs
 10000 Jobs
2 Navigate to Jobs panel Access the List of Jobs.
Accepted response time for 100 jobs: 1 sec

2.1.4.3 Test Case 3: Verify response time when users access a graph with different number of links
and nodes
Test ID: 3
Test Description : Verify response time when user access a graph with different number of links
and nodes
Test Steps:
 Graph with 100 nodes/

1000 links
 Graph with 200 nodes/
20000 links
 Graph with 500
nodes/50000 links
2 Navigate to Graph Access the Graph.
Accepted response time for Graph with 100 nodes/ 10000 links: 5 min
2.1.5 Stress Testing

The stress testing was used to test the SPIRIT system with unusual parameters or conditions that are
unlikely to occur in a real scenario. It was used to find defects in unexpected scenarios like the test
cases described in the next paragraphs:
2.1.5.1 Test Case 4: Check CPU and memory usage when 3 different jobs insert a large volume of data
to the database simultaneously
Test ID: 4
Test Objective: Check CPU and memory usage of the system and the database under peak
load conditions.
Test Description : Check CPU and memory usage when 3 different jobs insert a large volume of
data to the database simultaneously

Test Steps:
 A Crawler Job with a Graph

with 200 nodes/ 20000
links
 A Refined Search Job with
Graph with 200 nodes/
20000 links
 A Third Party Job with a
Graph with 200
nodes/20000 links
2 Navigate to System Settings and Access System Settings:
Docker Engine.
CPU allocation: 70%
Memory allocation: 70%
Access Docker Engine:
Docker Engine Wealth: Database container is running
3 Navigate to Graphs Access each Job’s Graph:
All Graphs produced.
2.1.6 Scalability Testing

Scalability Testing was used to measured the application’s performance in terms of its ability to scale
up or scale down the number of user requests or other such performance measure attributes.
2.1.6.1 Test Case 5: Determine how the system scales with increasing workload
Test ID: 5
Test Objective: Determine how the system scales with increasing workload
Test Description : Ensure that the system is scalable after a particular load
Test Steps:
1 Run Test Cases 1 -3 (load testing Accepted response time: 55 mins

test cases) simultaneously.

2 Navigate to System Settings and Access System Settings:

Docker Engine.
CPU allocation: 75%
Memory allocation: 75%
Network allocation: 10%
Access Docker Engine:
Docker Engine Wealth: All SPIRIT containers are running
2.2 Trial Runs and Functional Validation

2.2.1 LEA’s Requirements and Universal Use Case Scenario
According to Deliverable D8 (i.e. D2.4), where functional requirements were elicited, we have formed
a set of specifications for the forthcoming development cycles of Year 1. These are based to, a so
called, Universal Use Case Scenario, the following table presents these requirements which are
integrated into trial runs and functional validation of the SPIRIT platform.
Table 1: LEA’s Requirements
No. Requirement
1 Each partner will have a nominated Administrator who will be responsible for managing
their respective users and permissions (Admin; admin). Need to display both User ID & User
Name?
2 Administrators will set up all user’s access rights for logging in, creating incidents, viewing
incidents, searches etc on a global basis
3 Administrators can delete other users
4 With correct permissions, a user can start a new investigation. A Name, Description, Case
Number and Authority options are required.
5 Only a user who has created the investigation can open it as an Existing Investigation; unless
the person has been permitted to do so in the permissions table.
6 Only investigations that have the “Current” flag set are displayed
7 Existing Investigation: only see those in the list that have been set up in permissions. Case
reference to be displayed with a limited Description which is searchable.
8 Log Off will close the investigation. Quit will close the system

9 SIO will have the permissions to Manage the Investigation
10 SIO can add users and user permissions to the investigation – those users must already be
set up in the system
11 SIO can assign one or more searches to one or more users – those users must already be set
up in the system
12 SIO can delete investigations – This action will Set the “Current” flag to false so that it is not
displayed in any listing
13 Every action undertaken will be written to a database (transactions database)
14 Separate database for WP9? (Security database)
15 What database(s) to be used
16 Who will set them up?
17 Manage Operation to include Authority for Deep/Dark web search
18 Dark/Deep web authority set up at the initial investigation creation
19 Dark/Deep web authority set up by the SIO at any other time during the investigation
20 Search Authorisation and justification to be entered at this stage to cover searching

throughout the investigation
21 Ability to enter one or more search terms each of which separated by a semi colon (;)
22 Level of searching will default to 3. This will be set for Prototype 1
23 Exact search will use the search engine(s) / api’s on the words/phrases and each search item
will be AND to ensure all are combined
24 Fuzzy search will use the search engine(s) / api’s on the words/phrases and each search item
will be OR to ensure all terms are searched
25 Social Media searches will only retrieve publicly available information.
26 Deep Web searches cannot proceed without relevant authorisation
27 Dark Web searches cannot proceed without relevant authorisation. Authority for both to be
moved to Investigation level rather that search level. Added at creation or added later
though Op. Manage.

28 Authorisation details to be saved in database
29 All criteria will be saved in transaction database
30 Surface Web search engines for Prototype 1 will be Google / Bing (Same as Yahoo)/ Yahoo
31 Social Media search engines for Prototype 1 will be Facebook / Twitter New API means
virtually nothing from Facebook
32 Deep/Dark Web search engines for Prototype 1 will be….. LUTEC to be contacted
33 Dark Web searching for Prototype 1 will….
34 Run search will produce a list of results at Level 1
35
Example: Google results from all 10 pages will be returned
36 LEA intervention (re Ethics): None of the items return will be selected. The investigator will
“examine” each item and manually include it for the next set of searching.
37 Click on an item to display an information box
38 Click on the information box url to display the page
39 Information box: Click “Include” to add to the next set of searching
40 Search Screen: Click “include” on the search screen to add to the next set of searching
41 Click Continue to search all levels; transactions logged in database
42 Numbers 35 to 41 above on the final search set ready for entity extraction; transactions
logged in database
43 Reason for each search to be recorded on the search screen
44 Drop down list of choices – end users to discuss
45 Only searches created by the investigator, or those where the investigator has suitable
permissions, within the current investigation will be displayed
46 Click “Hide” to remove searches from the list, transactions logged in database
47 View Hidden Searches will list those that have been hidden, click check box to unhide,
transactions logged in database

48 Show Searches, add the unhidden to the search list
49 Select a search: “View Level 1 Search Results” see numbers n to n above, transactions
logged in database
50 Select a search to rerun the search (possible new information): “Re-Run Search” see
numbers n to n above, transactions logged in database
51 Re-run search only shows new results
52 Add selected sites to existing search page entries.
53 Surface Web Search Flow: Search criteria & sites entered; Level 1 results returned; Level1
results filtered by LEAs; Those Level 1 searches selected will be used for levels 2 & 3
searching; the results of which are returned; LEAs filter the level 2/3 results; Data
“cleansed”; (NLP; Sentiment; Entity Extraction); Graphing
54 Apply sentiment analysis to search results, transactions logged in database
55 Entities extracted from the search results, transactions logged in database
56 Images, videos (media) entities are stored, transactions logged in database (Is there a
separate media database?)
57 Entities including media must be linked to the original web page(s)
58 Entities must be labelled where possible
59 Entities added to ontology?
60 Graph database that works in Windows?
61 Recognise faces in images
62 Match a face to same faces in dataset
63 Acoustic scene recognition
64 Match speech to a known speaker
65 Is known speaker speaking
66 Multiple speakers identified throughout the sound bite
67 NLP – Entity and relationship extraction from text

68 Limit display to 2D
69 Graph should auto-arrange with minimal cross overs
70 Ability to draw a boundary around a section of the graph to select all entities inside
71 Ref. above – click on an entity to move the group
72 Ref. above – Delete key to delete entire group
73 Deleted entities should be placed in the trash area; see below
74 Ability to zoom in/out
75 Ability to click on the image and move the image
76 Refine graph though selection criteria
77 Add shapes/colours to nodes
78 Explore and save different views of graph
79 The link between entities should be labelled where possible
80 Click on the link and the relevant web page will be displayed
81 Click on the relevant entity and the web page will be displayed
82 Click on the video entity and the video will play
83 Inferring links in the data, transactions logged in database
84 Permitting the Investigator to add notes to an entity or link, transactions logged in database.
85 Remove an entity and the links are also removed, transactions logged in database
86 The entity is placed in the Trash area, transactions logged in database
87 Entities in the Trash area can be replaced on the screen and their links will automatically be
reinstated, transactions logged in database
88 If an entity is removed then all orphan entities are also removed, transactions logged in
database
89 Click on entity will show all info retrieved relating to that entity
90 Draw inferred link – text box for reasons

91 Add a node – text box reasons
2.2.2 Authentication & Authorisation Service

2.2.2.1 Test Case 1: Successful login
Test ID: 1
Test Objective: Successful Login
Requirement fulfilled (no): 1, 2, 13
(According to Table 1: LEA’s

Requirements)
Test Steps:
1 Login to the SPIRIT Platform by using User successfully logged into the SPIRIT Platform.
correct credentials
2 Go to: http://localhost:80/ to verify that The following info recorded:

this action recorded to the Activity Log
Viewer service Username: the logged user, action: login-submit
Username: the logged user, action: login-success
2.2.2.2 Test Case 2: Logout

Test ID: 2
Test Objective: Logout
Requirement fulfilled (no): 8, 13

Requirements)

Test Steps:
1 Login to the SPIRIT Platform by using User logins into the SPIRIT Platform.
correct credentials
2 User logouts from SPIRIT Platform The login page is displayed

Viewer service Username: the logged out user, action: logout-submit
2.2.3 General User Interface

2.2.3.1 Test Case 3: List of active investigations
Test ID: 3
Test Objective: List of active investigations
Requirement fulfilled (no): 1, 2, 6, 7, 13

Requirements)
Test Steps:
1 Navigate to Investigations panel List of active investigations either created by user or the user had been
authorized to view them
2 Go to: http://localhost:80/ to The following info recorded:

verify that this action recorded
to the Activity Log Viewer The logged user has one of the following permission for all listed investigations:
service ‘hasAdminPermision’, ‘hasWritePermission’, ‘hasReadPersmission’
2.2.3.2 Test Case 4: Hide investigation

Test ID: 4
Test Objective: Hide an investigation
Requirement fulfilled (no): 1, 2, 9, 11, 12, 13

Requirements)
Test Steps:

1 Navigate to Investigations panel. Select an Selected investigation is not listed.

investigation and press hide button
2 Navigate to Inactive Investigations tab The hidden investigation is listed
3 Go to: http://localhost:80/ to verify that this The following info recorded:

action recorded to the Activity Log Viewer
service Username: the logged out user, action: investigation hided
2.2.3.3 Test Case 5: List of inactive investigations

Test ID: 5
Test Objective: List of inactive investigations

Requirements)
Test Steps:
1 Navigate to Investigations panel List of inactive investigations either created by user or the user had been
and go to ‘Inactive’ tab. authorized to view them

verify that this action recorded
to the Activity Log Viewer The logged user has one of the following permission for all listed
service investigations: ‘hasAdminPermision’, ‘hasWritePermission’,
‘hasReadPersmission’
2.2.3.4 Test Case 6: Unhide investigation

Test ID: 6
Test Objective: Unhide an investigation
Test Steps:
1 Navigate to Investigations panel. Select an Selected investigation is not listed in the Inactive
investigation and press unhide button investigations tab
3 Navigate to Active Investigations tab The unhidden investigation is listed

3 Go to: http://localhost:80/ to verify that this The following info recorded:

action recorded to the Activity Log Viewer
service Username: the loggin out user, action: investigation unhided
2.2.3.5 Test Case 7: Create an investigation

Test ID: 7
Test Objective: Create a new investigation

Requirements)
Test Steps:
1 Select ‘create investigation’ button A form is displayed asking the end-user to complete it
2 Complete the displayed form and press The new investigation is listed in the list of active investigations
‘create investigation’ button

Viewer service Username: the logged user, action: investigation created
2.2.3.6 Test Case 8: Edit an investigation

Test ID: 8
Test Objective: Modify an investigation
Requirement fulfilled (no): 1, 2, 3, 4, 5, 9, 13

Requirements)
Test Steps:
1 Select ‘edit investigation’ button A form is displayed asking the end-user to modify the fields of the
selected investigation
2 Modify a field and press ‘Update The modified investigation is listed in the list of active
investigation’ button investigations. The modified field is as specified by end-user


Viewer service Username: the logged user, action: investigation edited
2.2.3.7 Test Case 9: Authorise an investigation

Test ID: 9
Test Objective: Authorize an investigation
Requirement fulfilled (no): 1, 2, 4, 5, 9, 13, 17, 18, 19, 20, 28

Requirements)
Test Steps:
1 Select ‘authorize investigation’ button An empty authorization form is displayed asking the end-user to
complete it
2 Complete the displayed form and press UI lists the active investigations.
‘authorize investigation’ button
3 Select ‘authorize investigation’ button The authorization form contains the information specified by end-
user

Viewer service Username: the logged user, action: investigation authorised
2.2.3.8 Test Case 10: Edit authorisation form

Test ID: 10
Test Objective: Edit the authorization form

Requirements)
Test Steps:
1 Select ‘authorize investigation’ button A completed authorization form is displayed
2 Modify fields in the authorization form and UI lists the active investigations.
press ‘edit authorization’ button

3 Select ‘authorize investigation’ button The authorization form contains the modified fields as specified by
end-user

Viewer service Username: the logged user, action: authorisation form edited
2.2.3.9 Test Case 11: Manage investigation permissions

Test ID: 11
Test Objective: Give to another user read permissions to a selected investigation

Requirements)
Test Steps:
1 Select ‘user management’ button The User Management form is displayed
2 Give ‘hasReadPermission’ to another user UI lists the active investigations.

and press ‘save user permissions’ button
3 Select ‘user management’ button In User Management form the ‘hasReadPermission’ field of the
other user is ‘ticked’

Viewer service Username: the logged user, action: hasReadPermission’ to selected
user
2.2.3.10 Test Case 12: Read only permissions on investigation

Test ID: 12
Test Objective: A user with read only permissions to a selected investigation
Requirement fulfilled (no): 1, 2, 11, 13

Requirements)
Test Steps:

1 Login to the SPIRIT platform as the user with The selected investigation from the TC9 is also listed in the active
‘hasReadPermission’ (TC 9) investigations list.
2 Select ‘User Management’ button A message ‘You are not allowed to set user permissions to this
investigation’ appears.
3 Select ‘Edit investigation’ button A message ‘You are not allowed to edit this investigation’ appears.
4 Select ‘Authorize investigation’ button A message ‘You are not allowed to authorize this investigation’
appears.
5 Select ‘Hide investigation’ button A message ‘You are not allowed to hide this investigation’ appears.
6 Select ‘Jobs’ button The ‘Jobs’ form will list all the investigation’s jobs. UI does not
provide the ‘Proceed to Analysis’ button.
2.2.3.11 Test Case 13: View jobs

Test ID: 13
Test Objective: View jobs
Requirement fulfilled (no): 1, 2, 5, 9, 11, 45, 46, 47, 48

Requirements)
Test Steps:
1 From the active investigations tab, select an The ‘Job’s form is displayed
investigation and press the ‘Jobs’ button
2 Select refined search jobs The listed refined search jobs are identical to those listed in the
‘Refined Search’ section of ‘Jobs’ form
3 Select crawler jobs. The listed crawler jobs are identical to those listed in the ‘Crawler’
section of ‘Jobs’ form
4 Select third party jobs The listed third party jobs are identical to those listed in the ‘Third
Party’ section of ‘Jobs’ form
2.2.3.12 Test Case 14: Access the Graph

Test ID: 14
Test Objective: Access graph
Requirement fulfilled (no): 55, 56, 58, 67, 68, 69, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90,
91


Requirements)
Test Steps:
1 From the active investigations tab, select an

investigation and press the ‘Jobs’ button.
The Graph is displayed.
Select a Job type from the list and press the
action ‘View Graph’ button.
2 Select to add a new node on the Graph. A form is displayed to fill the new node’s information.
Add relative information (name, type, The new node is added on the graph.
reason for addition) on the form.
3 Select to add an inference between two (2) A form is displayed to fill the inference’s information.
nodes on the Graph.
The new inference is added on the graph connected the selected
Add relative information (reason for nodes.
addition) on the form.
4 Select to view a node’s information. A dialog box is displayed providing the node’s information.
5 Select to hide a node from the Graph. The selected node is displayed as grey-out.
6 Delete nodes. Undo the action(s) Deleted nodes are displayed into the graph
7 Perform some modifications (such as create Reload the graph. The modifications should be made.
a node, delete a link) and save the graph
8 Select to filter based on node’s type. The nodes with the selected type are displayed as red
2.2.4 Logging and Privacy Control Service

In order to perform trial runs for Logging and Privacy Service, please proceed to test cases 1-11.
2.2.5 Refined Search Service

2.2.5.1 Test Case 15: Level 1 refined search
Test ID: 15
Test Objective: Level 1 Refined Search
Requirement fulfilled (no): 13, 21, 22, 23, 24, 25, 30, 31, 37, 38, 39, 40, 41, 34, 35, 49, 45, 46, 47, 48


Requirements)
Test Steps:
1 Go to the ‘Jobs’ form and press the ‘Initiate The ‘Initiate an analysis process’ form is displayed.
an analysis job’ button
2 Select ‘Keyword’, ‘Refined Search’ and fill in The Level 1 Results are displayed
the following fields: Reason, Keyword and
Google.
Press ‘Analyze’ button

Viewer service Username: the logged user, action: ‘refinement-stepone’ to
2.2.5.2 Test Case 16: Level 2 refined search

Test ID: 16
Test Objective: Level 2 Refined Search

Requirements)
Test Steps:
1 From the ‘L1 Results’ form, select one or The Level 2 Results are displayed
more results and press ‘Proceed to L2’
2. Go to: http://localhost:80/ to verify that The following info recorded:

Viewer service Username: the logged user, action: ‘refinement-steptwo’ to
2.2.5.3 Test Case 17: Select level 2 refined search results and proceed to further analysis
Test ID: 17

Test Objective: Select level 2 results and proceed to further analysis

Requirements)
Test Steps:
1 From the ‘L2 Results’ form, select one or An “Analysis processes have been created” message is displayed
more results and press ‘Proceed to analysis’

Viewer service Username: the logged user, action: ‘analysis-submit’ to selected
investigation
3 When all analysis processes complete, go to UI visualizes the produced social graph
‘Jobs’ form, select the initiated refined
search jobs and press ‘View Graph’ button
2.2.6 Crawler Service

2.2.6.1 Test Case 18: Initiate a crawler job
Test ID: 18
Test Objective: Initiate a Crawler Job
Requirement 13, 21, 22, 25, 29, 53, 54, 55, 56, 57,67
fulfilled (no):
(According to Table
1: LEA’s
Requirements)
Test Steps:
1 Go to the ‘Jobs’ form and press the The ‘Initiate an analysis process’ form is displayed.
‘Initiate an analysis job’ button
2 Select ‘URL and specify a list of urls. An “Analysis processes have been created” message is displayed
Press ‘Analyze’ button


investigation
4 Go to ‘Jobs’ form to view the created The created crawler job is visualized and contains the list of urls
crawler job specified by end user.
5 When all analysis processes complete, go UI visualizes the produced social graph
to ‘Jobs’ form, select the initiated refined
search jobs and press ‘View Graph’
button
2.2.7 Third Party Service

2.2.7.1 Test Case 19: Initiate a third party job (twitter search by keyword)
Test ID: 19
Test Objective: Initiate a third party api job (Twitter search by keyword)
Requirement fulfilled (no): 13, 21, 22, 25, 29, 31, 53, 54, 55, 56, 57,67
(According to Table 1: LEA’s Requirements)
Test Steps:
1 Go to the ‘Jobs’ form and press the ‘Initiate The ‘Initiate an analysis process’ form is displayed.
an analysis job’ button
2 Select Keyword and Automated Search. Fill An “A Third Party Job initiated” message is displayed
in the keyword field, select ‘Twitter’ and
press ‘Analyze’ button

investigation
4 Go to ‘Jobs’ form to view the created third- The created third-party job is visualized
party job
5 When all analysis processes complete, go to UI visualizes the produced social graph
‘Jobs’ form, select the initiated third-party
job and press ‘View Graph’ button
2.2.7.2 Test Case 20: Initiate a third party job (google search by keyword)
Test ID: 20

Test Objective: Initiate a third party api job (Google search by keyword)
Requirement 13, 21, 22, 29, 53, 54, 55, 56, 57,67
fulfilled (no):
(According to Table
1: LEA’s
Requirements)
Test Steps:
1 Go to the ‘Jobs’ form and press The ‘Initiate an analysis process’ form is displayed.
the ‘Initiate an analysis job’
button
2 Select Keyword and Automated An “A Third Party Job initiated” message is displayed
Search. Fill in the keyword field,
select ‘Google’ and press
‘Analyze’ button

verify that this action recorded to
the Activity Log Viewer service Username: the logged user, action: ‘analysis-submit’ to selected investigation
4 Go to ‘Jobs’ form to view the The created third-party job is visualized

created third-party job
5 When all analysis processes UI visualizes the produced social graph

complete, go to ‘Jobs’ form,
select the initiated third-party job
and press ‘View Graph’ button
2.2.7.3 Test Case 21: Initiate a third party job (twitter search by twitter id)
Test ID: 21
Test Objective: Initiate a third party api job (Twitter search by twitter_id)
Requirement 13, 21, 22, 25, 29, 53, 54, 55, 56, 57,67
fulfilled (no):
(According to Table
1: LEA’s
Requirements)
Test Steps:

1 Go to the ‘Jobs’ form and press The ‘Initiate an analysis process’ form is displayed.
the ‘Initiate an analysis job’
button
2 Select Twitter_id, fill in twitter ids An “A Third Party Job initiated” message is displayed
and press ‘Analyze’ button

verify that this action recorded to
the Activity Log Viewer service Username: the logged user, action: ‘analysis-submit’ to selected investigation
4 Go to ‘Jobs’ form to view the The created third-party job is visualized

created third-party job
5 When all analysis processes UI visualizes the produced social graph

complete, go to ‘Jobs’ form,
select the initiated third-party job
and press ‘View Graph’ button

3 Test Results
3.1 Non-functional validation (to be completed by technical consortium)

Ref. Test Test Results Expected Results Comments
Case no
1 Access the list of active investigations. Access the list of active investigations. Test passed.
Accepted response time for 100 investigations: 1 Accepted response time for 100 investigations: 1 sec
sec
Accepted response time for 1000 investigations: 3
sec Accepted response time for 10000 investigations: 5
sec
Accepted response time for 10000 investigations:
5 sec
2 Access the List of Jobs. Access the List of Jobs. Test passed.
Accepted response time for 100 jobs: 1 sec Accepted response time for 100 jobs: 1 sec
3 Access the Graph. Access the Graph. Test failed.
Accepted response time for Graph with 100 Accepted response time for Graph with 100 nodes/ Need to be improved
nodes/ 10000 links: 7 min 10000 links: 5 min

Ref. Test Test Results Expected Results Comments

Case no
Accepted response time for Graph with 200 Accepted response time for Graph with 200 nodes/
Accepted response time for Graph with 500 Accepted response time for Graph with 500 nodes/
4 Access System Settings: Access System Settings: Test passed.
CPU allocation: 68% CPU allocation: 70%
Memory allocation: 67% Memory allocation: 70%
Access Docker Engine: Access Docker Engine:
Docker Engine Wealth: Database container is Docker Engine Wealth: Database container is running
running
5 Access System Settings: Access System Settings: Test passed.
CPU allocation: 745% CPU allocation: 75%
Memory allocation: 75% Memory allocation: 75%
Network allocation: 9% Network allocation: 10%
Access Docker Engine: Access Docker Engine:
Docker Engine Wealth: All SPIRIT containers are Docker Engine Wealth: All SPIRIT containers are
running running

3.2 Functional validation - Questionnaire (completed by LEAs during the Y1 testing evaluation)
The following questionnaire was sent to the LEAs during the testing and evaluation of the Year 1 SPIRIT Prototype. As soon as they filled the
questionnaire, they sent it back to the WP7 leader. The technical consortium will consider the test results during the lifecycle of Year 2
Prototype.
Name:
Date:
Ref. Test Results User Think Time Comments

Case no
(* select an option from the list) (* complete estimation time (* complete with additional comments or
for the specific action) remarks)
1-2 I can log in and log out to the Spirit Platform without
encountering any problems.
Yes
1 - 13 I can access investigations that I think I should have

access to, and I do not see and cannot access
investigations that I should not have access to.
Choose an item.
3 - 20 I find it easy to find and access the different

functionalities of the system through the user
interface.

Name:
Date:

Case no
Choose an item.
3 -20 I can navigate through the system and use its

functionalities quickly and efficiently.
Choose an item.
1 - 20 As a user with audit privileges, I can view detailed

information on the actions that have been carried
out by the system and its users
Choose an item.
1 - 20 As a user with audit privileges, I can view a sufficient

amount of detail concerning the logged actions that
I am interested in.
Choose an item.
3 - 20 I can access all system functionalities that are

available to me through the user interface without

Name:
Date:

Case no
encountering any problems such as the functionality

not responding or having to restart the system.
Choose an item.
14 - 20 I can view, access and filter graphs created by the

Spirit Platform in a way that supports my work.
Choose an item.
14 - 20 The processing status and progress are

communicated to me clearly and in an easy to
understand way so that I always know the status of
the requests that I have submitted.
Choose an item.
14 - 20 The web search returns an amount of results that

conforms to what I expect from a web search
functionality

Name:
Date:

Case no
Choose an item.
14 - 16 The refined search function as a step-by-step

refinement process works as I would expect it to
work in terms of the results returned and the ability
to drill down on interesting results.
Choose an item.
14 - 20 Entities such as persons, places and times that are

extracted are extracted accurately and identified as
the correct type of entity (i.e. a person is identified
as a person and not as a place etc.)
Choose an item.
14 - 20 Relations between entities that are shown in graphs

are useful for supporting my activities.
Choose an item.

Name:
Date:

Case no

functionality.
Choose an item.
18 - 20 I can access messages posted on Twitter as part of

an investigation in line with our relevant policies
concerning social media access.
Choose an item.
18 - 20 I can access functionalities that search by keyword

as I would expect them to work
Choose an item.

3.2.1 Evaluation from Hellenic Border Police (HPB)

Name: Hellenic Border Police (HBP)
Date:

Case no
1-2 I can log in and log out to the Spirit Platform without 1 minute Upon entering the application, an
encountering any problems. unnecessary message is shown to
the user for a successful entry,
Yes
which requires a button be pressed
in order to close the window.
1 - 13 I can access investigations that I think I should have 2-3 minutes No comments
Yes
3 - 20 I find it easy to find and access the different 3-4 minutes No comments
interface.
Yes

Date:

Case no
3 -20 I can navigate through the system and use its 3-4 minutes
Yes
1 - 20 As a user with audit privileges, I can view detailed 2 minutes

Yes
1 - 20 As a user with audit privileges, I can view a sufficient 1 minute

amount of detail concerning the logged actions that
I am interested in.
Neutral
3 - 20 I can access all system functionalities that are 4-5 minutes A system restart required once.

Date:

Case no

Neutral
14 - 20 I can view, access and filter graphs created by the 2 minutes New filters should be supported.
Neutral
14 - 20 The processing status and progress are 4-5 minutes When performing a refined exact
communicated to me clearly and in an easy to search, the results did not return
understand way so that I always know the status of sorted by relevance, when two or
the requests that I have submitted. more keywords were used.
Yes When performing a fuzzy search,

the results did not return sorted by
relevance, when two or more
keywords were used.

Date:

Case no
14 - 20 The web search returns an amount of results that 4-5 minutes

functionality
Yes
14 - 16 The refined search function as a step-by-step When performing an automated

refinement process works as I would expect it to search, the application froze each
work in terms of the results returned and the ability time the user pressed the search
to drill down on interesting results. button, making the re-entry in the
application inevitable.
No
14 - 20 Entities such as persons, places and times that are 3-4 minutes
Yes

Date:

Case no
14 - 20 Relations between entities that are shown in graphs 3-4 minutes New relations should be added.
Neutral
14 - 17 The web search returns an amount of results that 4-5 minutes No comments
functionality.
Yes
18 - 20 I can access messages posted on Twitter as part of Unable to initiate a search on

an investigation in line with our relevant policies Twitter.
No
18 - 20 I can access functionalities that search by keyword 2-4 minutes


Date:

Case no
Yes
3.2.2 Evaluation of Thames Valley Police (TVP)

Name: Thames Valley Police
Date:

Case no
1-2 I can log in and log out to the Spirit Platform without 5 Mins Initially three attempts without
encountering any problems. changing login details, successfully
logged in on third attempt
Neutral

Date:

Case no
1 - 13 I can access investigations that I think I should have 2 mins Yes

Yes
3 - 20 I find it easy to find and access the different ? 10 mins At times it’s difficult to understand
functionalities of the system through the user what area of the database you are
interface. in
No
3 -20 I can navigate through the system and use its 10 mins I can navigate through the system;
functionalities quickly and efficiently. however, it needs to be made
easier for the user to interrupt
Yes
where they are.
1 - 20 As a user with audit privileges, I can view detailed 5 mins I did not have this capability.

Date:

Case no
No
1 - 20 As a user with audit privileges, I can view a sufficient N/A I did not have the capability to do
amount of detail concerning the logged actions that this
I am interested in.
No
3 - 20 I can access all system functionalities that are N/A As above, I did not have to restart
available to me through the user interface without the system
Yes
14 - 20 I can view, access and filter graphs created by the N/A

Yes

Date:

Case no
14 - 20 The processing status and progress are 5-7 mins This needs to be clearer, and when
communicated to me clearly and in an easy to you click ok when the message is
understand way so that I always know the status of displayed to say its initializing,
the requests that I have submitted. everything disappears and you have
to go back to the job and reopen.
Neutral
14 - 20 The web search returns an amount of results that 3 mins for SPIRIT Search I did not run a comparator search
conforms to what I expect from a web search directly on google
functionality
Neutral
14 - 16 The refined search function as a step-by-step 3-5 mins Yes

Yes

Date:

Case no
14 - 20 Entities such as persons, places and times that are N/A No Extraction identified with the
extracted are extracted accurately and identified as search criteria I used?
No
14 - 20 Relations between entities that are shown in graphs N/A Unable to display the graph.
No
14 - 17 The web search returns an amount of results that N/A Unable to determine this at this
conforms to what I expect from a web search stage.
functionality.
No

Date:

Case no
18 - 20 I can access messages posted on Twitter as part of N/A No

No
18 - 20 I can access functionalities that search by keyword N/A Unable to determine this.
No

3.2.3 Evaluation from Serbian Police (SERBLEA)

Name: Serbian Police
Date:

Case no
1-2 I can log in and log out to the Spirit Platform without In a few seconds No additional comments
Yes
1 - 13 I can access investigations that I think I should have In a few seconds No additional comments
Yes
3 - 20 I find it easy to find and access the different - The user interface is concise and
functionalities of the system through the user intuitive
interface.
Yes
3 -20 I can navigate through the system and use its In a few seconds Navigation through the system is
functionalities quickly and efficiently. satisfactorily efficient.

Date:

Case no
Yes
1 - 20 As a user with audit privileges, I can view detailed - When trying to access localhost:80,
information on the actions that have been carried the Internet browser displays the
out by the system and its users error "Unable to connect."
No
1 - 20 As a user with audit privileges, I can view a sufficient - When trying to access localhost:80,
amount of detail concerning the logged actions that the Internet browser displays the
I am interested in. error "Unable to connect."
No
3 - 20 I can access all system functionalities that are - No additional comments

Yes

Date:

Case no
14 - 20 I can view, access and filter graphs created by the In a few seconds The activity does not last long, even
Spirit Platform in a way that supports my work. with graphs of several hundred
nodes and a few tens of thousands
Yes
of connections.
The filter function should hide

everything except the selected type
of entity, such as a person, to make
the content clearer and more
comfortable to analyze.
14 - 20 The processing status and progress are - When initiating the analysis, the
communicated to me clearly and in an easy to system informs that the process
understand way so that I always know the status of had started, but not whether and
the requests that I have submitted. when it was completed. In other
cases, everything is clear.
Neutral

Date:

Case no
14 - 20 The web search returns an amount of results that - In a way, but there is a lot of room
conforms to what I expect from a web search for improvement.
functionality
Yes
14 - 16 The refined search function as a step-by-step - But slightly better results are
refinement process works as I would expect it to obtained in direct work with
work in terms of the results returned and the ability Google.
Yes
14 - 20 Entities such as persons, places and times that are - There is a lot of room for significant
extracted are extracted accurately and identified as improvement of this functionality,
the correct type of entity (i.e. a person is identified and a larger number of errors is still
as a person and not as a place etc.) noticeable.
Neutral

Date:

Case no
14 - 20 Relations between entities that are shown in graphs - Relations increase confusion as to
are useful for supporting my activities. the number of nodes increases.
No
14 - 17 The web search returns an amount of results that - In a way, but there is a lot of room
conforms to what I expect from a web search for improvement.
functionality.
Yes
18 - 20 I can access messages posted on Twitter as part of - No additional comments

No
18 - 20 I can access functionalities that search by keyword - No additional comments


Date:

Case no
Yes
3.2.4 Evaluation from West Midland Police (WMP)

Name: West Midland Police
Date:

Case no
1-2 I can log in and log out to the Spirit Platform without
Under 1 minute No issues
Yes

Date:

Case no
1 - 13 I can access investigations that I think I should have

Under 1 minute No issues
Yes
3 - 20 I find it easy to find and access the different

None Intuitive and easy to use
interface.
Yes
3 -20 I can navigate through the system and use its

Under 1 minute Intuitive and easy to use
Yes
1 - 20 As a user with audit privileges, I can view detailed Struggle with this at present as
information on the actions that have been carried unable to view localhost:80. May be
N/A

Date:

Case no
No due to local firewall settings which I

am unable to alter.
1 - 20 As a user with audit privileges, I can view a sufficient Struggle with this at present as
amount of detail concerning the logged actions that unable to view localhost:80. May be
N/A
I am interested in. due to local firewall settings which I
am unable to alter.
No
3 - 20 I can access all system functionalities that are N/A With the exception of the auditing
available to me through the user interface without facilities referred to above.
Neutral
14 - 20 I can view, access and filter graphs created by the Easily viewed and accessible –
Spirit Platform in a way that supports my work. content is a little confusing if the
Under 1 minute
result has significant entities.
Yes

Date:

Case no
14 - 20 The processing status and progress are

N/A Fairly easy to know when
something has worked and when it
has not.
Yes
14 - 20 The web search returns an amount of results that Not really – the results can be
conforms to what I expect from a web search slightly confusing but understand
N/A
functionality that the full functionality is not
there at present. For example
No
typing in Garry Mills brings back a
lot of information about a musician,
but little else.
14 - 16 The refined search function as a step-by-step As above.


Date:

Case no
No
14 - 20 Entities such as persons, places and times that are As above.

No
14 - 20 Relations between entities that are shown in graphs I think that this will be useful in the
are useful for supporting my activities. fullness of time, but at present with
the full results not being returned,
Neutral
it is difficult to fully assess.
14 - 17 The web search returns an amount of results that Seems to be same as 14-20
functionality.
Neutral

Date:

Case no
18 - 20 I can access messages posted on Twitter as part of Unable to access any real content
an investigation in line with our relevant policies that relates to Twitter. API issue is
concerning social media access. not resolved at present.
No
18 - 20 I can access functionalities that search by keyword Function works – but again it does
as I would expect them to work not necessarily bring back anything
that you would expect at present.
Yes
See reference to Mills above.
3.2.5 Evaluation from STAD Antwerp Police (LPA)

Name: STAD Antwerp Police
Date:

Case no
1-2 I can log in and log out to the Spirit Platform without 1 hour in total with We had a lot of difficulties because
encountering any problems. rebooting and trying we had to keep logging in and out of
again, 5-10 minutes for the system (because we clicked the
Yes
the system to react to a ‘login’ button twice and then the
successful logging login was unsuccessful)
- It takes a long time in between

clicking ‘sign in’ and a possible
response and again after the
notification that we had a
successful login to get into the
actual system.
1 - 13 I can access investigations that I think I should have Few seconds - When making a new investigation,
access to, and I do not see and cannot access the proportionality options seems
investigations that I should not have access to. more account/tool based then
investigation based
Yes
- The proportionality options seem
long to read them every time,

Date:

Case no
maybe using a key word before

starting the sentence would make
sense.
3 - 20 I find it easy to find and access the different Few seconds - When entering the user
functionalities of the system through the user management and no records can be
interface. shown, the system keeps on loading
which can be confusing.
Yes
- Going back functions doesn’t
work, you always have to cancel
your action. May be confusing as
well.
3 -20 I can navigate through the system and use its 1 minute - The loop-symbol for jobs feels
functionalities quickly and efficiently. more like a secondary option while
it’s the most important after all.
Yes

Date:

Case no
1 - 20 As a user with audit privileges, I can view detailed - - We assume that this option works
information on the actions that have been carried with http://localhost:80/ but this
out by the system and its users host does not react.
No
1 - 20 As a user with audit privileges, I can view a sufficient - - We assume that this option works
amount of detail concerning the logged actions that with http://localhost:80/ but this
I am interested in. host does not react.
No
3 - 20 I can access all system functionalities that are - - We had to restart the system once
available to me through the user interface without when going back with the ‘going
encountering any problems such as the functionality back’ arrow.
No

Date:

Case no
14 - 20 I can view, access and filter graphs created by the - I cannot seem to get a graph out of
Spirit Platform in a way that supports my work. the data I collected.
No
14 - 20 The processing status and progress are Few seconds

Yes

functionality
Yes
14 - 16 The refined search function as a step-by-step - We think it’s confusing that when
refinement process works as I would expect it to selecting a result from the refined
search, the selecting is colored red

Date:

Case no
work in terms of the results returned and the ability which seems that an error has
to drill down on interesting results. occurred.
No - After a few efforts to get enough

data for a graph we got the
notification that there was an error
called (Refined search – L1Results
Jobs’.
14 - 20 Entities such as persons, places and times that are

Choose an item.
14 - 20 Relations between entities that are shown in graphs

Choose an item.

Date:

Case no

functionality.
Yes
18 - 20 I can access messages posted on Twitter as part of

Choose an item.
18 - 20 I can access functionalities that search by keyword - When using the Third Party Jobs I
as I would expect them to work get a successful initiation but don’t
see any data in the ‘Third Party Job’
Choose an item.
afterwards.

4 APPPENDIX A – SPIRIT INTERNAL REVIEWING PROCESS REVISING DELIVERABLES

SPIRIT plenary meetings, of October 8th and October 22nd, decided to organise an Editorial Board to
revise all pending Deliverables. This process has been agreed and it will proceed throughout the rest
of the project life.
In this current session (November 2020) the Editorial SPIRIT Board for Deliverables’ Revision (ESB)
undertook the task on the following documents:
 D1.6 Management Systems Report (b) (WP1)

 D2.4 Requirements Analysis Document (b) (WP2)
 D2.6 System Design Document (WP2)
 D3.2 Multi-Purpose Semantic crawler (Report and software) (WP3)
 D7.1 System Integration (Report) (a) (WP7)
 D7.2 Trial Run Integration and Performance Testing (Report) (a) (WP7)
 D10.2 Exploitation Plan and IPR Management Report (WP10)
ESB configuration for this session (November 2020):
1. Geoff Robinson, TVP

2. Rick Adderley, AES,
3. Christian Weigel, FHG
4. Olaf Hartig, LIU
5. Emma Teodoro, UAB
6. Dave Hollies, WMP
7. Marcin Golizda, CEPIC
8. Costas Davarakis, SLG
ESB takes the process of distributing work and documents to each ESB member and respects SPIRIT
internal deadlines, EU review comments and workplan goals.
It is important to mention that ESB also moderated the process of passing the revised deliverables
to the Security and Independent Ethics Boards for clearance.

END OF DOCUMENT

PTC 46

Uploaded by

Copyright:

Available Formats

You might also like

PTC 46

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

PTC 46

Uploaded by

Copyright:

Available Formats

Ref.

Planned delivery date: 31/3/2020

Submission date: 31/3/2020

Work package: WP7/Task: 7.3

Work package leader: AES Deliverable leader: NST

Dissemination level ( where

Project funded by H2020 Programme – Grant Agreement number: 786993

VERSION MODIFICATION(S) DATE AUTHOR(S)

Project funded by H2020 Programme – Grant Agreement number: 786993

Project funded by H2020 Programme – Grant Agreement number: 786993

2.2.3.9 Test Case 11: Manage investigation permissions ................................................... 23

Project funded by H2020 Programme – Grant Agreement number: 786993

4. Using existing intelligence sources

Project funded by H2020 Programme – Grant Agreement number: 786993

5. Reliable & Timely

 D7.2 is based on other ethical deliverables documentation regarding security.

8. Training & Evaluation

D7.2 links to the exploitation plan, as follows:

No. Linking to exploitation area Comment Level of contribution

2 The other consortium D7.2 clarifies performance expectations, 5

3 Other LEAs and governmental D7.2 clarifies performance expectations, 4

4 Further research opportunities See No. 2 3

5 Commercial opportunities See No. 3 4

Project funded by H2020 Programme – Grant Agreement number: 786993

2 Trial Runs and Performance Testing

2.1 Performance Testing Methodology

2.1.1 Types of Performance Testing

Project funded by H2020 Programme – Grant Agreement number: 786993

2.1.2 Common performance problems

Below is a generic process on how performance testing performed:

Figure 1 - SPIRIT Performance testing process for Y1 prototype

Project funded by H2020 Programme – Grant Agreement number: 786993

Run the tests - Executed and monitored the tests.

Project funded by H2020 Programme – Grant Agreement number: 786993

# Description Expected Result

1 Prepare Test Data for:

Accepted response time for 100 investigations: 1 sec

Accepted response time for 1000 investigations: 3 sec

Accepted response time for 10000 investigations: 5 sec

# Description Expected Result

1 Prepare Test Data for:

Accepted response time for 100 jobs: 1 sec

Accepted response time for 1000 jobs: 3 sec

Accepted response time for 10000 jobs: 5 sec

Project funded by H2020 Programme – Grant Agreement number: 786993

# Description Expected Result

1 Prepare Test Data for:

 Graph with 100 nodes/

2.1.5 Stress Testing

Project funded by H2020 Programme – Grant Agreement number: 786993

# Description Expected Result

1 Prepare Test Data for:

 A Crawler Job with a Graph

Memory allocation: 70%

Access Docker Engine:

Docker Engine Wealth: Database container is running

3 Navigate to Graphs Access each Job’s Graph: