Professional Documents
Culture Documents
WSI User Guide CIS 1.1 (WS Competition)
WSI User Guide CIS 1.1 (WS Competition)
1 User Guide
CIS NG is the Next Generation of the Competition Information System (CIS) software. It has been re-
written from the ground up to provide a better user experience, as well as better performance and
stability. The software handles many of the tasks involved in running a WorldSkills Competition,
including creating marking schemes, holding Competitor information, storing Competitors’ marks,
producing medal results and many other activities.
This document will guide you through the installation and use of CIS NG. It details functions of the
software in chronological order of when they would most likely be used, not necessarily in the order
they appear in the menu.
2.1.1 Database
CIS NG gives you the choice of two different databases that can be used – MySQL or H2.
The MySQL database has the advantage of being a well-known and well-supported database
with many third party tools available for doing backups and other maintenance. If you wish to
use MySQL with CIS NG you must install it separately prior to installing CIS NG.
The H2 database requires no installation procedure for use with CIS NG. If you wish to use
this database CIS NG will create the necessary files in the databases folder of your
installation directory.
The first package assumes that you have a Java 6 JDK (not JRE) already installed on your
system. If you don’t have the JDK installed (or you don’t know if you do), choose one of the
other two packages. The other two packages come bundled with a JDK which will install during
the CIS NG installation process. One of the packages contains the JDK for 32-bit systems, the
other for 64-bit systems.
Download the package you require from http://cis.worldskills.com and save it to your hard
drive.
On this screen if you wish to upgrade an existing installation without adding any new CIS NG
instances, simply uncheck the checkbox and press the “Next” button. You will skip to step
2.1.12.
On this screen you also need to select what type of database you want to use for the new
instance of CIS NG. You can choose to create a new database in an existing MySQL
installation, or choose the H2 database which will create a file in the databases folder of your
installation directory.
Competition Name – This name appears on the top of every page, the login screen and at the
top of every PDF generated by CIS NG.
URL Path – Each CIS NG instance is accessed at a different URL by a web browser. The part
entered here comes after the machine name/address. For example, if the CIS NG screenshot
above were being installed on the WorldSkills International server it would be accessed at
http://www.worldskills.org/antarctica
Host Name (MySQL only) – This is the host name of the computer where MySQL is installed.
If it is installed on the same machine as the one CIS NG is being installed on, then it can be
left at the default name of localhost. If MySQL is on a different machine, enter the host
name or IP address of that machine. If the H2 database were selected in the previous step this
field will not appear.
Port Number (MySQL only) – This is the port number that the MySQL database is listening
on. In most cases this will be the standard MySQL port 3306.
Database Name – This is the name of the database that the new CIS NG instance will use. If
the database doesn’t exist then it will be created. If it does exist then it must be empty.
Username & Password fields – This is where you enter the username and password of the
database user that will access the CIS NG database. For MySQL installations the user must
already exist and have full permission to create and edit the database. For H2 installations
these details will be used to create a new user to access the new database that will be
created.
2.1.14 Shortcuts
For setting up shortcuts on your machine
You can also access the CIS NG instance using the IP address of your machine, for example
http://192.168.0.1/antarctica. Depending on your network configuration you may also have
other names mapped to your machine. For example, you may have domain name configured
to point to your machine, so http://www.yourdomain.com/antarctica could also point to CIS NG
The Linux installer is an executable jar file. This requires that you already have a Java 6 JDK (not JRE)
installed on the system. You have two options for running the Linux installer.
See the Windows installation documentation above for the data required at each step. The
Linux installer differs in that it does not show the “Processing” screen at 2.1.13 because it does
the processing in the background. It also skips the shortcut screen at 2.1.14 because the
automatic creation of shortcuts is unavailable for the Linux installation.
The installer will ask for the same information as the Windows installer, so the above
documentation can be used as a guide.
You can, however, still have URLs without the port number in them by redirecting traffic from
port 80 to the port that CIS NG is listening on. Here are a couple of ways you can do it:
To upgrade an existing installation run the installer for the latest version of CIS NG. Select the path of
the existing installation as the installation path in the new installer. The installer will detect that the path
contains an existing installation of CIS NG and upgrade the installation. Also remember to uncheck the
“Create new instance” checkbox in step 2.1.8 above so that it doesn’t add another database to your
installation.
CIS NG can run multiple instances (or competitions) from the one installation. To add a new instance to
your installation you need to run the configUpdater utility. It is located in the installation directory and is
named configUpdater.exe on Windows and configUpdater.sh on Linux.
CIS NG has a configuration file cisconfig.xml that it reads every time it is started. You shouldn’t
normally need to touch this file, but there are a few things that should be covered just in case you do.
Here is the file that has been created so far during installation and after adding an extra CIS NG
instance.
2.5.2 Version
This tag contains the version number of the CIS NG software.
The shutdown script works by connecting to a special port that CIS NG is listening on and
sending it a shutdown message. CIS NG will only accept connections on that port that
originate from the same machine. Any attempt to connect to it from another machine will be
rejected.
This section of the configuration file allows you change the default shutdown port and the
message that is shutdown message sent to CIS NG.
Now that CIS NG has been installed and configured, you can start up the CIS NG server. The CIS NG
server must be started before any users can login to the system.
If you chose to create shortcuts during installation you can start the CIS NG server by double clicking
on the desktop icon or clicking on the CIS NG application in your Start menu. Alternatively you can
double click on the cisng.exe file in the installation directory.
Starting the CIS NG server will create a minimised window that sits in the system tray and has the CIS
NG icon. The server is now running, and after a few seconds will be ready to accept requests from a
web browser. If the icon in the system tray is clicked it will maximise a window that looks similar to the
screenshot below:
This message shows that the configuration file has been read and all CIS NG instances have been set
up and are ready to accept connections from web browsers. It gives the URL for each instance that is
running in this installation. IThe screenshot above shows that there are two competitions running in this
CIS NG installation.
Installations on Linux should be started using the startup.sh script found in the installation directory.
The same output as shown in the screenshot of the Windows startup above can be found in the
nohup.out file.
The CIS NG server should always be shut down using the shutdown script. On Windows this is the
shutdown.exe file in the installation directory. There is also a shortcut to this file from the Windows
Start menu. On Linux the shutdown.sh script found in the installation directory should be used.
Shutting down the server in this way allows CIS NG to gracefully invalidate the users’ sessions and
clean up any locks or other resources they may be holding. Just closing the CIS NG window or killing
the process risks leaving mark entry screen for a particular Competitor locked until the user who was
entering the marks logs back in and goes the mark entry screen to reclaim the lock.
The log files for CIS NG can be found in the logs folder within the installation directory. Each instance
will have its own log file where any errors or warnings are logged. There is also a log file for the server
itself called server.rlog.
3.5 Troubleshooting
To find out what may be causing the problem, open up the server.rlog file located in the
“logs” folder of the installation directory. Most commonly you will be presented with a message
such as the following:
This indicates that some other software installed on your machine is currently using the port
that CIS NG has been configured to use, so CIS NG is unable to start. Webserver software
such as Apache, IIS or Tomcat can often cause this conflict. To resolve the conflict you can
either change the port that CIS NG uses (see section 2.5.1) or re-configure the other software
that is using the port.
Another piece of software that causes problems in this regard is Skype. It uses port 80 by
default. To turn this off, do the following:
1. Select Tools->Options from the menu.
2. Click on "Advanced" then "Connection".
3. Uncheck the box "Use port 80 and 443 as alternatives for incoming connections".
4. Click Save.
5. Restart Skype.
4.1 Logging in
Once the CIS NG server has been started a user can go to the login screen with their web browser. The
user interface has been designed to work with Firefox and this is the recommended web browser,
although other browsers should work as well.
In our case we’ll go to the login URL for our competition http://localhost/antarctica
You can see that the competition has the name that was supplied during installation. Enter the admin
username and password that you provided during installation and press enter or the login button to
login to CIS NG.
Each user of CIS NG can only have one active session at a time. If you attempt to login with a user that
has a currently active session, the active session is destroyed and the new session becomes the active
one.
Upon successfully logging in, the user is presented with the CIS NG homepage.
At the top is the header with the competition name and the WorldSkills International logo. On the left is
the menu that appears on every page of CIS NG. At the bottom of the menu are a few interesting
things.
Here you can see the language select box. The default language is English. Changing the language
results in the text of the user interface changing to the selected language.
Also shown here is the user that you’re currently logged in as, as well as a “Change Password” and
“Logout” button.
The “Change Password” screen is reached by clicking the “Change Password” button at the bottom of
the menu.
Once a new password has been saved it can be used the next time you log in.
The first item in the menu is “Config”. Various settings for the competition can be set on the Config
screen.
You need only upload one image here. CIS NG will use the uploaded version in the PDF files
and create resized versions to fit the user interface. It is recommended that an image in PNG
format be used. The file size should be kept to a minimum to avoid the size of the generated
PDF documents getting overly inflated. About 20K seems to give a good balance between
image quality and file size.
On Windows Vista and Windows 7 installations the default behaviour will likely be sufficient
because there are many Unicode fonts installed by default with these versions of Windows.
Other operating systems may need to have a font installed that will handle the characters. You
can let CIS NG know about these fonts by putting the names of the fonts in a comma-
separated list in this field, for example “MS Song, Baekmuk Batang, Kochi Mincho”.
The “Competition Dates” link in the menu will take you to the screen for setting the dates for each day of
competition.
Click the “Add” button to add a new day. The date can be changed by typing it in the text field or clicking
the calendar icon and selecting a date from the calendar. When you’ve finished adding dates hit the
“Save” button. Dates can be removed by clicking the red minus icon on the left.
The “User Groups” menu item provides a screen for managing user groups and what they have
permission to do within CIS NG. By default there is only the “Admin” user group that has permission to
every part of CIS NG. This is the user group that the admin user belongs to. If you’re planning on
allowing other people to log in to CIS NG, you will need to create some new user groups and set what
parts of the software they are allowed to use. To delete an existing user group, select it from the drop
down box and click the “Delete User Group” button with the red minus sign.
For our example competition we’ll click the “New” button and create a new group called “Experts”. They
will be allowed to edit their own skill’s information, edit their own skill’s marking scheme, view the PDF
marking forms for their own skill, enter marks for their own skill and view total marks for their own skill.
This screen is for defining the different relationships that people in CIS NG have to a skill.
Each position has a name and a position type. The position type is a category for the position. The
screenshot above shows the positions that are created automatically during installation. You can see
that there is an “Expert” position and a “Chief Expert” position. Both have the position type of “Expert”.
This will allow CIS NG to be able to retrieve a list of all Experts for a skill by retrieving all people with a
position type of Expert for that skill. There are three possible position types for a position – Competitor,
Expert and Non Participant.
Let’s say that each skill has an assistant that assists with mark entry. They will need to have a login to
CIS NG and have access to just the skill they have been assigned to. Obviously they shouldn’t have a
Competitor position or else they will appear on the marking forms and CIS NG will expect them to
receive marks. They shouldn’t be an Expert or Chief Expert because then they will appear in places like
the subjective mark entry forms as a judge for subjective marking. A new position called “Assistant”
should be created, with a position type of “Non Participant”. When a user login for each assistant is
created, they should be assigned to their skill with the “Assistant” position.
The “Edit Members” screen allows you to add, edit and delete members from the system. A newly
created competition has all the WorldSkills International members loaded by default. Individual
members can be removed by clicking the red minus icon next to the member. All members can be
removed at once by clicking the “Delete All” button at the bottom of the page. New members can be
added one at a time using the form or imported from an Excel or CSV file on the “Import Data” page
(see section 6.7). Once a Member has been added, their abbreviation code cannot be changed.
This screen is where the set of rules for subjective marking are configured.
Each rule group is required to have a name. You also need to set the minimum score and the maximum
score. These are usually 1 and 10 respectively.
You also need to define at least one (and usually only one) rule. In the screenshot above the rule is for
five judges. The maximum allowed difference between the judges’ scores is set to 4. It has also been
configured to ignore the highest score and the lowest score. So any skills that have the above
subjective judging rules applied to them need to have five judges assessing the subjective aspects of
Competitors’ work.
This screen allows you to create and delete skills from the system. You can also set various high level
details about a skill.
For our example competition two skills will be created: Snowman Building and Ice Sculpting. The Ice
Sculpting skill will be a team skill.
5.1.2 Name
This is the name of the skill. You can add a different piece of text for the name for each of the
different languages. It is mandatory to enter a name for at least one language.
The “Add Person” link under the “Personnel” section of the menu takes you to a screen where you can
add the details for a person to CIS NG. Only the “First Name” and “Last Name” fields are mandatory.
5.2.1 Picture
A picture of the Competitor can be uploaded that will appear on the mark entry screen when
marks are being entered for that Competitor.
5.2.2 Positions
This is used to map the person to a skill. The list of positions are the same that appear on the
“Positions” screen (see section 4.7). The list of skills are those created on the “Skills & Skill
Rules” screen. To create the link between the person and the skill, click the “Add” button. To
remove a link to a skill, click the red minus icon next to the skill name.
In the example screenshot above Expert One has been added as an Expert in the Snowman
Building skill.
5.2.6 Language
This is the language of the CIS NG user interface for this user. In the example screenshot
above, Expert One will see the user interface in German by default when she logs in.
The “Person Search” screen allows you to find people already entered in to CIS NG.
Clicking the “Find” button without filling in any part of the form will return all users in the system. Any
information entered in the form is used to narrow the results. In the screenshot above there has been a
search for all Competitors in the Snowman Building skill.
The “First Name” and “Last Name” fields will match names that start with the supplied text and are case
insensitive. For example, if “t” were added to the Last Name field it would return Competitor Two and
Competitor Three in the search results.
People can also be deleted from CIS NG on the search results page. The people are selected and
ticked in the checkbox. Clicking the “Delete Selected” button will delete all the currently selected people.
Clicking on any of the text in the search results will take you to Edit Person page. The Edit Person page
is very similar to the Add Person and contains the same fields as well as a Delete button.
The “Manage Teams” screen is for grouping individual Competitors into teams.
To group Competitors in to a team, click on the Competitors in the left column. The selected
Competitors will have green ticks next to their name. Click the “Group” button in the middle. The new
team will appear in the right column and the Competitors will disappear from the left column.
To remove a team, click on the team in the right column. The selected team will have a red tick on the
left. Click the “Ungroup” button. The team will be removed from the right column and all the Competitors
from the team will appear back in the left column.
People can be deleted individually from the Edit Person page by clicking the “Delete” button. Multiple
people can be deleted from CIS NG at the same time on the Person Search page. Simply search for the
people to be deleted, tick the checkbox for each person then click the “Delete Selected” button.
The “Assessment Criteria” link in the “Competition” section of the menu leads to a screen where you
can enter each criterion of the skill. As the mark for each criterion is entered it is added to the total at
the bottom of the screen.
The “Marks Defined” column contains the total of the aspect marks within that criterion. When creating a
new criterion as in the screenshot above, this value will obviously be zero. Once your marking scheme
is complete, the number of “Marks Defined” should match the “Max Mark” for the criterion. If it doesn’t
it’s a good indication that there aspects missing from the criterion.
When the “Save” button is pressed or the enter key is pressed the data is saved. Once a new criterion
is saved the delete and edit icons for the criterion will appear on the right. Clicking the edit icon will take
you to a new screen that allows you to enter sub criteria and aspects.
At the top right of the screen there are two boxes. The “Total Sub Criterion marks” box is the total
number of marks assigned to Sub Criteria in this Criterion. The “Total cumulative marks” box is the total
number of marks assigned to Aspects across all Sub Criteria on this screen.
Click the “Add Sub Criterion” button with the green plus sign to add a new sub criterion with the next
available ID. In the screenshot above sub criterion A1 has been created. There are a number of fields
for a sub criterion.
Marking Day – This can be entered on this screen or on the separate “Marking Days” screen. See
section 6.8 for further details.
Max Mark – The maximum possible mark for this sub criterion.
Marks Defined – This is the total of all aspects defined for this sub criterion and should never be more
than the “Max Mark” field. As aspects are added to the sub criterion, this field is updated.
Remove Button – The red minus icon removes the sub criterion from the criterion.
Click the “Add Subjective Aspect” button with the green plus sign to add a new subjective aspect. In the
screenshot above aspect A1.S1 has been created. There are a number of fields for a subjective aspect.
Add Row – This button adds an extra line of text to the aspect description. You can have as many of
these as you like.
Add Page Break – The first icon under the “Actions” heading will add a page break after the aspect
description in the generated marking form PDF documents.
Remove Button – The red minus icon removes the aspect from the sub criterion.
Click the “Add Objective Aspect” button with the green plus sign to add a new objective aspect. In the
screenshot above aspect A1.O1 has been created. There are a number of fields for an objective aspect.
Marking Type – For configuring mark calculations for the aspect. The default marking type is “Direct
Mark Entry”, which means that there are no calculations performed for this aspect. See section 6.5 for
further information on the different marking types.
Add Row – This button adds an extra line of text to the aspect description. You can have as many of
these as you like.
Add Page Break – The first icon under the “Actions” heading will add a page break after the aspect
description in the generated marking form PDF documents.
Remove Button – The red minus icon removes the aspect from the sub criterion.
Aspects can be re-ordered within a Sub Criterion or be moved from one Sub Criterion to another by
dragging the aspect to its new location. To drag an aspect, click near the symbol shown in the
screenshot below.
While still holding down the mouse button, you can drag the aspect to its new location within the Sub
Criterion. In the screenshot below, aspect A1O2 is being moved to the location after A1O5. Once the
mouse button is released the aspects will be re-ordered as well as being re-numbered to reflect the new
order.
In the same way, aspects can be moved from one Sub Criterion to another. Drag the aspect to
anywhere on the target Sub Criterion while holding down the mouse button. When the button is
released the aspect will be added as the last aspect in the Sub Criterion and re-numbered
appropriately. In the screenshot below aspect A1O5 is being dragged to Sub Criterion A2, where it will
become aspect A2O4.
Remember when moving aspects between Sub Criteria that the total marks for each Sub Criterion may
need to be adjusted.
CIS NG has the ability to perform various calculations on the values entered for an aspect to produce a
mark. For example, an aspect may measure the time taken to complete a task and award different
marks based on that time. At the point of entering data for each Competitor for this aspect, only the
value for the time taken would be entered and CIS NG would calculate the correct mark for the
Competitor.
Mark Deduction Type – In each value range, whether to deduct a number of marks or a
percentage of marks
Range 1 .. 10 – A number of ranges that define how many marks (or what percentage of
marks) are deducted for values that fall within the range
It is also worth noting that if a range defines a mark deduction that is greater than the
maximum mark for the aspect, a mark of zero will be awarded. It will not award negative
marks.
Selecting “Percent Score Comparison” from the marking type select box pops up the window
shown in the screenshot above. This allows you to determine whether the best value is the
highest value (for example furthest distance travelled) or the lowest value (for example the
fastest time).
Let’s demonstrate this with an example. An aspect has been defined where the maximum
mark (5 marks) is awarded to the Competitor with the fastest time and zero marks are awarded
to the Competitor with the slowest time. To set up the calculation, the “Percent Score
Comparison” option is selected from the marking type drop down. In the parameters window
that then pops up, the option for “Highest Mark to” should be set to “Lowest Value”.
During the competition the fastest Competitor takes 10 seconds to complete the task, and the
slowest Competitor takes 26 seconds, giving us a time difference of 16 seconds. After the
times for all Competitors have been entered in to CIS NG the marks will be calculated as
follows.
Highest Mark to – whether the maximum mark for the aspect goes to the highest value or the
lowest
Marks Deducted per Difference – how many marks are deducted per step
In the screenshot above maximum marks are awarded to the fastest time, so the highest mark
is awarded to the lowest value. From the next three parameters it is determined that for every
5 seconds slower than the fastest time, the Competitor will lose one mark. Since the
deductions have not been marked as proportional, the marks are calculated in a stepped
fashion. So times 0 – 4 seconds slower than the fastest lose no marks, 5 – 9 seconds lose one
mark, and so on. If the maximum mark for the aspect is 4 there might be marks such as:
To import data from a file, click the “Import Data” menu item in the “System” section. There are a
number of types of data that can be imported.
CIS NG will import .csv, .xls and .xlsx files containing the marking scheme defined using the
Excel template.
Select “Marking Scheme” from the select box on the “Import Data” screen to produce the form
shown in the above screenshot.
Select Skill – The skill to import the marking scheme to. If the current login only has access to
one skill, the drop down box will be replaced with the name of the skill.
Click the “Import” button to import the file. Any existing marking scheme for the skill will be
erased during import and replaced with the marking scheme in the file.
Please note that some users have encountered an error when importing a spreadsheet that is
currently open in Excel, so it is recommended that you close the spreadsheet in Excel before
importing the marking scheme.
Select “People” from the select box on the “Import Data” screen to produce the form shown in
the above screenshot.
CSV Separator – If importing a CSV file, enter the field separator character.
CIS NG will import files in the format used by the old CIS 2.x software or the new format. The
first row of a file in the new format is a row of headers that indicate to CIS NG what the content
of the column is. All headers are optional unless otherwise stated. The headers are:
title – The title of the person, for example Mr, Ms, Dr, etc
member – The abbreviation of the member that the person belongs to, for example “AE”
position – This is the name of a position defined on the “Position” screen (see section 4.7).
skill number – The skill number of the skill that the person holds the position in.
username – If the user is to have a login you can supply their username.
password – If the user is to have a login you can supply their password.
usergroup – If the user is to have a login you can supply the name of the usergroup to which
they will belong.
Select “Members” from the select box on the “Import Data” screen to produce the form shown
in the above screenshot.
CSV Separator – If importing a CSV file, enter the field separator character.
abbreviation – The abbreviation of the member, for example their two digit ISO code.
Each Sub Criterion needs to provide information about which day it should have marking completed.
This can be done in the “Assessment Criteria” part of CIS NG (as shown in section 6.2) or it can be
done on the “Marking Days” screen as shown below.
When the “Per Competitor” option is selected from the select box the following window will be
displayed.
Before any marks can be entered for a skill it must be locked. Once all Competitors and Experts have
been linked to the skill and the marking scheme is complete, go to the “Skills & Skill Rules” page (see
section 5.1) and check the “Prevent further editing of marking scheme” checkbox. Click the “Save”
button. The marking scheme for the skill can no longer be changed.
To check that a skill is ready for mark entry it is easiest to view a report that gives us an overview of the
status of all skills’ marking schemes. To view the report, click on the “Reports” menu item located in the
“Competition” menu section. Then select the “Marking Scheme Defined” report.
This report shows for each skill whether the marking scheme has been locked, information about the
marking scheme such as the number of criteria, sub criteria and aspects it has, as well as information
about the marking days for the sub criteria. Skills that do not have a total of 100 marks defined appear
in a row with a red background.
In the above screenshot the Snowman Building skill has had its marking scheme locked as indicated by
the lock icon next to its name. The report also shows that all of the marking days have been defined by
the presence of the word “Complete” in the “Marking Day” column. This skill is ready to have marks
entered.
The Ice Sculpting skill in the screenshot also has a total of 100 marks defined. However, the marking
scheme has not been locked and sub criteria E4 and G1 don’t have marking days selected. This skill is
not ready for mark entry.
CIS NG can generate a number of different PDF forms related to marking, such as forms for Experts to
record marks on while marking, forms with completed marks as recorded in the CIS, and a number of
signoff forms that are used to record Experts’ approval of marking schemes and mark entry.
The generated PDF will contain one of each type of selected marking form containing no
marking scheme information. The three types of marking forms are:
Mark Summary Form – This gives an overview of the marks achieved for each criterion and
sub criterion and indicates which day the tasks were assessed.
This option creates a single complete set of forms filled with criteria, sub criteria and aspect
information. Competitor information and marks are not included in these forms.
If you have access to more than one skill you will see a select box for choosing the skill to
generate forms for. You can also choose whether to generate forms that include all sub criteria
or just a single sub criterion. The forms can also be filtered to be generated for a single day of
the competition.
This set of forms is for Experts to use on the workshop floor when recording marks. These
forms contain Competitor information at the top of each form.
The default options on the form will generate a document that contains marking sheets for
every Competitor in the skill. The form can be filtered to generate for a single sub criterion,
competitor, day or combination of all three.
The “Collate Marking Forms By” drop down box allows the user to choose how the forms are
grouped within the generated PDF document. The forms can be grouped by Sub Criterion,
Competitor or by Day.So in the screenshot above, the PDF document would contain forms for
The “Hide Requirements data from Objective Marking Forms” checkbox will cause the
requirements column in the generated Objective Marking Forms to be greyed out. This option
exists because the requirements field may contain information that should not be revealed to
the Competitor, so it needs to be removed from the copy given to the Competitors at the start
of the Competition.
Some skills may require a marking form that allows them to mark multiple competitors for the
same aspect at the same time. This form lists the aspects down the left of the page, has space
for multiple competitors across the page and is in landscape format (as opposed to portrait).
These forms are harder to use when entering the data in to the CIS and should only be used if
absolutely necessary.
This option creates a set of forms that contains all the criteria, sub criteria and aspect
information, as well as mark information. It will create a set of forms for each Competitor.
If you have access to more than one skill you will see a select box for choosing the skill to
generate forms for. You can also choose whether to generate forms that include all sub criteria
or just a single sub criterion. You can choose to generate forms for all Competitors or a single
selected Competitor. You can also choose to generate forms for a particular day.
If the marks have not been locked (see section 8.4), a “Results Incomplete” watermark is
added to all pages to indicate that the marks have not been finalised.
This form is a PDF document that lists each Expert for the skill and provides a space for them
leave a signature to indicate their approval of the marking scheme. This form can only be
generated once the marking scheme has been locked.
At the top of the form is the timestamp of when the marking scheme was locked. This
timestamp also appears on each of the marking forms outlined above. This allows us to check
that the aspects on the marking forms being used to mark the Competitors’ work are exactly
the same as the ones signed off by the Experts. If the timestamps do not match, then the
marking scheme has been altered in some way since the Experts signed the form.
This form is a PDF document that lists each Expert for the skill and provides a space for them
leave a signature to indicate their approval that marks for their compatriot Competitor have
been entered in to the CIS correctly.
The user can select which day they would like to generate the form for. Days only appear in
the drop down once marks have been locked for that day.
At the top of the form is the timestamp of when the mark entry was locked for that day. This
timestamp also appears on each of the marking forms outlined above. This allows us to check
that the marks on the marking forms being are exactly the same as the ones signed off by the
Experts. If the timestamps do not match, then the marks may have been altered in some way
since the Experts signed the form.
Multi-Skill competitions allow the combination of several skills form a multi-skill. Multi-Skills don’t have a
marking scheme and do not have marks entered for them. Instead they combine the results from each
skill for a “Multi-Skill Team” to arrive at a mark.
For example, in the EuroSkills competition there is a multi-skill named “Cross Media Publishing” that is
comprised of the Web Designer, Graphic Designer and Print Technician skills. To determine the mark
for the Belgian multi-skill team, the CIS takes the marks for the Belgian Competitor in Web Design, the
Competitor in Graphic Design and the Competitor for the Print Technician skill, weights them
accordingly and arrives at a mark out of 100 for the Belgian team.
Once multi-skills have been created, they will appear in various reports and results pages along with
ordinary skills.
The multi-skill functionality must activated on the Config screen before any multi-skills can be set up.
Once activated the Multi-Skill menu section will be available. See section 4.4.7 for more detail on the
Config screen
The “Multi-Skills” link in the menu leads to a screen that allows the creation and editing of multi-skills. In
the screenshot below a multi-skill called “Frozen Water Construction” has been created.
The first drop down field at the top of the page contains a list of the existing multi-skills. To edit a multi-
skill select it from this drop down.
The multi-skill has been assigned a skill number and given a name in English. The “Exclude from
member results comparison” checkbox will exclude that particular multi-skill from the member results
(see section 10.8). To exclude all multi-skills from the member results, simply leave the relevant
checkboxes on the Config screen unchecked. See section 4.4.8 and 4.4.9.
The next section of the screen is for adding ordinary skills to the multi-skill. All of the ordinary skills
appear in the drop down box. The “Proportion” field indicates what proportion of the multi-skill marks are
taken from that skill. Any whole number can be entered in this field. When calculating the multi-skill
mark for a team, the amount from each skill is calculated by taking the competitor result for the skill and
multiplying it by the proportion divided by the total of all the proportion values. So using the example in
the screenshot above, the Snowman Building skill has a proportion of 2 and the total of all the
proportion values is 3 (2 for Snowman Building plus 1 for Ice Sculpting), meaning that two thirds of the
marks for the multi-skill are taken from Snowman Building. Since any values can be used for the
proportion, it would also be valid to use values like 66 for the Snowman Building and 33 for Ice
Sculpting, approximating the percentage values. The result would be the same.
To remove a skill from the multi-skill, simply click the red minus button to the right of the proportion field.
The buttons at the bottom of the screen are for creating a new multi-skill, saving the current multi-skill
and deleting the current multi-skill.
The “Multi-Skill Teams” link in the menu takes the user to a page that allows them to select the
competitors for each skill to assemble in to a multi-skill team.
The multi-skill that teams are to be made for must be selected before the rest of the form will appear.
The “Multi-Skill Team” drop down box contains a list of existing teams for the multi-skill. Select one to
edit it.
A name for the team can be specified. If the name field is left blank, the name of the Member is used
instead.
Each multi-skill team must have a Member. Often this is as straight forward as choosing one of the
regular Members from the Member drop down, but occasionally teams need to be created that contain
Competitors from different Member organisations. In this case a “pseudo-member” is required. A
pseudo-member is a type of Member that only exists in the context of multi-skills and is used solely for
grouping together competitors from different Members. Once a pseudo-member has been created it will
appear in the drop down lists for multi-skill screens, so it can be re-used for other teams if desired. For
example, in the EuroSkills competition they often use a pseudo-member named “Europe (EUR)” when
creating teams that have Competitors from different Member organisations.
When multi-skills are included in the member results (see section 10.8), the points and medal points for
pseudo-members are divided among the members of the teams.
The “Assign Team Members” section is for selecting the Competitors for each of the skills to be
included in the multi-skill team. The drop down box for each skill contains a list of Competitors that
haven’t been assigned to any multi-skill teams for this skill.
The “New” button creates a new empty team, the “Save” button saves the current team, and the
“Delete” button deletes the current multi-skill team.
When viewing the marks for a multi-skill it will only display the total marks, not the marks for
the daily totals and not the marks per Criterion.
All the operations that you would perform on this screen on an ordinary skill (see section 9.2)
are available with multi-skills as well. The CIS NG software treats the multi-skill as if it were an
ordinary skill.
The “Manage Application” menu item in the “System” section of the menu takes you to a screen that
gives an overview of all the users logged in to CIS NG and some basic server information.
In the above screenshot the admin user and Expert One are logged in. The report shows that Expert
One is an Expert in the Snowman Building skill. If the user’s name is clicked on the user interface is
redirected to the “Edit Person” screen (see section 5.3) for that user. Either of the users can be logged
out by clicking the “Logout” link at the end of their row.
From this screen it also possible to send a message to a group of users. Select the users you wish to
send a message to, type the message in the text area and press the “Send” button. Each of the users
will receive the message in a red box in the top right of the screen, similar to how an error message
looks. The message will disappear after a short period of time, or the user can close the message
themselves.
At the bottom of the screen is the version of the CIS NG software and the version of the Java virtual
machine that the software is using.
Click on the “Enter Marks” menu item in the “Assessment” section of the menu to start entering marks.
The first box indicates how many aspects have been marked in the current sub criterion for the
selected Competitor. The second box tells us how many marks have been entered for the
Competitor overall. The third box represents the amount of marks entered as a percentage.
The final box gives us the Competitor’s name. If there has been an image uploaded for the
Competitor, it will appear here above their name.
If the user has access to more than one skill they will see the select box as in the screenshot
above to allow them to select the skill to enter marks for. Otherwise the name of the skill the
user has permission to enter marks for will appear there.
The “Assessment Sub Criterion” select box will only list sub criteria available for the selected
Competitor on the selected day. This information comes from the Marking Day information
entered for each sub criterion (see section 6.8). Changing the selected day or Competitor may
result in a completely different list of sub criteria in the select box. Selecting a sub criterion will
update the objective marking form and subjective marking form with the aspects contained
within the sub criterion.
Either side of the “Competitor” and “Assessment Sub Criterion” select boxes are arrow
buttons. These can be used to scroll to the next or previous item in the list.
This form is for entering values and marks for objective aspects. Value fields may be left blank
unless they are calculated marks (see below). The arrow keys or tab key can be used to move
between fields. While marks can be entered by simply typing in the digits, there are shortcuts if
the Competitor received full marks or zero marks. If the Competitor received full marks for the
aspect, simply type “y” in the mark field. CIS NG will convert it to the maximum mark for the
aspect. If the Competitor received zero, an “n” can be entered in the field and it will be
converted to zero. This allows marks to be entered quickly by simply typing “y” or “n” and using
the down arrow key.
In the screenshot above the first aspect has a triangle icon before the description. This
indicates that the description can be expanded to see extra description lines. Clicking on the
triangle expands the description.
This form is for entering scores for subjective aspects. CIS NG calculates the final mark based
on the subjective judging rules for the skill.
In the above screenshot is a skill that has a subjective marking rule set that requires five
judges, so there is space for five scores for each aspect. Once all five scores for the aspect
have been entered and saved, the clock icon in the “Mark Awarded” column will be replaced
with the resulting mark. The mark is calculated according to the subjective marking rules
selected for the skill (see section 5.1.3). So for our example competition, the highest score and
lowest score will be ignored. The average of the remaining three scores will be used to
determine what percentage of the maximum mark for the aspect is awarded to the Competitor.
This skill has also been set to identify judges when entering subjective scores (see section
5.1.4) so the member abbreviation of the Expert who gave the score is shown next to each
score entered. In the screenshot the Austrian Expert gave a score of 5 and the Japanese
Expert gave a score of 8. How is the Expert set? At the top of each column there is an icon
that resembles a person. If this icon is clicked a drop down list of all Experts for the skill
appears. When an Expert is clicked their member abbreviation appears next to the icon. In the
screenshot above the third judge has been set to the Mexican Expert (MX). Any new scores
now entered in the third column will be attributed to the Mexican Expert. Note that this only
applies for entering new scores. For example, if the judge for the first column is changed to the
Indian Expert, the first score for aspect A1.S1 will still be attributed to the Austrian Expert. If a
score is now entered in the first column for A1.S2 it will be attributed to the Indian Expert. See
the screenshot below.
If a Competitor has not attempted a subjective aspect, they can be given a mark of zero by
checking the “Non Attempt” checkbox that appears to the left of the score fields as shown in
the screenshot below.
There are two reports that are useful for tracking mark entry for each skill. They can be accessed from
the “Reports” screen.
The screenshot above shows the marks entered for the two skills in the example competition.
The numbers are arrived at by adding the number of mark entries for each Competitor. For
example, there are 5 aspects contained in criterion H of the Snowman Building skill and there
are 5 Competitors, giving us a total number of expected marks of 25.
Cells that are highlighted in yellow indicate that the marks for that criterion are incomplete. In
the screenshot above there haven’t been many marks entered so all cells are highlighted.
The last cell in the row indicates what percentage of marking has been done for the skill in
total.
Clicking on the contents of a cell pops up a window that breaks down the number of marks
entered for each aspect of the criterion. In the screenshot below there have only been marks
entered for 4 of the 5 Competitors in aspect A2.O1.
In the above screenshot Competitor Five is missing a mark from Day 1. Clicking on the
contents of the cell for Competitor Five for Day 1 will pop up a window showing all the aspects
the Competitor was supposed to receive marks for.
This shows that Competitor Five is missing the mark for aspect A1.O3.
Once all marks have been entered for a day they should be locked to prevent further mark changes. On
the “Enter Marks” screen all objective and subjective mark fields for locked days will be disabled, so
marks can be viewed but not changed.
To get to this screen click the “Lock Marks” menu item in the “Assessment” menu section.
Days that have had all marks entered and are ready to be locked appear with a green background. To
lock the marking for that day, click on the word “Unlocked” in that cell. The word will change to “Locked”
and the cell background will change to grey. If there are no marks to be entered for a day then the cell
will be blank.
By default the Lock Marks screen will show the locks for the first skill in the drop down list. There is an
option in the drop down to show the lock status of all skills at once, but this may take some time to
generate if there are many skills.
Once all days have been locked the 500 scale results will be generated (see section 9.2).
Marks on the 100 scale for each Competitor can be seen on the “View Marks” screen located in the
“Assessment” section of the menu.
Next to the skill select box is a second select box that provides several options for viewing the marks:
Total Marks Only – Shows just the total for each Competitor
Include Daily Totals – Shows the total mark as well as the totals for each day
Include Marks For Each Criterion – Shows the total mark as well as the total marks for each criterion
The “View Marks” screen can also be seen during the course of the competition to show marks to that
point in time.
Once the marks for all days have been locked for a skill CIS NG will generate the 500 scale marks.
These can be viewed by clicking the “Results” menu item in the “Assessment” menu section.
CIS NG will automatically assign medals to the appropriate Competitors, as can be seen in the
screenshot above. These can be changed manually by selecting a different medal from the select box
and clicking the “Save” button.
Occasionally it may be necessary to adjust a Competitor’s mark, for example to apply a penalty. To do
this, click the green plus icon in the “Adjustments” column. The following window will pop up.
Percent Deduction – Deduct the specified percentage of marks from the 500 scale mark.
When the adjustment is saved, the marks are recalculated and medals re-assigned. The adjustment
appears in the “Adjustments” column. In the screenshot below a 15% penalty has been applied to the
Mexican Competitor.
CIS NG can generate a certificate for each Competitor that shows a summary of their marks, their final
500 scale score and their position in the skill. Clicking on the “Certificates” link in the menu presents the
following screen.
The form can be left blank to generate a certificate for every Competitor in the competition, or list of
certificates can be filtered by skill and by Member.
To export data to a file, click the “Export Data” menu item in the “System” section. There are a number
of types of data that can be exported.
This report provides an analysis of the subjective marking of each Expert for a skill. It shows the
number of assessments performed by each Expert, the number of times they gave the highest and
lowest scores, and the number of times they were above or below the average score. Clicking on the
name of an Expert will display the “Marks by Single Expert” report for that Expert.
This report provides an analysis of subjective marking performed by a single Expert. It shows the
number of assessments the Expert performed for each Competitor, the number of times they gave the
Competitor the highest or lowest score, and the number of times they gave scores above or below the
average score.
This report is another that provides analysis of subjective marking. There are four different
measurements that can be viewed: highest score, lowest score, above average score, below average
score.
This report is for comparing statistics such as highest mark, average mark, etc between skills. Any
number of skills can be added to the report.
This report shows the number of objective and subjective aspects for each skill. It can display the
overall number for the skill or break it down to show per criterion.
This report is used to assess whether the skill had an appropriate level of difficulty and produced a
sufficient spread of marks. Each Competitor’s mark is broken down by criterion and displayed as a
chart, as well as having various statistics like average mark and median mark for each criterion.
This report determines the top Competitor across all skills by comparing marks in the 500 scale.
This report screen is used to generate PDF documents that rank the various members. There are five
different comparison reports: Member (ordered alphabetically), Total Medal Points, Average Medal
Points, Total Point Score and Average Point Score. Each report shows the number of Competitors for
each member and the total number of each medal type the member gained.
This report screen is used to generate a PDF document that lists the Competitors for each member and
what their 500 scale mark was and what, if any, medal the Competitor won.
This report screen is used to generate a zip file containing all of the official results PDF documents. It
contains reports such as the Albert Vidal Award, the member comparison reports, the marking forms for
each skill grouped by member and skill, etc.
The “File Prefix” field supplies a prefix for each of the generated PDF file names. The field accepts
standard filename characters and has no restriction on length.
There is also a field for selecting a temporary folder to be used while generating the results. By default it
selects a path within the competition’s images folder. The folder used should be empty. Since the
generation of the official results can be a lengthy process, CIS NG assumes that if there are already
files in the folder then someone else is in the middle of generating official results using the same folder
and it won’t proceed. The user will receive an error notifying them that it cannot generate the results.
This report screen is used to generate an Excel spreadsheet that shows the percentage of marks
entered for each skill over time.