Rules of LND

1.
Introduction 2
2. The Rating Tool 2
3. Guidelines 4
3.1.Attached Rating Guidelines 4
3.2.Inline Guidelines 4
3.3.Inline Instructions 5
4. Map-Speci c Rating Tools 6
Using In Task Street Imagery 12
5. Polygon Drawing 14
6. Media Annotation 16
6.1.Audio Annotation 16
6.2.Video Playback Interface (video annotation only) 18
6.3.Media Data Labeling 22
6.4.Image and Video Data Labeling 22
7. Token Labeling Tool 24
8. Side by Side 25
9. Media Recorder 27
9.1.Audio Recorder 28
9.2 Video Recorder 28
9.4.Allowing Access to your Microphone 30
10. How to take/upload screenshots 33
11. Release Survey 35
12. Troubleshooting - Browser Developer Tools 36
Page 1
fi
1. Introduction
Welcome to TryRating! This is the tool you'll use to make and submit ratings for every task you
work on.
Once you log into the tool, click “Get a Survey” from the Home screen or Survey from the
sidebar navigation area and tasks will automatically be assigned to you. You will not be able to
select or change them.
Note: The words “task,” “survey” and “project" are used interchangeably in this document.
When you nish working, log out by clicking the small circle containing your initials at the bottom
left of the tool. It’s OK to log out even if a new survey is displayed on your screen.
2. The Rating Tool

General Anatomy of the Tool
Name Screenshot Description

The topmost icon will expand or collapse
the sidebar.
The Home icon will take you to the Home

screen.
Sidebar The Survey icon will open a new task .

Navigation Note, you will get a message if you are
switching task types.
The Announcements screen will show

any active announcements.
Announcements will appear at the top of
the page when they are rst published.
Click on “Get a Survey” from the Home or
Survey screen to start working.
Get a Survey
Page 2
fi
fi
The task bar shows:
• Task Type: The name of the task.

• Task ID: A number that identi es the
speci c task item. Use this number
Task Bar
whenever you communicate about a
task or rating.
• Test Locale: The country and
language combination this task is for.
Rating The Rating Guidelines shortcut button

Guidelines opens the guidelines for this task.
Inline The Inline Guidelines button will appear

Guidelines when inline guidelines are available.
Allows you to skip a survey in rare cases

Release when a technical or other issue prevents
Survey you from rating. See Release Survey for
complete instructions.
Click Submit to save your ratings and

Submit
display the next available task.
Click the small circle containing your initials at

Reset
the bottom left to see the Reset Password
Password
option.
Click the small circle containing your initials at

Logout
the bottom left to see the Logout option.
Page 3
fi
fi
3. Guidelines
Before submitting ratings for a task, always review the project’s inline
instructions and/or guidelines. Guidelines provide essential rating information,
including descriptions of project work ows, explanations of any necessary
research steps, and de nitions of speci c rating criteria. Guidelines may also
include rating examples that illustrate rating scenarios you may encounter when
working on a project.
Guidelines are generally provided using one or more of three methods. These
will vary from project to project, so look for each of the following:
3.1. Attached Rating Guidelines

Guidelines for some projects may require in-depth documentation in the form of
a .pdf document. When .pdf guidelines are available, the blue “Rating
Guidelines” will be visible in the Rating Tool’s Task Bar. Access the document by
clicking on the “Rating Guidelines” button.
3.2. Inline Guidelines

Shorter sets of guidelines, or project-speci c supplements to attached .pdf
guidelines, may be found on the Inline Guidelines tab.
When Inline Guidelines are available, they will be open by default in the rating
tool the rst time that you open a new task type.
To open or close the inline guidelines, either click on the “Inline Guidelines”
button in the Task Bar, or toggle the carrot on the Inline Guidelines tab.
Page 4
fi
fi
fl
fi
fi
3.3. Inline Instructions
Projects may also provide instructions or supplemental guidance within the
project survey page. For example, this project has an attached set of .pdf
project guidelines, but also includes question-speci c task rules and instructions
within the rating interface.
You may also receive inline instructions or warnings when ratings or text eld
content must adhere to speci c conditions or follow a speci c format. For
example, the url in the text eld has been incorrectly entered. A warning
message provides an example of the expected input format.
Page 5
fi
fi
fi
fi
fi
4. Map-Speci c Rating Tools
This tool will move the map to the location

Go to Lat/Lng
of the coordinates you enter.
Viewport shortcut buttons provide

convenient controls to pan or zoom to
common map viewport settings. Center
Center User
User moves the result viewport to the
user’s location without changing the zoom
level (map scale).
Center Viewport moves the result

Center viewport to the center of the user viewport
Viewport without changing the zoom level (map
scale).
Use Show All to return from any zoom

Show All level to the initial display showing the
user, the viewport, and all the results.
Page 6
fi
User viewport: A transparent purple box
centered over a device icon shows the
user’s viewport at the time the query was
issued (if known). The user viewport
represents the map area the user was
looking at when making the query.
The shape of the box may vary depending

on the size and orientation of the device
User
being used.
Viewport
Note: In some cases, the user viewport
may be so small that it cannot be seen,
but (if the viewport information is
available) the device icon will always be
visible, regardless of the map scale.
Note: The viewports you see in the

examples in these guidelines may be grey.
This device icon shows the center of the
user viewport at the time of the user
query. The purpose of the device icon is to
make it easier to nd the user viewport. It
does not indicate the user's location.
This icon should not be used for

Device Icon
measuring distance. To measure
distance, start from the outer edge of the
user viewport.
This icon has no rating signi cance. Its

only purpose is to help you nd the
viewport on the map.
A yellow bust+star icon indicates the user

location at the time of the query (if known).
User
Location Note: Clicking on the icon reveals the
coordinates (latitude/longitude) of the
user’s location.
Page 7
fi
fi
fi
A pin indicates a result’s location. The

head of the pin is only an indicator. The
tip marks the actual location.
Each pin on the map has a unique color,

which matches the color of its
Result Pin
corresponding result heading.
Note: The pins you see in the examples

in these guidelines may look different
from the the pin shown in this chart and
the ones in the tool.
Clicking any individual result or pin causes

a pop-up box to appear over the pin. The
text within the box may show additional
information and should only be used for
research purposes to help understand the
result:
Pop-up
Information • Phone number (for research only,
Box never call a business)
• URL
• Coordinates (latitude/longitude)
Note: Do not rate the information in the

pop-up box. Rate only the information in
the result.
Allows you to select which map layer is
displayed. Available map layers may vary,
depending on the project that you are
working on. The most common layers are:
• Standard - depicts streets and

Map Layer highways, towns and cities, business
Selector names, green spaces, and bodies of
water, etc.
• Hybrid - elements from the Standard
Map, including road/street names and
town/city names, are superimposed
onto a Satellite View.
• Satellite - An unlabelled,
photographic satellite view.
Page 8
Allows zooming in and out on the map.

Zoom
Change the zoom level by clicking the +
Controller
(zoom in) or – (zoom out) buttons.
Copy to
Click to copy information to a clipboard
Clipboard
The Compass will help to orient you with

respect to the four cardinal directions.
Compass
The map defaults to North. Click the
compass icon and move around to rotate
the map.
Select the Drop Location Pin icon and

click on the map to drop a pin. Once the
pin has been placed, a ag showing the
pin address and coordinates (latitude and
longitude) will appear.
Click on the copy icon shown to the

immediate right of the address of the
dropped pin to copy its address and
coordinates.
Select ‘Delete Pin’ from the pin ag or

Drop click on the trash bin icon shown on the
Location far right of the address of the dropped pin
Pin(s) to remove it from the Map.
Page 9
fl
fl
Select multiple pins:
Click on the pin icon next to “Drop” in the
tool ag and click on the Map again to
drop another pin.
The color of the pin will match the color of

the respective address listed in the tool
ag.
The maximum number of pins that can be

dropped is 10 pins.
Distance Tool shows the distance between
a Start and an End point. The tool displays
both imperial (feet) and metric units
(meters).
Select the Distance Tool icon and click on

two points on the Map. Once two points
have been selected, a dashed line will be
drawn between those points and the
measurement will appear at the tool ag
under the Distance Tool icon.
Select the Distance Tool icon again to

Distance toggle it off.
Measurement
Tool Draw a circle:
After the Start and End point are selected,
toggle on “Draw Circle”. A circle will be
drawn with the Start point as the center of
the circle and the line connecting the Start
and End point as the radius of the circle.
Toggle off “Draw Circle” to remove the

circle.
“Draw Circle” can only be toggled on after

the Start and End point have been
selected.
Page 10
fl
fl
fl
This tool is used to draw a polygon or a

polyline. More instructions can be found in
Polygon Drawing.
Draw tool is used to annotate features on

the map. This tool also provides additional
functionality such as:
1. Polygon tool allows you to create
custom shapes. You can modify the
polygon shape with the Edit tool.
Draw Tool
2. Polyline tool can be used to create a
line segment. You can modify the
number of vertices using the Edit tool.
3. Point tool is used to drop single points
on the map.
4. Selection mode will display all
features (polygons, polylines, and
points) as a list. Click on a feature’s
respective checkbox to select it.
5. Edit tool allows you to add/delete/
move vertices.
6. Copy features.
7. Delete all features.
Select the Look Around icon to open the Look

Around menu to fetch street view imagery for
your currently viewed map.
Look Around
Note: This icon will only be available for routes
with fetchable street view imagery and look
around enabled.
Page 11
Using In Task Street Imagery
In tasks with Lookaround enabled, for areas where street imagery is available directly in
the tool, the Look Around icon will appear in the map toolbar. For these areas, the map
will typically load with the current viewport’s imagery locations pre-fetched and the Look
Around tool already open.
The Look
Around
toolbar has
four tools:
Tool Name Icon Description

Hide Hides all fetched clickable panoramic street imagery
Collection points.
Points
Set Zoom Sets zoom to level necessary to fetch panoramic street
imagery.
Delete All Removes all fetched street imagery points

& Reset
Fetch Gets clickable panoramic street imagery points for the

current map viewport.
The panoramic street

image has three tools:
the zoom in, zoom out,
and full screen image
buttons.
To view a panoramic street view

image, click on one of the fetched
location dots to open the panoramic
image. The image to the right shows
the task map with clickable, purple
street imagery dots. Note: Street
imagery dots will vary in color. Do
not use street imagery dot color to
Page 12
in uence your rating.
These panoramic street images can be clicked on and dragged to rotate to the view.
It’s important to get your bearings and makes sure you understand the viewing
orientation of the panoramic image. Compare against the road geometry visible in the
map and use context clues in the image to make sure you have a good understanding
of the road network.
Page 13
fl
5. Polygon Drawing
To draw a polygon on the map, follow the instructions below:
1. Make sure you are in satellite view

2. Select the polygon drawing tool at the top right of the screen
3. Click where you would like

to drop the rst pin. Always
drop the rst pin of your
polygon in the southwest corner of the feature, or the closest equivalent when the
feature is diagonally shaped
4. Continue to draw the polygon by dropping pins around the feature in a clockwise
direction (see images below)
5. Close the polygon by dropping the nal pin on top of the rst pin. The tool will
automatically connect the pins and form a polygon. Once the polygon has been closed and
completed, the line around it will become bolder and brighter
Page 14
fi
fi
fi
fi
In the event where you need to label your polygons, you should:
1. Click on the selection mode
2. Select the polygon
3. Choose the label you want to apply to the
polygon
4. Draw additional polygon(s) if applicable
5. Select the additional polygon(s) for
labelling
6. Choose the label you want to apply to the
additional polygon(s)
Note: To remove a completed polygon, click

on trash bin directly beside the polygon you
want to delete. Note that clicking on the trash bin at the upper right of the draw tool
deletes all polygons.
Page 15
6. Media Annotation
6.1. Audio Annotation

Audio annotation tasks involve highlighting sections of audio waveforms. Goals of this
task type commonly may involve, but are not limited to, identifying speci c noises or
speci c parts of a song. In some cases, the task will also require labeling annotations
with provided categories.
Audio Annotation Interface
Page 16
fi
fi
Displays a visualization of the audio le.
The slider will move as the audio
progresses but it can also be moved by
clicking on the desired part of the
Audio Track
waveform. Annotations also appear here
Waveform
as rectangles overlayed on the waveform.
• Selection annotations will have a
yellow highlight, whereas unselected
annotation are blue.
Similar to the Audio Track Waveform, this
shows how much of the audio le has
progressed. This display will always
maintain the default zoom and can be
Waveform
used for more context. The yellow outline
Navigator
represents how much of the Audio Track
Waveform is visible. When the waveform
is zoomed in, the yellow outline will
appear smaller
Page 17
fi
fi
Information about how much time has
passed at the current point of the audio
waveform (represented by the vertical red
Time bar and how long the audio le is. This
can be helpful context when making frame
by frame adjustments. The time here is
displayed in milliseconds.
Control the zoom of the Audio Track
Zoom Slider Waveform. This can be used to help with
more nuanced annotation.
Click and drag on the desired portion of
Range the audio track to create new annotations.
Annotation • Range Annotations can be modi ed by
dragging either end of the annotation.
Double click on desired part of the audio
track to create a new point annotation.
Point • Point Annotations can be modi ed by
Annotation dragging the annotation itself or the
annotation number ag at the top of
each annotation.
Play/Pause Play or pauses the audio.
Each annotation will automatically create

a layer, or Track, when drawn. The oldest
( rst created) tracks will show at the top,
while the newest tracks will appear at the
Annotation
bottom. This area shows the beginning
Layers
(light blue bar) and end (red bar) of a track
in relation to the video.
• Deletes the track
6.2. Video Playback Interface (video annotation only)

The annotation interface will differ slightly for tasks that require video annotation. These
additional controls allow for video scrubbing and annotation on speci c moments of the
video.
Page 18
fi
fl
fi
fi
fi
fi
Play/Pause Plays or pauses the video. The arrows

and Frame can be used to rewind or advance the
Control video by 1 frame at a time.
This shows how much of the video has

Video progressed. The slider will move as the
Timeline video progresses but can also be dragged
to show a desired part of the video.
Information about which frame is currently

showing and how many total frames there
Video Frames
are. This can be helpful context when
making frame by frame adjustments.
Each polygon will automatically create a

layer, or Track, when drawn. The oldest
( rst created) tracks will show at the top,
while the newest tracks will appear at the
bottom. This area shows when each
polygon has a change, known as a Key
Key Frame Frame, in relation to the video. Every time
Layers the polygon is adjusted, a new key frame
is added.
• Adds a Key Frame (light blue bar)
• Adds an End Key Frame (red bar)
• Deletes the track
Key Frames are automatically created whenever an annotation is created or when an

action is taken on an annotation. For example, a Key Frame will be placed when an
annotated polygon is made smaller to represent an object moving away from the
camera. There are two types of Key Frames:
• Regular Key Frames mark the frame where an annotation is created or adjusted to
account for change of a subject in the video. Keyframes appear as light blue vertical
bars in a track’s keyframe timeline.
• Key Frames can be added automatically by modifying an annotation in the editor.
Creating, resizing, moving an annotation will cause a Key Frame to appear in the
Key Frame Layers Interface by itself.
• Key Frames can also be added manually by clicking on the Add Key Frame button
for each track in the Layers interface.
Page 19
fi
• When a Key Frame has been added, navigating to that frame will turn the Add
Key Frame button into a Remove Key Frame button instead.
• End Key Frames signify the frame where an annotation appears in the video for the
last time. End keyframes appear as light red vertical bars in a track’s keyframe
timeline.
• Annotations cannot disappear and reappear. A new annotation will have to be
created if, for example, a subject that earlier moved out of the video re-enters.
• End Key Frames are created by clicking on the Add End Key Frame button in the
Layers interface.
• End Key Frames can be removed by navigating to the relevant frame that the
End Key Frame is located on and clicking the Remove Key Frame button.
Alternating click the End Key Frame will convert the existing End Key Frame to a
regular Key Frame.
• The position of an End Key Frame can also be adjusted if the End Key Frame
button is used before the current location of the End Key Frame. However, End
Key Frames cannot be added to a frame in the video that takes place after the
current End Key Frame.
Annotation Interpolation (video annotation only)

To avoid the necessity for frame by frame modi cation of video annotations, areas in-
between Key Frames are interpolated. This means that the interface will estimate how
to smoothly transition from the initial Key Frame to the subsequent Key Frame if the
annotation was modi ed.
• For example, the goal is to annotate a car driving from left to right. The start of
the movement occurs at the 125th frame and the end of the movement occurs at
the 166th frame.
• A bounding box and its respective Key Frame were added at the 125th frame to
annotate the vehicle. On the 166th frame, the annotation was moved to the right
Page 20
fi
fi
to cover the vehicle before it makes a turn and a Key Frame was added
automatically because of the change to the annotation.
• Because the annotation was modi ed, the annotation itself is interpolated
between the 125th and 166th frame. Notice how the annotation is almost halfway
between the original and ending positions even though the 145th frame does not
have a Key Frame.
As seen in the above example, interpolation is not perfect. A small tweak on the 145th
frame would improve the interpolation as there is now another reference point.
Interpolation would then occur in-between the 125th frame and the 145th frame, as well
as between the 145th frame and 166th frame. Additional strategically placed Key
Frames would further improve the accuracy of the annotation. By only adding Key
Frames when necessary, it’s possible to create accurate annotations for moving objects
without frame by frame adjustment.
Page 21
fi
6.3. Media Data Labeling
Some tasks will require that annotations are labeled. This could be presented in a
dropdown list of provided categories, or even a free-form text eld to write custom
descriptions. Data labelling is supported for audio, image, and video annotation tasks.
Though these components are highly customizable and will likely be different project to
project, the general work ow should remain the same.
1. Create an annotation.
2. Select the annotation.
3. Use the labeling component to select or input a label.
4. If the labeling process was successful, that annotation should now be associated
with the label. The header of the annotation should display the selected label.
6.4. Image and Video Data Labeling

1. Use one of the shape tools and draw bounding polygon around the object (in this
example, the box tool is used to draw a bounding box around the cat)
2. Make sure the polygon you want to annotated is selected. The polygon should be
highlighted in green (as shown in the image above).
3. Select the proper label from the accompanying rating component (in this case a
dropdown) to assign a label for the annotation.
Page 22
fl
fi
4. After you select the label, the label will show up on the bounding polygon and in the
annotation list con rming that the annotation has been labeled.
5. Depending on the task, you may need to select more than one object in an image/
video. In this case, repeat steps 1-5, and the following objects will appear like this
below.
Note: For video labeling tasks, make sure to set key frames as seen in 6.3. Video
Playback Interface so that the annotation polygon tracks the object throughout the
video.
Page 23
fi
7. Token Labeling Tool
The Token Labeling tool (also known as the "Chunk
Tool") is an advanced rating tool that is used in
combination with other label selection options to
apply one or several labels to either a single token
or group of tokens (also referred to as a span or
span of tokens).
An unlabeled token selection tool that uses a
dropdown menu for applying labels
Selecting a Token
To select a token for labeling, click on the token to be

labeled. A gray border should appear. Move your
mouse to the bottom of the gray border and the option
to "Select Chunk" will appear. After clicking "Select
Chunk", the selected token will be ready to accept
labels from the task's other labeling tools. In this
example task shown, selecting an option from the
"Token Label Type" will apply it to the token. The same
drop down selection tool will be used to label all tokens. After initially clicking on a token, it should turn
blue and a gray border appears around it.
After labeling the selected token, simply repeat the
process for the next token to be labeled.
Labeling and Removing Labels
After selecting an option from the applicable selection tool, the label will appear beneath the
selected token. To remove the label, click the right-aligned blue X across from the label to be
deleted. Click the red X in a Circle to remove the selection and all applied labels. If a selection
was already present in the task tools used in combination with the Token Labeling tool and you
want to apply that answer to the token, simply change the selected answer and then change it
back to the correct answer. For example, if you want to label POI to the token "of" in "Museum
of Pop Culture" but Token Label Type already had POI selected when you hit "Select Chunk" for
of, you would rst need to change the Token Label Type eld away from POI by selecting any
other answer and then back.
Some tasks may ask you to answer multiple questions about a single token. Doing so will result
in multiple labels being applied to the token.
Page 24
fi
fi
A token labeled using two dropdown selection tools
Selecting Multiple Tokens At Once
Some tasks will have you label a group of tokens (or

"chunks") with a uni ed label rather than labeling
individual tokens. The token or chunk groups will
often be referred to as a span in tasks.
To select multiple tokens, rather than clicking

"Select Chunk" after selecting a single token, click
on the additional adjacent tokens rst. You can then
click "select chunk" and apply a label or labels as A multiple token span ready to be selected and
normal. have a label applied
Follow task speci c guidelines to determine when to

label tokens individuals or in groups.
8. Side by Side
Side by Side is a speci c type of task where two sides (A/B or Left/Right) are rated
against each other. There are usually questions for each single result/item in each side
as well as an overall judgment.
Understanding what quality aspects are taken into account to what extent is very
important. Please read the guidelines very carefully and follow along with the given
examples. Below is a detailed list of Side by Side components:
Page 25
fi
fi
fi
fi
The top of the page usually shows the

Query / Input
query or input to be rated.
Each side provides a set of results or

Results /
items. Often times there will be individual
Items
questions, like Relevance in our example.
Results or Items that are similar are

Similar marked “Same as <Side+Index>”. In
Results / many cases the rating automatically
Items applied for your convenience as well as
to ensure consistent ratings.
When a map is present, selecting the

header will move the map over the result.
Map
Where results are “same”, the pins are
Component
on top of each other. Selecting a visible
part of the pin will cause it to appear in
front.
Page 26
One of the most important parts of a Side

by Side task is the overall rating scale.
Overall
Rating Scale Take all aspects into consideration and
review the guidelines before providing a
rating.
Some tasks provide a summary table that

show all provided ratings.
Summary
Table This enables making an informed
decision with regards to the overall
rating.
9. Media Recorder
Use the Media Recorder to collect audio and video recordings directly from your
device. Please contact your manager if you would like to work on or be removed from
these projects. You will need to voluntarily opt in to work on Projects utilizing this tool.
Page 27
9.1. Audio Recorder
Certain tasks may ask you to upload audio recordings. The Media Recorder
component captures .mp4 audio les directly from your microphone. This type of data
may be used in the context of, but not limited to, reading out loud on-screen text for
the purposes of collecting information such as pronunciation and patterns of speech.
Audio Recorder Interface
1. The Media Recorder will display an audio waveform even when you are not
recording. Notice how the waveform uctuates depending on your volume.
2. When the you are ready, click the Record button. The waveform will turn green
color to indicate that the audio is being recorded. You can click the Pause button
to temporarily halt the recording or click Stop to end the recording.
3. Prior to upload, the component will allow you to playback your audio le. You can
then decide to Upload the recording or Clear the recording to restart the process.
4. After successfully uploading the recording, you will still have the ability to Delete
the recording and restart the process prior to submitting the task.
9.2 Video Recorder
Page 28
fi
fl
fi
The Media Recorder tool supports video recording as well, capturing .mp4 video les
directly from your webcam and microphone. The quality and size of the video will
depend on your hardware.
Video Recorder Interface
1. The Media Recorder will display a preview of the webcam capture as long as the
task is open. When the you are ready, click the Record button.
2. You can click the Pause button to temporarily halt the recording or click Stop to
end the recording.
Page 29
fi
3. Prior to upload, the component will allow you to playback your video. You can then
decide to Upload the recording or Clear the recording to restart the process.
4. After uploading the recording, you will still have the ability to Delete the recording
and restart the recording process prior to submitting the task.
4.4. Allowing Access to your Microphone

You will need to grant TryRating Access to use your computer’s camera and microphone
hardware. This is usually set at the browser-level. If permission is not granted, the tool will
display the following image and you will not be able to work on the task.
Page 30
Different browsers may require different methods enable recording. Please reference the
following instructions for each browser:
Chrome
1. A pop-up window should appear the rst time you open a task containing the Media
Recorder tool.
2. Click “OK” to let “tryrating.com" access your microphone or video.
3. If this window does not appear, the setting can also be directly changed through the
browser settings:
1. While on the task, nd the microphone or camera icon on the search bar.
2. Select the rst option to provide tryrating.com with access
Page 31
fi
fi
fi
Safari
1. A pop-up window should appear the rst time you open a task containing the Media
Recorder tool.
2. Click “Allow” to let “tryrating.com" access your microphone or video.
3. If this window does not appear, the setting can also be directly changed through the
browser settings.
1. Access Safari’s preferences.
2. Click on the “Websites” tab and select “Camera” or “Microphone”
3. Find “tryrating.com" and select either “Ask” or “Always”, depending on your
preferences
Page 32
fi
10. How to take/upload screenshots
Certain tasks may ask you to upload images of your screen. Keep in mind the following rules
when you are uploading images.
• Do not show information such as, but not limited to, your browser user account,
bookmarks, your taskbar, etc.
• Most operating systems will allow users to choose between capturing the entire
desktop, a speci c window, or a custom portion of your screen. Unless otherwise
speci ed, capturing a portion of your screen is preferred.
• Image must be saved as les and not copied on your clipboard.
• Acceptable le formats will vary per task, but generally JPEG and PNG les should be
acceptable.
The steps to take a screen capture of the your screen may depend on your operating system.
Instructions for Chromebooks, macOS, and Windows devices can be found below.
Different operating systems require different methods to take screenshots. Please reference the
following links to identify the best method for your device:
Chromebook - of cial resource
1. Use Shift-Control-switch windows ( )to enable screen capture
2. The mouse should turn into a crosshair pointer. Click and hold the mouse to create a frame
that will capture all contents within it as an image.
3. Release the click to save the picture. The image should be stored in your downloads folder.
A oating thumbnail will also appear so that you can preview the image.
macOS - of cial resource

1. Use Shift-Command-5 to open the built-in Screenshot tool.
2. Click on the “Capture Selected Portion” option, which is represented by a rectangle icon with
the dashed borders.
3. A frame should appear. Drag the edges of the frame to capture the desired portion of your
screen.
• Reminder: Your image should not contain any personal information that may be located
on the webpage or browser toolbar.
Page 33
fl
fi
fi
fi
fi
fi
fi
fi
4. Click “Capture”.
5. A oating thumbnail will appear, click on the thumbnail to preview the image.
6. Click “Done” to save the image.
• By default, the image will be saved to your desktop.
Alternatively, using Shift-Command-4 instead of Shift-Command-5 will directly allow you to

capture a selected portion of your screen as well.
1. Use Shift-Command-4 to directly open the built-in Screenshot tool.
3. Release the click to save the picture.
5. Click “Done” to save the image.
• By default, the image will be saved to your desktop.
Windows - of cial resource
Windows has a new built-in screenshot tool called Snip & Sketch. This is the recommended tool
to use as it is faster to open. Note that certain Windows updates may need to be installed to
access this tool.
1. Use Windows-Shift-S to open the Snip & Sketch tool.
2. Ensure that the Rectangular Snip option is selected. This should be the left most icon and is
represented by a rectangle icon with a plus symbol.
5. Click the save icon or use Control-S to save the image to the desired directory.
You may also use the Snipping Tool if your computer does not have the Snip & Sketch tool.
1. Open the Windows Start menu and look for the Snipping Tool.
• Alternatively, you can also access the Search menu and type “Snipping Tool” to nd the
tool quickly.
2. Click on the Mode icon and ensure that the Rectangular Snip is selected.
3. The rest of the screen should now have a semi-transparent overlay. If this is not the case,
click “New”.
Page 34
fl
fl
fl
fi
fi
5. A new Snipping Tool window will appear. Click on the save icon or use Control-S to save the
image to the desired directory.
11. Release Survey

You may occasionally be unable to rate a survey. To skip it, click Release Survey at the top
right of the tool.
A pop-up window with a list of reasons for release will appear. You must choose one before
you can release the survey.
Note: If you change your mind and decide to rate the survey, click Cancel at the bottom left of
the pop-up window.
Reasons to Release
1. Adult Content
You are not comfortable rating queries and/or results that include businesses offering adult
services or entertainment, like strip clubs.
2. Technical Issue
Technical issues that prevent rating include:
• Map or map features not loading.

• Query eld is empty.
• Tool is not interactive. You cannot zoom in and out or move the map at all.
• Cannot submit ratings.
• Pins are missing or do not stay in place.
• Receiving 5 consecutive surveys with all results in China, if China, or Hong Kong, are
not the markets you are rating.
If you choose this option, leave a comment describing the issue before releasing the survey.
3. Not Enough Time Allocated
Select this option if the Estimated Rating Time is not suf cient to complete the task.
Remember, the Estimated Rating Time is the average time over multiple tasks.
Page 35
fi
fi
4. Other
Issues not mentioned above that prevent you from rating.
If you choose this option, you must leave a comment before you can release the
survey.
Do Not Release
These issues will not prevent rating and are not reasons to release surveys:
• Poor satellite image, including clouds over target area or lack of image detail
• You can zoom in and out on map, but not as much as you'd prefer
• Rating is too dif cult
• Query does not appear to be related to maps
• Query or results are from outside your market
• Query or results in unexpected languages and scripts:
12. Troubleshooting - Browser Developer Tools

On the occasion that a project is blocked by technical issues, your manager(s) may ask you to
provide screenshots of your internet browser’s developer tools to help debug the issue for a
particular project.
Different browsers have different ways to access developer tools.
Chrome
1. While on the TryRating task, click on the Chrome menu (the 3 vertical dots) on the top right
of the browser window.
2. Click “More Tools” to open the expanded menu.
3. Click “Developer Tools” (also accessible by Option-Command-I on macOS or Shift-Control-
I on Windows).
Page 36
fi
4. By default, the developer tools will appear as a tab on the right side of the browser window.
Because the end goal here is to generate screenshots, we want to make the developer
tools as large as possible so we can capture more information. To do this, the settings
must rst be accessed by clicking the “Customize and control DevTools” (also represented
by 3 vertical dots)
5. Under the “Dock side” option, select the “Undock Into Separate Window” option.
Page 37
fi
6. In the developers window, click the “Console” tab (also accessible by Option-Command-J
on macOS or Shift-Control-J on Windows).
7. The console should ll up the whole window. Enlarge the window to cover your whole
desktop. Try to t as much of the console content and take a screenshot. Please feel free
to take more screenshots if you can’t ll all the content in one image.
• For instructions on how to take screenshots, refer to the How to take/upload
screenshots section.
8. Send your screenshots to your manager(s) so that they can escalate the issue.
Safari (on macOS)
1. First, access Safari’s preferences by clicking on “Safari” in the menu bar and selecting
“Settings” or Command-,
2. Navigate to the “Advanced” tab.
3. Make show the “Show Develop menu in menu bar” is checked. The “Develop” button
should now be visible in the menu bar.
Page 38
fi
fi
fi
4. Back to the browser window, check that the tab with the TryRating task is selected and
click the “Develop” menu button.
5. Click“Show Javascript Console” or by using the Option-Command-C hotkey.
6. By default, the developer tools will appear as a dock on the bottom of the browser window.
Because the end goal here is to generate screenshots, we want to make the developer
tools as large as possible so we can capture more information. To do this, click on the
“Detach into separate window” icon.
7. In the developers window, click the “Console” tab.
Page 39
8. The console should ll up the whole window. Enlarge the window to cover your whole
desktop. Try to t as much of the console content and take a screenshot. Please feel free
to take more screenshots if you can’t ll all the content in one image.
• For instructions on how to take screenshots, refer to the How to take/upload
screenshots section.
9. Send your screenshots to your manager(s) so that they can escalate the issue.
Page 40
fi
fi
fi

Rules of LND

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Rules of LND

Uploaded by

Copyright:

Available Formats

1.

2. The Rating Tool

Name Screenshot Description

The Home icon will take you to the Home

Sidebar The Survey icon will open a new task .

The Announcements screen will show

• Task Type: The name of the task.

Rating The Rating Guidelines shortcut button

Inline The Inline Guidelines button will appear

Allows you to skip a survey in rare cases

Click Submit to save your ratings and

Click the small circle containing your initials at

Click the small circle containing your initials at

3.1. Attached Rating Guidelines

3.2. Inline Guidelines

This tool will move the map to the location

Viewport shortcut buttons provide

Center Viewport moves the result

Use Show All to return from any zoom

The shape of the box may vary depending

Note: The viewports you see in the

This icon should not be used for

This icon has no rating signi cance. Its

A yellow bust+star icon indicates the user

A pin indicates a result’s location. The

Each pin on the map has a unique color,

Note: The pins you see in the examples

Clicking any individual result or pin causes

Note: Do not rate the information in the

• Standard - depicts streets and

Allows zooming in and out on the map.

The Compass will help to orient you with

Select the Drop Location Pin icon and

Click on the copy icon shown to the

Select ‘Delete Pin’ from the pin ag or

The color of the pin will match the color of

The maximum number of pins that can be

Select the Distance Tool icon and click on

Select the Distance Tool icon again to

Toggle off “Draw Circle” to remove the

“Draw Circle” can only be toggled on after

This tool is used to draw a polygon or a

Draw tool is used to annotate features on

Select the Look Around icon to open the Look

Tool Name Icon Description

Delete All Removes all fetched street imagery points

Fetch Gets clickable panoramic street imagery points for the

The panoramic street

To view a panoramic street view

To draw a polygon on the map, follow the instructions below:

1. Make sure you are in satellite view

3. Click where you would like

Note: To remove a completed polygon, click

6.1. Audio Annotation

Audio Annotation Interface

Play/Pause Play or pauses the audio.

Each annotation will automatically create

• Deletes the track

6.2. Video Playback Interface (video annotation only)

Play/Pause Plays or pauses the video. The arrows

This shows how much of the video has

Information about which frame is currently

Each polygon will automatically create a