Mars Webcam Semseg 

Spec Document
2D Semseg | IN PROGRESS
Oct. 7, 2020 Update: Edge cases and clarification
March 30,2021 update : Image change and content redistribution
Contents

 Task summary
 Labels & Attributes Table
 Edge cases and clarifications
 Annotation rules
 Labels & Attributes Description/Examples
 Critical vs. Non-Critical Errors
 Edge Cases

Task summary
label people in the webcam images, making sure to
include accessories this people may have or anything directly in front
of them (occluding the person), otherwise leave unlabeled.
Labels & Attributes Table
Label

Unlabeled

Person

Edge cases and clarifications

Image Explanation

 Label anything in front of a person

as Person when that Person is the
background of that thing.

 Label any head accessories on the person

as Person

Example: headphones/earphones
(excluding wires), glasses, and hat

 DO NOT label anything the person is

holding as Person unless it is directly in
 front of the person

DO NOT label the wires if no human

background in the back

Please keep estimates of the remainder of the

person's body.

Annotation Rules
Rule Description Example

“Person” - Please annotate any visible part of the subject as “Person” Here is an example of what
vs. - Additionally, if the subject is wearing should be labeled as
“Unlabeled any headsets/headphones, glasses, microphones, accessori “Person” and what should
” es or jewelry - please include them in the “Person” be left unlabeled
annotation
- Gaps or spaces between the subject and the objects they
are wearing (such as a headset seen in the image to the **notice the microphone
right) should be left unlabeled included in the headset
- If the subject is attached to or holding an object, only label should also be labeled as
the parts of the object that occlude the person. Do not person
label other parts of the object. See example here.
“Person” - Accessories or objects that occlude (overlap) with the
vs. subject should be labeled as “Person”
“Unlabeled - If an object is partially occluding the subject, only the
” cont. part that is in front of the subject should be labeled as
(For objects “Person”
occluding
the subject)

Additional If there are other people in a task, please annotate as ‘Person’

Subjects only if they are approximately <4 ft. from the camera (ego)
AND looking at the camera.
If the subject is too far away or too unclear to accurately
annotate - they can be left unlabeled

Details Particular things to annotate as “Person”: Here is an example of how to

annotate accessories (headset,
 hands/fingers eyewear, etc.) as “Person”
 glasses/eyewear Correct annotation
 headphones/microphones example:
 accessories/jewelry Notice how we include the
 Parts of an object that directly occlude microphone of the head et,
the person) but NOT the cable

You DO NOT need to label thin strands of hair in the

“Person”. See example to the right for reference. Incorrect annotation
example:

Here are examples of how

NOT to annotate thin
strands of hair:
CORRECT annotation
example:

INCORRECT annotatio
n example:

Pixel - Please make sure annotations are within a 2 pixel threshold Example: Below is an example
Threshold in relation to the subject! of a task that stays true to the
(+/- 2px) - Each annotation should be as accurate as possible to avoid +/- 2 pixel threshold
segmentation bleeding or misclassification errors CORRECT:
 INCORRECT:

Labels & Attributes Description/Examples

Labels/Attributes Description Example

Person - Any visible part of the subject’s

body, including head and torso
area(sometimes legs)
- Additionally, if the subject is
wearing any
headsets/headphones, glasses,
microphones, accessories or
jewelry - please include them in
the “Person” annotation
- Gaps or spaces between the
subject and the objects they are
wearing (such as a headset seen
in the image to the right) should
be left unlabeled

 If an object is attached
or held by the
subject, only label the parts
of the object that
occlude(overlap)the
person. Do not label other
parts of the object.

Unlabeled - Miscellaneous objects that

are not attached or held by the
subject
- Wires to headphones/headsets
- Gaps between
headset/headphones and subject
- People far away from camera (> 4
ft away)
- People whose face is outside of
image
- Furniture or other objects the
subject might be sitting
in/on (includes the chair a subject
might be sitting in)

Critical vs. Non-Critical Errors

Critical Errors Non-Critical Errors

- Make sure all parts of a subject is - Exclude wires from

labeled as “Person” headsets/headphones as “Person”
- Make sure you do not label gaps in (remember they should be unlabeled if
between headsets, and other head-wear as they are not occluding the subject!)
‘Person’ - Stray hairs are non-critical
- Make sure you do not label chairs,
couches, or other pieces of furniture as
“Person”
-Make sure you label the microphone
included in the headset even if they
don't occlude the person

Edge Cases
Edge Case How to Handle It Example

Overlapping textures/colors
If there are any overlapping - Notice how the color of subject’s
textures/colors between headset blends in with the monitors
“Person” and “unlabeled,” behind him. Exclude the monitors
please label as accurately as from the “Person” label.
possible.
In the example to the right, the
subject’s headset should be
labeled as “Person” while the
monitor behind him should be
left unlabeled.

Image Blur and Quality
If there are any images that have
poor quality or image blur please
annotate as accurately as
possible. Annotate as close to
the refined pixels as possible.
- Notice how the subject’s hand is
blurred and might give us difficulty
when trying to annotate. Try your
best to annotate as accurately as
possible in these situations,
and make sure you don't leave
them without annotation

Subject in the dark Annotate outline of subject

using your best judgement.