Bee Lidar Update

4/28/2021 https://docs.google.com/document/d/1cGZTK_BH8-UJWSaJstgMFg_9_TLMMDPrMcxiqnOTGPc/pub?
embedded=true#…
Return to top
Bee Lidar - Cuboid

Annotation Instructions Version 6
Task Overview
The objective of Bee Lidar - Cuboid Annotation is to locate the position of objects and place cuboid
labels on top of them in a 3D lidar annotation scene for 320 frames. Objects can be either moving
(dynamic) or not moving (static). Objects to label include vehicles, pedestrians, static road objects,
and non-physical objects called Ghosts. Just as important as selecting the right label for each object
is, so is selecting attributes which best describe aspects of the objects. Attributes can be either used
to describe objects in each specific frame or on all 320 frames.
LABEL TABLE
Type Class Type Class Type Class

Pedestrian Adult Vehicle Other Other Dynamic Other Dynamic
(Dynamic) (Dynamic or (Motorized) Object Object
Stationary) Riding (Dynamic)
Child Car Construction Construction Cones

Object and Poles
(Stationary)
Traffic Van Construction Zone
Controller Signs and
Construction Sign
Boards
Truck
Specialty Tow Truck Temporary

Construction Barrier
Pedestrian Adult Bus Temporary Fence

with Object
(Dynamic) Child Bendy Bus Other
https://docs.google.com/document/d/1cGZTK_BH8-UJWSaJstgMFg_9_TLMMDPrMcxiqnOTGPc/pub?embedded=true#h.2vhm067… 1/72
4/28/2021 https://docs.google.com/document/d/1cGZTK_BH8-UJWSaJstgMFg_9_TLMMDPrMcxiqnOTGPc/pub?embedded=true#…
Specialty School Bus Static Road Pushable or

Obstacle Pullable
(Stationary)
Animal Dogs/Cats Construction Other
(Dynamic)
Other Mobile Traffic No Label Zone No Label Zone
Control (Stationary)
Vehicle Trailer Other Ghost Reflection

(Dynamic or (Dynamic)
Stationary) Wheelchair Emergency Police Vehicle Sensor Artifact
(Dynamic or
Other Personal Stationary) Police Retroreflector
Mobility Motorcycle
Bicycle Other Dust/Spray/Smoke
Motorcycle Railway Railway Vehicle

Vehicle
(Dynamic or
Stationary)
ATTRIBUTES TABLE
Towing Rider Attribute Occlusion Turn Signal Lights Intent to Cross

Attribute (Adjusted once) (Adjusted each (Adjusted each (Adjusted once)
(Adjusted frame) frame)
once)
Any Lights Brake Lights Reverse Lights Parked
(Adjusted (Adjusted each (Adjusted once) (Adjusted once)
once) frame)
GLOBAL RULES & GUIDELINES
Minimum LiDAR Points & 0-point

rule:
Label all objects that have 1 LiDAR point

anywhere in the scene.
Any object that has 1 LiDAR point at any

time in the scene needs to be annotated
the entire time they are in camera view,
even if they have 0 points. Use
interpolation, camera view, and your best
judgment on correct cuboid position
Occlusion
If a cuboid becomes fully occluded

during a frame and becomes visible
again, annotate the object through the
occlusion. Use your best judgment on
correct cuboid position and heading
If you cannot annotate the object due to

high uncertainty (for example very long
occlusions), but can re-identify it later,
start a new cuboid path with the next
frame where the object is visible. Be sure
to merge the two cuboid paths afterward,
using the Merge This Cuboid Path with
Another Path button
Echo Sizing and Positioning
INCLUDE echo inside of the cuboid.

Make the cuboid larger if necessary to
include echo. (Front and Back, NOT the
sides.)
If the echo is only visible for a few

frames align the cuboid with the points
closest to the ego vehicle.
Use Grid View and Dynamic APC to

check the sizing and positioning of
cuboids affected by echo
CUBOID SIZING
DO NOT leave more than .2m of space

between the cuboid and points on object
DO NOT INCLUDE side view mirrors or

antennas
INCLUDE echo inside of the cuboid.
INCLUDE large objects on the back or

on top of the vehicle, like luggage,
lumber, bicycles…
INCLUDE any open doors, trunks, and

hoods in the cuboid
INCLUDE any ramps & lifts on the back

of large trucks in the cuboid
150 Meter Rule
Annotate the entire path of any object

that was within 150m of the ego vehicle
at any point.
If there is an object that was never within

150m of the ego vehicle, do not
annotate it.
Towed objects
Annotate the towing vehicle and the

towed object with separate cuboids.
Annotate each cuboid with their
respective label.
It is fine for the cuboids to overlap.
2 Pedestrians with 1 Object:
If more than one person are carrying a

large object together, divide the object
(approximately) evenly between the
persons.
Example: If two people are carrying a

couch, create two cuboids, each of them
with a pedestrian and half of the couch.
Both cuboids get the label ‘Pedestrian
with object’
Labels Changing Mid-Scene

If an object needs to change labels mid-

scene, create a new cuboid in its path
with the new label, and new sizing, if
necessary.
Example:
If a person is holding a large

object - label them Pedestrian with
Object - Adult.
If they throw that object, start two

new cuboids - one Pedestrian - Adult
and one Other Dynamic Object
Using Visual Changes to Determine

the Label for Attributes
In the example provided to the right, from

frames 1-39 the brake lights of this
vehicle appears to be OFF or Can’t Tell,
but in frame 40 we can see the brake
lights actually turn off (darken), therefore,
we should change the labels for frames
1-39 to ON from OFF given this new
information.
If there is clear proof that an attribute is

either On or Off after seeing a visible
change in later frames, we set the
attributes accordingly.
Only make these changes if you can
visually confirm the differences.
On/Off because of Clear Status change even if blurry
If Vision for an Attribute Changes

Throughout a Scene Change the
Labels Accordingly
In the example to the right we can see

that the brake lights for this van is ON in
certain frames due to the angle of the
camera, but in other frames it is difficult
to see the brake lights due to the camera
angle and be labeled as Can’t Tell. Once
the angle makes the brake lights difficult
to see and you cannot say with certainty
whether they are ON or OFF you should
label it as Can’t Tell.
Once an attribute becomes difficult to

confirm due to camera angle, glare,
blurriness, etc., you should consider
using the Can’t Tell label.
Signal Lights - Both
When selecting Signal Lights - Both,

check that both indicator lights are
switching On and Off between frames,
vs. remaining constantly on.
When Signal Lights remain constantly
on, DO NOT select Signal Lights -
Both, select Signal Lights - Off.
NO LABEL ZONE (NLZ) RULES & GUIDELINES
No Label Zone (NLZ) Basics
When creating a ‘No Label Zone’, create

one large cuboid to cover the entire area.
DO NOT label any other classes within

the ‘No Label Zone’ unless it is an object
that enters, exits or is waiting to exit a
NLZ (examples)
DO NOT cover gas stations, auto repair

shops (ie. oil changers), and fast food
drives with a NLZ; we should always
label all objects in these areas
individually
DO NOT annotate house driveways with

a NLZ
NLZ Sizing & Positioning
NLZs can generally be left at the default

height of 2m
NLZs should be as long/wide as

necessary to cover the area.
The NLZ MUST be level with the
ground surface (not floating or sinking)
DO NOT cover any vegetation or
sidewalks with NLZs.
Embedded (built-in) NLZ /

Geofence
Some tasks will contain built-in No

Label Zones.
DO label objects completely outside

built-in NLZs as usual.
DO label static (stationary) objects
partially in the built-in NLZs.
DO label the entire path of dynamic
objects that enter/exit built-in NLZs.
DO label NLZ areas as normal with
NLZ if uncovered by E-NLZ
DO NOT label static objects
completely inside the NLZs.
If an object enters a NLZ, annotate
the entire path of the object (including
when it is inside the NLZ)
If an object is waiting to exit a NLZ,
annotate the entire path of the object
(including when it is inside the NLZ)
larger image
larger image
larger image
Same Rule applies for Autobody shops
NLZ Parking Lots
The ‘No Label Zone’ should cover all

stationary vehicles, alleys within the
parking lot
If the parking lot has an abnormal shape,

use as many cuboids as necessary to
cover the entire area without going over
buildings, sidewalks, or any other area
that does not represent the zone being
annotated
Annotate all parking lots that the Ego

Vehicle isn’t entering, driving in, or
exiting, with a NLZ
If the Ego Vehicle is driving in, exiting, or

entering a parking lot, ALL objects in the
parking lot need to be annotated.
If there are any other parking lots in the

scene that aren’t connected to the
parking lot the Ego Vehicle is in,
annotate those parking lots with a NLZ
DO NOT annotate parking lots that do

not have any vehicles in them
DO NOT annotate parking lots that are

right next to the street (without driveway
or curb separating them) with a NLZ
DO NOT label vehicles parked along the

road with a NLZ
DO NOT cover cars in very small parking

areas outside or inside of Auto Repair
Shops garages or Gas Stations.
Auto Repair Shops with large parking

lots should be covered with NLZs
Objects near NLZs
Annotate any objects on sidewalks,
vegetation, or any other surface outside
of the parking lot NLZ area
Objects entering/exiting NLZs
If an object enters a NLZ, annotate the

entire path of the object (including when
it is inside the NLZ)
If an object exits a NLZ annotate the

entire path of the object
If an object is waiting to exit a NLZ,

annotate the entire path of the object
Grouped Objects
Use one or more ‘No Label Zone’

cuboids to annotate groups or crowds.
THERE MUST BE 5 or more objects, no

more than 10m radius apart, and further
than 10m from the drivable surface.
When an object exits from a ‘No Label

Zone’ group, then it needs to be
annotated in the affected frames.
Car Dealerships
Use No Label Zones to cover any parked

vehicles - including parked cars at
dealerships
Underpasses
If the Ego Vehicle is on an overpass

(road that passes over another road that
does not physically join with the
overpassing road), annotate vehicles
with a ‘No Label Zone’ on the lower
roads which pass under the Ego
Vehicle.
DO NOT use a NLZ for vehicles above

the Ego Vehicle
DO NOT use a NLZ for onramps or

offramps to the highway.
LABELS (Click on the label name to view more example images of the label)
Pedestrian - Adult: Human adult or older child (>8

years old) on foot.
INCLUDE arms and legs in the cuboid
INCLUDE any everyday items (backpacks,

cellphones…)
INCLUDES standing, sitting and lying humans.
Pedestrian - Child: Human child (≤8 years old) on

foot.
INCLUDE arms and legs in the cuboid
INCLUDE any everyday items (backpacks,

cellphones…)
INCLUDES standing, sitting and lying humans.
Pedestrian - Traffic Controller: Human Adult or older
child (> 8 years old) on foot, controlling traffic.
INCLUDES traffic police, construction workers

controlling traffic, crossing guard, or any other person
controlling traffic.
MUST INCLUDE signaling device, like batons, flags

and signs.
Pedestrian - Specialty: Police officer, construction
worker, firemen, ambulance personnel, etc.
DOES NOT INCLUDE traffic police, crossing guards or

any other human directing traffic
Pedestrian with Object - Adult: Human adult or older

child (>8 years old) on foot, holding a large object.
INCLUDES babies, large boxes, or open umbrellas;
pushing a bicycle, motorcycle, cart, stroller, knee
scooter, or trash can; pulling a suitcase.
MUST INCLUDE both the adult and the object.
DOES NOT INCLUDE adults carrying normal day-to-
day items (e.g. backpack, tote bag, purse), or pet’s
leash
Pedestrian with Object - Child: Human child on foot

(≤8 years old), holding a large object.

MUST INCLUDE both the child and the object.

DOES NOT INCLUDE children carrying normal day-to-
day items (e.g. backpack, tote bag, purse), or pet’s
leash
Pedestrian with Object - Specialty: Police officer,

construction worker, firemen, ambulance personnel
holding a large object.

MUST INCLUDE both the specialty person and the

object.
DOES NOT INCLUDE specialty people carrying

normal day-to-day items (e.g. backpack, tote bag,
purse), or pet’s leash
DOES NOT INCLUDE traffic police, crossing guards or

any other human directing traffic
Animal - Dogs/Cats: Small domestic animals such as

dogs and cats, on foot.
INCLUDES animals on a leash (do not include the leash)
DOES NOT INCLUDE dead animals (roadkill) or animals

being carried.
EXAMPLE: An adult walking a dog would yield two

cuboids, one “Animal - Dogs/Cats” and one “Pedestrian -
Adult”.
Animal - Other: Non-domestic animals of any size on foot.
INCLUDES larger “domesticated” animals such as sheep,

horses, cows and donkeys, birds on foot like ducks, deer,
squirrels, skunks and raccoons
If the animal has a rider (like a horse or donkey), the

cuboid MUST INCLUDE both animal and rider.
DOES NOT INCLUDE dead animals (roadkill) or flying

birds
Vehicle - Trailer: Any object designed to be towed by

another vehicle. Object cannot move under its own
power
INCLUDES boat on trailer, flatbed trailer, cement mixer,

caravan, shipping container on a trailer, trailers of 18-
wheelers, bike trailer
DOES NOT INCLUDE boats, shipping containers etc.
not currently on a trailer. Does not include cars being
towed
EXAMPLE: Trailers with other vehicles on top of them
should be annotated with only single cuboid for both
the trailer and the vehicles atop of it, labeled as
‘Vehicle - Trailer’.
See Towed Objects for examples.
Vehicle - Wheelchair: Wheelchair.
INCLUDES both wheelchair with rider and without

rider.
If the wheelchair has a rider, the cuboid MUST

INCLUDE both the human and the wheelchair.
Vehicle - Other Personal Mobility: Personal mobility

vehicle like a skateboard, Segway, roller blades,
scooters.
INCLUDES both vehicles with rider and without rider.
If the object has a rider, the cuboid MUST INCLUDE

both the human and the vehicle.
DOES NOT INCLUDE wheelchairs, knee scooters,

bicycles, vespas, sitting scooters, all-terrain vehicles,
trikes, and motorcycles.
Vehicle - Bicycle: Bikes, including tandem and

recumbent bicycles.
INCLUDES both bikes with rider and without rider.
If the bike has a rider, the cuboid MUST INCLUDE both

the human and the bike.
DOES NOT INCLUDE pedestrians pushing bicycles
STANDARD SIZE:
Standalone: .60m x 1.0-1.5m x 0.8-1.2m

Bicycle with Rider: .60m x 2.0m x 1.8-2m
Vehicle - Motorcycle: Motorcycle.
INCLUDES both motorcycles with rider and without

rider, vespas, and sitting scooters.
If the motorcycle has a rider, the cuboid MUST

INCLUDE both the human and the motorcycle.
DOES NOT INCLUDE all-terrain vehicles, trikes, and

motorcycles with sidecar, or pedestrians pushing
motorcycles
Vehicle - Other (Motorized) Riding: Three or four

wheeled vehicles with the rider outside (on top).
INCLUDES all-terrain vehicles (ATVs), quads, trikes,

motorcycles with sidecar, vehicles with rider and
without rider.
If the vehicle has a rider, the cuboid MUST

INCLUDE both the human and the vehicle.
DOES NOT INCLUDE convertible cars, construction

vehicles, tractors, or any two-wheeled vehicles like
motorcycles without sidecar, vespas, sitting scooters.
Vehicle - Car: Normal sized vehicles, such as cars,

pickup trucks, SUVs, small passenger minivans, small
flatbed pickup trucks.
INCLUDES cargo on the roof or trunk, open doors,

hoods or trunks.
DO NOT INCLUDE antennas or side-view mirrors

STANDARD SIZE:
Sedan: 1.5-1.8m x 4.5-4.8m x 1.6-2m
SUV: 1.5-2.0m x 4.6-5.0m x 1.6-2.0m
Pick up truck/ Minivan: 1.80-2.0m x 5.0-6.0m x 2.0m
Vehicle - Van: Large passenger vans, moving vans,

delivery vans, news vans.
DOES NOT INCLUDE small passenger minivans.
STANDARD SIZE:1.80-2.0m x 5.0-6.0m x 2.0m
Vehicle - Truck: Large truck shaped vehicles such as

freight trucks, oil tank trucks, large flatbed trucks, large
delivery trucks, large moving trucks, tractor units,
garbage trucks, cement trucks, RVs.
INCLUDES open lift gates / lifts on the back of the

truck
Note: A good rule to follow for larger/modified pick-up

trucks that can be labeled as trucks instead of cars is if
it’s a lot larger than a regular pickup truck (2.0, 5.0-5.5, 2.0)
then label it as truck
STANDARD SIZE: 2-3m x 5m x 1.8-2.5 m
See Towed Objects for examples.
Vehicle - Tow Truck: Large vehicle designed to tow

other vehicles, like cars or trucks.
DOES NOT INCLUDE objects being towed by the tow

truck - label these objects separately
STANDARD SIZE: 2-3m x 5m x 1.8-2.5 m
Vehicle - Bus: Bus or shuttle designed to transport

large numbers of people.
DOES NOT INCLUDE bendy (articulated) busses or

school busses.
STANDARD SIZE: 2.5-3m x 12-18m x 2.5-3m
Vehicle - Bendy Bus: ‘Vehicle - Bus’ that is composed

of two or more connected segments.
Each segment of the bendy (articulated) bus MUST BE

ANNOTATED with its own cuboid.
Vehicle - School Bus: ‘Vehicle - Bus’ that transports

children to school. Typically a yellow bus, with flashing
stop signs when dropping off/pickup up children.
Vehicle - Construction: Vehicles primarily designed

for construction. Typically very slow moving or
stationary. Includes cranes, caterpillars, bulldozers,
excavators.
DOES NOT INCLUDE forklifts, powerline utility trucks,

or cement trucks
Vehicle - Mobile Traffic Control: Mobile signs or

boards (including electronic signs) that are meant to
direct traffic. Typically mounted on a trailer or on the
back of a pickup truck.
Cuboid MUST INCLUDE the entire object (sign and

towing hitch if on a trailer).
DOES NOT INCLUDE traffic control signs or boards

that cannot move (i.e., have no wheels).
Vehicle - Other: Unusual vehicles such as forklifts,

tanks, street sweepers, duck boats, golf carts, tuk-tuks.
Emergency - Police Vehicle: Police vehicles. Does

not include police motorcycles.
Emergency - Police Motorcycle: Police officer on a

motorcycle. The cuboid must include both the police
officer and the motorcycle.
Emergency - Other: Non-police emergency vehicle.

Examples include ambulances and firetrucks.
Railway Vehicle: Any vehicle that travels on rails, like

light rail, tram or train. For trains that consist of several
linked units, annotate each segment with a separate
‘Railway Vehicle’ cuboid.
Other Dynamic Object: Any object in motion left on

the driveable surface that is too large to be safely
driven over.
INCLUDES balls rolling on the street, runaway strollers

or shopping carts.
STATIONARY LABELS
Construction Object - Construction Cones and

Poles: Any temporary cone or short temporary pole
(usually orange or striped and used in construction),
placed to redirect traffic.
MUST BE WITHIN 5m of the diveable surface
DOES NOT INCLUDE permanent structures like

bollards or delineators
STANDARD SIZE: 30m x .30m x .80m
Construction Object - Construction Zone Signs and

Construction Sign Boards: Any construction related
signs or boards (including electronic signs) that are
meant to direct traffic.
(includes directing pedestrian and bicycle traffic)
Construction Object - Temporary Construction

Barrier: Temporary small walls used to block off
construction zones. Typically plastic, sometimes
concrete or metal, and brightly colored about knee or
waist height.
DO NOT ANNOTATE if there is no visible construction

area in the scene.
Construction Object - Temporary Fence: Temporary

walls used to block off construction zones for buildings.
Typically metal or wood and 2-3 meter high.
DO NOT ANNOTATE if the fence is further than 5

meters from the driveable region.
YOU CAN draw as many cuboids or as little cuboids to

annotate temp fences, as long as it's within reason.
Construction Object - Other: Any other large (larger

than 0.5m high) construction object. Includes stationary
cement mixers, wreckage, generator, wheelbarrow.
Object MUST BE INSIDE the work zone area (marked

by indicators like barrier or cones).
DO NOT ANNOTATE if the objects are not on the

normal drivable region (road work, etc.)
DOES NOT INCLUDE smaller items such as

sandbags, tools.
Static Road Object - Pushable or Pullable: Objects

that can be moved by a pedestrian.
INCLUDES garbage bins with wheels, shopping carts,

wheelbarrows, moving carts, dumpsters with wheels.
Object can be on the road, on the sidewalk, or away
from the driveable surface.
Object MUST BE larger than .5m high and must

currently be stationary.
DOES NOT INCLUDE garbage bins that do not have

wheels, or any garbage bins that are permanently in
the ground.
Static Road Object - Other: Any object larger than

.15m obstructing the normal drivable region.
INCLUDES dumpsters without wheels, dead animals,

tree branches, trash bags, mattresses, ladders,
shipping containers not on a trailer, and temporary
objects on driveways.
MUST BE obstructing the normal driveable region
No Label Zone (NLZ): Large cuboid used to cover

parking lots, grouped objects, and highway
underpasses.
For NLZ rules, please see the No Label Zone Rules &
Guidelines
Note: Occlusion attribute on No Label Zones should
ALWAYS be set to 'Can't Tell'
Ghost - Reflection: A
physically real object
seen in a reflection off of
a surface, making visible
lidar points.
INCLUDES reflections of
cars, pedestrians, static
road obects, etc.
Reflection MUST HAVE

AT LEAST 10 LiDAR
points
CAN intersect with other

real objects or
annotations, but as little
as possible.
Ghost - Sensor Artifact
Ghost Sensor Artifacts

are clusters of lidar
points created from
non-physical objects
that float around the
front, back and sides of
the self-driving
car(SDC).
Points located <10m from

ego vehicle that do
NOT correspond with a
physical object. These
typically follow the ego
vehicle around.
Annotate these points if

they have at least 10
LiDAR points.
Size the cuboid so that all

of the points from the
Ghost - Sensor Artifact
are contained within it.
If the Ghost - Sensor

Artifact points are
moving, the cuboid(s)
MUST BE dynamic.

real objects or
as possible.
Ghost - Retroreflector
LiDAR points associated

with retroreflective road
signs or on-road markers.
Ghost points will be <1m

away from the physical
retroreflector
Annotate any Ghost -

Retroreflector if it has at
least 10 points in a
single frame.
Ghost -
Dust/Spray/Smoke/Fog
LiDAR points (and

sometimes visible in
Camera View) produced
by real physical
environmental factors,
such as rain, dust, fog,
smoke. These physical
environmental factors can
be caused by cars too, for
example exhaust smoke
and tire spray.
Annotate any Ghost -

Dust/Spray/Smoke if it
has at least 10 points
DO NOT annotate Ghost

- Dust/Spray/Smoke if it
is never higher than the
vehicle’s wheels.

real objects or
as possible.
ATTRIBUTES TABLE
Towing Attribute Rider Attribute Occlusion Turn Signal Lights

Any Lights Brake Lights Reverse Lights Parked
Intent to Cross
TOWING ATTRIBUTE
For each vehicle, select the attribute "Is the vehicle towing or being towed?"
Being Towed: The vehicle/trailer is being

towed by another vehicle, or you are
reasonably sure that it is being towed (you
can see a moving shipping container)
(More examples / Edge Cases here)
Towing: The vehicle is towing another

vehicle
Neither: The vehicle is not towing, is not

being towed, or you are not certain if it is
being towed or not towed.
RIDER ATTRIBUTE
For animals and vehicles select the attribute "Does the vehicle or animal have a rider?"
With Adult Rider: Vehicle currently has a rider.

Rider is an adult or older child (>8 years old)
With Child Rider: Vehicle currently has a rider.

Rider is a child
Without Rider: Vehicle does not have a rider
INTENT TO CROSS ATTRIBUTE

For all pedestrians and following Vehicles with Rider (Wheelchair, Other Personal Mobility,
Bicycle with Child Rider), select the attribute "Does the pedestrian/vehicle intend to cross the
street at any point in the scene?"
***Does Not Include Bicycle with Adult Rider
Note this captures intent, not necessarily whether they are actually crossing, but can include crossing
objects.
Yes: Pedestrian or Vehicle with Rider indicates

intent to cross a street in at least one frame during a
scene.
Solid “Yes”:
● Persons standing within a crosswalk, or

crossing a street (not just walking parallel
with traffic along street)
Ex: within 3 meters of corner and facing the

● Persons within approx. 3 meters of a road
corner (unless clearly always walking away
from crosswalk during sequence)
● Persons who look like they intended to

cross at a corner, but end up rounding the
corner/changing direction
Usually “Yes”:
Ex: persons facing/looking at the road

● Persons facing/looking at the road, while
on sidewalk or standing on road surface
● Err towards “Yes” for intent if in doubt
Ex: within 3 meters of corner and facing the

road
Ex: persons actively crossing the road
No: Pedestrian or Vehicle with Rider does not

indicate any intent to cross a street in at least one
frame during a scene.
Solid “No”:
● Sitting down, whether on sidewalk or curb
Ex: persons sitting down whether on sidewalk

● Persons facing completely away from the or curb
road, during the entire sequence
Usually “No”:
● Persons walking on sidewalk, parallel to

traffic, not near an intersection
Ex: persons sitting down whether on sidewalk
● Persons walking parallel to road while on or curb
the road
● Err towards “Yes” for intent if in doubt
Ex: persons parallel to the road, while on or

off the road
Ex: persons parallel to the road, walking on

the sidewalk/driveway
Ex: persons parallel to the road, facing away

from intersection
Ex: persons parallel to the road, on the road
Can't Tell: Pedestrian or Vehicle with rider intent to

cross is not determinable due to camera blurriness,
lack of visibility, too far away, or too occluded by
other objects for the entirety of the scene
● Must check All Frames that Intent to Cross

is indeterminable before selecting 'Can't
Tell'
Ex: no clear view of intent to cross due to
distance for all frames
Ex: no clear view of intent to cross due to

occlusion for all frames
Ex: no clear view of intent to cross due to low
visibility for all frames
OCCLUSION
For ALL OBJECTS, select the attribute "What percent of this object is occluded (blocked by other
objects in camera view)?"
Occlusion is how much the object is blocked from our view in Camera View by other objects (ie. walls,
vegetation, cars, etc.)
If the object is visible in more than one camera view, select the least occluded (lowest %) answer.
If the object is behind glass treat the glass as a wall.
Annotate each frame’s occlusion individually.
Examples
0-20%: Object is not occluded at all, or

slightly occluded
20-40%: Object is slightly - moderately occluded
40-60%: Object is moderately - more than half

occluded
60-80%: Object is more than half - almost completely

occluded
80-100%: Object is very occluded to completely

occluded - not visible in any camera image -
Can’t Tell: Object that cannot be occluded (eg. No

Label Zone) or object is too far away or blurry to
determine the occlusion percentage (rare)
DOES NOT INCLUDE objects that aren’t visible due
to being fully occluded (use 80-100% if we cannot see
the object)
09/15/20
Note: If a scene is at night, adjust camera brightness
and/or contrast to determine Occlusion on object
before selecting Can't Tell.
TURN SIGNAL LIGHTS

For every vehicle, select the attribute “Which turn signal indicator light is on? (if any)”.
These are from the perspective of the vehicle’s driver.
Note: If Signal lights DON'T blink between frames, do not consider signal lights to be active.
BOTH On -
The vehicle’s right turn signal is visible,
and is active in this frame.
AND
The vehicle’s left turn signal is visible,
1 On: Left On, Right Off -
AND
The vehicle's right turn signal is
visible and is inactive in this frame.
1 On: Left On, Right Can't Tell -

AND
The vehicle's right turn signal is not
visible in this frame.
1 On: Right On, Left Off -

AND
The vehicle's left turn signal is visible and
is inactive in this frame.
1 On: Right On, Left Can't Tell -

AND
The vehicle's left turn signal is not
BOTH Off
and is not active in this frame.
AND
Left Off, Right Can't Tell -

AND
The vehicle's right turn signal is not
Right Off, Left Can't Tell -

AND
The vehicle's left turn signal is not
Both Can’t Tell -

The vehicle’s right turn signal is not
AND
The vehicle’s left turn signal is not visible
in this frame.
ANY LIGHTS
For vehicles, objects, animals and pedestrians select the attribute “Are any lights on? (in at least
1 frame)”
Any lights includes all lights of a vehicle or object (tail light, head light, turn signal, or any other light
on the exterior OR interior of the vehicle) and any lights that can be held or found on the body of a
pedestrian or animal (flashlights, cell phones, lights attached to pets, etc.)
If a light is on / visible in one frame, mark as On in ALL frames
On (Yes) - If any lights are on at ANY point in the

task - even for 1 frame
INCLUDES any lights that are visibly on through
windows/glass
Off (No) - If lights are visible and NO lights are on at

any point in the scene
INCLUDES any lights that are visibly off through

windows/glass
Can’t Tell - Use Can’t Tell if...

- The object doesn’t have any lights that can be
turned On/Off (cones/poles, garbage cans)
- The object’s lights are occluded at every point
during the scene.
- We are uncertain if any lights were
On/Off due to blurriness, distance, glare.
- Any Lights on Pedestrian objects must
always be 'Can't Tell'
BRAKE LIGHTS
For every vehicle, select the attribute “Are the object’s brake lights on?”
On (Yes)- The brake lights are active in this

frame
INCLUDES brake lights that are visibly
on through windows/glass
Off (No) - The brake lights are not active in

this frame
INCLUDES brake lights that are visibly
off through windows/glass
Can’t Tell - Use Can’t Tell if…

- The object doesn’t have brake lights
- The object’s brake lights aren’t
visible in this frame
- Vehicles which are faced
perpendicular to the camera view
should be labeled as “Can’t Tell”
- We are uncertain whether the
object’s brake lights are On/Off due
to blurriness, angle, glare, or
distance.
REVERSE LIGHTS ATTRIBUTE

For every vehicle, select the attribute “Are the object’s reverse lights on? (in at least 1 frame)”
This is asking whether or not the reverse lights were active AT ANY POINT in the scene.
On (Yes) - The reverse lights were active

at ANY point in the task - even for 1
frame
INCLUDES reverse lights that are visibly
on through windows/glass
Note: If at no point the back of a vehicle

is visible, please do not set Reverse
Lights to 'Off', set it to 'Can't Tell'.
Off (No)- The reverse lights were never on

in this task - not even 1 frame
INCLUDES reverse lights that are visibly
off through windows/glass
Can’t Tell - Use Can’t Tell if...

- The object’s reverse lights aren’t
visible at any point in the scene
- The object doesn’t have reverse
lights
- We are uncertain whether the
reverse lights were On/Off due to
distance, glare, angle, or blurriness.
PARKED ATTRIBUTE
For every vehicle, select the attribute “Is the object parked in every frame?”
A vehicle is parked if stopped or standing still, whether or not the driver is visible, and does not have
the intention to start moving again FOR ALL frames it is visible.
If a vehicle is parked for almost all frames, but begins to move on the last frame, mark Parked as No.
Yes - The object was parked in ALL of the

frames of the task
INCLUDES cars stopped on the side of the
freeway with or without hazard lights on
INCLUDES cars parked in driveways, cars
against the curb, or in a parking lot the
SDC is in.
No - The object was not parked in EVERY

FRAME during the scene or if the object is
parked in some frames, but begins to move
later on
INCLUDES Any cars stopped with the
intention to move again, (ie. cars stopped
at a stoplight or intersection, brake lights
on), their Parked attribute should be set
to 'No'.
INCLUDES cars waiting to go through a toll
booth
INCLUDES cars that are stopped by traffic

controllers
INCLUDES cars that are pulled over to make
way for emergency vehicles
INCLUDES cars that are stopped while
loading / unloading passengers or goods
Can’t Tell - Object is too far away to tell

what the parked status is.
LABEL EXAMPLES (click the label to return to the label definition)
Pedestrian - Adult:
Pedestrian - Child:
Pedestrian - Traffic Controller:
Pedestrian - Specialty:
Pedestrian with Object - Adult:
Pedestrian with Object - Child:
Pedestrian with Object - Specialty:
Animal - Dogs/Cats:
Animal - Other:
Vehicle - Trailer:
Vehicle - Wheelchair:
Vehicle - Other Personal Mobility:
Vehicle - Bicycle:
Vehicle - Motorcycle:
Vehicle - Other (Motorized) Riding:
Vehicle - Car:
Vehicle - Van:
Vehicle - Truck:
Vehicle - Tow Truck:
Vehicle - Bus:
Vehicle - Bendy Bus:
Vehicle - School Bus:
Vehicle - Construction:
Vehicle - Mobile Traffic Control:
Vehicle - Other:
Emergency - Police Vehicle:
Emergency - Police Motorcycle:
Emergency - Other:
Railway Vehicle:
Other Dynamic Object:
Construction Object - Construction Cones and Poles:
Construction Object - Construction Zone Signs and Construction Sign Boards:
Construction Object - Temporary Construction Barrier:
Construction Object - Temporary Fence:
Construction Object - Other:
›Static Road Object - Pushable or Pullable:
Static Road Object - Other:
No label Zone (grouped objects)
No Label Zone (parking lot separate from SDC):
No Label Zone (SDC on Overpass):
Annotate all other objects not in no label zone (ex. cars on on and off-ramps)
ATTRIBUTES EXAMPLES (click the attribute to return to the attribute definition)

Occlusion
0-20%: Object is completely visible, or slightly occluded
20-40%: Object is slightly - moderately occluded
40-60%: Object is moderately - more than half occluded
60-80%: Object is more than half - almost completely occluded
80-100%: Object is almost occluded - not visible in any camera image
Split Occlusion: Add up the surface area of the poles/fences (The amount of space that the
poles/fences cover) and treat that value as you would normally for occlusion.
An acceptable guess would be 0-20% OR 20-40% occlusion.
Updated Indicator Lights
Example OLD NEW
Can't Tell: Object is a 'No Label Zone' or the Object is Too Far Away
🚨 PAST ALERTS & UPDATES 🚨

For current updates, visit Current Updates
NOVEMBER 2020
Please review the video or slides on how to set Brake Lights in nighttime scenes.
SEPTEMBER 2020
October 20 E-NLZ: Please review updated clarifications on E-NLZ rules

Oct 06 Common Errors: Please review Bee Common Error Report for tips on better
accuracy, especially on Attributes
Sep 24 Common Errors (Updated Daily): Please review Bee Common Errors 09/23 for
tips on better accuracy, especially on Attributes.
help on annotating and selecting attributes. 🔥

Sep 18 Clarifications/Instructions: Please review this Attribute/Annotation Guide for
09/19 Update
Sep 15 IMPORTANT NOTE: Clarifications made on Occlusion - Can't Tell in the

instructions. (see here)
AUGUST 2020 🚨
Sep 6 UPDATE: You may see different 9 indicator light choices now! This is NEW.
Please review the updates closely. (See more examples here)
Previous Indicator Choice New Indicator Choice

Both (Both On) "BOTH On"
Left "1 On: Left On, Right Off"
Left "1 On: Left On, Right Can't tell"
Right "1 On: Right On, Left Off"
Right "1 On: Right On, Left Can't tell"
Off (Both Off) "BOTH Off",
Off "Left Off, Right Can't tell"
Off "Right Off, Left Can't tell"
Can’t Tell "BOTH Can't tell"
CLARIFICATION: Annotate Objects on gas stations covered by E-NLZs!(example)
UPDATE: You may see 9 indicator light choices now! This is NEW.

Please review the updates closely. (See more examples here)
Previous Indicator Choice New Indicator Choice

Both (Both On) Left On, Right On (both On)
Left Left On, Right Off
Left Left On, Right Can’t Tell
Right Left Off, Right On
Off Left Off, Right Off (both Off)
Off Left Off, Right Can’t Tell
Right Left Can’t Tell, Right On
Off Left Can’t Tell, Right Off
Can’t Tell Left Can’t Tell, Right Can’t Tell (both Can’t Tell)
JULY 2020 🚨
REMINDER: If built-in NLZ is on a non-NLZ area, label as normal.
PLEASE REVIEW: Clarification on Signal Lights - Both (example)
JUNE 2020 🚨
UPDATE: Intent to Cross does not include Bicycle with Adult rider
UPDATE: New Pedestrian / Vehicle w/ Rider Attribute: Intent to Cross (example)
🚨 MAY 2020 🚨
ADDED: Standard Sizes on some Labels
UPDATE: New Guidelines for Embedded NLZs (example)

Bee Lidar Update

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bee Lidar Update

Uploaded by

Copyright:

Available Formats

4/28/2021 https://docs.google.com/document/d/1cGZTK_BH8-UJWSaJstgMFg_9_TLMMDPrMcxiqnOTGPc/pub?

Bee Lidar - Cuboid

Type Class Type Class Type Class

Child Car Construction Construction Cones

Specialty Tow Truck Temporary

Pedestrian Adult Bus Temporary Fence

Specialty School Bus Static Road Pushable or

Vehicle Trailer Other Ghost Reflection

Bicycle Other Dust/Spray/Smoke

Motorcycle Railway Railway Vehicle

Towing Rider Attribute Occlusion Turn Signal Lights Intent to Cross

GLOBAL RULES & GUIDELINES

Minimum LiDAR Points & 0-point

Label all objects that have 1 LiDAR point

Any object that has 1 LiDAR point at any

If a cuboid becomes fully occluded

If you cannot annotate the object due to

Echo Sizing and Positioning

INCLUDE echo inside of the cuboid.

If the echo is only visible for a few

Use Grid View and Dynamic APC to

DO NOT leave more than .2m of space

DO NOT INCLUDE side view mirrors or

INCLUDE echo inside of the cuboid.

INCLUDE large objects on the back or

INCLUDE any open doors, trunks, and

INCLUDE any ramps & lifts on the back

150 Meter Rule

Annotate the entire path of any object

If there is an object that was never within

Annotate the towing vehicle and the

It is fine for the cuboids to overlap.

2 Pedestrians with 1 Object:

If more than one person are carrying a

Example: If two people are carrying a

Labels Changing Mid-Scene

If an object needs to change labels mid-

If a person is holding a large

If they throw that object, start two

Using Visual Changes to Determine

In the example provided to the right, from

If there is clear proof that an attribute is

On/Off because of Clear Status change even if blurry

If Vision for an Attribute Changes

In the example to the right we can see

Once an attribute becomes difficult to

Signal Lights - Both

When selecting Signal Lights - Both,

NO LABEL ZONE (NLZ) RULES & GUIDELINES

No Label Zone (NLZ) Basics

When creating a ‘No Label Zone’, create

DO NOT label any other classes within

DO NOT cover gas stations, auto repair

DO NOT annotate house driveways with

NLZ Sizing & Positioning

NLZs can generally be left at the default

NLZs should be as long/wide as

Embedded (built-in) NLZ /

Some tasks will contain built-in No

DO label objects completely outside

NLZ Parking Lots

The ‘No Label Zone’ should cover all

If the parking lot has an abnormal shape,