Goat_LiDARCuboids_V2.

0 Spec Document
“LiDAR Cuboid Annotation” | IN PROGRESS

Contents
● Labels & Attributes Table -- needs to be locked by end of Qualification Run
● Annotation Rules -- needs to be locked by end of Qualification Run
● Attributes Description and Choices -- needs to be locked by end of Qualification Run
● Label Description/Examples - needs to be locked by end of Qualification Run
● Obstacle Attribute Description/Examples - needs to be locked by end of Qualification Run
● Critical vs. Non-Critical Errors - needs to be locked by end of Qualification Run
● Edge Cases - iterative

Note: in this task you will be drawing 3D LiDAR cuboids on pedestrians, vehicles, animals, obstacles, and
traffic signs.

Labels & Attributes Table

Category Label Attributes and Attribute Option Choices

Pedestrian Pedestrian

4 Wheel Vehicles Medium Vehicle Lane Change [options: Actively Changing Lanes, Staying in
Current Lane],

Different_Road_Physically_Separated [options: Yes

Separated, No],

Regular Lights [options: On, Off],

Brake Lights [options: On, Off],

Turn Signals [options: Left, Right, Both, Off],

Reverse Lights [options: On, Off],

Doors [options: Open, Closed],

Trunk [options: Open, Closed],

Large Vehicle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk

Extra Large Vehicle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk

Bus Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk,

School Bus [options: School Bus with Stop Sign Extended,

School Bus no Stop Sign Extended, NOT School Bus]
 Other 4 Wheeled Vehicle Lane_Assignment, Lane Change,
Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk

Unknown Vehicle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk

Emergency Vehicle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights, Doors, Trunk,

Emergency Vehicle Type [options: Police, Ambulance,

Firetruck, Other],

Emergency Vehicle Siren [options: On, Off]

Trailer Need to ask customer

Parking Lot

2 Wheel Vehicles Motorbike Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights

Manual/Electric Scooter Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights

Manual/Electric Bicycle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights

Other 2 Wheeled Vehicle Lane_Assignment, Lane Change,

Different_Road_Physically_Separated, Regular Lights,
Brake Lights, Turn Signals, Reverse Lights

Road Obstacles Road Obstaclesˇ Road Obstacles [options: Cone, Barrel, Potholes, Temporary
Sign, Trash Can, Other]

Animal Animal Animal Type [options: Small Animal, Large Animal, Manned
Animal, Other Animal]

Traffic Sign Traffic Sign Traffic Sign Type [options: Stop, Yield, Speed Limit]

Annotation Rules

Rule Description Example

Minimum LiDAR points 1 point

 Extremities Label the vehicle or pedestrian
only.
Extremities such as taxi hats and
extra load should not be included
in the cuboid.

Trailers Trailers will be annotated with a

dedicated cuboid, separated
from the towing vehicle. Include
all elements on the trailers.

Parked Vehicles Individual parked vehicles

(anywhere in the image), should
be annotated as regular vehicles.

Densely packed vehicles, far

away or behind a barrier should
be labeled as parking lot.
ID Consistency ID’s must be kept consistent for
an object across all the frames
when they are visible.

Make sure IDs do not switch

between objects. If an object
appears then disappears across
frames, that object must keep the
same ID.
Echo Points Include echo points and drifting
points in the cuboid. It is OK if the
cuboid is larger to include these
points.
Pitch and Roll When the cuboid is in an area that
is not completely flat. The cuboids
should be adjusted to the slant of
the landscape.

Good way to check if the cuboid

needs to be adjusted is using
dAPC on objects and determine if
the object does not fit within the
cuboid.
Camera View If the object is hard to identify with
LiDAR points and is not visible or
clear in camera view, do NOT
annotate the object

Attributes Description and Choices

Attribute Category Description Choices

Ignore For pedestrians that are too

small/grouped together to
annotate well with cuboids.

Add a pedestrian cuboid in the

area and mark the Ignore attribute
as Yes.

If you can draw a cuboid on

individual pedestrians and each
has at least 1 LiDAR point,
annotate individually and mark
Ignore=No.

Age Baby - can’t walk on its own Baby

Child - can walk on its own,

usually shorter than a regular
sedan car

Adult

Old - older person who may walk

more slowly than others
Child

Old

Assisted Is the person using something to Stroller

help them move?
 Include the
stroller/wheelchair/other device in
the annotation.

Wheelchair

Other

Lane Change Is the vehicle actively changing ● Actively changing lanes

lanes? ● Staying in current lane

As long as the vehicle is still

moving into the center of its new
lane, use “Actively changing
lanes.”
 ^Staying in current lane

^Actively changing lanes

^Actively changing lanes

Road Obstacles What type of obstacle is on the ● Cone

road or road shoulder? All ● Barrel
examples here. Obstacles are ● Potholes
objects that may prevent vehicles ● Temporary Sign
from driving on the road and are ● Trash Can
not part of the road structure. A ● Other
reasonable driver would avoid
these obstacles.

DOES NOT INCLUDE:

- Curbs
- Objects NOT on road
surface/shoulder ^Other (leaves)

^Barrel
 ^Barrel

Animal Type What type of animal is it? Small Animal

If you see a large animal that is

manned (has a person riding it),
mark “Manned Animal.”

Large Animal

Manned Animal (with person riding

it)

Other Animal
Traffic Sign Put a tight cuboid around the sign
plate of Stop, Yield, and Speed
Limit signs.

Do NOT label any other signs.

Do NOT label the sign pole.

Different_Road_Physically_Sep Is the labeled object on a

arated physically separated road or lane?

For example:
● Oncoming traffic on the
highway

Regular Lights Are the front or rear lights lit? On

NOT brake lights.

Off

Brake Lights Are the brake lights on? On

If you can’t see the back lights,

mark “Off.”
 Off

Turn Signals Is the turn signal blinker on for left,

right, or both sides?

Mark “Left” if you see the left

blinker actively flashing.

Note: you DON’T have to change

the status in each frame as the
turn signal turns on and off. Just
mark “Left” as long as the left turn
signal is active, and mark “Off”
when the signal is no longer
active.

Do the same for “Right” and

“Both.”

Mark “Off” if you can’t see the

status, or the turn signals are not
on.

Reverse Lights Is the reverse light on? These are On

usually white.

If you can’t see, mark “Off.”

Doors Is at least one door open? Open

If you can’t see any doors open,

mark “Closed.”

Trunk Is the trunk open? Open

School Bus Is the bus a school bus (usually School Bus with Stop Sign
yellow)? Extended

If it is a school bus, is it extending

a stop sign in the frame?
 School Bus no Stop Sign
Extended

NOT School Bus

Emergency Vehicle Type Police

Ambulance
 Firetruck

Emergency Vehicle Siren Is the emergency siren on in the

scene? You should see flashing
lights.

Label Description/Examples

Labels/Attributes Description Example

Pedestrian A Person (adult, child,

or baby) in any
position.

Make sure that the

cuboid is big enough to
contain all of the
pedestrian’s body and
limbs.

Do NOT include
anything the person is
carrying, pushing, or
pulling.

If the person is riding a

vehicle (scooter, bike,
motorcycle, etc)
annotate together with
the vehicle and mark
as the vehicle type. Do
 NOT mark as
pedestrian.

Medium Vehicle Medium vehicles

including:
● Coupes
● Sedans
● Convertibles
● Hatch-Backs
● Station-Wagons
etc..

Large Vehicle Large vehicles

including:
● SUVs
● Vans
● Pick Up Trucks,
etc...

Extra Large Vehicle Extra large vehicles

including:
● Box Trucks
● Light Duty
trucks
● Heavy Duty
trucks
● Semi Truck
without Trailer
● Semi Truck
Trailer (labeled
separately)
Bus A bus is a large motor
vehicle that is usually
capable of carrying at
least 10 passengers.

Also, this includes

“bendy buses” and
buses with 2 parts -
use one cuboid for both
sections.

Other Vehicle Other Vehicles

including:
● Farm Tractor
● Concrete-mixer
● Boat-loader, etc

Trailer Objects that are being

towed by medium or
large vehicles.

Formerly labeled as
Extra Large Vehicle.
Unknown Vehicle Any vehicles that are
too small/blurry to
distinguish, and do not
fit into categories listed.

Emergency Vehicle Police, ambulance,

firetruck, and other
emergency vehicles.

Parking Lot Densely packed

vehicles, far away or
behind a barrier should
be labeled as parking
lot.

Motorbike A two-wheeled vehicle

that is powered by a
motor and has no
pedals.

Include the rider.

Manual / Electric Foot Operated or

Scooter Electric, two-wheeled
scooter.

Include the rider.

 Foot operated or
Manual / Electric Electric, two wheeled
Bicycle bicycle.

Include the rider.

Other 2 Wheeled Other types of 2-

Vehicle wheeled vehicles.

Include the rider.

Animal Animals, from cats to

horses.

Obstacle Attribute Description/Examples

Road Object Description Example

Cone Small cones that are used to guide

traffic, especially near construction
zones.

Must be on the road or road

shoulder.
Barrel Medium sized barrels that are
used to guide traffic, especially
near construction zones and
highways.

Must be on the road or road

shoulder.

Potholes Ground indentations (road that is

curving downwards) that are found
on the road bed or on the side.

These should be large enough

that a driver would want to avoid
them.

Must be on the road or road

shoulder.

Temporary Sign Signs placed on roads that are not

permanently installed, usually to
warn or guide traffic.

Must be on the road or road

shoulder.

Trash Can Must be annotated only when on

road or road shoulder
 Other Any object that does not fit into
said categories.

Must be on the road or road

shoulder.

Critical vs. Non-Critical Errors

Critical Errors Non-Critical Errors

Edge Cases
Edge Case How to Handle It Example