cupcake-oxen-refresher-course-h-itzy-0

Instructions
Please read the following instructions carefully and then proceed to the practice scenarios.

Updates:

- Update July 2022: Updated minimum Box Size rule and annotation rule for roads. .

Overview
This task involves drawing 2D bounding boxes around individual objects in an image that fall into

one of seven categories specified below. The categories include common types of road users. The

images being annotated are taken from traffic surveillance cameras monitoring highways.

Here's an example of what you'll be doing in this task.

All visible Road Users on the main highway must be annotated.

Both moving and stationary/parked objects should be annotated.

Minimum Box Size Rule: If the object is under 15 pixels x 15 pixels for Car, Bus and Commercial

truck, you do not need to annotate it, and if the object is under 10 pixels x 10 pixels for Pedestrians,

Motorbikes, Bicycles and Emergency Vehicle, you do not need to annotate it. Note that these are
very small sizes, and would be rare to leave objects unlabelled. Please use the Ruler tool to get

precise measurements.

Do NOT annotate vehicles outside of the main road.

Using the rule Tool:

If you are uncertained if a car should be label you can use the Rule Tool

By pressing "R" on your keyboard the tool should be ready to use, just drag the mouse and measure

the car you are uncertain.

By pressing "R" again the tool will be deactivated so you can continue working on the task:

Example of the Rule Tool:

Label Taxonomy
The chart above shows the taxonomy of class labels. Seven (7) types of common road users have

been identified to be labeled in this task. These road users comprise two super-categories: vehicles

and vulnerable road users. The vehicles super-category includes five (5) classes: car, bus, truck,

motorbike, Emergency Vehicle. The vulnerable road users super-category includes two (2) classes:

pedestrian and bicycle.

Label Definitions
A description of each category is provided below. This includes the category definition, edge cases

that may be present in a traffic surveillance scene, and examples of the class taken from a traffic

surveillance dataset.

Vehicles
 Any vehicle driving on the main road should be annotated.

 Any vehicle parked on the main road (i.e., parallel parking, disabled vehicles) should be

annotated.

 Both empty and occupied vehicles should be annotated. Any parked or disabled vehicle

without users still needs to be annotated, including motorbikes without a rider.

 Vehicles parked in parking lots or driveways should not be annotated.

Car
The Car class is defined as a four-wheeled vehicle which can carry a small number of people. This

includes sedans, convertibles, coupes, SUVs, crossovers, jeeps, minivans, light trucks, vans, and

most medium pickup truck style trucks (e.g, Chevy Silverado, Ford F-150, Dodge Ram). Taxis and

town cars are included in this category. If a car object is towing a trailer, the trailer should not be

included in the car bounding box. The trailer should not be annotated. Examples of correct

annotations for cars with trailers can be found under "Special Vehicles."

Figure 1: Examples of Cars from a traffic surveillance dataset

Bus
The Bus class is defined as a large, elongated communal vehicle used for the transportation of

people. This includes small buses like an airport shuttle, municipal articulated buses, and large

coach buses.

Figure 2: Examples of Buses from a traffic surveillance dataset

Commercial Truck
The commercial truck class is defined as medium and heavy vehicles with separate tractor (cab) and

trailer used for the transportation of goods. By the FHWA vehicle category classification, this class

includes all Class 6, 7, 8 trucks (“Heavy Trucks”) as well as Class 4, 5 (“Medium Trucks”) with

Trailers. This class does not include Class 3, 4, 5 “Medium Trucks” without trailers; pickup truck style

trucks are cars and should not be labeled as commercial trucks.

Examples of trucks include moving trucks, large tractor-trailer trucks, dump trucks and other

specialized trucks like car carriers. Both the truck’s cab and truck’s trailer(s) should be included in

the bounding box. Trucks ("cabs") without a trailer should still be annotated as trucks. Delivery trucks

(like FedEx or UPS) and food trucks should also be labeled as trucks.

Figure 3: Examples of trucks from a traffic surveillance dataset

Note that trailers without a cab should not be annotated. An example where such a case may be

seen is a loading dock or other parking area off of the main roadway.

Figure 4. Trailers without a cab should NOT be annotated.

Motorbike
The Motorbike class is defined as a two- or three-wheeled motorized vehicle without a cab. This

includes motorcycles, like Harley Davidsons or Suzukis, and motor scooters, like Vespas. Three-

wheeled motorbikes (“trikes”) should also be labeled as a motorbike. Electric scooters, like Lime or

Spin, and electric bikes are not included in this category. The rider(s) should be included in the

motorbike bounding box; they should not be annotated separately as pedestrian(s).

Figure 5: Examples of motorbikes from a traffic surveillance dataset

Emergency Vehicles
Emergency vehicles are any vehicles used to respond to an incident, which includes police cars,

police motorcycles, ambulances, and fire trucks. Emergency vehicles are generally fitted with visual

warning devices (i.e., lights) which they use when responding to an emergency. Emergency vehicles

both with and without their lights on should be annotated.

Figure 6: Examples of emergency vehicles from a traffic surveillance dataset.

The table below shows some of the diversity of each class label.

Vulnerable Road Users

Bicycle
The Bicycle class is defined as a two-wheeled, pedal-driven, single-track vehicle. Both human-

powered and electric bicycles fall into this category. Scooters of any kind are not a part of this class.

The entire bicycle and human(s) riding the bicycle should be annotated. The human should not be

annotated separately as a pedestrian.

Figure 6: Examples of bicycles from a traffic surveillance dataset

Pedestrians
The Pedestrian class is defined as any person who is not operating a motor vehicle or riding a

bicycle. This includes persons who are walking, sitting, or standing. A person riding a small scooter,

such as a Lime electric scooter, skateboard, roller skates, etc. who is on the sidewalk should also be

annotated as a Pedestrian. The equipment they are using (i.e., skateboard) should be neglected

from the bounding box if possible.

Pedestrians often have other items with them. The following rule should be used to determine if any

objects with them should be included in the bounding box:

 An item, such as a handbag or umbrella, held by a person should be included in the

Pedestrian bounding box.

 Luggage, carts, and items being pulled or pushed by a person should not be included in

the Pedestrian bounding box.

 If there is free space between a person and an object they are interacting with, the object

should not be included in the bounding box.

Figure 7: Examples of pedestrians from a traffic surveillance dataset

Special Vehicles
Trailers
Trailers, such as livestock trailers or campers, which are being towed by a car such as an SUV or

pickup truck are handled differently than the commercial truck class. Trailers towed by cars should

not be annotated. Trailers should not be annotated as “truck” if they are being towed by a car. The

car towing the trailer should be annotated as a car. Trailers should not be included inside the

bounding box with the car. Trailers being towed by cars are not within the scope of this task.

Figure 8: Example of correct annotations for cars towing trailers.

Recreational Vehicles (RVs)

RV classification is based on chassis size. RVs can fall into the bus, van, or truck category. The

example photos included in the Appendix will be especially helpful in classifying RVs.

 RV classification is based on chassis size. RVs can fall into the bus or truck category.

However, campers which are being towed by a car should follow the rules presented

above for trailers. This means that towed RVs should not be annotated, but the car

towing them should be labeled.

  Class A RVs should be labeled as a bus.

 Class B RVs should be labeled as car.

 Class C RVs should be labeled as a truck.

 RVs or Campers being towed by a car are trailers and should not be annotated.

RVs or Campers being towed by a Car are trailers and should not be annotated.

Tow Trucks and Car Carries

Tow trucks and car carriers should be labeled with their appropriate label (truck or car). Any visible

vehicles being towed or carried by the truck should be annotated with a bounding box and their

appropriate class as well.

Figure 9: Every vehicle being transported by tow truck or car carrier should also be annotated as an

object instance itself.

Other Edge Cases

 If the class label of a vehicle is ambiguous, such as vehicles in very low-resolution

images, the annotator should use their best discretion. A good rule of thumb is to use the

relative vehicle size to assign a label. The most common class labels are Car and

Commercial Truck.

 Small, motorized carts such as golf carts, ATVs, and UTVs should not be labeled.
Fig. 10 Golf Cart Fig. 11 ATV Fig. 12 UTV

 Unattended scooters, Segways, and other equipment that may be used by pedestrians

should not be labeled. If a human is riding a scooter, the human should be labeled as a

pedestrian. If a scooter is not being ridden, it should not be labeled.

 Any vehicle that is not explicitly defined above should not be left unlabeled. It should still

be annotated with a bounding box and given the most appropriate label at the

annotator’s discretion. The chassis size of the vehicle is the best indicator of the

appropriate class.

 A note should be made of the example containing the object, a short description of the

object in question, and the label assigned by the annotator. These cases can then be

more closely reviewed and additional notes can be added to these guidelines if needed.

 Vehicles in the shadow of other vehicles should be annotated, even the portion of the car

that is occluded.

Fig. 13 Vehicle in shadow of other vehicle

 Be really careful with commercial trucks, do not misclassify them as buses.

Fig. 14 Commercial Truck Fig. 15 Bus

Bounding Box Guidelines

For this task, individual object instances are annotated with a 2D bounding box. Examples are

provided to visualize proper bounding box annotations.

Label every object instance in every image that falls under one of the seven listed categories and

has a height and width of at least 15 pixels for Cars, Commercial Trucks or Bus and 10 pixels for

Pedestrians, Motorbike, Bicycles and Emergency Vehicles. This means that objects very far from the

imager will not be annotated.

Figure 16: This scene is properly annotated. Notice that very small objects far from the imager are

not annotated as they don’t meet the pixel size requirement.

Label the entirety of each object instance. For instance, the mirrors of a vehicle should be included

in the bounding box.

Figure 17: The right image shows the correct bounding box annotation for this truck, which includes

the mirror.

Objects that are partially out of frame should still be annotated. The entirety of the object that falls

within the frame should be annotated.

Figure 18: The bounding boxes in this scene show the proper annotation for objects that are partially

out of frame.

The bounding box for a partially-occluded object should be drawn as if the entire object is visible.

This means that the extent of the object should be estimated, even where the object is occluded.
Figure 19: Left: incorrect annotation--only the visible portion of the taxi is annotated. Right: correct

annotation--the entire taxi, included occluded portions, are annotated.

The images being annotated often have text overlays including timestamps, banners, and

watermarks. If an object is occluded by an overlay, it is acceptable to include that portion of the

overlay in the bounding box.

Figure 20: The overlays which occlude a portion of the objects are appropriately included in the

bounding boxes of the Cars and Commercial Truck.

Bounding boxes should be drawn tight around each object. The amount of empty space within each

bounding box should be minimized. However, bounding boxes should never cut off portions of an

object.

Figure 21: The bounding box on the left has too much free space around the truck. The right image

shows the correct bounding box annotation.

Each object instance should be annotated with its own bounding box. Multiple objects, even of the

same type, should never be labeled together in the same bounding box. It is acceptable for one

object that occludes another to be present within the bounding box of the occluded object, but it

should also be annotated with its own box and label.

Figure 22: The right image shows the correct bounding boxes for a group of pedestrians.

Annotations in Challenging Weather and

Lighting
Images are captured at all times of day and in all weather conditions. Some images may be

challenging to annotate due to low light or low visibility conditions, but they should still be annotated

following the guidelines presented above. Examples for how to handle some of these conditions are

presented below.

In low light conditions, common challenges include:

Glare from headlights

Objects under heavy glare from headlights at night or the sun during the day should still be

annotated. As it can be difficult to distinguish the object class in these cases, one might try to use

the relative size of an object to determine the appropriate class.

Figure 23: Bounding boxes are drawn correctly around the full extent of the object, even though it is

difficult to distinguish its class due to glare. Because of the relative size, these objects are labeled as

cars.

Motion blur of fast-moving objects

Bounding boxes should still be drawn around the extents of the object as before. Motion blur can

sometimes make it difficult to distinguish the class of the vehicle, in which case one might try to use

the relative size of an object to determine the appropriate class.

Figure 24: Despite motion blur, the cars pictured below are still annotated with a bounding box

drawn around the full extent of the object.

In low visibility conditions from weather events, common challenges include:

Reflections from a wet roadway

Generally, the reflection of headlights should be neglected from the bounding box of an object when

possible. In places where it is too difficult to distinguish between glare, reflections, and the object

itself, it is okay to include the extents of the lights in the bounding box.

Figure 25: This scene is properly annotated despite reflections from the wet roadway. The reflections

are mostly neglected from the bounding boxes.

Snow that covers the roadway and stopped vehicles

Figure 26: The image on the left contains three cars, one moving and two disabled. The image on

the right is correctly annotated, with bounding boxes drawn around the moving object and both

disabled vehicles, including the one covered in snow.

Parking lots and Junkyards (DO NOT ANNOTATE)

Figure 27: The image on the left contains a parking lot that has many vehicles parked and

annotated, this specific parking lot must not be annotated since these vehicles will not move. The

image on the right shows how it should look.

Residential Areas

Figure 28: THe image on the left contains some cars in a Residential Area. These areas should not

be labeled. The Image on the right shows no annotations as how it should look

Correct Examples

Show examples (10)

Common Errors

Show examples (16)

BackContinue
Remotasks