INTRODUCTION TO AUTONOMOUS SYSTEMS

LESSON 1: INTRODUCTION.
- Autonomous system: system that has the capacity and freedom to choose its own actions
and behaviour independently of direct human control.
- An autonomous system can:
 Perceive its environment.
 Operate robustly on its owns means and capacity over long periods of time.
 Learn to adapt and develop in changing situations.

1. Key challenges.
- Operation challenges:
 Safety.
 Reliability.
 Critical decision making.
- Ethical challenges:
 Dilemas (train dilemma).
 Alignment with societal norms: incorporating ethical principles.
 Social effects.

2. Misconceptions.
1.1 Autonomy is unidimensional:
- REALITY: Autonomy is muli-dimensional and encompasses various aspects of decision-
making:
 Decision-making.
 Perception.
 Learning.
 Action.
1.2 Autonomy levels are reliable.
- REALITY: Not all aspects of autonomy are equal, and they are rated differently on the
autonomy scale.
- All aspects don’t matter equally, rated higher or lower in the autonomy scale.
- Activities are parallel and not sequential.
- SAE levels: Defines the defree of automation in vehicles.
 Level 0 (No Driving Automation):
Description: Vehicles at this level are manually controlled by humans, with no automation
involved in the driving task.
Example: Traditional, manually operated vehicles. Systems like emergency braking, while
assisting the driver, don't qualify as automation at this level.
 Level 1 (Driver Assistance):
Description: The vehicle has a single automated system for driver assistance, such as steering
or acceleration. The driver monitors other aspects of driving.
Example: Adaptive cruise control, where the vehicle can maintain a safe distance behind
another car, with the driver handling steering and braking.
 Level 2 (Partial Driving Automation):
Description: Advanced driver assistance systems (ADAS) allow the vehicle to control both
steering and acceleration/deceleration. The driver can take control at any time.
Examples: Tesla Autopilot and Cadillac Super Cruise are Level 2 systems.
 Level 3 (Conditional Driving Automation):
Description: Level 3 vehicles have environmental detection capabilities and can make
informed decisions. However, they still require human override, and the driver must remain
alert.
Example: The 2019 Audi A8L with Traffic Jam Pilot, though its Level 3 functionality is not fully
utilized in the U.S. due to regulatory constraints.
 Level 4 (High Driving Automation):
Description: Level 4 vehicles can intervene in case of system failure, and they don't require
constant human interaction in most circumstances. They operate in self-driving mode within
specific areas (geofencing).
Examples: NAVYA shuttles, Waymo's self-driving taxi service, and collaborations like Magna
and Lyft for Level 4 capabilities.
 Level 5 (Full Driving Automation):
Description: Level 5 vehicles do not require human attention for any driving tasks. They lack
steering wheels and pedals and are not bound by geofencing.
Status: Currently in testing, with no Level 5 autonomous vehicles available to the general
public yet.
 1.3 Autonomous Systems are autonomous.
- Any system can appear autonomous if the task is confined enough.
- All machines are supervised by humans to some degree.
- Edge cases always require some user supervision.

1.4 Once achieved, full autonomy obviates the need for human-machine collaboration.
- REALITY: Even in fully autonomous systems, collaboration with humans still be necessary for
various reasons.
1.5 As machines acquire more autonomy, they will work as simple substitutes (or
multipliers) of human capability.
- REALITY: Full autonomy doesn't eliminate the need for collaboration; instead, it may require
a different form of collaboration.

3. Components of AS.
3.1 Sensors and Actuation.
- Sensors: Serve as the sensory organs.
 LiDAR: Provides high-resolution 3D mapping of the surroundings. BUT higher cost
and computational cost.
 Cameras: Capture visual data for interpretation.
 Radar: Uses radio waves to detect objects and their movements.
 IMUs: Measure acceleration, angular rate, and sometimes include GPS for location
data.
- Actuation: Mechanical orchestration.
 Motors.
 Servos.
 Mechanisms.
3.2 Locomotion and its low-level control.
- Locomotion Modes are Tied with Application/Environment: the choice of how a robot moves
is closely connected to the specific purpose it serves (application) and the conditions in
which it operates (environment).
- Different locomotion modes, such as wheels, tracks, aerial capabilities, or legs, are selected
based on the requirements of the task a robot needs to perform.
 Wheeled Robots:
- Traction vs Speed must balance the need for sufficient traction, ensuring grip on surfaces,
with the desire for speed in their movement.
- Trade-offs: The level of traction affects slippage, which, in turn, can impact the accuracy of
movement.
 Tracked Robots:
- Skidding leads to Computational Costs:Tracked robots, while offering stability, may
experience skidding, leading to additional computational costs for correction and control.
 Aerial Robots:
- Operation Time vs Capabilities: need to find a balance between the time they can operate
and the capabilities they can offer.
- Maneuverability in the air must be weighed against the overall efficiency and operational
time of the aerial vehicle.
  Legged Robots:
- Smooth Motion vs Adaptability to Different Terrains: Achieving smooth motion requires
balancing it with the adaptability required to navigate different and challenging surfaces.
- Specifically for UAV control (Unmanned Aerial Vehicles):
 Physics and Aerodynamics.
 Modeling and Control:
- Differential equations and matrix transformations.
- Model predictive control (MPC): A technique that optimizes performance by
considering the predictive model of the system.
 Control:
- Proportional Integral Derivative (PID) Control:
- Adaptive Control:Real-time adjustment of control to adapt to changes in flight
conditions.
- Optimal Control Techniques: such as MPC, to achieve efficient and precise
performance.

3.3 Perception.
- Sensor selection and data acquisition
 Sensor Selection: Choosing appropriate sensors (such as cameras, LiDAR, radar, etc.)
based on the specific requirements of the perception task.
 Data Acquisition: Gathering data from selected sensors to capture information
about the environment.
- Data processing and fusion:
 Data Processing: Analyzing and interpreting the raw data collected from sensors.
 Data Fusion: Integrating information from multiple sensors to create a
comprehensive and accurate representation of the surroundings.
- Object detection and tracking:
 Object Detection: Identifying and recognizing objects in the environment, often
using computer vision techniques.
 Tracking: Following the movement of detected objects over time, enabling the
system to understand their trajectories.
- Scene understanding and mapping:
 Scene Understanding: Interpreting the context of the environment, including the
relationships between objects and their significance.
 Mapping: Creating a spatial representation of the environment based on the
perceived data.
- Self-localization: Determining the precise location of the perception system or the vehicle
within the mapped environment.
- Deep learning and perception: Leveraging deep learning techniques, such as neural
networks, for more advanced and intricate perception tasks.
- CHALLENGES:
 Lighting conditions.
 Weather conditions.
 Computational limits.
3.4 Sensor fusion.
- No single sensor provides an all-encompassing perspective.
- Combining information from multiple sensors enhances the overall understanding of the
environment.
- Data alignment and calibration:
 Self-Supervised: The system internally adjusts and aligns sensor data.
 External Target: Calibration using external references or targets for alignment.
- Sensor data fusion techniques: Methods and algorithms for integrating data from different
sensors.
- CHALLENGES:
 Resolution.
 Accuracy.
 Timing.
 Noise description.
3.5 Mission and behavioural planning.
- Mission planning: Establishing the overarching goals and tasks that the autonomous system
is designed to achieve.
- Behavioral planning: Adapting the system's behavior based on real-time changes in the
environment and handling unexpected situations.
- Path planning: Determining the optimal path for the autonomous system to follow,
considering obstacles, constraints, and the mission objectives.
- Trajectory generation: Creating a detailed, time-parameterized path for the system to follow,
often considering dynamic aspects of the environment.
- Decision making:
 Finite state machines (FSM): A computational model used to represent different
states of a system and transitions between those states based on certain conditions.
 Behavior trees: Hierarchical structures used to model complex decision-making
processes.
 Markov decision processes (MDP): A mathematical framework used for modeling
decision-making problems, where an agent makes decisions in an environment.
- Reactive vs. Deliberative planning:
 Reactive Planning: Rapid decision-making based on immediate sensor inputs and
real-time feedback. Well-suited for dynamic environments.
 Deliberative planning: In-depth decision-making involving strategic analysis and
planning, often considering a broader context.
- Reinforcement learning: A machine learning paradigm where an agent learns to make
decisions by interacting with an environment, receiving feedback in the form of rewards or
penalties.
3.6 Learning Based Autonomy.
- Data-Driven: Autonomous systems leverage data to learn and improve their performance
over time. Data-driven approaches involve training models on large datasets to enable the
system to make informed decisions.
- Inference Models for Learning-Based Perception: Allow the system to perceive and
understand its environment through learning.
- Probabilistic and Uncertain Operation:
 Bayesian Networks: To model and analyze probabilistic relationships among variables in
the system, allowing for reasoning under uncertainty.
 Probabilistic Inference: Applying probabilistic reasoning to make decisions, considering
uncertainties in the environment or sensor measurements.
- Bioinspired Learning:
 Spiking Neural Networks: Replicating the spiking behavior observed in biological
neurons, which can contribute to more realistic and biologically-inspired
computational models.
- Transfer Learning and Domain Adaptation:
 Transfer Learning: Adapting knowledge learned in one context to another related
context, improving the efficiency of learning in new environments.
 Domain Adaptation: Adjusting a model trained in one domain to perform effectively
in a different, but related, domain.
3.7 Fault diagnosis.
- Refers to the process of identifying, analysing, and responding to deviations or abnormalities
in the behaviour or performance of the system.
 Identification of deviations: Analysing the data provided by the sensors to identify
any deviations.
 Model-Based Approaches: Mathematical or computational models that represent
the expected behaviours of the system under normal circumstances.
 Dara-Driven Approaches: Analyse historical and real-time data to detec patterns or
anomalies.
 Decision Logic and Confidence Estimation: Implements rules to guide the decision-
making process and assesses the level of certainty or confidence in the diagnosis.
- The integration is made by:
 Adjusting control inputs.
 Rerouting mission plans.
 Triggering safety protocols.
LESSON 2: SENSOR AND ACTUATOR INTEGRATION.
1. Sensor properties.
- Sensitivity: High sensitivity will exhibit noticeable changes in output for small changes in
input.
- Influence by noise or disturbances: they affect the accuracy of the sensor readings.
- Non-linearity with respect to the measurement range.
- Resolution and quantisation errors: Higher resolution generally results in more accurate
measurements.
- Internal sampling and aliasing: Internal sampling refers to how frequently the sensor
samples or measures the input. Aliasing occurs when the sampling rate is too low, leading to
inaccuracies in representing signals.

2. Common sensors in physical AS.

2.1 Distance sensors: the measure the distance between the sensor and an object.
- Ultrasonic sensors: Uses sound waves to determinate the distance.
- LIDARs: Uses laser light to measure the distance.
- RADARs: Use radio waves to detect presence, distance and speed.

2.2 Visual sensors:

- Cameras for texture information.

2.3 GNSS/GPS.
- Provide accurate location and positioning information by receiving signals from satellites.

2.4 Orientation sensors.

- Inertial measurement units (IMUs): Combine accelerometers and gyroscopes to measure
acceleration and angular velocity.

3. Analog to Digital representation.

3.1 Analog representation:
- Time-varying.
- Represents physical values like temperature, velocity and weight.

3.2 Digital representation:

- Based on two fixed values: 0 and 1.
- Used in computers and microcontrollers.
-
3.3 ADC quantification.
- Process of converting continuous analog signals into digital representation.
- This will result in quantification errors due to the finite resolution of the converter.
- Step-size (Q):

- Increasing the number of bits, we increase the resolution which will reduce the step-size and
the quantification error. BUT it also increases the complexity of the ADC.
4. System integration.
- Is the process of interconnecting different sub-components to create a full system.
- Considerations:
 Levels of importance.
 Robustness against faults.
 Timing in control loops.

5. System integration.
5.1 Publish/subscribe interconnection.
- The components communicate through a publish/subscribe pattern.
- A middleware is used to facilitate the communication.
- Example: ROS is an example of a middleware.

5.2 Raw intercommunication.

- Direct connection using TCP/IP sockets.
- Involves a client and a server.
 Client: Iniciates connection by specifying the IP and port.
 Server: Exchanges information/data.
LESSON 3: MOBILE ROBOT LOCOMOTION.
1. NASREM software model.
- It’s a software model designed for autonomous robots.

1.1 Sensor processing: Collects and process the data.

- Situation Sensor: Detects the robot's current situation.
- Obstacle Sensor: Identifies obstacles in the robot's path.
- Pose Sensor: Determines the robot's position and orientation (pose).

1.2 World model: The processed sensor data is used to update the robot's internal model of
the world, which consists of:
- Situation State: The robot's interpretation of its current situation based on sensor
data.
- Obstacle Map: A map indicating the location of obstacles.
- Robot State: The status of the robot including its pose, velocity, etc.

1.3 Behaviour Generation: Based on the world model, the robot generates behaviours or
actions to achieve its mission objectives. This involves:
- Behaviour Decision: Selecting the appropriate behaviour to execute.
- Obstacle Avoidance: Specific behaviours to navigate around obstacles.
- Drive Control: The component that directly controls the robot's movement.
LESSON 4: UNMANNED AERIAL VEHICLES (UAV).
1. Classes of UAVS
- Based on landing:
 HTOL: Horizontal takeoff and landing.
 VTOL: Vertical takeoff and landing.
- Based on aerodynamics:
 Fixed wings.
 Flapping wings.
 Ducted fan.
 Multi-rotor: Tricopter.
- Multi-rotor:
 Trocpter.
 Quadcopter.
 Hexacopter.
 Octacopter.

2. UAVs flight mechanics.

- A rotating propeller generates a lift force and a drag torque.
- Flight physics include:
 Lift Force: Arises from propeller rotation.
 Drag Torque: Opposes propeller rotation.
 Force Vectors: Dictate overall UAV movement.
 Zero Lift Line: Indicates no lift generation.
 Flow Vectors: Affect lift and drag.

- Propeller speed influence: The force and torque generated are directly proportional to the
square of the propeller's rotational speed.
- Position is maintained by balancing gravity and torque.
- Altitude changes via:
 Increase propeller speed to ascend.
 Decrease propeller speed to descend.
- Directional control achieved by varying the speed in opposing propellers.
Forward Backward Left Right
 3. Control mechanism in UAVs.

- For controlling the attitude and the position we need to measure both of them.
- IMPORTANT: quadrotor is underactuated. We cannot move laterally, unless we tilt. This
means that in order to control the position of the quadrotor, we first have to control its
attitude.

Underactuation means that the system must use dynamic interactions (such as tilting or pitching) to
achieve certain movements since it doesn't have direct control over all possible motions.

4. Quadrotor hardware components.

1. Actuators: Electric motors.
1.1 DC brushed motors.
- Easy to control.
- Simple electronics.
- When small dimensions and low electronic complexity is needed.
- Operated using a H-bridge.

1.2 Brushless motors.

- 3 electric phases.
- Complex control.
- Higher mechanical durability and power density (they give more power).
- Operated using an ESC.
- 6 transistors regulate the voltage.
- Sensors or voltage monitoring can determine the position of the rotor.

2. Sensory systems:
2.1 IMU (Inertial Measurement Unit): for orientation and movement tracking.
- Gyroscopes:
 Measure rotational velocity in radians per second across three axes.
 Provide orientation data.
 Subject to drift if not corrected by kinematic models and algorithms: This
means they can make small mistakes over time (drift), but we have clever
methods (kinematic models and algorithms) to correct these mistakes and
keep our measurements accurate.
- Accelerometers:
 Measure acceleration in meters per second squared (m/s²) in three
dimensions (x, y, z).
 Detect changes in velocity due to user movement.
- Compass:
 Provides magnetic heading information in radians.
 Essential for navigation and maintaining directional control.
 2.2 Ultrasound sensors.
- Operate above human audible range (>40 kHz) using sound pulse echoes.
- They receive and evaluate the properties of the echo pulse.
- Working principles:
 Time of flight: Measures distance based on echo return time.
 Doppler shift: Measures object’s velocity via frequency shift.
 Amplitude attenuation: Evaluates distance, directionality, or attenuation by
measuring pulse intensity.

2.3 GPS (Global Positioning System).

- Gives information about your location and the current time.
- To work accurately, it needs a clear line of sight to at least four GPS satellites.

2.4 Visual sensors.

- RGB Camera:
 Captures light in the visible spectrum.
 Image distortion affected by optics.
 The distortion can be corrected by calibrating the camera.
- RDBD Camera:
 Utilizes an IR pattern for depth estimation.
 Compares captured IR image to reference patterns to minimize projection
error.

3. Navigation, guidance and control.

3.1 Navigation:
- Determining the current and future position and orientation through measurements.
3.2 Guidance:
- Establishes the desired trajectory for the aircraft to follow in real-time and for future
manoeuvres.
3.3 Control:
- Implements actions to align the aircraft's current state with the planned trajectory as
determined by navigation and guidance.
LESSON 5: IMAGE AND VIDEO CODING.
1. Image acquisition. Requirements and system architecture.
- Navigation: Essential for proper orientation and location awareness during image
acquisition.
- Video transmission: Enables real-time or near-real-time communication of video
information. Prioritizes achieving low latency/delay in transmitting video data.
- Video compression: Reduces the size of video data for efficient storage and transmission.
- Image analysis: Extracts valuable information, identifies patterns, or performs specific tasks
based on the content of the images.

2. Lossless image compression.

- Compression technique that reduces the size of an image file without losing any information.
- Retains the original image quality and allows for exact reconstruction.
- Suitable for situations where preserving every detail is crucial.
2.1 Sample:
- Samples are the individual pixel values or elements of the image.
2.2 Prediction (Transform):
- Predictions estimate a pixel's value based on neighboring pixels.
2.3 DPCM (Differential Pulse Code Modulation):
- DPCM encodes the difference between predicted and actual pixel values (residuals).
- The differences (residuals) are smaller and better for compression.
- Encoder:
 Calculates the residuals, encodes them and applies entropy coding (Huffman)
2.4 Reconstructed Sample:
- After compression and decompression, you get the reconstructed sample.
- It's generated by adding decoded residuals back to the predicted values.
- Decoder:
 The decoder decompresses image data.
 It reverses the encoding process, decoding residuals, and reconstructing the
original image.
 3. Lossy image compression.

- Process that achieves higher compression by sacrificing some details.

- For example, JPEG.

3.1 Key components.

- Transform (DCT - Discrete Cosine Transform):
 Function: Converts spatial image data into frequency components.
 Process: Decomposes the image into different frequency patterns.
 Result: Represents the image in a way that high-frequency components can be
efficiently compressed.
- Quantization (Uniform):
 Function: Reduces the precision of the frequency components obtained from
DCT.
 Process: Divides the frequency spectrum into intervals and assigns the same
value to all components within each interval.
 Result: Achieves compression by discarding less noticeable details, exploiting
human perceptual limitations.
- Entropy Coding (Huffman):
 Function: Further compresses the quantized frequency components.
 Process: Assigns variable-length codes to different frequency components
based on their probabilities of occurrence.
 Result: Optimizes the representation of data, assigning shorter codes to more
frequent components, reducing overall bitstream size.

DIFFERENCES BETWEEN BOTH COMPRESSIONS:

- Lossless: Retains all original information, resulting in a perfect reconstruction.
- Lossy: Sacrifices some details for higher compression, leading to a reconstructed image that
approximates the original.
 4. Discrete Cosine Transform (DCT).
- It takes a block of pixel values and transforms them into a set of coefficients.
- These coefficients represent how different spatial frequencies are present in the block.
- Higher coefficients capture important image details, while lower ones represent less
important details.
- DCT helps in compactly representing image information, making it suitable for compression
techniques like JPEG.

5. Quantization.
- Process of reducing the precision of the coefficients obtained during the DCT.
- It involves rounding off the coefficients to a smaller set of values.
- This is done to compress the image and reduce the amount of data needed to represent it.

5.1 Quantization matrix (QM).

- The quantization process is guide by the QM.
- It specifies the amount of quantization to apply to each coefficient position.
- Higher values in the matrix lead to more aggressive quantization, resulting in a greater
loss of detail, while lower values lead to less quantization and better image quality.
- By adjusting the values in the quantization matrix, we can control the trade-off
between image compression (smaller file size) and image quality.

6. Entropy.
- Entropy (H(X)) quantifies the average self-information of independent events in a set A.
- This means that it represents the minimum amount of space needed to store or compress
information from a source that generates random symbols. The smallest file size we can
achieve when saving this information without any loss.
- Formula:

6.1 Constant Length Coding:

- Always uses the same number of bits to represent each symbol.
- Uniform, fixed-sized encoding for all symbols.
6.2 Variable Length Coding:
- Uses different numbers of bits to represent different symbols.
- Allows more efficient encoding by using shorter codes for more frequent symbols and
longer codes for less frequent ones.

7. Huffman coding.
- A type of variable length coding.
- It gives shorter binary codes to more common symbols and longer codes to less common
ones.
- By using fewer bits for common symbols, it efficiently compresses data.
- It forms a binary tree where each symbol has a unique path from the root to a leaf node.
- The tree is built based on symbol frequencies, with common symbols closer to the root for
quicker representation.
 8. MPEG video coding.
- This compression is crucial for efficient storage and transmission of video content.
- It offers a balance between video quality and compression efficiency.
- The MPEG video coding process involves several key steps:
8.1 Motion Estimation:
- Definition: Analyzing the movement of objects between successive frames to create motion
vectors.
- Motion Vectors: Each vector represents the shift of an object or block from one position to
another in the image sequence.

8.2 Motion-Compensated Prediction:

- Utilization of Motion Vectors: Leveraging motion vectors to predict the current frame from
previous and/or subsequent frames.

8.3 Residual Compression:

- Residual Calculation: Determining the residual signal by subtracting the prediction from the
current image.
- Residual Compression: Compressing the residual, which signifies the difference or error
between the predicted and actual frames.
LESSON 6: MULTIMODAL SENSOR FUSION.
1. Multimodal sensor fusion: combining diverse sensor data.
- Sensor Fusion: Combines data from various sensors to create new or refine existing data.
- Multimodality: Involves using different types of sensors that measure diverse data types.
- Sensor Fusion Benefits: Reduces uncertainty in all sensor types by combining their data.
- It is used for solving problems like localization, navigation, and target tracking.

1.1 Triangulation: Sensor fusion example.

- Vessel equipped with both radar and a camera.
- RADAR'S CONTRIBUTION: provides accurate information about the distance to an object.
- CAMERA'S CONTRIBUTION: provides information about the direction or bearing of an
object. Not as precise in determining distances.
- To fuse these measurements, we use a mathematical technique called weighted least
squares. It assigns different weights to the measurements based on their accuracy.
- More weight is given to the accurate range measurements from the radar. This means the
system relies more on the precise distance information provided by the radar when
determining the position of an object.

1.2 Internal navigation.

- Estimates a vehicle's position, orientation, and velocity using measurements of linear
accelerations, velocities, and angular velocities.
- These estimations start from the knowledge of an initial position and orientation.
- Dead-Reckoning:
 Fundamental concept in inertial navigation.
 The position and orientation of the vehicle is available only at the initial time, and
they are updated based on measured velocity, heading and elapsed time.
 Keeping track of position and orientation as you move forward.

Dead-Reckoning is like navigating without GPS or landmarks. You start from a known
point, keep track of your movements, and estimate your current position based on those
movements. It's an ongoing process, but without external checks, there's a potential for
accumulating errors over time.

 Sensors Involved:
 IMU (Inertial Measurement Unit): Measures accelerations and angular
velocities.
 Doppler Velocity Log: Measures the velocity of the vehicle.
 Magnetic Compass/Gyro Compass/Satellite Compass: Provides heading
information.
 Uncertainty: Because you're not checking your position against any external
references, there's a chance of accumulating errors.
- Aided inertial navigation:
 Combines measurements of position and heading with previous measurements at
regular intervals.
 Additional sensors involved:
 IMU (Inertial Measurement Unit): Continues to measure accelerations
and angular velocities.
 GNSS (Global Navigation Satellite System): Provides global position
information.
 Doppler Velocity Log: Continues to measure velocity.
 Magnetic Compass/Gyro Compass/Satellite Compass: Continues to
provide heading information.
1.3 Localization.
- Figuring out exactly where a vehicle is in a specific place by aligning its motion with a local
map or reference frame.
- Map available a-priori:
 The position and orientation of the vehicle is estimated with respect to static
landmarks whose position is known and given in the map.
 Sensors involved:
 Map (the known points).
 IMU (Inertial Measurement Unit).
 Doppler Velocity Log.
 Magnetic compass/Gyro compass/Satellite compass.
 RADAR.
 LIDAR.
 Cameras.
 Ultrasonic distance sensor.
- Simultaneous Localization and Mapping (SLAM):
 When there’s no map or only a partial map.
 The vehicle builds or refines it while moving, estimating positions of fixed points
using sensors.
 Sensors involved (same as above):
 Map (the known points).
 IMU (Inertial Measurement Unit).
 Doppler Velocity Log.
 Magnetic compass/Gyro compass/Satellite compass.
 RADAR.
 LIDAR.
 Cameras.
 Ultrasonic distance sensor.
1.4 Target tracking.
- Predicts future positions and properties of moving objects based on measurements, motion
models, and knowledge of the observer's own motion.
- A-priori knowledge of own pose and velocity: assumes known or estimated position,
velocity, and orientation.
- Motion models: Hypotheses about how the target object is moving over time.
- Target measurements: Range and bearing measurements for position, velocity, and
orientation estimation.
- Sensors involved (same as above):
 Map (the known points).
 IMU (Inertial Measurement Unit).
 Doppler Velocity Log.
 Magnetic compass/Gyro compass/Satellite compass.
 RADAR.
 LIDAR.
 Cameras.
1.5 Bayesian framework for sensor fusion.
- Bayestian filtering: mathematical framework for combining data from various sensors,
providing a systematic way to estimate the state of a system over time.
- Based on Bayes' theorem, a rule for updating probabilities or beliefs when new data is
obtained:

 P(A): Prior probability - the belief about event A before considering new evidence.
 P(B|A): Likelihood function - the probability of observing evidence B given that A is
true.
 P(A|B): Posterior probability - the updated belief in A after taking into account the
evidence B.

1.5.1 Maximun a posteriori estimate (MAP).

- Method to estimate the most probable state of a system based on both current data
(likelihood) and prior information (a priori).

- We combine the current information in the likelihood function p(y|x) with the a
priori information p(x).
- By letting the prior p(x) come from past observations, a natural recursive algorithm is
obtained with two stages:
 Measurement update: here the prior is updated based on the current
information.
 Time update: a new prior is computed based on the Markov model and the
current estimate.
- Optimal Solution: The Kalman filter is the optimal solution to the Bayesian filtering
problem when the system's model is linear, and uncertainties follow a Gaussian
(normal) distribution.
In Simple Terms:

Step 1: We start with an initial belief about the state (prior) based on past observations.

Step 2: In the measurement update, we refine our belief using the current observation. This is like
adjusting our expectations based on the new information.

Step 3: In the time update, we predict the future state based on the current estimate and system
dynamics. This is like anticipating how the system will evolve over time.

2. Kalman filter.
2.1 Discrete-Time Kalman Filter (DTKF):
- Optimal technique used to estimate the state of a system by intelligently combining
measurements with predictions.
- Relies on Bayesian filtering.
- Two stages:
 Measurement Update: In this stage, the filter incorporates the most recent
measurement data to adjust and refine the current estimate of the system's state.
 Time Update: In this stage, the filter predicts the future state of the system based on
its current estimate and the known dynamics of the system.

2.1.1 Example: Tracking the position of a moving car:

- Step 1 - Initialization: Begin with an initial estimate of the car's position and velocity. This
could be based on the last known state or an initial assumption.
- Step 2 - Prediction (Time Update): Project where the car will be in the next time step using its
current state and knowledge of its motion model (e.g., constant velocity).
- Step 3 - Measurement (Measurement Update): Receive a new measurement, such as a GPS
reading, which is inherently noisy.
- Step 4 - Kalman Gain Calculation: Calculate the Kalman Gain, a crucial factor that determines
how much weight should be given to the prediction versus the measurement.
- Step 5 - Update Estimate (Posterior Estimate): Adjust the car's position estimate based on
the Kalman Gain and the difference between the predicted and measured values.
- Step 6 - Repeat: Repeat this process for each new time step, continually updating the
estimate based on the latest measurements and predictions.

Result:

- Over Time: As the Kalman Filter iteratively processes new data, it effectively mitigates the
impact of noise in the measurements and provides a refined and accurate estimate of the
car's position.
- Noise Handling: The filter's ability to adapt and weigh measurements based on their
reliability allows it to maintain accuracy even in the presence of noisy data.
3. Multi-Modality for coastal target tracking
- The use of multiple sensing modalities (multi-modality) is employed for tracking targets in
coastal areas, providing a more comprehensive and accurate understanding of the
environment.
- Correlating data from RADAR, cameras, and LIDAR checks RADAR reliability.
- Challenges:
 Precise calibration needed.
 Advanced machine learning, like deep fusion, helps fuse data without prior sensor
calibration knowledge.
LESSON 7: AUTONOMOUS SYSTEMS PERCEPTION WITH LIMITED
DATA.
1. Data efficiency and transfer learning.
- Context: When there's limited data available, one approach is transfer learning.
- Mechanism:
 Pre-training: Initiate training a neural network on a large, diverse dataset like COCO,
captures general concepts of objects and class differences.
 Fine-tuning: For your specific task with limited data, fine-tune the pre-trained
network. While the core features (backbone or feature extractor) stay fixed, adjust
the classification layers to align with your specific task.
- This method optimizes the use of available data by leveraging knowledge gained from a
broader dataset, reducing the need for an extensive task-specific dataset.
 Transfer learning involves pre-training a neural network on a large dataset like COCO.
 The initial training captures general concepts of objects and class differences.
 Fine-tuning occurs when applying this pre-trained network to a new problem or dataset.
 During fine-tuning, the backbone or feature extractor remains unchanged, and only the
classification layers are adapted.
 This approach leverages pre-learned features, requiring less data to achieve comparable results
since the network already understands general concepts.

2. Few-Shot learning: Learning with very little data.

- Few-shot learning helps a network adapt to new tasks with very little data.
- One method is Prototypical Networks:
o It uses average "prototypes" for each class.
o New data is classified based on how close it is to these prototypes.
- The network learns to adapt by practising with different tasks in training episodes.
LESSON 8: OPERATION OF AUTONOMOUS SYSTEMS IN AN
UNCERTAIN WORLD.
1. How do you structure autonomy?
- Rule-based systems.
- Hierarchical task networks.
- Skill-based systems.
- Behaviour trees.
- Learning-based autonomy.

2. Rule-based Systems:
- Use IF-THEN setup (specific actions are triggered if certain conditions are met).
- Pair situations (IF) with actions (THEN).
- Common in simpler AI systems (e.g., tic-tac-toe).
- Relies heavily on expert knowledge.

3. Hierarchical Task Networks (HTN):

- HTN involves creating plans with tasks.
- It breaks down down high-level tasks into smaller ones recursively until simple action
primitives.
- HTN method is defined by a 4-tuple: m=(name, task, subtasks, constraints). This HTN
structure helps in organizing and planning complex tasks hierarchically, making it more
manageable for robotic systems.

4. Skill-based approaches:
- Simplify complex actions by breaking them into smaller, understandable parts.
- Used in collaborative robots, especially in industries.
- Robots perform actions that can be customised for specific tasks.
- Programming involves arranging these actions.
- Includes checking prerequisites, performing the action, and verifying outcomes.

5. Behavior trees:
- Function like decision trees for behaviour execution.
- Execution order determined by leaf composition (usually left to right).
- Organize how a system or agent switches between behaviours.
- Updated recursively at specified frequencies (ticks).
- Node Types:
 Root Node: The initial state of the program with exactly one child.
 Control Flow Nodes:
 Sequence: Execute children left to right until one returns running or
failure, only returning success if all children succeed.
 Fallback: Execute children left to right until one returns success or running,
only returning failure if all children fail.
 Parallel: Execute all children concurrently, returning success only if a
specific number of children succeed.
  Execution Nodes:
 Action: Execute complex operations, returning success, failure, or running.
 Condition: Perform simple checks, returning success or failure.
 Decorator: Modify the child's return value based on custom policies.

6. Learning-based Approaches - Reinforcement Learning:

- Uses a reward signal to evaluate performance after actions.
- Objective: Maximize the value function (V) by optimising the policy.
- Challenges:
 Balancing Exploration vs. Exploitation: Deciding between trying new states and
exploiting known actions.
 Sample Efficiency and Training Time: Often requires extensive trial-and-error
before meaningful learning.
 Designing Reward Functions: Determining how to define rewards for tasks like
pick-and-place or human locomotion.
 Addressing Continuous Functions vs. Sparse Rewards: Handling situations where
rewards are not easily obtained.

7. Robot Operating System (ROS):

- Widely used framework for robots.
- Offers many ready-to-use tools.
- Flexible and modular.
- You can replace parts with your own code.
LESSON 9: LEARNING-BASED SITUATIONAL AWARENESS.
1. Robot localization.
1.1 Frames and transformations.
- A vector represents a point.
- The reference frame is crucial: Frames define both translation and rotation.
- We use indices to indicate which frame is relevant:
 For points, it's the frame where the point is defined.
 For transformations, it's the pose of one frame relative to another (from A to
B).
- The strategy is to follow the arrows.
- Pose transformations can be combined sequentially.
- They can also be reversed by taking the inverse.

1.2 Laser based solution.

- We can form a nonlinear solution for the Rotation R, translation T and Laser scans at
t0 and t1.
- Many solutions to this problem: ICP, polar velocities non-linear optimization…
- Iterative closest point (ICP):
 Matching Points: For each object point (p) in Model 1, find the nearest point
(q) in Model 2.
 Transformation Estimation: Use all pairs (p, q) to estimate the transformation
from Model 1 to Model 2.
 Applying Transformation: Apply the estimated transformation to the points
of Model 1.
 Iterative Process: Repeat steps 1-3 until convergence or a predefined stop
criterion is met.

1.3 Camera/3D based solution.

- Visual-inertial odometry (VIO):
 Estimates the state (pose and velocity) of an agent, like an aerial robot.
 Uses inputs from one or more cameras and one or more Inertial Measurement
Units (IMUs).
 Cameras are slow but provide information-rich data.
 IMUs are fast but have high noise.
 Two main filtering paradigms for estimation:
 Loosely coupled: Limited integration of visual information.
 Tightly coupled: Extensive integration of visual information into the
state.
 2. Disambiguation and Quick word on SLAM.
2.1 Pose-Landmark graph-SLAM.
- Represents the Simultaneous Localization and Mapping (SLAM) problem as a Bayesian
network graph.
 At each location (xt), the system observes nearby features in the map (m).
 Movement (ut) is also considered.

- Arrows in the graph indicate causal relationships between variables.

2.2 Graph-based SLAM.

- Uses a graph to solve the SLAM problem.
- Nodes on the graph represent robot positions or locations.
- Edges on the graph represent observed landmarks and movement information.
- The goal is to adjust node positions to reduce errors in observed data.

2.3 SLAM constraints optimization.

- When the robot observes areas it has seen before, it creates constraints between non-
successive positions.
- These constraints, both from robot motion and measurements, are like "soft" elastic springs.
- The goal is to minimise the overall energy in these springs, which means making the robot's
estimated path consistent with the observations it makes.
LESSON 10: FUNDAMENTALS OF CONTROL AND FAULT DIAGNOSIS
IN DYNAMICAL SYSTEMS.

1. Dynamical Systems.
- Autonomous systems are not static but dynamical systems:
 They have an internal behaviour.
 They respond to stimuli (inputs, disturbances, initial conditions).
 They produce outputs (measurements).
1.1 Anatomy.
- System States (x):
 Variables representing the system's condition or behaviour.
 Examples: Position, temperature, speed, etc.
- Control Inputs (u):
 Deliberate signals or actions applied to the system to influence its behaviour.
 Examples: Force, voltage, current, etc.
- Exogenous Inputs (w):
 External factors affecting the system that are not under direct control.
 Examples: Wind, gravity, market fluctuations, etc.
- System Outputs (y):
 The measurable responses from the system resulting from various inputs.
 Examples: Position, velocity, pressure, etc.
We need to be able to calculate how their states will change given their current values and the set
of inputs.

1.2 State-Space Representation.

- System of ordinary differential equations (ODE) that relate, derivatives (rate of change) of
states, the states and the inputs.

1.3 Simulating dynamical systems.

- We use state-space models to predict and/or control the future.
- Euler's Method: A first-order numerical procedure for solving ordinary differential equations
(ODEs) with a given initial value.
 2. Feedback Control.
- Elements of feedback control:

 State-Space Models: Tools to describe the dynamics of systems in terms of their states
and inputs.
 Control Input (u): The mechanism we use to influence or shape the system's
behaviour towards a desired outcome.

3. Fault diagnosis in dynamical systems.

3.1 Why?
- Personal Safety.
- Machine Protection.
- Availability.
- Maintenance.
- Environmental Protection.

3.2 What?
- Fault: A deviation from the expected or nominal operation of the system.
- Change in System Equations: A fault typically alters the structure or parameters within the
system's governing equations.
- Manifestation in Signals: Faults are reflected in the system's output signals and the
behaviour of its equations.

3.3 How?
- Fault Detection: The process of determining the existence of a fault within the system.
Compute the same thing from several independent sources and check.
- Key element: Information redundancy.

What we learned in Lecture 10

1. What is a dynamical system and how it is mathematically described?

– It is a system that responds to stimuli and produces outputs.

– We use state-space representation (ODEs) to describe it.

2. How can we simulate a simple dynamical system in Matlab?

– Using Euler forward integration method.

3. What is feedback control?

– It is a way to shape a system’s dynamics by adjusting the input at each time instant based on how

far we are from reaching our “target”.

4. What is fault diagnosis and how do we represent a fault in dynamical systems.

– It is the way to find faults (violations of system’s equations) by using known signals and equations.

5. How can we design basic residual generators for fault detection?

– By applying the ranking algorithm and using the redundant (unmatched) constraints.

LESSON 11: BIOINSPIRED LEARNING AND CONTROL.

1. Neurorobotics:
- Combines neuroscience, robotics, and artificial intelligence (AI).
- Goals:
 Understand biological systems and the mechanisms of intelligent behaviour in
humans and animals.
 Abstract general principles of intelligent behaviour.
 Apply these principles to design useful artifacts.

2. Motor Control:
- Examines postures, movements, and their underlying mechanisms.
- Types of Motor Control:
 Closed-loop Motor Control: Involves slow, precise movements.
 Open-loop Motor Control: Deals with skills that require rapid execution.
2.1 Neurocontrol.
- Explores how open-loop voluntary movements are generated.
- Involves experiments like those with a decorticated cat to understand movement
without the cortex's involvement.
2.2 Brain regions involved.
- Cerebral Cortex: Involved in complex motor control and learning.
- Motor Cortex: Facilitates the planning, control, and execution of voluntary movements.
- Basal Ganglia: Involved in the regulation of movement initiation and the learning of
movements.
- Cerebellum: Coordinates fine motor control, balance, and motor learning.

3. Robot Learning
- A model describes essential predictive information about the behaviour of the environment
and the influence of an agent on this environment.
3.1 Purpose of Models in Robotics:
- Enable model-based control for high compliance.
- Manage the complexity of modern robots where standard models are insufficient.
- Address interactions with unpredictable and dynamic environments.
- Account for wear and tear that leads to model drift over time.
- Complex Body Models: Essential for realistic brain simulations and understanding brain-
environment interactions.
3.2 Characteristics of robot learning.
- Utilizes various machine learning approaches.
- Defined by the type of feedback, data generation process, and data type.
- The data type influences the robot learning approach used.
3.3 Modeling levels (Marr 1982):
- Goal Level (the what):Define the desired outcome (e.g., smooth robot arm movement).
- Algorithmic Level (the how): Choose the learning method (e.g., trial and error or
example-based). Decide on control strategy (e.g., reactive or pre-planned actions).
 - Implementation Level (the where and with what): Determine the learning environment
(e.g., simulation or real-world). Select the biological analogy (e.g., brain regions like
motor cortex or basal ganglia).
3.4 Model learning architectures.
- Forward Model:
 Predicts the next state of a system based on the current action and state.
- Inverse Model:
 Determines the action required to move from the current state to a desired future
state.
 Challenges arise when the motor command's complexity exceeds the desired
outcome's simplicity.
4. Computational models in motor control.
4.1 Feedback and Feed-forward Control
- Feedback Control:
 Acts on moment-to-moment sensory feedback to correct movements post-impact.
 Example: Adjusting the elbow angle after a ball impacts and perturbs it.
- Feed-forward Control:
 Essential for rapid responses, especially when sensory feedback is delayed.
 Prevents oscillations that can occur with delayed feedback.
 Relies on the Central Nervous System's (CNS) ability to predict outcomes of actions,
requiring adaptability and learning from experience.
 Need to be adaptive. Learns from experience.

5. Model learning architectures.

- Internal models:

 Mimic the body's responses to sensory input, adapting to changes and

disturbances.
 Replace external feedback in control systems.
- Learning Internal Models:
 Through supervised learning (from errors) or reinforcement learning (from
rewards and costs).
Cerebellar Theory of Learning: Applies these with feedback error learning (FEL)
and emphasizes the cerebellum's role in movement coordination, balance
stabilization, vision stabilization, and learning motor control for new situations by
signaling discrepancies between predicted and actual movement consequences.
5.1 Forward Models:
- Predict consequences of physical processes.
- Roles: control, planning, and sensory cancellation.
5.2 Inverse Models:
- Determine motor commands to achieve a desired sensory outcome.
- Challenges arise when the motor command's complexity exceeds the desired outcome's
simplicity.
5.3 Hybrid Control:
 - Combines feedforward (for fast, approximate targeting) and feedback (for precise
corrections) control, guided by internal models.

5.4 Learning methods in robotics.

- Learning from errors: Supervised learning.
- Learning from rewards and cost: Reinforcement learning.

Supervised Learning of Forward Models:

 System predicts action outcomes.
 Compares predictions with actual results.
 Adjusts to reduce prediction errors.
 Increases prediction accuracy over time.
Supervised Learning of inverse models.
 Combines feedback correction with feedforward planning.
 Starts with higher reliance on feedback error correction.
 Learns to minimize feedback over time.
 Gradually shifts towards accurate feedforward commands.

5.5 Cerebellar Control Theory.

- Cerebellum's Role:
 Maintains balance and coordinates movements.
 Stabilizes vision and learns motor control.
 Adapts movements to new situations, signaling discrepancies between predicted and
actual movement consequences.
- Cerebellar-Based Control:
 Functions as an inverse model, creating motor commands to achieve goals.
 Represents a forward model, storing motor primitives for coordinated movements like
intercepting a ball with a racket.
- Cerebellar Adaptive Filter Algorithm:
 Processes inputs and produces outputs through Purkinje cells.
 Uses an adaptive learning rule where the error between desired and actual output drives
the learning process. This is a mathematical representation of cerebellar learning.
What we learned in Lecture 11
 Motor Control (MC):
o Studies postures, movements, and their mechanisms.
o The cortex, basal ganglia, and cerebellum play roles in unsupervised, reinforcement, and
supervised learning, respectively.
 Robot Learning:
o Necessary due to environmental complexity.
o Utilizes models that provide predictive information.
o Forward models predict future states; inverse models specify actions to achieve those
states.
 Learning Internal Models:
o Can be done through supervised or reinforcement learning.
o Cerebellar theory of learning is applied using supervised learning for both inverse and
forward models, coupled with feedback error learning (FEL).
The cerebellar adaptive filter algorithm mathematically represents cerebellar learning.

What is motor control (MC)?

MC is the study of postures and movements and the mechanisms that underlie them.

What are the main learning roles of the cortex, basal ganglia, and cerebellum?

Unsupervised learning to establish relationships between sensory and action patterns.

Reinforcement learning to filter out unwanted movements.

Supervised learning of motor patterns and coordination.

How do we achieve robot learning?

Using a model that describes essential predictive information.

LESSON 12: SAFETY.
1. Definition of safety.
- Safety is defined as a judgment of the acceptability of risk.
- Risk is quantified as the probability and severity of harm to human health.
- An entity or system is considered safe if its associated risks are deemed acceptable.
- Importance of safety in Autonomous Systems: Ensuring that autonomous systems do not
pose undue risks to human health.
- Ensuring and Improving Safety: Assessing and managing the risks associated with
autonomous systems.
1.1 Safety as a Research Field in Autonomous Systems
- Safety Engineering: application of basic scientific and technical principles to the mitigation or
reduction of loss of life, property, and the environment by recognized hazards.
- System Safety: application of engineering and management principles, criteria, and
techniques to achieve acceptable risk.

2. Challenges in safety for Autonomous Systems.

- Current Standards Limitations: Existing safety standards may not fully cover autonomous
systems.
- Digitalization and increased automation bring new safety issues.
- Safety and risk assessment methods for autonomous systems are still evolving.

3. Directive standards and CE Marking

3.1 Machinery Directive (Directive 2006/42/EC).
- A European Union directive.
- Its main intent is to ensure a common safety level for machinery marketed or used in EU
member states.
- Promotes freedom of movement within the European Union.
3.2 CE Marking.
- Represents European Conformity.
- Indicates a product's compliance with health, safety, and environmental protection standards
within the European Economic Area (EEA).
- CE marking is recognizable worldwide and also found on products outside the EEA that are
manufactured in or designed for the EEA.
3.3 Standards Under the Machinery Directive.
- Risk assessment is a crucial part of the directive and must be properly performed and
documented.
- The directive may be too general for some products, so harmonized standards are used to
achieve conformity.
- Three types of standards:
 Type-A (basic safety standards).
 Type-B (generic safety standards).
 Type-C (machine-specific safety standards).
 4. Functional Safety and Safety Performance Level.
4.1 Functional Safety.
- General standard: IEC 61508.
- Objective: To achieve freedom from unacceptable risks of physical injury or health damage.
- Implemented through one or more automatic protection functions.
4.2 Safety Performance Levels (ISO 13849).
- Used to evaluate a safety function’s contribution to risk reduction.
- Levels range from L (low contribution) to H (high contribution).
- Performance Level required (PLr) is based on:
 Severity of injury (S1: slight, S2: serious or fatal).
 Frequency/exposure to hazard (F1: seldom-to-less-often, F2: frequent-to-
continuous).
 Possibility of avoiding hazard or limiting harm (P1: possible under specific conditions,
P2: scarcely possible).