Welcome to the mid-term presentation of my bachelor thesis.

My name is Nico Wertli and I’m

working on laser-guided mobile manipulation.

Project Description

What is it all about?

This project aims to develop an intuitive way to guide robotic construction tasks, for example
drilling holes. A construction worker sets up a laser marker and specifies the desired hole
dimensions. The SWACO robot should then execute the task fully autonomously.
The kinova end-effector is equipped with one color- and two monochrome-cameras. However,
for this bachelor thesis, I’m only using the color camera.

Image Processing

To start off with computer vision, I wrote a first software module that analyses the footage.
The image on the top left corner represents the input feed. The image processing breaks down
into a red filter mask, a canny edge detector and a Hough transformation to detect a good
match of the actual laser lines.

Line Triangulation
Now what will we do with these line segments?
In order to determine the three dimensional position [next] we triangulate the lines found of
two convenient perspectives. The position of the laser line can be estimated by intersecting the
two epipolar beams. [next] Displayed in red you can see the actual laser cross and in blue our
estimation. This approximation will be used as a first guess to initialize a state estimator.
Next steps

The plan for my next steps is to define a cost function. Therefore, we project our estimation into
each feasible image. The deviation of the projection from the actual cross in the image should
be used to formulate a cost function.

Furthermore, these three modules (image processing, line triangulation, back-projection) should
be implemented in a state estimator. This state estimator iterates through our entire image
data set to optimize the position as accurate as possible.

Once this package is running it would be useful to have an adaptive motion planner. In a first
approach SWACO should look for the laser cross and capture it from different angles. If the
position is sufficiently determined, the robot can start executing the drilling task. This motion
planner will be combined with a whole-body MPC controller to ensure a dynamic interaction of
end-effector and base.

Discussion

Now I’d like to move over to the discussion where I’d be pleased to hear your questions and
inputs. Thank you for the attention.