BOSNA I HERCEGOVINA

FEDERACIJA BOSNE I HERCEGOVINE

TUZLANSKI KANTON

JU MJEŠOVITA SREDNJA ELEKTROTEHNIČKA ŠKOLA

T U Z L A

PID regulation using WebCam data

Students:
Bakalović Dino
Omerdić Samir
Mentor:

Mr.sci. Halilović Muamer dipl.ing.el

Ul. Muhameda Hevaija Uskufija br.2, 75 000 Tuzla

Tel/Fax : +387 35 281-167; 280-012
Tel/Fax : +387 35 282-009
E – mail : ets.tz@bih.net.ba
www.etstuzla.edu.ba
 PID regulation using WebCam data
January 22, 2013

Table of Contents
Abstract .................................................................................................................................................. 3
Hardware ................................................................................................................................................ 4
Arduino ............................................................................................................................................... 6
Servo ................................................................................................................................................... 7
WebCam ............................................................................................................................................. 9
Ball and Beam ....................................................................................................................................... 10
Processing ............................................................................................................................................. 11
JMyron .................................................................................................................................................. 15
Arduino PID library ................................................................................................................................ 21
What Is PID? ...................................................................................................................................... 21
Tuning Parameters ............................................................................................................................ 21
Conclusion............................................................................................................................................. 28
Reference .............................................................................................................................................. 29

ETS TUZLA | 2
 PID regulation using WebCam data
January 22, 2013

Abstract

Well our mentor wondered if it was possible to train high school students to solve
complex problems, such as machine vision, if you add the low budget and the use of open
source tools, it almost seems impossible, but you are wrong.

The ball and beam system consists of a long beam which can be tilted by a servo
together with a ball rolling back and forth on top of the beam. It is a popular textbook example
in control theory, and usually use some kind of optical sensor on the one end of a beam. Our
idea was to use a cheap WebCam to track a ball on a beam, using JMyron library for Processing,
and pass data form application of a ball position via serial port to Arduino. Serial data is actual
positions on the screen in pixels, respectively by x-Axis.

Using this data as Input data for Arduino PID library we could define Setpoint for ball
and controller will attempt to minimize the error by adjusting an Output, or servo angle.

Setting PID parameters is a science for itself so we use a simple 20 iteration method and
succeeded to get decently response.

ETS TUZLA | 3
 PID regulation using WebCam data
January 22, 2013

Hardware
First we needed to buid up a construction starting from a scratch, for that we use a
ArtCAM which provides a unique CAD modelling and CNC machining software. It’s easy to use
and customizable so you get the freedom to design and manufacture high-quality 2D or 3D
products from their artwork real fast.

Servo motor holder 2D model

Construction rack 2D model

ETS TUZLA | 4
 PID regulation using WebCam data
January 22, 2013

After we create 2D model we are going to create toolpaths for cutting parts of
construction in forex. Cutting is done using CNC and Mach3 software.

Mach3 with loded G-code

Final result

ETS TUZLA | 5
 PID regulation using WebCam data
January 22, 2013

Arduino
As we mentioned the start point for our project was low budget, so the Arduino
platform with his price of 14$ for China clone, has imposed as first choice.

Arduino is a single-board microcontroller designed to make the process of using

electronics in multidisciplinary projects more accessible. The hardware consists of a simple
open source hardware design for the Arduino board with an Atmel AVR processor and on-board
input/output support. The software consists of a standard programming language compiler and
the boot loader that runs on the board.

An Arduino board consists of an 8-bit Atmel AVR microcontroller with complementary

components to facilitate programming and incorporation into other circuits. An Arduino's
microcontroller is pre-programmed with a boot loader that simplifies uploading of programs to
the on-chip flash memory, compared with other devices that typically need an external
programmer.

ETS TUZLA | 6
 PID regulation using WebCam data
January 22, 2013

Servo

For controlling a position of a ball on a shaft we use a servo motor, specifically FUTABA
S3001 from disassembled demining machine. So servo is a small device that has an output
shaft. This shaft can be positioned to specific angular positions by sending the servo a coded
signal. As long as the coded signal exists on the input line, the servo will maintain the angular
position of the shaft. As the coded signal changes, the angular position of the shaft changes.

A Futaba S-148 Servo

The guts of a servo motor are shown in the picture below. You can see the control
circuitry, the motor, a set of gears, and the case. You can also see the 3 wires that connect to
the outside world. One is for power (+5volts), ground, and the white wire is the control wire.

ETS TUZLA | 7
 PID regulation using WebCam data
January 22, 2013

A servo disassembled.

So, how does a servo work? The servo motor has some control circuits and a
potentiometer (a variable resistor, aka pot) that is connected to the output shaft. In the picture
above, the pot can be seen on the right side of the circuit board. This pot allows the control
circuitry to monitor the current angle of the servo motor. If the shaft is at the correct angle,
then the motor shuts off. If the circuit finds that the angle is not correct, it will turn the motor
the correct direction until the angle is correct. The output shaft of the servo is capable of
travelling somewhere around 180 degrees. Usually, its somewhere in the 210 degree range, but
it varies by manufacturer. A normal servo is used to control an angular motion of between 0
and 180 degrees. A normal servo is mechanically not capable of turning any farther due to a
mechanical stop built on to the main output gear.

Servo mounting

The amount of power applied to the motor is proportional to the distance it needs to
travel. So, if the shaft needs to turn a large distance, the motor will run at full speed. If it needs
to turn only a small amount, the motor will run at a slower speed. This is called proportional
control.

How do you communicate the angle at which the servo should turn? The control wire is
used to communicate the angle. The angle is determined by the duration of a pulse that is
applied to the control wire. This is called Pulse Coded Modulation. The servo expects to see a
pulse every 20 milliseconds (.02 seconds). The length of the pulse will determine how far the
motor turns. A 1.5 millisecond pulse, for example, will make the motor turn to the 90 degree
position (often called the neutral position). If the pulse is shorter than 1.5 ms, then the motor
will turn the shaft to closer to 0 degress. If the pulse is longer than 1.5ms, the shaft turns closer
to 180 degress.

ETS TUZLA | 8
 PID regulation using WebCam data
January 22, 2013

As you can see in the picture, the duration of the pulse dictates the angle of the output
shaft (shown as the green circle with the arrow). Note that the times here are illustrative, and
the actual timings depend on the motor manufacturer. The principle, however, is the same.

WebCam
The Eye 312 web cam sold for 14$ is used for taking picture capturing, it’s low quality
webcam for instant messaging application equipped with a 640 x 480 (VGA) CMOS sensor, and
with a handy snapshot button located at the tip of the Eye 312, simply click to take photos at up
to 1.3 mega pixel resolution, and do the job just well.

ETS TUZLA | 9
 PID regulation using WebCam data
January 22, 2013

Ball and Beam

The ball and beam system is one of the most enduringly popular and important
laboratory models for teaching control systems engineering. The ball and beam system is
widely used because it is very simple to understand as a system, and yet the control techniques
that can be studied it cover many important classical and modern design methods. It has a very
important property – it is open loop unstable.

Ball and beam system

The system shown above is very simple – a steel ball rolling on the top of a long beam.
The beam is mounted on the output shaft of an electric motor and so the beam can be tilted
about its centre axis by applying an electrical control signal to the motor amplifer. The position
of the ball on the beam can be measured using a special sensor, and in our case using ball
position in pixel on X-axis. The control job is to automatically regulate the position of the ball on
the beam by changing the angle of the beam. This is a difficult control task because the ball
does not stay in one place on the beam but moves with an acceleration that is proportional to
the tilt of the beam. In control techology the system is open loop unstable because the system
output (the ball position) increases without limit for a fixed input (beam angle). Feedback
control must be used to keep the ball in a desired position on the beam.

There is a quite complex mathematical model for explaining forces and ball acceleration,
but as we are just high school student we use a PID algorithm to do all the job of process
control for us.

ETS TUZLA | 10
 PID regulation using WebCam data
January 22, 2013

Processing

For displaying a webcam captured data we use a processing which is an open source
programming language and environment for people who want to create images, animations,
and interactions. Initially developed to serve as a software sketchbook and to teach
fundamentals of computer programming within a visual context, Processing also has evolved
into a tool for generating finished professional work. Today, there are tens of thousands of
students, artists, designers, researchers, and hobbyists who use Processing for learning,
prototyping, and production.

We only use several processing function so we are going to eexpalin some of them.

ellipse()

ellipse(56, 46, 55, 55);

Description Draws an ellipse (oval) to the screen. An ellipse with equal width and height is
a circle. By default, the first two parameters set the location, and the third and
fourth parameters set the shape's width and height. The origin may be
changed with the ellipseMode() function.

Syntax
ellipse(a, b, c, d)
afloat: x-coordinate of the ellipse

bfloat: y-coordinate of the ellipse

Parameters
c float: width of the ellipse by default

dfloat: height of the ellipse by default

Returns void

ETS TUZLA | 11
 PID regulation using WebCam data
January 22, 2013

line()

line(30, 20, 85, 75);

Draws a line (a direct path between two points) to the screen. The version of line()
with four parameters draws the line in 2D. To color a line, use the stroke()
function. A line cannot be filled, therefore the fill() function will not affect the
color of a line. 2D lines are drawn with a width of one pixel by default, but this can
Description
be changed with the strokeWeight() function. The version with six parameters
allows the line to be placed anywhere within XYZ space. Drawing this shape in 3D
with the z parameter requires the P3D parameter in combination with size() as
shown in the above example.

JavaScript The "@pjs crisp" directive affects how lines are rendered. You can set it with the
Notes "Playback Settings" tool in the "JavaScript" menu.

line(x1, y1, x2, y2)

Syntax
line(x1, y1, z1, x2, y2, z2)

x1float: x-coordinate of the first point

y1float: y-coordinate of the first point
Parameters x2float: x-coordinate of the second point
y2float: y-coordinate of the second point
z1 float: z-coordinate of the first point
z2 float: z-coordinate of the second point
Returns void

ETS TUZLA | 12
 PID regulation using WebCam data
January 22, 2013

text()

textSize(32);
Examples text("word", 10, 30);

fill(0, 102, 153);

text("word", 10, 60);

fill(0, 102, 153, 51);

text("word", 10, 90);

Draws text to the screen. Displays the information specified in the first parameter
on the screen in the position specified by the additional parameters. A default font
will be used unless a font is set with the textFont() function and a default size will
be used unless a font is set with textSize(). Change the color of the text with the
fill() function. The text displays in relation to the textAlign() function, which gives
the option to draw to the left, right, and center of the coordinates.

DescriptionThe x2 and y2 parameters define a rectangular area to display within and may only
be used with string data. When these parameters are specified, they are
interpreted based on the current rectMode() setting. Text that does not fit
completely within the rectangle specified will not be drawn to the screen.

Note that Processing now lets you call text() without first specifying a PFont with
textFont(). In that case, a generic sans-serif font will be used instead. (See the third
example above.)

JavaScript Please see this page for details on font handling in JavaScript mode:
Notes http://wiki.processing.org/w/Font_handling_in_JavaScript_mode

text(c, x, y)
Syntax
text(c, x, y, z)

ETS TUZLA | 13
 PID regulation using WebCam data
January 22, 2013

text(str, x, y)

text(chars, start, stop, x, y)

text(str, x, y, z)

text(chars, start, stop, x, y, z)

text(str, x1, y1, x2, y2)

text(num, x, y)

text(num, x, y, z)

c char: the alphanumeric character to be displayed

x float: x-coordinate of text

y float: y-coordinate of text

z float: z-coordinate of text

charschar[]: the alphanumberic symbols to be displayed

start int: array index at which to start writing characters

Parameters
stop int: array index at which to stop writing characters

x1 float: by default, the x-coordinate of text, see rectMode() for more info

y1 float: by default, the x-coordinate of text, see rectMode() for more info

x2 float: by default, the width of the text box, see rectMode() for more info

y2 float: by default, the height of the text box, see rectMode() for more info

num int, or float: the numeric value to be displayed

Returns void

ETS TUZLA | 14
 PID regulation using WebCam data
January 22, 2013

JMyron
For catching webcam data in form of array of array pixel color value we used a JMyron is
the cross-platform, cross-language, open source, video capture and computer vision plugin.
These core C object gets cross-compiled on a Handful of high level language "wrapper" libraries.
The wrapper for Java and Processing is called JMyron.

Jmyron has a powerful range of functions with which you can easily solve problems for
Motion Detection, ColorTracking and Glob Distinction.

Well let see some basic reference for JMyron witch help as solve this seemingly complex
task, and we will explain only one we used.

void start(int width,int height) initializes the xtra. This should be the first function you call just
following instantiation, otherwise the plugin will malfunction.

Processing / Java:
JMyron m = new JMyron();
m.start(320,240);

void stop() will close the xtra at the end of your movie. call this or else you won't be able to call
start again when the movie plays the next time.

void update() is called once per frame in order to run the vision processing and camera
updating.

void trackColor(red,green,blue,tolerancewill tell webcamxtra which color you want it to match

when it's looking for globs. Tolerance will set the "sensitivity" of the color matching.

In Processing, returned images are int arrays whose item lengths are the width*height. This is
the same format as the Processing "pixels" array.

cameraImage() or image() returns the regular camera image object.

Glob will analyze the scene with the help of your parameters, and return a list of geometries
concerning the multiple objects in the camera's view. Glob processing is always run on the
differenceImage, which is defaultly equivalent to the plain image of the camera. (See MOTION)

findGlobs(int state) if state is 0, glob processing is disabled. if state is 1, then glob processing is

ETS TUZLA | 15
 PID regulation using WebCam data
January 22, 2013

enabled. Parts of webcamxtra will become inactive when you disable globs, such as globsImage,
globQuads, globBoxes, etc.
Java: int[][2]

Here is a minimum code for finding white object and drawing a rectangle around it.

import JMyron.*;

JMyron m;

void setup(){
int w = 640;
int h = 480;

size(w,h);
m = new JMyron();
m.start(352,288);
m.findGlobs(1);

void mousePressed(){
m.settings();
}

void draw(){
m.trackColor(255,255,255,255);

m.update();
int[] img = m.image();

//first draw the camera view onto the screen

loadPixels();
for(int i=0;i<width*height;i++){
pixels[i] = img[i];
}
updatePixels();

noFill();
int[][] a;

//draw bounding boxes of globs

a = m.globBoxes();
stroke(255,0,0);
for(int i=0;i<a.length;i++){
int[] b = a[0];
ETS TUZLA | 16
 PID regulation using WebCam data
January 22, 2013

rect(b[0], b[1], b[2], b[3]);

}

public void stop(){

m.stop();
super.stop();
}

And code for processing is show bellow, there is still room for aprovement, like idea is to
change Setpoint position with a mouse click, and get update of set position diagram on the fly,
but

/*
Ball Balance Machine by ETS

*/

//========================PREPROCESSOR=========================

import processing.serial.*;
import JMyron.*;

Serial MySerialport;
JMyron MyWebCam;

//_______________________________________

// chage this for diferent setpoint

int Xtarget = 320; //pixels from left edge
//_________________________________________

int Ytarget = 150; //pixels from top edge

int[] podaci = new int[640];//array for storing response data

int[] XX=new int[640];//array of X pixel from 0-640
int i=0, k=0;

//=================================================================
ETS TUZLA | 17
 PID regulation using WebCam data
January 22, 2013

//============================SETUP============================
void setup(){
MySerialport = new Serial(this,"COM5",57600);//serial port setup
size(640,480);//screnn size in pixels

MyWebCam = new JMyron();//call JM

MyWebCam.start(640,480);//
MyWebCam.findGlobs(1); // 1 is on, 0 is off

PFont font; //create font

font = loadFont("Arial-BoldMT-16.vlw");
textFont(font);

//=================================================================

//=============================LOOP MAGIC=========================

void draw()

//background(0); // clear the text from the previous frame

MyWebCam.trackColor(0,0,0,150);//track black colored ball with tolerance of 100

MyWebCam.update();
int[] img = MyWebCam.image();

//first draw the camera view onto the screen

loadPixels();
MyWebCam.imageCopy(pixels);

updatePixels();

noFill();

ETS TUZLA | 18
 PID regulation using WebCam data
January 22, 2013

//initiate a two dimensional array that will hold the glob start point
int[][] a;
//track glob center
a = MyWebCam.globCenters();
stroke(255,255,0);
if (a.length >0){
int[]b = a[0]; //look at the first glob only (0)
//draw big circle arround glob
ellipse(b[0], b[1],60 , 60);
//draw small circle arround glob
fill(255);
ellipse(b[0], b[1],10 , 10);

MySerialport.write(b[0]+"\n\r");//send X centar of glob pixel position to serial port

//rect(0, 200, width, height);

String x = "X: " + b[0] +"\n\r";//show X pixel position on screen
text(x, 5, 15);

//draw graph
podaci[i]=b[0];
XX[k]=k;
strokeWeight(5);
for(int xo=0;xo<i;xo++)
point(XX[xo],podaci[xo]);
//point(XX[k],podaci[i]);

if(i>= 639)
{i=0;
//updatePixels();
}
if(k>= 639)
{k=0;}
k++;
i ++;
}

//data display

ETS TUZLA | 19
 PID regulation using WebCam data
January 22, 2013

//show target posn

strokeWeight(1);
line(Xtarget, Ytarget-10, Xtarget, Ytarget+10);
line(Xtarget-10, Ytarget, Xtarget+10, Ytarget);
String Setpoint = "Setpoint:"+Xtarget;//show Setpoint value
text(Setpoint, 15, 305);
line(0,Xtarget,width,Xtarget);//setpoint
text("("+Xtarget+")", Xtarget-20, Ytarget-15);
String t = "Webcam Ball Ballance Machine - ETS Tuzla";
text(t, 180, 20);

String l = "Legenda:";
text(l, 530, 400);
ellipse(530, 430,15 ,15);
String odziv = "PID Output";
text(odziv, 550, 435);

//=================================================================

//========================WHAT IF==================================
void mousePressed()
{
////reset target position
//Xtarget = mouseX;
//Ytarget = mouseY;
MyWebCam.settings();

//===================STOP CLOSE FREE RESOURCES==================

public void stop(){
MyWebCam.stop();
super.stop();
}

//=================================================================

ETS TUZLA | 20
 PID regulation using WebCam data
January 22, 2013

Arduino PID library

What Is PID?

From Wikipedia: "A PID controller calculates an 'error' value as the difference between a
measured [Input] and a desired setpoint. The controller attempts to minimize the error by
adjusting [an Output]."

So, you tell the PID what to measure (the "Input",) Where you want that measurement to be
(the "Setpoint",) and the variable to adjust that can make that happen (the "Output".) The PID
then adjusts the output trying to make the input equal the setpoint.

For reference, in a car, the Input, Setpoint, and Output would be the speed, desired speed, and
gas pedal angle respectively.

Tuning Parameters

The black magic of PID comes in when we talk about HOW it adjusts the Output to drive the
Input towards Setpoint. There are 3 Tuning Parameters (or "Tunings"): Kp, Ki & Kd. Adjusting
these values will change the way the output is adjusted. Fast? Slow? God-awful? All of these
can be achieved depending on the values of Kp, Ki, and Kd.

So what are the "right" tuning values to use? There isn't one right answer. The values that work
for one application may not work for another, just as the driving style that works for a truck
may not work for a race car. With each new application you will need to try Several Tuning
values until you find a set that gives you what you want.

For tuning PID parameters (Kp, Ki and Kd)we used a table below witch provides an
example of using this method to quickly tune a servo motor. This example took 20 iterations to
arrive at a reasonably well tuned system. The gains used in each iteration are shown as well as
the step response graph and characteristics. The screenshots from this tuning are also shown
in the animated image of the Step Response window

ETS TUZLA | 21
PID regulation using WebCam data
January 22, 2013

ETS TUZLA | 22
PID regulation using WebCam data
January 22, 2013

ETS TUZLA | 23
PID regulation using WebCam data
January 22, 2013

ETS TUZLA | 24
 PID regulation using WebCam data
January 22, 2013

Example code for Arduino using PID and Servo library.

//-------------------------------PREPROCESSOR-----------------------------
#include <PID_v1.h>
#include <Servo.h>

// Change this setings if you want another Setpoint (X: 0-640; Y:0-480)

int SetX=320;

// create servo object to control a servo

Servo myservoX;

//Define Variables we'll be connecting to

double Setpoint, Input, Output;

//Specify the links and initial tuning parameters

PID myPID(&Input, &Output, &Setpoint,2.6,8,0.5, REVERSE);

//----------------------------------------------------------------

ETS TUZLA | 25
 PID regulation using WebCam data
January 22, 2013

//-------------------------SETUP---------------------------------
void setup()
{
pinMode(6,INPUT);
Serial.begin(57600);
// attaches the servo on pin 9 to the servo object X axis
myservoX.attach(9);
//map pixels input to 0-255
Setpoint = map(SetX, 0, 640, 0, 255);//test

//set output limits

//myPID.SetOutputLimits(30, 80);
myPID.SetSampleTime(10);

//turn the PID on

myPID.SetMode(AUTOMATIC);

}
//----------------------------------------------------------------

//-------------------------------LOOP----------------------------

void loop()
{

if (Serial.available()>0)

{
int data = Serial.parseInt();

if (data==0)
{myservoX.write(85);}
else{
// map the X pixel data (0-640) to 0 - 255
Input = map(data, 0, 640, 0, 255);
myPID.Compute();

int OutputX = map(Output, 0, 255, 70,100 );

//Write servo position
myservoX.write(OutputX);
}
}
else if((Serial.available()==0))
{
ETS TUZLA | 26
 PID regulation using WebCam data
January 22, 2013

myservoX.write(85);
}
}

ETS TUZLA | 27
 PID regulation using WebCam data
January 22, 2013

Conclusion

Well we hope that this document convinced you that a complex task of machine vision
could be solve using open source, low budget, little mathematics and we must say lot of will
and work from our side and lot of patient mentor :D.

ETS TUZLA | 28
 PID regulation using WebCam data
January 22, 2013

Reference
1. http://processing.org/reference/
2. http://webcamxtra.sourceforge.net/reference.shtml
3. http://playground.arduino.cc/Code/PIDLibrary
4. http://www.ni.com/white-paper/2923/en
5. http://www.control-systems-principles.co.uk/whitepapers/ball-and-beam1.pdf

ETS TUZLA | 29