Professional Documents
Culture Documents
Voice Activated Presentation Assistant
Voice Activated Presentation Assistant
EECS 129B Senior Design Project Professor Raymond Klefstad Winter 2007
.
Executive Summary The Voice Activated Presentation Assistant (or VAPA) allows the presenter to use his/her voice commands to control PowerPoint slides and activate a guiding countdown timer. Key Benefits Plug and Play Hands Free Easy to Use Wireless Modular
EECS 129B
Page 2
EECS 129B
Page 3
Table of Contents
Page
1. OBJECTIVE ................................................................................................................................................. 6 2. OVERVIEW .................................................................................................................................................. 7 3. SYSTEM ASSEMBLY.................................................................................................................................. 8 4. OPERATION INSTRUCTIONS .................................................................................................................... 9 5. SETUP........................................................................................................................................................ 10 6. TRAINING .................................................................................................................................................. 11 7. SYSTEM LEVEL BLOCK DIAGRAM AND COMPONENT DESCRIPTIONS .......................................... 12 8. CIRCUIT DIAGRAM................................................................................................................................... 13 9. SR-06 (SPEECH RECOGNITION) DIAGRAM ..........................................................................................14 10. COUNT DOWN TIMER DIAGRAM.......................................................................................................... 15 11. ARDUINO/DECODER CIRCUITRY......................................................................................................... 16 12. ARDUINO/KEYPAD INTERFACE........................................................................................................... 17 13. VOLTAGE THRESHOLD LEVELS FOR KEYPAD................................................................................. 18 14. USER LEVEL FLOW CHART.................................................................................................................. 19 15. SYSTEM LEVEL FLOW CHART............................................................................................................. 20 16. SOFTWARE SYSTEM OVERVIEW ........................................................................................................ 21 17. MASTER CONTROLLER SOFTWARE CODE ....................................................................................... 25 18. SLAVE CONTROLLER SOFTWARE CODE .......................................................................................... 37 19. COMPONENT DESCRIPTIONS.............................................................................................................. 45 20. ENCLOSED ASSEMBLY ........................................................................................................................ 47 21. COST SHEET .......................................................................................................................................... 48 22. TEST PLAN.............................................................................................................................................. 49 23. CONCLUSION ......................................................................................................................................... 52 APPENDIX ..................................................................................................................................................... 53
EECS 129B
Page 4
Table of Figures
Page
FIGURE 1 AVAILABLE VOICE COMMANDS ............................................................................................. 6 FIGURE 2 FUNCTIONAL OVERVIEW......................................................................................................... 7 FIGURE 3 PROJECT IMAGE (TOP VIEW).................................................................................................. 8 FIGURE 4 HIGH LEVEL BLOCK DIAGRAM ............................................................................................... 9 FIGURE 5 TRAINED WORD INDICES......................................................................................................... 9 FIGURE 6 SETUP OPTIONS...................................................................................................................... 10 FIGURE 7 KEYPAD DIAGRAM.................................................................................................................. 10 FIGURE 8 HOW TO TRAIN WORDS ......................................................................................................... 11 FIGURE 9 KEYPAD DIAGRAM.................................................................................................................. 11 FIGURE 10 SYSTEM LEVEL BLOCK DIAGRAM ..................................................................................... 12 FIGURE 11 CIRCUIT DIAGRAM ................................................................................................................ 13 FIGURE 12 SPEECH RECOGNITION CIRCUIT DIAGRAM ..................................................................... 14 FIGURE 13 COUNTDOWN TIMER DIAGRAM .......................................................................................... 15 FIGURE 14 ARDUINO/DECODER INTERFACE ....................................................................................... 16 FIGURE 15 ARDUINO/KEYPAD INTERFACE .......................................................................................... 17 FIGURE 16 VOLTAGE THRESHOLD LEVELS......................................................................................... 18 FIGURE 17 USER LEVEL FLOW CHART................................................................................................. 19 FIGURE 18 SYSTEM LEVEL FLOW CHART ............................................................................................ 20 FIGURE 19 TOP LEVEL COMPONENT DESCRIPTIONS ........................................................................ 45 FIGURE 20 ENCLOSED ASSEMBLY........................................................................................................ 47 FIGURE 21 LED TRANSISTOR CIRCUIT ................................................................................................. 49 FIGURE 22 DECODER PIN LAYOUT DIAGRAM...................................................................................... 50 FIGURE 23 SPEECH RECOGNITION PIN LAYOUT................................................................................. 53 FIGURE 24 SPEECH RECOGNITION PIN DESCRIPTIONS .................................................................... 54 FIGURE 25 KEYPAD PIN DESCRIPTIONS............................................................................................... 55 FIGURE 26 ARDUINO / SPEECH RECOGNITION INTERFACE.............................................................. 56 FIGURE 27 -- TRANSMITTER CIRCUIT BOARD......................................................................................... 57 FIGURE 28 MICROPROCESSOR PIN LAYOUT (ON TRANSMITTER) ................................................... 58 FIGURE 29 FUNCTIONAL BLOCK DIAGRAM ......................................................................................... 59 FIGURE 30 TRANSMITTER PIN LAYOUT ................................................................................................ 60 FIGURE 31 FUNCTIONAL BLOCK DIAGRAM ......................................................................................... 61
EECS 129B
Page 5
1. Objective
Purpose of Project To design a voice activated presentation assistant with a count down timer. This will allow the user to control PowerPoint without using his/her hands and also be able to note the amount of time left on his/her presentation by using the count down timer. Main Features Voice Activated PowerPoint Commands Keypad Interface Large Countdown Timer Keypad Input Five Unique PowerPoint Commands
EECS 129B
Page 6
2. Overview
Summary of Functionality Speaker says command into microphone Timer begins countdown sequence if Begin command received Binary code is relayed from voice recognition board to remote control Remote control sends signal to laptop with receiver PowerPoint responds by performing action
Speaker
EECS 129B
Page 7
3. System Assembly
EECS 129B
Page 8
4. Operation Instructions 1. 2. 3. 4. 5. 6. Power-Up Unit Press Reset button on Master and Slave Arduino Verify that LEDs on Timer blink Once and then Twice Set mode of operation using Keypad Train Commands or Set Time Begin Presentation
VersaPoint Tranmitter
Arduino MicroProcessor
Word Index 01 02 03 04 08 09
EECS 129B
Page 9
5. Setup
Key 1 2 3 4 5
Option Speech and Timer Speech Only Timer Only (Speech Activated) Timer Only (Keypad Activated) Speech and Timer (Skip Training)
Figure 6 Setup Options
1 4 7
2 5 8
3 6 9
CLR
TRN
EECS 129B
Page 10
6. Training
Action Trains First Pattern Trains 21st Pattern Clears memory Clears 1st pattern
1 4 7
2 5 8
3 6 9
CLR
TRN
EECS 129B
Page 11
Countdown Timer
Arduino Controller
Keypad
Transmitter
Arduino Controller
Component Keypad Main Arduino Microcontroller Secondary Arduino Controller Digital Displays Decoder and PNP Transistors PNP Switch Array Speech Recognition Board Transmitter
Functional Description Allows user to train words into memory and set timer Controls timer and accepts input from keypad Inputs commands to Speech Recognition circuit and transmitter Displays remaining time to presenter Deocdes binary code from Arduino to PNP transistors which control transmitter Activates buttons on speech recognition ports from decoder output Processes speech recognition functions and outputs status codes to Second Arduino Tranmits RF signlal to USB reciever module to control PowerPoint
EECS 129B
Page 12
8. Circuit Diagram
EECS 129B
Page 13
EECS 129B
Page 14
Quantity 2 4 1 28 1
Item LED (Green) 74HCT4543E (Texas Instruments) 74LS373 110 Ohm Resistor 520 Ohm Resistor
Description Bright Green LED 7-Segment Decoder Driver with Latch 8 Bit Register 110 Ohm Resistor 520 Ohm Resistor
EECS 129B
Page 15
Figure 14 Arduino/Decoder Interface Digital Pin 8, 9, 10, 11 on Arduino connect to Pins on 1, 2, 3, 4 on Decoder (74HC154)
Quantity 5 1 1 5 1
Description Resistor Microcontroller 4-16 Decoder PNP Transistor RF Transmitter for PowerPoint
EECS 129B
Page 16
Arduino
ADC0 1nF 1K 820 1K 15K 3.3K 680 180 +5 VDC
1 4 7 *
2 5 8 0
3 6 9 #
Keypad
EECS 129B
Page 17
Key 1 2 3 4 5 6 7 8 9 * 0 #
Voltage at ADC0 Min Avg Max 0.225 0.248 0.272 0.396 0.434 0.474 0.588 0.641 0.698 0.930 0.969 1.048 1.468 1.522 1.627 1.959 2.020 2.139 2.563 2.688 2.809 3.285 3.396 3.500 3.740 3.832 3.917 4.170 4.237 4.298 4.507 4.550 4.588 4.671 4.700 4.726
EECS 129B
Page 18
Start
Begin Presentation
Train Mode?
Continue Presentation
No Yes No
Train Words Is Presentation Over?
Yes
Set Time End
EECS 129B
Page 19
Master
Start
Handshake
Slave
Start
Select Mode
NO
Mode 1, 2, 3, 5
YES Keypad buttons To SR
Train Words
Press 5,6
Train Words
Press 5,6
Mode 2, 3, 4, 5
YES NO
Recognize Words
From SR
Set Time
CLR/Begin Press CLR
Valid word?
YES To To Transmitte Transmitter r
Count Down
TRN/ Time TRN/ Time To Timer NO
Transmit
Paused
Time/Begin
YES
EECS 129B
Page 20
Main Components:
Keypad The keypad, which is connected to the master Arduino through one analog pin, transmits the different button values using a resistor network. The Arduino will read the analog pin and depending on the voltage value it gets, will assign a value that corresponds to the pressed button to a global variable and will return 0 if there was a valid key pressed, and -1 otherwise. To distinguish between a key being pressed and one that is held down, the analog values is compared to the previous value and only if the two are not equal will it be possible for a valid press to register. This is because when no buttons are pressed, the analog pin would read a different value from the keypad. Therefore, buttons must be released in between presses.
Speech Recognition Circuit The speech recognition (SR) circuit is connected to the slave Arduino for both input as well as output. Originally, the keypad was connected directly to the speech recognition circuit. A button press would short two of the pins on the input bus of the SR, resulting in an input. Since the keypad is used in all aspects of VAPA, it cannot be connected directly to the SR. Instead, transistors that connect the pins of the SR input bus allow the Arduino to simulate the button presses by sending a signal, which activates the transistor and shorts the two pins together.
EECS 129B
Page 21
The output of the SR is also connected to the slave Arduino. It reads the data through its connection to six of the eight data pins of the output bus, which are in binary coded decimal. This allows the Arduino to read numbers up to 39.
Timer The timer consists of four seven-segment displays, each with a display driver, which also serves as a latch. The master Arduino interfaces with the timer using 10 digital pins. Four pins are used as a data line to be able to send the numbers 0 through 9 in BCD to the timers universal data bus that connects to all four display drivers. Another four pins are used to connect to a latch disable pin on each of the 4 display drivers. Sending a HIGH signal to the latch disable pin will all the input of the driver to pass to the output. A LOW on the latch disable will capture what is on the input during the high to low transition, hold it on the output, and not let any inputs pass through. One pin is used to connect to the blank pin on all of the latches, which turns off the displays. Finally, one pin is used to control the : between the minutes and seconds. Displaying a number on one of the displays of the timer is done as followed: Put the number of the data bus Set the latch disable pin of the appropriate driver HIGH, and then LOW This is repeated for the remaining three displays.
Decoder To save pins, a decoder is used to connect between the slave Arduino and the transistors for simulating the button presses on the SR and the transmitter. The decoder is a 4-to-16 decoder and has 13 of its outputs used by the VAPA. Five of the outputs are used for the five buttons on the transmitter. Eight outputs are used for the input bus of the SR. These are 1, 2, 3, 4, 8, 9, TRN, and CLR. 5, 6, and 7 are unnecessary and therefore not used.
EECS 129B
Page 22
Mode Selection Once the system is ready the user will first press a key on the keypad to select a mode of operation. The entered key will set a global variable on the master to determine which of the SR and timer will be used. In each function to train words and to set the time on the timer, this global variable will be checked before proceeding. Train Words In this mode, keypad buttons are checked to ensure they are valid values for the SR, and will then be sent to the slave Arduino. The slave Arduino will then activate the corresponding buttons on the SR. When the user presses the keys 5 and 6 in succession, training will be complete and the master will no longer send button values to the slave. Set Time After commands are trained, the user will enter the time of the presentation length on the timer. Times are entered in a similar way as on a microwave. Numbers are shifted in from right to left with each entry, until four numbers have been entered. Each time a number is entered, variables for minutes and seconds are updated. When the CLR buttons is pressed during set mode, numbers are all shifted right by one, discarding the rightmost number. Command Recognition Mode Once a time has been entered, the user can immediately begin speaking voice commands. The slave Arduino handles the outputs of the SR. If the slave reads a valid command it will take one of the following actions:
Command Begin Next Previous Blank Exit Time Transmit YES YES YES YES YES NO Timer Start time
Pause/Restart
The first 5 commands will all cause their corresponding commands to be sent by the transmitter. If begin is spoken during set time mode, the timer will also start counting down. If the Time command is spoken while the timer is counting down, it will pause the timer, while speaking Time when it is paused will start the timer counting down again. The SR works by placing the value it gets on the output bus and keeping it there until a different word is recognized or an error code is output. This presents the problem of not being able tell when a certain word is spoken twice in a row because there would be no EECS 129B Voice Activated Presentation Assistant Page 23
change in the output when the word is recognized the second time. To alleviate this problem, upon recognizing a valid word, the slave Arduino will send the pattern 1, 8, CLR to the SR, which serves to separate back-to-back recognitions. Timer Countdown Mode Once a time has been entered, countdown can begin by either speaking the word Begin, or by pressing the TRN button on the keypad. In either case, the time will be checked to ensure it is valid by using the variables for minutes and seconds. Minutes must be between 0 and 99 while seconds must be between 0 and 59, providing a maximum time duration of 99:59. A time of 00:00 will also not be a valid time. Once the time runs out, the timer will blink 00:00. The software for the timer works by using a pair of nested while loops. The outer while loop is for minutes which checks minutes > 0 while the inner loop checks seconds > 0. For both loops, their respective time variable, minutes or seconds, is decremented after one iteration. Each time the while loop for seconds exits (seconds = 0), seconds is set to 59. When the time variables are updated, they are printed to the displays and master delays for 1 second. During a standard delay, the Arduino cannot do anything. However, the timer needs to be able to accept input from the keypad and slave to start, pause, or clear the time. Therefore, VAPA uses a special function, which breaks up the 1-second delay into 40 equal time chunks. During each time chunk, the Arduino will delay for 25ms and then perform a non-blocking read of the keypad and the serial port. If it gets input, it will take the appropriate action. Otherwise, it will continue delaying and checking for inputs until the second is over. By checking for inputs 40 times per second, the timer is very responsive to inputs from either the keypad or the serial port. The timer can be cleared during countdown or while it is paused by pressing CLR on the keypad. This puts the timer back into set time mode. When the time finishes and the displays are blinking 00:00, pressing CLR will put the timer into set time mode as well.
EECS 129B
Page 24
= = = = =
1; 2; 3; 4; 5;
decoder inputs int D3 = 7; int D2 = 6; int D1 = 5; Voice Activated Presentation Assistant Page 25
EECS 129B
int inChar = -1; int sysState = STATE_MODE; boolean hasRun = false; /////////////////////////////// //--------PROTOTYPES----------// void modeSelect(); void trainWords(); void setTime(); void getKey(); int getKeypadButton(); boolean voiceButton(char); //Timer Functions void ready(); EECS 129B Voice Activated Presentation Assistant Page 26
EECS 129B
Page 27
EECS 129B
Page 28
) ) ) ) ) ) ) ) ) )
Page 32
dBus(M1); latch(LD3); dBus(M0); latch(LD2); dBus(S1); latch(LD1); dBus(S0); latch(LD0); } void polledDelay(int length) { int period = 25; int times = length/period; while( times > 0 ) EECS 129B Voice Activated Presentation Assistant Page 35
EECS 129B
Page 36
= = = = = =
1; 2; 3; 4; 8; 9;
= = = = = =
2; 3; 4; 5; 6; 7;
//Transmitter decoder inputs const int DEC_BEGIN = 3; const int DEC_NEXT = 4; const int DEC_PREVIOUS = 2; const int DEC_BLANK = 1; const int DEC_EXIT = 5; //Decoder const int const int const int const int const int const int const int const int Inputs SR_1 SR_2 SR_3 SR_4 SR_8 SR_9 SR_TRN SR_CLR
= = = = = = = =
EECS 129B
Page 37
const int STATE_INIT = 0; const int STATE_TRAIN = 1; const int STATE_RUN = 2; ////////////////////////////// //--------VARIABLES---------// char inChar = -1; boolean hasRun = false; int sysState = STATE_INIT; int srVal = 0; int srPrevVal = -1; ////////////////////////////// //---------PROTOTYPES-------// void trainWords(); void slideNav(); void srPress(char); void getInput(); void readSR(); void interruptSRCom(); int binToInt(int, int, int, int); ///////////////////////////// void setup() { pinMode(DEC_0, pinMode(DEC_1, pinMode(DEC_2, pinMode(DEC_3,
OUTPUT); OUTPUT); OUTPUT); OUTPUT); INPUT); INPUT); INPUT); INPUT); INPUT); INPUT);
digitalWrite(DEC_0, LOW); digitalWrite(DEC_1, LOW); digitalWrite(DEC_2, LOW); EECS 129B Voice Activated Presentation Assistant Page 38
val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0; srPrevVal = srVal; srVal = val; int i = 0; for(i = 0; i <= 5; i++) { if( val == command[0][i] ) { delay(75); sr0 sr1 sr2 sr3 sr4 sr5 = = = = = = digitalRead(SR_OUT_0); digitalRead(SR_OUT_1); digitalRead(SR_OUT_2); digitalRead(SR_OUT_3); digitalRead(SR_OUT_4); digitalRead(SR_OUT_5);
val0 = binToInt(sr3, sr2, sr1, sr0); val1 = binToInt(sr5, sr4); val = (val1 * 10) + val0;
EECS 129B
Page 42
EECS 129B
Page 44
4 5 7 8 6
EECS 129B
Page 45
1. Timer 2. Master Controller 3. Keypad Interface 4. Transmitter Switches 5. Transmitter 6. Speech Recognition Switches 7. Speech Recognition Board 8. Slave Controller
EECS 129B
Page 46
EECS 129B
Page 47
Arduino Microcontrollers VersaPoint Transmitter Speech Recognition Board 74HC154 Decoder PNP Transistors 74HC4543 Drivers 7-Segment Displays Breadboards Power Supply Resistors and Wires Cover and Wooden Support Total Cost
$60.00 $40.00 $100.00 $2.00 $5.00 $8.00 $10.00 $30.00 $20.00 $15.00 $60.00 $350.00
EECS 129B
Page 48
22. Test Plan Check for Damaged Transistors Verify that both PNP and MOSFET devices are working. Construct simple LED Transistor circuit as shown below to determine if LED turns on. Replace defective transistors.
EECS 129B
Page 49
Check Decoder Voltages on Input / Output Lines Apply Voltmeter on pins 1 to 13 of Decoder and verify that the voltages are 5V for an input of 0000 on the input pins. Note that the decoder outputs active low output. Verify that E1 and E0 are both grounded.
Test Speech Recognition Circuit Connect keypad and status display board to SR-06 and train words. Power on circuit using external power source or 9V battery. Train words by pressing pattern number followed by TRN button. Speak command into microphone. Make sure command is no more than 1.92 seconds. If word is successfully processed the red LED will blink once. Test that the words will be recognized by saying the word clearly. The pattern number of the word should appear on the status display board. If the word is not recognized, too long, or too short the following error codes should display on the board. SR-06 Error Codes: 55 = word too long 66 = word too short 77 = word no match
EECS 129B Voice Activated Presentation Assistant Page 50
IMPORTANT Never short power and ground on SR-06 as this will rapidly heat up voltage regulator and may cause permanent damage to the circuit.
Verify that VersaPoint Communicator can transmit Turn on PowerPoint and open a presentation. Connect 3V source to Transmitter and short each pair of terminals (red and black) Check that LED on the other side will blink to indicate successful transmission of signal. Test Countdown Timer Select Timer Only Mode and Set time with Keypad. Press TRN to start countdown. Verify that Keypad inputs work Use a test program that will read inputs from keypad and output both to status board and timer displays Check power and ground connections Verify that power source is being properly distributed to all major components. Measure input voltage on transmitter from 3V regulator. Measure input voltage on SR-06 from Arduino. (Verify that it is 9V) Measure output voltage across 5V regulator on SR-06. Measure input voltages on Timer and Keypad.
EECS 129B
Page 51
23. Conclusion Overall we have concluded that our project works successfully. Before reaching this milestone we encountered several learning experiences that will be discussed. We realized that the Microcontroller Interface between the Arduino and Speech Recognition circuit could have been easily done after trying to directly interface the Speech Kit to the Microcontroller in CPU mode. We debugged the system for about 3 weeks after we abandoned the decision to use CPU mode. We used a much easier interface which connects the Speech Kit to the Microcontroller using switches. The mode of operation for the Speech Kit was Manual mode which was much easier to work with. In addition to this problem we had to debug the noise on the board. We determined that there was noise causing the timer to flicker values. However, once the Arduinos reset we noticed that the timers blanked out which was good. We learned about Common Annode and Common Cathode configurations for Display Components and learned how to use them with our LCD drivers. Eventually with some ingenious programming we successfully programmed a countdown timer with reset capabilities. We learned that we could use a keypad configured with a resistor matrix to reduce the number of ports used for the keypad and programmed a driver program to accept the commands from the keypad. After noticing that the Speech Kits onboard regulator heated up considerably we purchased a clip on heat sink and mounted it on the regulator after relocating the regulator to the breadboard.
EECS 129B
Page 52
Appendix
HM2007 Pin Descriptions
Pin Names SA0-12 D0-7 MR/MW and ME X1 and X2 S1-3 K1-4 TEST LINE MICin WLEN CPUM
Description Memory address bus connections are between the HM2007 and the static RAM IC Data input bus three connections for each line: the HM2007, the static RAM, and the microcontrollers inputs Additional control lines for writing to and reading from external RAM connection is between HM2007 and RAM Connection pins for 3.58 MHz crystal Read/write control pins connection to microcontroller I/O Bi-directional data bus (K-bus) connection to microcontroller I/O Not used pull low Not used leave disconnected Microphone connection Word-length select pin. Pull high to use 1.92-second words, pull low to use 0.9-second words CPU/manual-mode select pin. Pull low for manual mode, pull high for CPU mode. Pulled high to select CPU mode in this report
EECS 129B
Page 53
Status LED
EECS 129B
S1 S2 S3 K1 K2 K3 K4
GND POWER D7 D6 D5 D4 D3 D2 D1 D0
Page 54
Keypad Diagram
Pin # 4 5 6 8 9 10 11 Corresponding Pin Name S1 S2 S3 K1 K2 K3 K4
EECS 129B
Page 55
SR-06/Arduino Interface
Figure 26 Arduino / Speech Recognition Interface Pin Mapping Data Bus On SR-06 D0 D1 D2 D3 D4 D5 Arduino Pins 2 3 4 5 6 7
EECS 129B
Page 56
EECS 129B
Page 57
EECS 129B
Page 58
Figure 29 Functional Block Diagram The instruction set in the microprocessor consists of 13-bit words that are composed of an OP code and several operands. Usually, the instructions are executed with one instruction cycle. However, if the PC becomes changed by a MOV, ADD, or by instructions of arithmetic or logic operation then the execution takes two cycles.
EECS 129B
Page 59
The Transmitter The TDA5102 is designed by Infineon Technologies and is an ASK, FSK transmitter whose transmission frequency band is between 905 and 925 MHz. Major features include High level of integration Need for only a few external components Integration of a PLL (frequency) synthesizer High efficiency power amplifier to drive a loop antenna Power down mode Selectable XTAL oscillator at 7.15MHz/14.3MHz
EECS 129B
Page 60
EECS 129B
Page 61