Workshop

HONG KONG INSTITUTE OF VOCATIONAL EDUCATION

AI Workshop: Snake Game by using Teachable machine and p5.js
Name:___Wan Sai Yin. Time Allowed: 90 min
Class:_______________________ Date:_____________________

Procedures
A. Create FIVE gestures image samples to control the snake movement

A1. Connect PC webcam to the computer or follow the instructions from

MobileWebcam.docx to use your smart phone as the webcam.

A2. Visit the link https://teachablemachine.withgoogle.com/ using the Chrome browser.

A3. Click the Get Started button to create the new project

A4. Click “Image Project” to create a new image project.

Page 1 of 12
 Workshop

A5. The web browser will open a page for you to create a model.

A6. Change Class 1 to Left in the Image Set Column.

Before After

A7. Click Settings Icon and change the setting as below.

Page 2 of 12
 Workshop

FPS: 24
Hold to Record: OFF
Delay: 2 seconds
Duration: 15 seconds

A8. Click Save Settings at the bottom.

A9. Click “Record 15 Seconds” to record a gesture for controlling the snake to the left as
below.

Tips: Try to move your hand in different positions/orientations, it helps to improve the
robustness the AI model.

Page 3 of 12
 Workshop

A10. Click “Add a class” to create another 4 image sets to control the snake movement.
(Order is NOT important.)

Right Up

Down Nothing

B Train your model and testing

B1 Now you can train your model by clicking the Train Model button.

Page 4 of 12
 Workshop

It may need a while… so you can take a break.

But remember don't switch tab or close the browser, you must leave the page open to train
your model.

B2 Capture and paste the whole screenshot below. [25%]

Any one of the Gesture (Left / Right / Up / Down / Nothing)

C Start to create a basic Snake game

C1. Go to the link https://editor.p5js.org/ to create a new p5js project.

C2. Create an account (optional – if you want to save your project.)

C3. Open the source codes sketch.js from the resources folder using Notepad++, copy
the source codes sketch.js and replace the existing one (sketch.js).

C4. Create a new file snake.js

i. Click the buttons below.

2 1
Page 5 of 12
3
Workshop

ii. Name it as snake.js

C5. Open the source codes snake.js from the resources folder using Notepad++, copy
the source codes snake.js and replace the existing one (snake.js).

C6. Open the index.html from Sketch Files. Copy the following code and add it at (line
12) as below. It aims to load the JavaScript “snake.js” into the index page.

<script src="snake.js"></script>

C7. Copy the following code and add it at (line 6) in the index.html file as below.

<script src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>

Page 6 of 12
 Workshop

C8. Check the Auto-refresh and click Play button. You can click the Preview screen
and use the four arrows ↑ ↓ ← → to control the snake movement.

C9. Capture and paste the Preview below. [10%]

Page 7 of 12
 Workshop

D Integrate the hand gestures into the snake game

D1. Goto the page Teachable Machine and click “Export Model”.

D2. Choose
Tensorflow.js and click
Upload my model.

D3. Copy “Your sharable link”.

D4. Select "sketch.js" file from the Sketch file.

Page 8 of 12
 Workshop

D5. Paste the copied link as below. This is going to apply the classifier that you trained
into your snake game.

*Note: Remember to keep "/model.json" at the end of the link.

E Control the snake with gestures

E1. Find and change the function name "keyPressed" to "controlSnake".

E2. Change LEFT_ARROW, RIGHT_ARROW, DOWN_ARROW, UP_ARROW to

"Left", "Right", "Down" and "Up" respectively. You may copy the code below and paste
into the (line 53 to 64)

// Step E2: Change this function from "keyPressed" to "controlSnake" and the keyCode to label

function controlSnake() {
if (label == "Left") {
snake.setDir(-1, 0);
} else if (label == "Right") {
snake.setDir(1, 0);
} else if (label == "Down") {
snake.setDir(0, 1);
} else if (label == "Up") {
snake.setDir(0, -1);
}
}

HINT

The text inside quotation (e.g "Left") should same as the Image Set name in Teachable
Machine.
Remember in programming world, all texts are usually case-sensitive.

Page 9 of 12
 Workshop

Before After

E3. Now, goto function "gotResults", add a call function " controlSnake(); " in line 42.

E4. Try to play your game with the gesture that you trained in Teachable Machine.
Capture the whole browser in the below box [25%]

Sorry for can’t playing the game with the al machine, I think it just something wrong with my
program
So it’s the old pic of previous question

Page 10 of 12
 Workshop

*Note: If you want to see display your cam in the output, you can add “//” to disable the
command video.hide();in line 20.

Short Question

Q1. List THREE types of classification that the Teachable Machine can work with.
[12%]
Images, Sounds and Poses

Q2. List THREE factors to improve the robustness of the AI model [18%]

1. More and clear model for the machine to learn, in this example, we can have
more model like wearing different colour dresses, more different hand shapes
or even people in different colour, which means giving more data for the
machine for learning the moves
2. More learning time of the machine, assuming the machine can learn be itself
like us, the learning time should be a important factor
3. Stronger computer, a stronger computer for calculating, the answers should
have a higher accuracy

Page 11 of 12
 Workshop

Two little ideas for training the machine:

Train the machine with another machine model. I’ve forget the name of the
pattern, so there’s an example : we can build up a another database , which
inputs to an other machine for generating more models to questioning our
machine, let’s call it ‘the questioner’, then our machine will answer the
questioner, the questioner will give the correct answers to our machine, in this
hypothesis, the more learning time and with stronger computer, the machine
should be smarter

Using the cloud computing system, and getting the data from the host.Example: for
this game, we can publishing it and letting different kinds of people to play with it,
getting the data of them and putting them inside or databases, which will let us have
a larger data for the machine, in theory, it should have more accurate to get the
correct answers.

Q3. Give an example where the Teachable Machine can be used in our daily life. [10%]

I’ve noticed that teachable machine can work with poses , which means it should
also working to noticing the tumble pose. It can be used to warn us if it found our elders fall
down inside the house , which is useful for the people who can’t caring them all day.
Submission of the CA Workshop
This CA Workshop contributes 10% of the module marks.
Submit this CA Workshop (word file) to Moodle.
Deadline: 05-OCT-2021 (Week 5)
- END -

Page 12 of 12