Professional Documents
Culture Documents
Fusing Virtual Game Environment and Reality World
Fusing Virtual Game Environment and Reality World
Fusing Virtual Game Environment and Reality World
CYBER-PHYSCAL
GAME
Interactive game
system
with
Physical game agents
Intelligent algorithms
Real players
Puppeteer
Marionette
Staging
Scoreboard Manipulator
Gamepad
Servos Controller
Marionettes and
Puppeteers
Sensory Interface Cyber-
Physical
Game
Processor
Staging Manipulators Manipulators Motion Planner
User Interface
User
Users
System Architecture
Physical System Cyber System
Servos Controller
Marionettes and
Puppeteers
Sensory Interface Cyber-
Physical
Game
Processor
Staging Manipulators Manipulators Motion Planner
User Interface
User
Users
Marionette Design
• Marionette figure
– 12 inches wood figure with
movable joints (18 DOFs)
– types of actions:
• Kicks and Punches (Left & Right)
• Fall back
– Installed with Reed sensors and
magnets for hit detection
Robotic Puppeteer Mechanism
Joint
Tubes
Hinder
After elongation
Before elongation
String attached to
shoulder and
elbow.
Joint
Rubber band
By pulling the string, the
String
arm is bent by string which
attached at shoulder and
forearm. Both rubber bands
Without pulling the string, at both joints are stretched.
the arm is straightened by
the pulling force of rubber
bands at both joints.
Hit Detection
• Reed Switch
– Activated when a magnet comes closely
– No interface circuitry and power supply required
– Installed on fist and foot
– Magnet installed on body
Reed Switch
(source: en.wikimedia.org)
Installation of magnet
Puppeteer Controller
• Manipulate strings to control the action of marionette
• Mounted on staging manipulators
• 4 Servo motors and four 50mm pulleys for each
puppeteer
• Interface with Phidget Servo Control
Servos Controller
Marionettes and
Puppeteers
Sensory Interface Cyber-
Physical
Game
Processor
Staging Manipulators Manipulators Motion Planner
User Interface
User
Users
Staging Manipulators Subsystem
• 6 PowerCube motors
• Two 3-degree of freedom planar manipulators
• Smooth trajectory movement in any direction with
specified facing angle for both marionettes
• Interface with CAN-bus to USB converter
Fixed link
Revolute Joint
PowerCube
(source: technikerprojekt.de)
Revolute Joint
Position of Marionette
Face-to-face Movements
Coordinate of Joint 1, (x1, y1)
Angle of Joint 1, θ1
Length of Link 1, L1
Angle of Joint 2, θ2
Length of Link 2, L2
Servos Controller
Marionettes and
Puppeteers
Sensory Interface Cyber-
Physical
Game
Processor
Staging Manipulators Manipulators Motion Planner
User Interface
User
Users
Multi-modal User Interface and
Interaction System
• Provides interface and interaction features
for users to communicate with Cyber-
Physical Game Processor
• Multi-modal interface
– Reads user’s instruction through gamepads
– Displays score for both users Gamepad
Scoreboard
System Architecture
Physical System Cyber System
Servos Controller
Marionettes and
Puppeteers
Sensory Interface Cyber-
Physical
Game
Processor
Staging Manipulators Manipulators Motion Planner
User Interface
User
Users
Implementation with Microsoft Robotics
Developer Studio
Software
System Architecture Implement as Purposes of services
Service
Processes for all inputs and outputs
Cyber-Physical Game
Main Makes major decisions
Processor Controls other services
Calculates trajectory of staging manipulators
Provides simulation of staging manipulators
Manipulators Motion Calculation Checks positions of all PowerCubes
Planner Prevents collision and out-of-boundary movement
Drives PowerCube motors
PowerCube Monitors PowerCube motors’ condition
Servo Controller Servo Drives servo motors
Game
Monitors and processes inputs from first gamepad
Controller
User Interface Driver
Game Monitors and processes inputs from second
Controller 2 gamepad
Pools status of sensors
Sensory Interface Processes raw values from sensors
Phidget Provides digital outputs for score board
Interaction Renderer Produces sound effect for different actions
Software Architecture
Message Purpose Content
walking directions that
Walk marionette A to walk
marionette A to follow. Game Game
Controller Controller2