Designing An Android

Sensor Subsystem
Pitfalls and Considerations

Jen Costillo
jen@rebelbot.com
Simple Choices
User
experience
Battery
performance
7/15/2012 2 Costillo- OSCON 2012
Established
or Innovative Product?
Established
Will I be making another
new product in 6
months?
Is the reference design
considered good enough
for the application?

Innovation-Driven
Do I have new sensors
types?
Are features more
important than release
date?
Are money and
resources no
problem?


7/15/2012 3 Costillo- OSCON 2012
Forsaking Reference Designs
7/15/2012 4 Costillo- OSCON 2012
Going On Your Own
If you make your own,
Youre on your own
Integration pains
Test time
Gesture testing
becomes a challenge
Calibration blues
Larger mechanical
footprint
But
power
Control code size
Control mechanical
footprint
In-house expertise


7/15/2012 Costillo- OSCON 2012 5
Android Universe
7/15/2012 Costillo- OSCON 2012 6
Sensor Hub/
Coprocessor
Sensors
Interface Kernel
Driver
Sensor Driver
Sensor HAL
Sensor Service Sensor Manager
Sensor JNI
SensorManager
Android Application
Hardware
Linux Kernel
Libraries
Frameworks
Application
HARDWARE
Hardware
7/15/2012 7 Costillo- OSCON 2012
Linux Kernel
Libraries
Frameworks
Application
Hardware Architecture
7/15/2012 8 Costillo- OSCON 2012
Sampling Rates:
The 3 Rates
Under-sampling
Inaccurate, sluggish
response
Slight power savings
Sampling
Rate
Over-sampling
Accurate, smooth
response
Power-hungry
7/15/2012 9 Costillo- OSCON 2012
Wake up events and power
considerations
Application
Processor only
Internal
Coprocessor
External
Processor
Reference supported
Most power hungry
Reference supported
Most work done for
you
More processor selection
More outcome control
Most customized
Footprint impact

7/15/2012 10 Costillo- OSCON 2012
Hardware Summary
Power
Consumption
= sensors
n
+
any dedicated
processor
Latency =
Max(sensors
n
)
+ dedicated
processing
time
Sensor
Solution
Use tie-breaker criteria

7/15/2012 Costillo- OSCON 2012 11
KERNEL
Linux Kernel
7/15/2012 12 Costillo- OSCON 2012
Hardware
Libraries
Frameworks
Application
Sensor
Kernel Driver
7/15/2012 13 Costillo- OSCON 2012
Coprocessor
Microcontroller
Application Processor
Peripheral
Interface
Shared
Memory
LIBRARIES AND
SERVICES
Libraries
7/15/2012 14 Costillo- OSCON 2012
Hardware
Frameworks
Application
Linux Kernel
Sensor HAL and Services
HAL
device/<vendor>/<board name>/libsensors

Service
frameworks/base/services/sensorservice

Manager
frameworks/base/libs/gui
7/15/2012
15
Costillo- OSCON 2012
Sensor Fusion
7/15/2012 Costillo- OSCON 2012 16
Sensors
Linux Kernel
Libraries
Sensor Hub
http://en.wikipedia.org/wiki/Sensor_fusion
https://www.llnl.gov/news/newsreleases/2010/NR-10-01-06.html
Gesture Detection Algorithm
Sensors
Sensor Hub
Application
Processor
Android
SensorService
Co-
Processor
MPU with
Gyro/Accel
Compass
Barometer Proximity
7/15/2012 17 Costillo- OSCON 2012
Gesture Detection Comparison
Make Buy
Complete
solution
Already Tested
&tuned
Minimal
Schedule Impact
Compact code
In-house
expertise
Off-load AppPro
Application
Powerful processor
Sensor hub
Off-load to cheaper
power
Wake up Event
Handling
7/15/2012 Costillo- OSCON 2012 18
Calibration
7/15/2012 19
Costillo- OSCON 2012
Use of Calibration Gesture in the
Compass App By Catch.com
FRAMEWORKS
7/15/2012 20 Costillo- OSCON 2012
Hardware
Application
Linux Kernel
Libraries
Frameworks
Virtual Sensors
Leverages 1+ physical or other virtual
sensors
Multiple Options
Google (version 3)
Reference Vendor
Sensor Vendors
7/15/2012 Costillo- OSCON 2012 21
Android Virtual Sensors
Gravity Accel
Linear
Accel
Accel Gravity
7/15/2012 Costillo- OSCON 2012 22
Android Virtual Sensors
Gyro Accel
Orientation
Compass Accel Rotation
7/15/2012 Costillo- OSCON 2012 23
Virtual Sensors Challenges
Garbage In- Garbage Out
Latency
Non-Synchronized
samples
Implementation
Dependencies
Multi-vendor problems,
verify Vendor ID
http://www.invensense.com/midc/presentations/James%20Lim.pdf
7/15/2012 24 Costillo- OSCON 2012
Sensor
1
Sensor
2
Virtual
Output
Sensor
1
Sensor
2
Virtual
1
Virtual
2
APPLICATIONS
7/15/2012 25 Costillo- OSCON 2012
Hardware
Linux Kernel
Libraries
Application
Frameworks
Using Sensors
7/15/2012 Costillo- OSCON 2012 26
mSensorManager =
getSystemService(Context.SENSOR_SERVICE);
mSensorManager.registerListener(
mSensorListener,
mSensorManager.getDefaultSensor(
Sensor.TYPE_ACCELEROMETER),
SensorManager.SENSOR_DELAY_NORMAL);

void onSensorChanged(SensorEvent event) {
// get sensor data
float x =
event.values[SensorManager.DATA_X];
}

Using Sensor (Continued)
SensorManager.getRotationMatrix(
m_rotationMatrix,
null,
m_Mag,
m_Accels);
SensorManager.getOrientation(
m_rotationMatrix,
m_orientation);

float yaw_deg = m_orientation[0] * 57.2957795f;
float pitch_deg = m_orientation[1] * 57.2957795f;
float roll_deg = m_orientation[2] * 57.2957795f;
7/15/2012 Costillo- OSCON 2012 27
Types of Sensor Problems
Bias
Drift
Settling Time
Jitter/Noise
Environmental
Interference
7/15/2012 Costillo- OSCON 2012 28
Bias
Problem: Data is off
by a constant value.
Sources:
static calibration
failure
Solutions:
Calculate linear
offset at start of
application
Recalibrate locally


7/15/2012 Costillo- OSCON 2012 29
Drift


Problem: Shift of
data without cause
Sources:
Magnetic
interference
Poor HW calibration
Solutions:
Increase smoothing
techniques


7/15/2012 Costillo- OSCON 2012 30
Settling Time
Problem: Extended
time before
finalized steady
data.
Sources:
Latency
Sensitivity
Solutions:
Limit additional
processing


7/15/2012 Costillo- OSCON 2012 31
Noise
Problem: Data jumps
around constantly
Sources:
Sensor
Calibration
Poor filtering
Solutions:
High pass filter
Linear averaging
FFT
7/15/2012 Costillo- OSCON 2012 32
Environmental Interference
Problem: Inconsistent
results
Sources:
Magnetometer
EMI
Solutions:
Reference Device
Calibration gesture


7/15/2012 Costillo- OSCON 2012 33
Best Practices in
Application Development
Select the right sensor for the job.
Use the Correct Data Rate.
UI or GAMING are the most common.
Use Sensor In Context
Customize for your hardware and system
capabilities
Magnetometer-based sensors are the most
touchy.
Keep the Gesture UI simple.

7/15/2012 34 Costillo- OSCON 2012
QUESTIONS?
JEN@REBELBOT.COM
Additional resources
http://processors.wiki.ti.com/index.php/Android_Sensor_PortingGuide
http://www.kandroid.org/online-pdk/guide/sensors.html
http://invensense.com/midc/
http://developer.android.com/reference/android/hardware/SensorEvent.html
7/15/2012 35 Costillo- OSCON 2012