Embedded Target Communications With TCF: Martin Oberhuber Michael Scharf Wind River

Embedded Target Communications
with TCF
Martin Oberhuber
Michael Scharf
Wind River
Tutorial Themes
How we’re going to run this:
• Practical
• Interactive
• Workspace Take-away
2 TCF Tutorial | © 2007, 2009 Wind River Systems, Inc. | all contents
except logos and trademarks made available under the EPL v1.0
The Stick
• All-in-one Eclipse ZIP
• QEMU/mini.zip
• Workspace.zip
• Compilers
– OPTIONAL - For the adventurous
– We provide pre-built executables
– If you want to build yourself, install the compiler EXE’s
• Tools
– Ingredients of the all-in-one. Install if you want your own
Eclipse.
• Tcf_source
– patches
Agenda
• TCF Overview
• Building and running TCF native
– Workspace Setup
– Command-line tools
– Protocol Basics
• Building and running embedded
– Source structure, Removing a Service
– Proxy Setup
• Adding a Service or Value-add
– The C side (client and server)
– The Java side (client)
System Debug: the Big Picture
Eclipse
SoC (Model)
3rd party TCF
SW TCF
CPU
CPU
Debugger Target Target
Description Server(s) TCF HW
IP-XACT, IP
3rd party XML
SW
TCF
DSP
TCF
DSP
Debugger
This slide
except © SPRINT
logos and Infineon
and trademarks made2007, 2008.
available All rights
under reserved.
the EPL v1.0
Today: Separate Communications per tool
Tool A Tool B Tool C Tool D

UI
P2
Value Add Value Add
Host P1 B C
P3
Agent A Agent B Hardware C

Target
Bad for the User
? ? ?
UI
P2
Value Add Value Add
Host P1 B C
P3
Agent A Agent B Agent C

Target
Bad Design!

UI
Value Add Value Add

Host B C
Agent A Agent B Agent C

Target
Limited eco system
• Too many different tools agents and protocols

• Add-on providers need to provide multiple integrations
• Huge effort putting it all together end-to-end
• Lock-in to single vendor for end-to-end solution
– No best-of-breed mashup solutions
Design Goals of TCF
• Protocol Framework provides common infrastructure

– communication protocol
– Agent: “Service container”
– Proxying
• Same protocol on all layers supporting value-add
– Support pass-through
• Tools can use services in uniform way
• Service implementers can focus on functionality
TCF vs. ECF
• TCF – Target Communication (Protocol) Framework
– Protocol, independent of API
– API in multiple languages (C, Java, Perl, …)
– One extendable Protocol (though supports multiple transports)
– Typically point-to-point only
• ECF – Eclipse Communication Framework

– API independent of Protocol
– API in Java - abstract specification of concepts:
• Message, Channel, Container, ID
• Datashare, Filetransfer, Directory Listing
– ONE API for multiple protocols (e.g. Files; FTP, HTTP, EFS, …)
– Very flexible one-to-many communications
• TCF and ECF are similar, but ECF is on a higher layer
TCF - Core Design Ideas
• Use the same extendable protocol end-to-end

– allow value-adding servers to intercept select services
• Extension: Abstract Services as building blocks
– Same tool for multiple targets (e.g. agent, OCD, simulator)
– Avoid tools specific agents
– Bridge gap with specific services to configure common ones
• Data-driven by target
– Service knows best how to represent the system
– If not possible, put the knowledge in the lowest possible layer and data
drive the layers above
• Support high latency communication links
TCF: Common agent and protocol
UI Tool A Tool B Tool C Tool D
Host Service 4
Value Add
Service 5
TCF Agent
Target Service 1 Service 2 Service 3
Peer is a Communication endpoint
Host Service 4
Value Add
Service 5
TCF Agent
Service
Host Service 4
Value Add
Service 5
TCF Agent
Message Channel
TCF Channel
Host Service 4
Value Add
Service 5
TCF Channel
TCF Agent
Channels and Messages
• Communication between peers use channels

• Channels abstract/hide the transport layer
– Currently TCP
– Possible: RS232, JTAG, USB etc
• Channels transmit Messages
TCF Communication Protocol
• Message
– A packet of data
– Transmitted over communication channel
• Multiple channels per peer
• Proxying/Tunneling
– Message forwarding
– “Decorator” can intercept communication
– “Value-add” services (e.g. debug info)
– No protocol conversion needed
The TCF Message Types
• Command (send)
– Request some action on remote peer
• Progress
– Long running commands may send “progress ticks”
• Result (reply)
– Remote peer sends one result for each command!
• Event
– Notify a change
• Flow control
– prevent congestion
Command
• Command is a message
• Sent to remote peer
• Remote peer must send one Result for each Command!
Result
• Result is a message
– Data, Errors
• Sent as a response to a command
• Remote peer sends one Result per Command!
Commands are Asynchronous
• Sender of a command returns immediately

– Fast on high latency connections
• Sender receives results
– asynchronously
Event
• Event is a message
• Notification about state changes
Message order guaranteed on channel
• Keep state of remote peer

• Without message ordering no consistent state
• Ordering per channel and direction
Command X=1
Result X=1
Event X=2
Command X=3
Event X=4
Result X=3
Event X=5
Flow Control
• Prevent congestion
TCF: Layered View
Host Service 4
Value Add
Service 5
TCF Agent
Communication Layers
Services
Messages
Channel
Transport Layer (e.g. TCP/IP)
Transport Layer: Byte Stream
Services
Messages
Channel
?????????????????????????????????????????????
Channel: Message Stream
Services
Messages
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Messages: Command Result Event Flow
Services
Messages C 17 SysMonitor getChildren ??????
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Message Type
Message Type
Services
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Token
Token: Channel Unique
To identify Result
Services
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Service Id
Service id
Services
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Command Name
Command name
Services
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Arguments (byte array)
Arguments (byte array)
Services
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
Argument Representation: e.g. JSON
Marshalling: JSON
Services C 17 SysMonitor getChildren [“p123”]
Channel ??????????????????????????????????????????
??????????????????????????????????????????????
The 5 TCF Messages
Command
C • <token> • <service> • <command> • <arguments>
Progress
P • <token> • <progress_data>
Result
R • <token> • <result_data>
Event
E • <service> • <event> • <event_data>
Flow
F • <traffic_congestion_level> •
TCF and the OSI Layers
7 Application Layer Services
6 Presentation Layer JSON
5 Session Layer Channel + Messages
4 Transport Layer
3 Network Layer
2 Data Link Layer
1 Physical Layer
TCF as Layer 3 and 4
7 Application Layer Services
6 Presentation Layer JSON
5 Session Layer Channel + Messages
4 Transport Layer
3 Network Layer
2 Data Link Layer
1 Physical Layer
Prototype implementation
• Lightweight C-based agent

– Minimal agent has < 4000 lines of C code
– Minimal impact on the target
• Eclipse plug-ins
– Java TCF implementation to access service
Getting Real: Development Tools
• Eclipse-TCF-Preinstalled.zip
– Eclipse SDK 3.5m6
– CDT 6.0m6
– RSE 3.1m6
– Optional: Subclipse with GEF or Subversive
– Target Emulator
• QEMU-arm-tcf.zip with mini Linux + ssh
• Compilers: *.exe with installer

– Cygwin, or MinGW 5.1.4+MSYS (with Wascana), or VS Express
– CodeSourcery g++ Lite for ARM
Workspace Setup and Build
• Import into Workspace: tcf.zip

– Import > General > Existing Projects > Archive
– Or import SVN Team Project Set
• Build TCF Agent native / cross
– User-defined build (hand written Makefile) – external shell
• Type “make” on host
• Type “make ARCH=ARM” for target ARM
– Visual Studio (Express): open agent.sln and build
• Need VS for Windows Debugging
• Need external dbghelp.dll for bug fixes – CQ 2553
TCF Sample Session
• Open Command prompt 1 for agent: Logging to stdout
– cd org.eclipse.tm.tcf.agent/Cygwin/i686/Debug
– ./agent -L-
• Open Command prompt 2 for client: Agent is auto-detected
– ./client -L-
Shortcut: connect TCP::
– help
– peers
– connect tcp:127.0.0.1:1534 JSON messages
Using an ID
– tcf FileSystem roots
– tcf FileSystem opendir "/root"
– tcf FileSystem readdir "FS0"
– tcf <serviceName> <command> <JSONargs>
Auto Discovery
• Discovers Peers and Services
• Simplifies setup
– Client UI can easily find peers and discover services
• TCF comes with a simple UDP auto discovery

– Other mechanisms possible and optional
How Discovery works
• First agent binds UDP::1534 to listen

– If port is not available, somebody else is master already
• Become slave: periodically send “Hello” to UDP::1534
• Master responds with list of known peers
• Master listens for new slaves announcing themselves
– Keep list of known slaves, broadcast to all when asked
• When master dies, a slave will get no response on poll
– After timeout on poll, slave becomes new master
– It has the full list of known peers already at this time
• Across networks: fixed master – tcfreg program
• Other implementations of Protocol / API are possible
Cross-compiling for QEMU ARM
• Define OPSYS, ARCH in the Makefile

• Not all Services may be supported on target Platform
– Use config.h to disable unwanted services
• Look at SVN diffs in your Workspace

• Once built, use RSE “SSH Only” to drag & drop
executables into the remote. Don’t forget chmod +x
Agent Structure
• Documented on TCF Agent Prototype docs
Services: filesystem, sysmon, processes;

main*, test, config, diag.
runctrl, memory, breakpoints, registers;

expressions, stacktrace, symbols
context, dwarf*, linen.,

memorymap, windbg
Core services
proxy, discovery*, streams
Core handlers / event dispatch

protocol, channel*, streams, event, inputbuf, ip_ifc tcf.h
Infra: cmdline, errors, exceptions, asyncreq

json / base64, mdep, myalloc, trace
Agent Structure (2)
• Main and config configures and instantiates framework

and services („hook up“)
– #define SERVICE_xxx
• Event loop sends request into service, service
responds immediately or asynchronously
• One event Thread only
• Service protocol spec relates to functions in code:
– command_roots, event_context_created, ...
• Client cmdline is just another source of events
Advanced: connecting into QEMU
• This is in run.bat, to redirect SSH (2222) and port 7000:

– qemu.exe –redir tcp:2222::22 -redir tcp:7000::7000
• Start run.bat -sURL
Server socket on port 7000
– Login with “root : root”
• Or ssh localhost –p 2222 –l root
– ./agent –L- -sTCP::7000
• RSE: New TCF connection to localhost:7000
– Show QEMU Linux processes
Connecting QEMU from outside
• This is in puppy_redir.bat:
– start puppy.exe -redir tcp:1534::1534
• 1534 is the TCF default port for discovery. QEMU
forwards it from the client to the host in both directions
• From Eclipse, launch RSE+TCF
– Run > Debug Configurations > Eclipse App
– Open RSE Perspective
– New Connection : TCF
– Files Subsystem Properties : Port : 7000
– Expand Processes / All Processes
 Shows QEMU Linux Processes
RSE sample
• Process service
• File service
Multiplexing: Tunnel into QEMU
• QEMU is not discoverable because it cannot talk to UDP::1534 –

since that port is not shared
– Use a remote value-add for dispatching multiple channels
• Inside QEMU:
Multiplexer on port 7000
– Stop agent.exe
– Shell 1: ./valueadd –L- -sTCP::7000
– Shell 2: ./agent –L- -sTCP::7010 Peer on 7010 is
• Using default port autodetected
• On local commandline
– ./client –L-
– connect TCP::7000 value-add supports multiple
– peers clients/channels
– tcf Locator redirect TCP::7010

– tcf FileSystem roots
More on command-line tools
• Tcfreg: dumbed-down agent as UDP discovery registry only

• Tcflog: no service itself: forwards requests and logs them
• Valueadd: provides Locator.redirect service only
• Code mostly common, in main_*.c

– l<num> : log level
– L<file> : log to file
– s<url> : for servers (agent, tcflog, valueadd): Port to expose
e.g. TCP:localhost:1534, default just TCP:
• Additional client option:
– S<scriptfile> : replay a script
• Additional agent options:
– i : interactive, -d : daemon, -t : test
The Java side
• org.eclipse.tm.tcf
– Plain Java, no dependency to Eclipse
– Protocol binding only
– Asynchronous, callbacks
Java: TCF Service Implementation
• Asynchronous: DoneMkDir is the Callback

• Commands go into a queue to run on Command Thread
Callback
More about Services
• Protocol Specification / Docs can be viewed locally

– org.eclipse.tm.tcf.docs project
• C Impl: Each has a macro define to enable/disable
– FileSearch “#if SERVICE_” to find impl.
– Common init function to register commands
– Common naming pattern
• Java Impl: Each service is an interface
– Extends Iservice
– Implemented by a Proxies
– Async callback mechanism
Required service: Locator
• Used to discover peers

• Peer lifecycle events
• When connected peers can list its services
TCF Services
• Run Control Service

• Breakpoints Service
• Memory Service
• Registers Service
• Processes Service
• Stack Trace Service
• System Monitor Service
• File System Service
Value-add
Host Service 4
Value Add
Service 5
TCF Agent
Use Case: SimpleJtagDevice
• Protocol
– TCP/IP
• Services
– Service Manager (returns fixed list of services)
– Debug (run-control, breakpoint, memory access)
– Possibly Others (flash programming, download, etc)
• No Dynamic Addition or Removal of Services
• No Multiplexing (single client)
• No Forwarding
• No Dynamic Discovery
except logos and trademarks made available under the EPL v1.0 60
Use Case: TestExceutionAgent
• Protocol
– Depends on OS configuration and board
• Services
– Service Manager (returns fixed list of services)
– Process launch and kill
– Standard I/O redirection
– File system access
• No Dynamic Addition or Removal of Services
• No Multiplexing (multiple clients)
• No Forwarding
• No Dynamic Discovery
Use Case: LinuxUserModeAgent
• Protocol
– Typically TCP/IP, but depends on OS configuration and hardware
• Services
– Service Manager
– Debug (run-control, breakpoint, memory access)
– OS Awareness (process/thread list, CPU utilization, etc)
– Process launch and kill
– Standard I/O redirection
– File system access
• Possibly Dynamic Addition or Removal of Services
• Possibly Multiplexing (multiple clients)
• Possibly Forwarding
• Possibly Dynamic Discovery
TCF Plugins
• Org.eclipse.tm.tcf – Core Java framework

• Org.eclipse.tm.tcf.agent – The agent (plain C)
• Org.eclipse.tm.tcf.debug.* - Debug Integration
• Org.eclipse.tm.tcf.docs –
• Org.eclipse.tm.tcf.dsf.* - DSF integration
• Org.eclipse.tm.tcf.examples.daytime.* - How to create
a custom Service (both agent and client)
• Org.eclpise.tm.tcf.rse – RSE Files and Processes
Debugging
• Run > Debug Configurations > TCF

– Select connection (auto-discovered)
– Program: /root/helloworld/helloworld
– Args: “tcf is cool”
– Debug
• Switch to “Debug Perspective”
• Show View “TCF Trace”
• Suspend / Resume, Registers
TCF Debugging example
TCF: Next Steps
• A large number of adopters today – “TCF Round Table”

– Mostly “whitebox adoption” today
– Conversion from TCF to legacy proprietary
• Protocol conversion agent (like valueadd)
• TCF C Agent binding to legacy implementation
• Core Protocol spec is frozen, working on Services
– Version / maturity / status info to be added to docs
• Getting involved:
– mailto:dsdp-tcf-dev@eclipse.org
– Bugzilla, Newsgroup, Wiki
Specification Status
• Review of current and specification of additional

services in power.org and Eclipse
• Transport Channel
• Current Services
– Run Control, Memory, Register, Breakpoint, Processes, Stack
Trace, File System, System Monitoring
Links
• TCF Homepage has all the pointers

– http://wiki.eclipse.org/DSDP/TM/TCF
– SVN Team Project Sets, ViewSVN
– Documentation
• Getting Started (less than what we did)
• Protocol Specification (messages, events, JSON)
• Services description
• Agent description
Questions?
Credits
• Idea:
– Felix Burton (Wind River),
– Eugene Tarassov (Wind River)
• Original implementation
– Eugene Tarassov

Embedded Target Communications With TCF: Martin Oberhuber Michael Scharf Wind River

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Embedded Target Communications With TCF: Martin Oberhuber Michael Scharf Wind River

Uploaded by

Copyright:

Available Formats

Embedded Target Communications

How we’re going to run this:

Tool A Tool B Tool C Tool D

Agent A Agent B Hardware C

Agent A Agent B Agent C

Tool A Tool B Tool C Tool D

Value Add Value Add

Agent A Agent B Agent C

• Too many different tools agents and protocols

• Protocol Framework provides common infrastructure

• ECF – Eclipse Communication Framework

• TCF and ECF are similar, but ECF is on a higher layer

• Use the same extendable protocol end-to-end

UI Tool A Tool B Tool C Tool D

UI Tool A Tool B Tool C Tool D

UI Tool A Tool B Tool C Tool D

UI Tool A Tool B Tool C Tool D

• Communication between peers use channels

• Multiple channels per peer

• Sender of a command returns immediately

• Keep state of remote peer

UI Tool A Tool B Tool C Tool D

Transport Layer (e.g. TCP/IP)

Messages C 17 SysMonitor getChildren ??????

Messages C 17 SysMonitor getChildren ??????

Messages C 17 SysMonitor getChildren ??????

Messages C 17 SysMonitor getChildren ??????

Messages C 17 SysMonitor getChildren ??????

Messages C 17 SysMonitor getChildren ??????

Services C 17 SysMonitor getChildren [“p123”]

Messages C 17 SysMonitor getChildren ??????

7 Application Layer Services

6 Presentation Layer JSON

5 Session Layer Channel + Messages

2 Data Link Layer

7 Application Layer Services

6 Presentation Layer JSON

5 Session Layer Channel + Messages

2 Data Link Layer

• Lightweight C-based agent

• Compilers: *.exe with installer

• Import into Workspace: tcf.zip

• Open Command prompt 1 for agent: Logging to stdout

• Discovers Peers and Services

• TCF comes with a simple UDP auto discovery

• First agent binds UDP::1534 to listen

• Define OPSYS, ARCH in the Makefile

• Look at SVN diffs in your Workspace

• Documented on TCF Agent Prototype docs

Services: filesystem, sysmon, processes;

runctrl, memory, breakpoints, registers;

context, dwarf*, linen.,

Core handlers / event dispatch

Infra: cmdline, errors, exceptions, asyncreq

• Main and config configures and instantiates framework

• This is in run.bat, to redirect SSH (2222) and port 7000:

• QEMU is not discoverable because it cannot talk to UDP::1534 –

– tcf Locator redirect TCP::7010

• Tcfreg: dumbed-down agent as UDP discovery registry only

• Code mostly common, in main_*.c

• Asynchronous: DoneMkDir is the Callback

• Protocol Specification / Docs can be viewed locally

• Used to discover peers