DCS Overview

MCS/DCS
Technical Interchange Meeting
August, 2000
 The Mission

The DCS shall provide the infrastructure

to support the operation of facility
instruments on SOFIA using best
current knowledge and tools, and
supporting continuous improvement in
an efficient, extensible, and modular
architecture.
-- DCS CoDR, June 1999.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Requirements

The recommendations of the CoDR

committee are the top level requirements
for the DCS.
-- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Requirements

The DCS shall record and archive all data

necessary for complete analysis,
calibration and future access to
observational data obtained with the FSIs.
The DCS shall support a select number of
observing modes with pipeline reduction,
including quicklook analysis, for each of
the FSIs.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Requirements

The DCS shall provide on-line

documentation necessary for the GI to
propose and carry out observations with
the FSIs.
The DCS shall be implemented as a
system of distributed modules,
integrated by an extensible hardware
and software infrastructure.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Requirements

Functionality mandated by SOFIA leaders

covers two functional domains:
SOFIA: Data acquisition from live
instrument, fast in-flight reduction
(quicklook).
SSMOC: Data acquisition from
simulators and other external sources,
full reduction of FSI data.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Requirements

Goal: Avoid building two separate systems if at

all possible.
Observation: The functional role is the same, only
the algorithms and equipment varies.
Solution: Two DCS systems, one in SOFIA and
one at SSMOC. Each instance of the DCS has
the pipelines and interfaces needed for its
environment. The core DCS is identical and
invariant.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Design Goals

The DCS must support and benefit from

continuous improvement, both in itself
and within the SOFIA program, over a
twenty year lifetime.
-- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Design Goals

The DCS design must be:

Modular
Extensible
Maintainable
This is the heart of continuous
improvement.
-- DCS PDR, March 2000.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Design Goals: Modular

The DCS must not be a monolithic

program.
The DCS must not be limited to a single
system.
The DCS shall be a collection of small
independent services, residing on one or
more machines, providing functionality
as needed.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Design Goals: Extensible

The DCS must support the easy

incorporation of new procedures and
techniques to its repertoire.
The DCS will provide for the safe test
and evaluation of new components,
while maintaining access to established
proven components.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Design Goals: Maintainable

The DCS must not be tied to any specific

vendor or platform.
The DCS must be based on open, widely
available standards and technologies.
The DCS must be well documented,
both in design and implementation.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Base Technologies

Technologies on which the DCS system is

based:
Distribution of and communication
between objects across a system.
(CORBA)
Extensible and flexible exchange of
information, both in data and
documentation. (XML)

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems

DCS Data
Storage Acquisition

User
Interaction
Layer

Task Data
Library Reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: User Interaction

Common interface to all DCS resources

The DCS experience is customizable
on a per user basis without affecting the
rest of the DCS
Leverages off the web and related tools
Access regardless of geographic location
Relies only on commonly available
technology at users end (e.g. web browser)

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: User Interaction

DCS Data
User profile Storage Acquisition

User
Interaction
Layer

User Proxy
Requests and Responses Task Data
Library Reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Task Library

Home to all core functionality on users

behalf.
Can build on simple tasks to provide
sophisticated sequences of actions.
Easily extended with new activities and
procedures.
Once you know how to do it, we can
automate it.
RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Task Library

DCS Data
Storage Acquisition

User Storage access Initiate,

Interaction is unique to monitor, and
tasks needs control data
Layer acquisition

User Proxy
Requests and Responses Task Data
Library Reduction
Initiate and
monitor data
reduction
RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Store

Includes retrieval and storage of raw

science data, but also much more.
Captures everything (e.g. raw FSI data,
reduced data, OBS plans, instrument modes,
pipeline parameters, science personnel).
Enables easy import & export in variety
of formats (e.g. general investigators, IPAC)
by being platform neutral.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Store
Obtain OBS
plan, store raw
DCS FSI data Data
User profile Storage Acquisition

User Storage access Obtain raw FSI

Interaction is unique to data, store
tasks needs reduced data
Layer

Task Data
Library Reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Acquisition
Data acquisition, like the rest of the DCS, is
composed of small modules working in
concert together.
Modularity insulates the DCS from
instrument specifics, and encourages re-use
between similar instruments.
Effectively, the DCS translates an
experiment to instrument specific
commands and needs.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Acquisition
Obtain OBS
plan, store raw
DCS FSI data Data
Storage Acquisition

User Initiate,
quick
Interaction monitor, and
look
control data
Layer acquisition

Task Data
Library Reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Reduction

Instrument science teams focus on

algorithm development, not the DCS.
Reduction is automatic, distributed,
pipelined, and relentless.
DCS Data Reduction removes the need
for every GI to have their own compute
engines, but does not prevent GI from
doing their own reduction.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems: Data Reduction

DCS Data
Storage Acquisition

User Obtain raw FSI

Interaction data, store
reduced data
Layer

Task Data
Library Reduction
Initiate and
monitor data
reduction
RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Subsystems

DCS Data
Storage Acquisition

User
Interaction
Layer

Task Data
Library Reduction

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Technologies

Built on top of the COTS technologies

CORBA and XML, the DCS team is
developing four new technologies that
enable the overall DCS:
DCS Registry
Data Typing
Data Connection Association Object (DCAO)
XML Parser Generator

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Technologies: DCS Registry

Enables various modules in the DCS to

locate each other based on combinations of
names, versions, and other criteria.
Allows simultaneous, non-interfering
deployment of modules with the same
functionality:
Modules that are proven and trusted
Modules undergoing evaluation and test

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Technologies: Data Typing
Provides a live dictionary for use in the DCS
that defines various data types and formats.
Best explained by example: data typing allows
objects to interact with each other using
rawFrameAIRES instead of
IEEEsingle[1024][1024].
More expressive than IDL; addresses more
portability and platform issues.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Technologies: DCAO

Provides common language, interface,

and linkages between groups of objects
that must locate and communicate with
each other at runtime.
Directly enables the Data Acquisition
and Data Reduction subsystems, both of
which rely heavily on dynamic
collections of objects.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Technologies: Parser Generator

Extends existing event-driven XML

parser (expat) by generating the
support code normally written by
engineers to use a particular DTD.
Software writing software, that
ultimately yields a parsed DOM-like
object for any given combination of
XML document and DTD.

RIT Center for Imaging Science SOFIA Data Cycle System: Overview
 DCS Rundown
Subsystems Technologies
User Interaction Registry
Task Layer Data Typing
Data Store DCAO
Data Acquisition XML Parser
Data Reduction Generator

RIT Center for Imaging Science SOFIA Data Cycle System: Overview