SDOD0060 P Guide To SDO Data Analysis Sdoguide

Guide to SDO Data Analysis
edited by Marc DeRosa & Greg Slater February 19, 2013
Contents
1 Introduction 1.1 Synopsis of the SDO Mission 1.2 About this Guide . . . . . . 1.3 Data Products from AIA . . 1.4 Data Products from EVE . . 1.5 Data Products from HMI . . How to Browse SDO Data 2.1 The Sun Now . . . . . 2.2 The Sun Today . . . . 2.3 SolarMonitor . . . . . . 2.4 The Helioviewer Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 4 4 4 10 10 10 10 10
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
How to Find SDO Data 17 3.1 Heliophysics Events Knowledgebase . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 iSolSearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 How to Get AIA and HMI Data 4.1 How Data are Organized at the JSOC . . . 4.2 The lookdata Tool . . . . . . . . . . . . . 4.2.1 Getting Dataseries Names . . . . 4.2.2 Selecting Records . . . . . . . . . 4.2.3 Exporting Data . . . . . . . . . . 4.2.4 JSOC Image Timing Details . . . 4.3 The Cutout Service . . . . . . . . . . . . 4.4 The Virtual Solar Observatory . . . . . . 4.5 Synoptic Data . . . . . . . . . . . . . . . 4.6 Uncompressing the Data . . . . . . . . . 4.6.1 Compiling and Installing imcopy 23 23 23 24 24 27 27 29 30 31 31 31
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
How to Use SolarSoft with SDO Data 41 5.1 Installing or Upgrading SolarSoft . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.1.1 Doing a Clean Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.1.2 Updating Packages for an Existing Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Guide to SDO Data Analysis version dated February 19, 2013 5.1.3 Using a Proxy Server with SSW . . . . . . . . Querying the HER . . . . . . . . . . . . . . . . . . . 5.2.1 Basic Query Syntax . . . . . . . . . . . . . . . 5.2.2 Adding Filters to Queries . . . . . . . . . . . . Retrieving Data from the JSOC . . . . . . . . . . . . . 5.3.1 Getting Dataseries Names . . . . . . . . . . . 5.3.2 Selecting Records . . . . . . . . . . . . . . . . 5.3.3 Exporting Data . . . . . . . . . . . . . . . . . 5.3.4 Exporting Specic Segments . . . . . . . . . . Requesting a Cutout Using SSW . . . . . . . . . . . . Using the SSW Interface to the VSO . . . . . . . . . . 5.5.1 Querying the VSO . . . . . . . . . . . . . . . 5.5.2 Downloading Data from the VSO . . . . . . . Uncompressing the Image Data . . . . . . . . . . . . . 5.6.1 Using read_sdo.pro . . . . . . . . . . . . 5.6.2 Using read_sdo.pro with a Shared Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 43 43 44 46 46 49 52 54 57 58 58 60 61 61 63 65 65 68 69 71 73 76 76 76 77 78 78 81 84 86 89
5.2
5.3
5.4 5.5
5.6
How to Process AIA Data 6.1 Converting AIA Data to Level 1.5 . . . . . . . . . . . . . . . . . . . . . . . 6.2 Co-Aligning HMI Data with AIA Data . . . . . . . . . . . . . . . . . . . . . 6.3 Despiking/Respiking AIA Data . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Point Spread Function Deconvolution . . . . . . . . . . . . . . . . . . . . . 6.5 Getting AIA Filter Response Data . . . . . . . . . . . . . . . . . . . . . . . 6.6 Miscellaneous AIA Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.1 Using AIA Standard Scaling and Colors . . . . . . . . . . . . . . . . 6.6.2 Creating AIA Tri-Color Images . . . . . . . . . . . . . . . . . . . . 6.6.3 Plotting an AIA Light Curve . . . . . . . . . . . . . . . . . . . . . . 6.6.4 Interfacing with plot_map.pro (Dom Zarros Mapping Software) 6.6.5 Checking the QUALITY Keyword . . . . . . . . . . . . . . . . . . . Frequently Asked Questions Lists of Useful Links Version History of this Guide
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
7 8 9
10 Contributors to this Guide
PDF version of this guide
Guide to SDO Data Analysis version dated February 19, 2013
1 Introduction
1.1 Synopsis of the SDO Mission
The Solar Dynamics Observatory (SDO) is the rst mission to be launched under NASAs Living With a Star (LWS) program. This program involves a diverse set of research topics that aim to provide a greater understanding how the activity and variability of the Sun affect life on Earth. The purpose of SDO is to understand how the Suns magnetic eld is generated and structured, and how this stored magnetic energy is converted and released into the heliosphere and geospace in the form of solar wind, energetic particles, and variations in the solar irradiance. SDO was launched aboard an Atlas V rocket on 11 February 2010 from Kennedy Space Center in Florida. It is now located in a nearly geosynchronous orbit that allows continuous contact with its dedicated ground stations in New Mexico. The observatory has been operating nominally since launch, without any major problems. The observatory contains three instruments (presented in alphabetical order): The Atmospheric Imaging Assembly (AIA) is an array of 4 telescopes that together provide full-disk images of the solar corona at 1 resolution (40964096-pixel images) in 10 UV and EUV wavelengths every 10 seconds; The Extreme ultraviolet Variability Explorer (EVE) measures the EUV spectral irradiance with unprecedented spectral resolution, temporal cadence, accuracy, and precision; and The Helioseismic and Magnetic Imager (HMI) provides full-disk, high-cadence Doppler, intensity, and magnetic images at 1 resolution (40964096-pixel images) of the solar photosphere, allowing studies of the sources and evolution of activity within the solar interior. Together, this suite of instruments enables the monitoring of the solar interior, chromosphere, and corona with high spatial and continuous temporal coverage, as described in the SDO overview paper (Pesnell et al. 2012; appearing in the SDO issue of Solar Physics). Lowlevel processing and calibration for data from the two imaging instruments (AIA and HMI) occurs at the Joint Science Operations Center (JSOC), located on the campus of Stanford University. EVE data are processed at the Laboratory for Atmospheric and Space Physics (LASP) at the University of Colorado. Brief descriptions of the data products produced by each of the three SDO instruments are given below in Sections 1.3, 1.4, and 1.5.
1.2 About this Guide

The massive amount of image data produced by SDO (of order several terabytes per day) requires new techniques for processing, browsing, retrieving, and analyzing these data. Indeed, signicant samples of full-disk, full-resolution images from either AIA or HMI can easily exceed hundreds of gigabytes. Consequently, remote users will not be able to download very much SDO image data before it is realized that either ones internet bandwidth is not sufcient to download the amount of data requested in a reasonable amount of time, or ones available on-site computing power and storage are not sufcient to interact or process the data once it resides locally (or both). The purpose of this guide is to demonstrate the ways in which users can more readily progress through the sequence of browsing, nding, downloading, and analyzing SDO data. To that end, the guide contains sections describing: How to browse data: Various Sun-in-time pages, summary images and movies, and popular event and feature listings (Section 2) How to nd data: Heliophysics Event Knowledgebase, a searchable database that is useful for nding events and features of interest during (and prior to) the SDO era for which the dates are not known (Section 3) How to get data: Multiple interfaces to the data centers through which SDO data can be downloaded (Section 4)
Guide to SDO Data Analysis version dated February 19, 2013 How to nd and get data using SolarSoft: Demonstration of SolarSoft IDL commands useful for nding and downloading data (Section 5) How to perform common AIA data-processing tasks: Demonstration of how to manipulate AIA data for scientic research (Section 6) Frequently Asked Questions list (Section 7), list of useful links (Section 8), and the version history of this guide (Section 9) The guide is a living document, and materials will be added as needed. The online version of this guide can be found either at http://www.lmsal.com/sdouserguide.html, or by navigating to the SDO Documentation (SDOdocs) webpage where it is the rst link in the list. Alternatively, a PDF version of this guide is also available. Both versions of the guide contain numerous links in order to make nding SDO-related information easier. In the PDF version, such links are active, with external links colored blue, and links to other items in the guide colored red. If you have questions, problems, or other feedback related to the analysis of SDO data or other related topics that do not seem to be covered in this guide, please send an e-mail to the address listed on this webpage.
1.3 Data Products from AIA

AIA obtains full-Sun images in multiple EUV and UV passbands, as summarized in Figure 1 and described in more detail on the AIA Instrument webpage. These images are used to generate higher-level data products, including browse-quality data and feature and event entries in the Heliophysics Events Knowledgebase (see Section 3.1), all of which in turn support the science requirements of AIA. A summary of this process appears in the owchart presented in Figure 2. For more details about AIA, refer to the AIA instrument paper (Lemen et al. 2012; appearing in the SDO issue of Solar Physics). The different data-processing levels for AIA are summarized as follows (more details are available from this document on SDOdocs): Data at Level 0 are images that have been constructed from the raw telemetry stream. Data at Level 1.0 are images that have been converted from Level 0, with processing including bad-pixel removal, despiking and at-elding. All higher-level products for AIA are based on Level 1.0 data. Data at Level 1.5 are images that have been adjusted to a common 0.6 plate scale, and that share common centers and rotation angles, but are not exposure-time corrected.
1.4 Data Products from EVE

EVE produces a multitude of spectral and photometer data products that support the EVE primary science objectives, as discussed in the EVE instrument paper (Woods et al. 2012; appearing in the SDO issue of Solar Physics). EVE data products are summarized in Figure 3 and available through the EVE data access webpage.
1.5 Data Products from HMI

HMI produces a multitude of data products that support the HMI science investigation. The four main observables are: Dopplergrams (maps of line-of-sight photospheric velocity), continuum images (images of the solar photosphere near the 6173 absorption line of Fe I), and both line-of-sight and vector magnetograms (maps of the photospheric magnetic eld). The images of Doppler velocity comprise the input for both global and local helioseismology analyses. Time series of these images are used to infer a mean rotation and sound speed in the solar interior. Such time series are also used to sense the presence of active regions either before they have rotated onto the Earth-facing half of the sun or before they have fully emerged onto the surface. The magnetic data products include line-of-sight and vector magnetic maps, synoptic maps, the locations of surface activity, and related space-weather indicators. Parameters and data cutouts associated with HMI Active Region Patches (HARPs), such as geometric extent, total ux, etc., have also been catalogued. Please refer to this webpage for additional information regarding HMI magnetic data, in 4
Guide to SDO Data Analysis version dated February 19, 2013 particular this webpage for detailed examples of HARPs, and lastly this webpage for a summary of the vector magnetic eld processing scheme. The different data-processing levels for HMI are summarized as follows, with more details available from this source: Data at Level 0 are images that have been constructed from the raw telemetry stream. Data at Level 1.0 are images that have been converted from Level 0, with processing including bad-pixel removal, at-elding, and quality assessment checks, but otherwise not having undergone any irreversible data alterations. Data at Level 1.5 are images of the physical observables (Dopplergrams, magnetograms, and continuum images), which were constructed from the individual Level 1.0 ltergrams. Data at Level 2 have been irrevocably ltered, time-sequence-merged, Fourier-transformed or otherwise changed from Level 1.5 data in a way that is irreversible. Level 2 products include intermediate products for later production of mission science data products, such as helioseismic inferences of solar subsurface ows. For more details and specications, see the HMI instrument paper (Scherrer et al. 2012; appearing in the SDO issue of Solar Physics), the HMI instrument overview webpage at Stanford, or the accompanying analysis pipeline owchart in Figure 4.
AIA Telescopes, Wavelengths, and Thermal Response

sp
+Y +Z
he
ric
I m a gin g
A
ss
em
At mo
bly
XX
XII
XIV
XVI
LMSAL
He II
1600 C IV + Cont. 1700 UV Cont. 4500 White Light
XV
III
IX
N
A
Fe XIV
Fe XVI
304
UV
211
335
94
171
193
131
4
Fe XVIII
3
Fe IX
2
Fe XII / XXIV
1
Fe VIII / XXI
Looking at the AIA from the Sun
http://aia.lmsal.com/
Figure 1: A summary of the passbands of AIA, showing the lter responses of each of the EUV channels, and their arrangement on the instrument as viewed from the sun (source: Summary of AIA telescopes, wavelength channels, and thermal responses at SDOdocs). See also Section 6.5.
Figure 2: AIA science analysis pipeline, illustrating how the main ltergram data, along with data from EVE and HMI produce the data products that support the science goals of AIA. Note that AIA Level 1.5 data are not currently being corrected to a standardized exposure time (as is incorrectly indicated in the gure).
Figure 3: Table of EVE data products (sourced here), showing the range of wavelengths covered, their resolution, and the different levels of processing applied.
Figure 4: HMI science analysis pipeline owchart (sourced here), showing how the four main observables are used to create the many data products that support the science objectives of the HMI instrument.
2 How to Browse SDO Data

In this section, we briey illustrate a few of the online resources that allow users to browse SDO data: The Sun Now (Section 2.1), The Sun Today (Section 2.2), SolarMonitor (Section 2.3), and Helioviewer and JHelioviewer (Section 2.4).
2.1 The Sun Now

The SDO mission webpage hosted by NASAs Goddard Space Flight Center contains a wealth of information about SDO. The rst entry under the Data tab is The Sun Now, which contains thumbnail images, links to full-size and downsampled images, movies, and composite images from AIA and HMI. Also present are a soft X-ray image from the pinhole camera on EVE as well as a heliographic representation of the corona assembled from the two STEREO spacecraft and AIA. The other entries under the Data tab provide additional functionality, including templates that enable users to generate a customized image sequence or movie. The top portion of The Sun Now webpage is shown in Figure 5. Users may also nd useful the array of self-updating 48-hr movies.
2.2 The Sun Today

The Sun Today, located at http://sdowww.lmsal.com/suntoday, is a webpage summarizing images, movies, events, and features that occurred on the Sun through any given day in the SDO mission. The page keys off the date indicated at the top of the page, which can be changed to any date of interest either by using the plus and minus buttons in the upper-right corner of the webpage, or by entering a URL with the suntoday_date parameter set to the date of interest. For example, the summary page for 1 August 2010 is located at this link (http://sdowww.lmsal.com/index.html?suntoday_date=2010-08-01), as illustrated in Figure 6. The page layout (currently) includes three panels that contain: a series of AIA and HMI images for the day of interest, the results of an iSolSearch query (see Section 3.2) for that day, and a plot of AIA light curves for that day. The image panel contains a row of four preview images, under which are lists labeling various wavelength channels from AIA and HMI. In each column of labels, the label of the channel that is underlined corresponds to the preview image atop the column. Mousing over any of the other wavelength channel labels changes the preview picture, enabling users to compare images from up to four channels at once. Clicking on the links next to each wavelength channel will open a new browser window containing a larger image from that channel. For example, the 1K link will open a 10241024-pixel image, the 4K link will open a 40964096-pixel image, and the PFSS link will open a 40964096-pixel image with eldlines from a potential-eld source-surface model overlaid. Links to summary movies of four of the wavelength channels appear above the row of preview images. There is also a link to an interactive eldline viewer that allows the user to manipulate a rendering of pre-computed PFSS eldlines from within WebGL-enabled web browsers. To put these images in some context, the results of an iSolSearch query and a series of selected AIA light curves for the date of interest are displayed in the panels underneath the image preview panel. For more information on iSolSearch, as well as instructions describing how to use the full search tool, please see Section 3.2.
2.3 SolarMonitor
The SolarMonitor website provides a straightforward way to browse a variety of solar imagery, including magnetograms, continuum images, H images, and images in ultraviolet and X-ray passbands. The front page for 3 August 2010 is shown in Figure 7 below. Zoomed-in summary images for each NOAA-identied active region are also available (search for these rst by date, and then use the links along the left-hand sidebar or underneath the image display to view the active-region summaries). Links along the right-hand sidebar will take the user to external websites that provide other ways to monitor the sun, space weather, and the heliosphere.
2.4 The Helioviewer Project

The helioviewer.org data browser is a user-friendly way to align and display multiple images from a selection of recent solar and heliospheric observatories, as shown in Figure 8. The tool consists of a web-based interface that enables users to interactively pan and 10
Figure 5: The top portion of the The Sun Now webpage showing the array of AIA image channels.
11
Figure 6: The top two sections of The Sun Today webpage for 1 August 2010, showing the assortment of preview images as well as the results of an iSolSearch query for that day.
12
Figure 7: The SolarMonitor.org front page for 3 August 2010.
13
Guide to SDO Data Analysis version dated February 19, 2013 zoom an image, overlay images from different observatories with the proper registration, and indicate the location of features and events of interest. The interface downloads image and event data asynchronously (using AJAX), which allows the user to continue browsing while the tool downloads additional data. For more information, please visit the website and/or refer to the Helioviewer.org User Guide. JHelioviewer is a Java/OpenGL-based application to browse time series of solar images. JHelioviewer overlays image series from SOHO and SDO in space and time and makes use of the JPEG2000 standard to stream large datasets interactively in real-time. Users can, for example, zoom and pan full-size SDO movies, apply basic image processing methods (including feature tracking) and overlay event markers from the Heliophysics Event Knowledgebase (see Section 3.1). JHelioviewer is open-source and has a plugin architecture that allows users to expand its functionality. More information can be found by visiting the website, and in particular reading the JHelioviewer Handbook.
14
Figure 8: The helioviewer.org display showing an eruption visible in LASCO C2 and C3 images, as well as a (mostly) co-temporal and co-aligned image from the AIA 171 channel. The three images are all taken within 15 minutes of 2 August 2010, 00h UT.
15
Figure 9: A screen shot of the JHelioviewer application illustrating a zoomed-in portion of an AIA image with icons associated with Heliophysics Events Knowledgebase overlaid on the image.
16
3 How to Find SDO Data

In this section, we will demonstrate how to use the Heliophysics Events Knowledgebase (HEK). The HEK is essentially a list of observational sequences combined with a catalog of features and events that have occurred on the sun. It is useful in situations when the user knows that a particular type of event occurred, but who cannot remember exactly when. It is also useful in situations when the user wants to know whether a particular sequence of observation has occurred. The HEK allows users to report new features and events, and to contribute information on existing ones. Consequently, the HEK is useful in an environment where browsing large amounts of data is cumbersome (as will be the case during the SDO era), and allows users to focus on establishing connections between similar events and features. More information on the HEK is available in this article (Hurlburt et al. 2012; appearing in the SDO issue of Solar Physics).
3.1 Heliophysics Events Knowledgebase

The HEK is comprised of two major components: the Heliophysics Events Registry (HER) and the Heliophysics Coverage Registry (HCR). The HER indicates what features and events have been identied, and the HCR identies which data sequences are available. There is an interplay between the HER and HCR (see Figure 10), as searches for entries in the HER also reveal related observations cataloged in the HCR. We now discuss each one in sequence. As illustrated in the schematic in Figure 11, the HER accepts data from a multitude of providers, and various Feature Recognition Methods (FRMs) operate on these data streams and report events and features of interest. FRMs encompass both automated featurending algorithms as well as human annotators. A consortium tasked with developing many of the FRMs that supply events to the HER is described in this article (Martens et al. 2012; appearing in the SDO issue of Solar Physics). Users access the HER by web portals such as the iSolSearch tool (see Section 3.2), or can use the specications given by the HEK Application Programming Interface (API). Additionally, for users familiar with SolarSoft, there exist SSWIDL routines for querying with the HER (see Section 5.2). Each feature or event entry in the HEK contains information about its data source (e.g., observatory, instrument, wavelength), its spatiotemporal location, and the FRM (automated or human) used to identify the item. The full list of event classes tabulates the required and optional attributes for each class of event, and it is these attributes than can be used to query the HEK. Because the name of the FRM is an attribute for each event or feature stored within the HEK, multiple FRMs can generate events belonging to the same event class. There are many cases where the same physical feature has been identied by different FRMs, but no choice is made as to which one is better or more useful as such qualitative assessments often depend on their intended use within a research program. Using coronal mass ejection (CME) events as an example, multiple catalogs of CMEs based on different observatories and/or event specications (such as, for example, the LASCO (CDAW) CME Catalog, the SEEDS Catalog, the LASCO ARTEMIS Catalog, the STEREO COR1 Catalog, and the Cactus CME Catalog) are already in use by the community, and it is likely that the research community has beneted from having more than one independently generated catalogs of the same event. The same benets can be expected by being able to query events from multiple FRMs seeking to identify each event class in the HEK. The HCR is a tool that tracks data requests. Users can, for example, view recent or view popular data sequences from AIA, or view the recent events from SDO reported by human observers. Users can make more advanced searches using the HCR search form. As data (especially, the starting and ending dates, instrument, and wavelength) are entered into the HCR search form, the number of observational sequences that satisfy the query is updated at the bottom of the page. Clicking on the Show Search Result Details button reveals more details about each observation, including pointing information and a brief description of the science goal of the observation. These observations should be readily available via the Get All Data link on the summary page for each observation.
3.2 iSolSearch
The easiest way to interact with the HEK is probably by using iSolSearch, which enables users to easily query the HEK for a listing of events between two points in time. In this section, the basic usage of iSolSearch is summarized. Alternatively, there is a brief
17
Figure 10: A schematic illustration of the operation of the HEK, showing the interrelationship between the HER and HCR.
YouTube video (with audio) that also demonstrates the basic usage of the iSolSearch tool. Users are encouraged to ask questions in the YouTube comments section. To query the HEK and learn what happened on the sun during a particular time frame, simply enter two dates in the search panel, choose a selection of event classes, and press the Search button underneath the event-class listing. Any events in the HEK that satisfy this query will appear as icons on the disk view panel, as well as in a listing in the results panel to the right of the disk view. The results from a typical search are illustrated in Figure 12. Mousing over a particular icon will on the disk view will cause it to be highlighted in yellow in the event listing. Likewise, mousing over an event in the event listing will enlarge the icon in the disk view. When clicking on an event (either its icon in the disk view, or its corresponding entry in the event listing), the tool will display more detailed information about this event in the gray information panel, as shown in Figure 13. There, the user will nd a more detailed description of the event, a summary image or movie (if available), and a link to the underlying XML code (in the VOEvent XML link) associated with this event. Users can also add a reference or comment regarding this event, and rate this event, by clicking on the small icons at the top of the information panel. Observational sequences that overlap the selected event in both space and time, as found from a HCR search based on the event properties, are listed under the Observations in the neighborhood heading in the information panel. Clicking on one of these related observational sequences will reveal a summary page as well as a link to related data sequences that already exist (thus allowing users to avoid longer wait times when requesting data). If none of the HCR observational sequences satisfy the needs of the user, clicking on
18
Figure 11: A schematic illustration of the operation of the HER.
19
Guide to SDO Data Analysis version dated February 19, 2013 the Request data link leads to the data-request form for the cutout service (see Section 4.3), where data customized by the user can be requested. To customize the event display, the disk view panel has clickable buttons that zoom in and out in the upper-left corner, and clickable buttons that switch between an orthographic and (Carrington) latitude-longitude projections of the sun in the upper-right corner. Zooming in and out can also be done using the mouse scroll wheel when the cursor is located within main portion of the disk view panel. In the lower-left corner of the disk view panel, clicking on the arrows will rerun the existing query after adding or subtracting one day (single arrows) or one week (double arrows) to or from the start and end dates of the existing query. Clicking on the key icon in the lower-right corner will take the user to the aforementioned list of event classes, where the icons in the disk view panel are dened in terms of the corresponding event classes. More complex queries can be constructed by clicking on the Filters tab in the search panel. Users can add multiple lters that screen for, or screen out, events that satisfy a set of user-dened conditions. As an example, suppose one wants to nd all ares having GOES Class M2 or greater that occurred in the year 2010. Under the Search tab, set the start and end dates to 2010-01-01 and 2010-12-31, respectively, and clear all of the event-class checkboxes except for Flare. Then, under the Filters tab, adding a constraint like FL_GOESCls >= M2.0 is achieved by choosing FL_GOESCls (are GOES class) from the Attribute drop-down menu, setting the operator (Op) to >=, choosing M2.0 from the Value drop-down menu, and then pressing the Add button. After pressing the Add button, the constraint should appear in the white box. After going back to the Search tab, press Search button to query the HEK. At the time of this writing, this query returned 18 events. After the query has produced a listing of results within iSolSearch, the query itself and the associated list of results are available by clicking on the export link at the top of the panel showing the event listing. In the menu that appears, the Query link will open a window containing a URL of the query and the SolarSoft call link will open a window containing the same query formatted as an SSWIDL command using the SolarSoft API (thus allowing the event listing to be exported to SolarSoft). The listing of results is available by clicking on the Table(CSV) link. In this way, queries and their results can be saved for later use, or published in journal articles so as to be repeatable by other researchers. Furthermore, users can even subscribe to the query (allowing the listing of results to be updated, and the user notied) by clicking on the RSS link in order to set up an RSS feed.
20
Figure 12: The iSolSearch tool, showing the results of a query for 1 August 2010. The tool displays an assortment events that happened on this day, including several lament eruptions, coronal holes, active regions, ares, and even a coronal dimming event. The various events are displayed as icons (based on event class) in the disk view, and as a list in the results panel to the right of the disk display.
21
Figure 13: The iSolSearch tool, showing the gray information panel that provides more details about the lament eruption event from the results of the query shown in Figure 12. In the right-most panel, more details about this event are shown, including a description, links to a summary frame and movie, the starting and ending time, observatory, and FRM. Additional links to related resources also appear, including a link to the cutout request form that will allow users to download the associated data underlying this event from AIA. The icons at the top of the information panel permit users to add references or comments and rate the particular event being displayed.
22
4 How to Get AIA and HMI Data

Low-level processing and calibration for all HMI and AIA observables occurs at the Joint Science Operations Center (JSOC), located at Stanford University. After such pipeline processing has taken place, the JSOC is responsible for cataloging and archiving the data products and serving it to all users. In this section, the basic elements of the JSOC are briey described (Section 4.1), followed by descriptions of various ways to retrieve data (Sections 4.2, 4.3, and 4.4). As the data are typically exported in a compressed format, the nal section (Section 4.6) illustrates the various methods of uncompressing and reading in the data. Additionally, for users familiar with SolarSoft, there exist SSWIDL alternatives for retrieving data from the JSOC (see Sections 5.3, 5.4, and 5.5).
4.1 How Data are Organized at the JSOC

To better comprehend what is happening when retrieving data, it is useful to have a general understanding of how the data are cataloged in the JSOC. The system employed by the JSOC is called the Data Record Management System (DRMS), and was created by software developers at Stanford to manage the large volume of data and metadata produced by HMI and AIA. In the DRMS, data are organized into dataseries, which is the name given to a sequence of related or similar items. These items are called records, which in turn are comprised of keywords, their values, and the associated data segments. Most dataseries of interest currently published by the JSOC are listed in Table 1. While most dataseries are essentially sequences of images in time, such as hmi.V_45s (HMI Dopplergrams with a 45s cadence), aia.lev1 (AIA Level 1.0 data), as well as historical data such as mdi.fd_M_96m_lev18 (full-disk MDI Level 1.8 magnetograms with a 96m cadence), this is not always necessarily the case. Some dataseries contain derived data products such as synoptic or diachronic charts, or cubes of remapped Dopplergrams for use in helioseismology. Several points about the DRMS cataloging system are worth mentioning: A key aspect of the way the DRMS catalogs data is that all records of the same dataseries have the same set of allowed keywords. Of course, for different records these keywords will take on different values. Together, the list of records and list of keywords (and their values) form a database that is easily searched by DRMS. The keywords and their values are stored separately from the data they describe. This allows the metadata, i.e., the values of the keywords, to be accessed efciently (and updated) without having to deal with large image les. In some dataseries, such as hmi.V_45s, records contain exactly one image segment (such as the Dopplergram image in this example). In others, records contain multiple segments of related data. It is important to note that, regardless of how many segments a record contains, all segments of a record share many, if not most, of the same keyword values. As an example, most of the records in the aia.lev1 dataseries contain a data segment (the image) and another segment containing a record of pixels that were changed during the despiking process. Another example of dataseries that contain records with multiple segments are the various HMI vector magnetogram dataseries, for which the HMI Level 1.0 ltergram data are processed to yield various magnetic eld parameters, including |Btotal |, inclination and azimuth angles, lling factors, and uncertainties. Each of these quantities will be stored in a different image segment, and yet because these segments will be part of the same record, they will share the same set of keyword data. For more detailed information on the JSOC and DRMS, we refer the reader to the JSOC home page, and in particular the JSOC wiki.
4.2 The lookdata Tool

The lookdata tool is an online Javascript front end to the DRMS at Stanford, and can be used to browse names of dataseries, to get dataseries keyword lists, to examine metadata for sets of records, and to generate export requests. The lookdata tool is comprised of several tabs that, when progressed from left to right, allow the user to locate and export data from the JSOC. Throughout this process, there are several steps; bold labels in the subsections below correspond to bold labels in the tool. The tool itself provides some additional help, available by clicking on the yellow or blue question marks that are visible from within the tool. 23
Guide to SDO Data Analysis version dated February 19, 2013 JSOC Dataseries Name aia.lev1 aia.lev1_euv_12s aia.lev1_uv_24s aia.lev1_vis_1h hmi.B_* hmi.Ic_* hmi.Ld_* hmi.Lw_* hmi.M_* hmi.ME_* hmi.Mharp_*; hmi.sharp_* hmi.S_720s hmi.Synoptic* hmi.V_* hmi.fsV* hmi.rdV* mdi.fd_M* mdi.fd_V* Description AIA Level 1.0 data AIA Level 1.0 data for the 7 EUV channels of AIA AIA Level 1.0 data for the 1600 and 1700 channels of AIA AIA Level 1.0 data for the 4500 channel of AIA HMI Level 1.5 full-disk disambiguated vector magnetograms (12m cadence) HMI Level 1.5 continuum intensity data (various cadences) HMI Level 1.5 line depth data (various cadences) HMI Level 1.5 line width data (various cadences) HMI Level 1.5 line-of-sight magnetograms (various cadences) HMI Level 1.5 full-disk Milne-Eddington inversions (12m cadence) HMI active region patch (HARP) cutouts (12m cadence) HMI Stokes parameters (IQUV) data (12m cadence) HMI synoptic charts of the radial and line-of-sight photospheric eld HMI Level 1.5 Dopplergrams (various cadences) HMI Dopplergram data associated with far-side analyses HMI Dopplergram data associated with ring-diagram analyses MDI full-disk line-of-sight magnetograms (various cadences) MDI full-disk Dopplergrams
Table 1: Alphabetical listing of the published dataseries of note available through the JSOC. The asterisk * in the dataseries names is used here as a wildcard character. The three indented aia.lev1_* series are subsets of the main aia.lev1 series.
4.2.1 Getting Dataseries Names The rst step is to determine the name of the dataseries containing the data you wish to examine (and possibly export). To search for available dataseries, enter a search string in the Seriesname lter eld (Step 1) under the Series Select tab on the lookdata webpage, as shown in Figure 14. Pressing the Fetch seriesname list button will return a list of all dataseries names containing that search string in the larger white box to the right of the Series Select eld. In this white box (Step 2), click on the desired dataseries name from the list, and the lookdata tool will switch to the RecordSet Select tab. Entering NOT before the search string in the Seriesname lter eld in Step 1 will return the list of dataseries names that exclude that search string. Prepending the caret (circumex) character to the search string in the Seriesname lter eld in Step 1 will return only those dataseries names that start with the search string. 4.2.2 Selecting Records The next step is to lter the full set of data records in the chosen dataseries down to a more manageable number. This is done under the RecordSet Select tab, shown in Figure 15, where a DRMS query can be entered into the white box (Step 3). You will notice that the query box is already pre-populated with the dataseries name selected in Step 2. Typically, users will want to rene the query by selecting a particular range of dates and times, and/or by ltering based on values of particular keywords. Filtering by date and time is achieved by adding to the dataseries name a date or dates enclosed in square brackets. For informational purposes, near the top of the RecordSet Select tab, the dates of the earliest and most recent records in the dataseries are displayed. The following examples demonstrate this syntax (here the hmi.M_45s dataseries is used, but the syntax is generally applicable to most other dataseries): 24
TECHNICAL NOTE : TECHNICAL NOTE :
Figure 14: The Series select tab from the lookdata tool, showing Steps 1 and 2. Here, in Step 1 the user has searched for dataseries names containing the search string hmi, and has selected the hmi.M_45s dataseries in Step 2.
hmi.M_45s[2011.01.01_01:30_TAI-2011.01.01_02:00_TAI] This query requests all HMI magnetograms for 1 January 2011 between 01:30 and 02:00 TAI. It will return 41 records (endpoints are inclusive). hmi.M_45s[2011.01.01_01:30_TAI/30m] This query is almost identical to the above query, except here the forwardslash modier is used to indicate the time range (the /30m means over the next 30 minutes). It will return 40 records (exclusive of the nal endpoint). hmi.M_45s[2011.01.01_01:30_TAI/30m@3m] This query is similar to the above queries, except that now the cadence is reduced from 45 seconds to 3 minutes (the @3m means once per 3 minutes). It will return 10 records. Queries can be further rened by using keyword logic. A list of the full set of keywords for the selected dataseries can be found in the Series Content tab, located between the Series Select and RecordSet Select tabs. More detailed information about HMI and AIA keywords can be found by referencing the JSOC keywords for metadata document (sourced from the All About JSOC Names page from the JSOC wiki) and the AIA/SDO FITS keyword list (sourced from the AIA Keywords entry on SDOdocs). The following examples demonstrate queries using keyword logic (using the aia.lev1 dataseries): aia.lev1[2011.01.01_01:30/30m] As above, this query requests all AIA image records for 1 January 2011 for the half hour starting at 01:30. It will return 1200 records (one image in each of 8 wavelength channels every 12 seconds). aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 ?] This query now subselects the data from the above query to include only those image records for which the WAVELNTH keyword equals 171 (i.e., images from the 171 channel of AIA). It will return 150 records (one image every 12 seconds). 25
Guide to SDO Data Analysis version dated February 19, 2013 aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 or WAVELNTH=304 ?] This query returns those image records for which the WAVELNTH keyword equals either 171 or 304. It returns 300 records. Subselecting the aia.lev1 dataseries using keyword logic is (admittedly) a bit cumbersome. Consequently, three additional dataseries for AIA data are available: aia.lev1_euv_12s, aia.lev1_uv_24s, and aia.lev1_vis_1h. These three dataseries can be thought of as sub-series of the all-encompassing aia.lev1 dataseries, but with the data grouped into EUV-, UV-, and visiblechannel subsets. Unlike the aia.lev1 dataseries, however, these dataseries allow users to query the JSOC and subselect by wavelength channel without the use of the explicit keyword logic demonstrated above, and in fact are typically faster when doing so. Additionally, the specication of a cadence using the @ character (as shown in the preceding HMI queries) is now more user-friendly when searching for AIA data. Such features are demonstrated in the examples below (using the aia.lev1_euv_12s dataseries): aia.lev1_euv_12s[2011.01.01_01:30/30m][171] This query selects all AIA image records for the 171 channel for 1 January 2011 for the half hour starting at 01:30. It returns 150 records (one image every 12 seconds). aia.lev1_euv_12s[2011.01.01_01:30/30m][171,304] This query selects all AIA image records for the 171 and 304 channels for 1 January 2011 for the half hour starting at 01:30. It returns 300 records (two images every 12 seconds). aia.lev1_euv_12s[2011.01.01_01:30/30m@3m][171,304] This query is similar to the above query, except that the cadence is reduced from 12 seconds to 3 minutes. It returns 20 records. aia.lev1_euv_12s[2011.01.01/10d@1h][171,304] This query selects AIA image records for the 171 and 304 channels for the 10 days beginning 1 January 2011, sampled once per hour. It returns 480 records. The elements in brackets following the dataseries name (and that do not involve keyword logic) in the queries shown here correspond to searches of prime keys for the dataseries. For nearly all dataseries of interest, either T_REC or T_OBS will be a prime key, and this is what enables quick date and time searching without the use of more complicated keyword logic. For the three AIA sub-series (and unlike aia.lev1), WAVELNTH is also a prime key, which is what enables the more straightforward wavelength subselection shown above. One aspect of prime keys is that no two records displayed in a set of records have the same set of prime key values (i.e., each record in the set is uniquely specied by the values of its prime keys). The prime keys associated with a selected dataseries are listed near the top of the RecordSet Select tab (see Figure 15). After entering a query, press the GetRecordCount button, and the tool will return the record count. It is often prudent to press the GetRecordCount button after modications to the query are made, in order to update the record count. Additionally, entering a number n in the (optional) Record Limit eld will limit the number of records displayed, with positive numbers limiting the display to the rst n records and negative numbers limiting the display to the last n records. The user is now able to select which keywords to inspect (Step 4), which segments are desired (Step 5), and which links are desired (Step 6). Selecting **ALL** in these boxes will return all of each of these, but to speed things up users may want to select only a subset of keywords, segments, and/or links. Users can select individual elements in these lists by holding down the COMMAND or ALT key while clicking the left mouse button, and ranges of keywords can be selected by holding down the SHIFT key while clicking. Note that the Select Links box will often be empty. Proceed to the next step by clicking on the Fetch Keyword Values for RecordSet, at which time the lookdata tool will switch to the Values Display tab, as shown in the example in Figure 16. The table shown in the Values Display tab allows users to inspect the keyword values in the ltered set of records (Step 7). Rening the selected set of records is achieved by going back to the RecordSet Select tab (by clicking on the tab itself), adjusting the query, and repeating Steps 3 through 6. It should be noted that selecting keywords, segments, and links in Steps 46 is entirely optional. If no selections are made, the default behavior is for all keywords, segments, and links to be displayed in the Values Display tab once the Fetch Keyword Values for RecordSet button is pressed. Furthermore, users can choose to skip the Values Display tab altogether, and instead proceed directly to the data-export process, as described in the next subsection, after entering a valid query in Step 3.
TECHNICAL NOTE :
26
Guide to SDO Data Analysis version dated February 19, 2013 Near the top of the RecordSet Select tab, the dates of the earliest and most recent records in the dataseries are displayed. However, in several dataseries (one of which is aia.lev1) the rst record is listed as being in the year 4712 BC. This (nonsense) date is used to indicate records that have blank values of T_REC. The workaround for determining the date of the actual rst record of a particular dataseries involves setting the date range of the query to be something like [2010-01-01/365d] (or whatever date range the rst record is thought to be located in, for the dataseries of interest) and placing a +1 in the Record Limit eld below the query. Pressing the GetRecordCount button should yield one valid record (if not, try increasing the record limit), and pressing the Fetch Keyword Values for RecordSet button will then display the prime keys for this record (one of which is usually T_REC or T_OBS). An explanation of JSOC image timing details (i.e., why HMI images are in TAI and why AIA images are not) is given in Section 4.2.4. 4.2.3 Exporting Data When the user wishes to export the selected set of records, clicking on the Export Data tab will bring up Step 8. After doublechecking that the record set query is correct in Step 8, clicking on the Export button will open a new webpage containing the JSOC Export Data form, the top section of which is shown in Figure 17. On the export form, the query and record count should already be pre-populated in the RecordSet and Record Count elds, respectively. The Method drop-down menu gives the user the option of choosing various export methods. For most remote users, we recommend the url-tar method. After selecting url-tar, several new elds will appear, of which Protocol should be set to FITS, and Compression set to compress Rice. Also of note is the Filename Format eld allowing users to specify the format of the lenames. For example, if it is desired that the wavelength of the image be included in the lename, then one can add {WAVELNTH} to this eld. After pressing the Submit Export Request button, the request will be given a RequestID tag. Users should make note of the ID tag that appears in the RequestID eld below the button. Entering the ID tag in the darker-blue area at the bottom of the form will provide either an estimated time to completion or (if completed) a link to the archive containing the data along with a list of les that comprise the archive. A completed export using the suggested url-tar/FITS/compress Rice selections is illustrated in Figure 18.
TECHNICAL NOTE : In Step 8 and on the export form, if there is an argument in the query contained in curly braces, this indicates a listing of segments. No such argument present corresponds to the default setting of {**ALL**} (indicating all segments). If a different set of segments is desired, going back to the RecordSet Select tab, selecting the segments of interest from the list in Step 5, and pressing the GetRecordCount button will update the query shown in Step 8. (One will need to press the Export button again to get the new query transferred to the export form.) For example, if one wants the list of despiked pixels for a set of records belonging to the aia.lev1 dataseries, the query will look like aia.lev1[2011.01.01_01:30/30m]{spikes}, since these data are contained in the spikes segment of the aia.lev1 dataseries. TECHNICAL NOTE : Users who already know which records they want and who can construct the relevant query can save time by bypassing the lookdata tool altogether and entering the query directly onto a blank export form (using the link above). TECHNICAL NOTE : TECHNICAL NOTE :
4.2.4 JSOC Image Timing Details This section gives an overview of some of the technical details regarding the various timing conventions used by the JSOC. In particular, the differences between the T_OBS, T_REC, DATE__OBS, and DATE-OBS keywords will be discussed below, and for additional information users may want to consult the JSOC keywords for metadata document (sourced from the All About JSOC Names page from the JSOC wiki) and the AIA/SDO FITS keyword list (sourced from the AIA Keywords entry on SDOdocs). For temporal co-alignments between AIA and HMI, the time contained in the T_OBS keyword should be used. This is because the T_OBS keyword in AIA and HMI image headers indicates the time of the center of the respective observation interval. For HMI Level 1.5 observables, which are constructed from a sequence of HMI Level 1.0 ltergrams, T_OBS reects the peak of the weighting function applied to the constituent images comprising the sequence. For AIA, T_OBS is the center of the time for which the camera 27
Figure 15: The RecordSet Select tab from the lookdata tool, showing Steps 3 through 6. Here, the user has entered the query hmi.M_45s[2011-01-01_01:30_TAI/30m@3m], and the tool has indicated that 10 records match this query.
28
Guide to SDO Data Analysis version dated February 19, 2013 shutter is open. In both cases, the most probable time with which each observation can be associated will be contained in the T_OBS keyword. The FITS standard requires the presence of the DATE-OBS keyword, indicating the beginning of the observation interval, in Coordinated Universal Time (UTC). Consequently, DATE-OBS will be different than T_OBS (since AIA and HMI observations occur over a nite interval). For AIA and HMI Level 1.0 images, the time contained in DATE-OBS is equal to T_OBS(0.5EXPTIME), where the EXPTIME keyword is the duration over which the camera shutter was open for that image, as one might expect. For HMI observables, the time contained in the T_REC keyword is offset from T_OBS by the light travel time between 1 AU and the location of SDO in its orbit. This offset is primarily due to the ellipticity of the Earths orbit around the sun, and thus varies on an annual basis by at most 8 seconds. However, this effect, if unaccounted for, introduces measurable errors in the helioseismology results. For AIA images, T_REC is the same as T_OBS after rounding to the nearest second. When querying the JSOC, users should be aware that in many cases (including queries of all dataseries listed in Table 1) it is the T_REC keyword, and not T_OBS or DATE-OBS, that is used to determine whether records fall into the time range specied in the query. The times contained in T_OBS and T_REC are tagged in either International Atomic Time (TAI) for HMI, or in UTC for AIA. The difference between TAI and UTC is due to leap seconds that have been added to UTC over the years (due to the gradual slowing down of the Earths rotation rate); such leap seconds have not been added to TAI. At the time of this writing, UTC lags TAI by 35 seconds. When specifying a JSOC query, a sufx added to the time indicates which time format is meant. Thus, the _TAI sufxes added to the query times in the examples shown above in Section 4.2.2 indicate a time in the TAI format. Alternatively, sufxes of _UTC, _UT or _Z, or omitting the sufx altogether, imply UTC. To put these timing details in perspective, a quick calculation indicates that, in the 35 seconds of time representing the offset between TAI and UTC, a point at the equator of the sun will have rotated through about 1/6 of one pixel (assuming an image having the canonical 0.6 AIA plate scale). As a result, many users will not need to worry about such small time differences, especially in light of the fact that inaccuracies in the values in the various instrument pointing keywords are likely at least as great as 1/6 of one pixel (especially for AIA data from the EUV channels).
TECHNICAL NOTE : The JSOC keyword naming convention does not allow for hyphens, and consequently the DATE-OBS keyword is represented as DATE__OBS (with two underscores) within the JSOC. However, upon export to FITS les, this keyword gets converted to DATE-OBS so as to be compliant with the FITS standard. Additionally, note that the keyword DATE_OBS (with one underscore) that was more common during the SOHO era is not used by the JSOC.
4.3 The Cutout Service

The cutout service enables users to request sequences of images from the various AIA wavelength channels as well as of HMI line-ofsight magnetograms and continuum intensity images. Users can access the cutout service directly using the link to the data-request form above. The cutout service is also employed when using the Request data link associated with events returned in iSolSearch queries (see Section 3.2), in which case the data-request form will be pre-populated with the spatiotemporal bounds of the event so that data associated with HER events is easily retrievable by users. As an alternative to this online form, there is also a SolarSoft interface to the cutout service that allows additional customization (see Section 5.4). As shown in Figure 19, the online form has a straightforward layout, containing input elds for specifying the spatiotemporal extent of the cutout, as well as the wavelength channels desired. The spatial extent can be specied either by drawing a bounding box on the sample image or typing in coordinates directly into the appropriate parameter elds. As an alternative to specifying a subeld, users can instead request full-disk data at various spatial resolutions. Clicking on the full-disk link (underneath the sample image) will bring up radio buttons that allow the resulting image size to be specied, enabling the user to specify full-resolution images (40964096 pixels) or images binned down by a factor of 2, 4, or 8. In the Service drop-down menu (underneath the sample image), users can select either SSW Service or, if available, JSOC Service. Selecting JSOC Service causes the form to construct a query as if the user had lled out the JSOC Export Data form described in Section 4.2. Upon successful submission, the user will be provided with RequestID tag(s) that can be used to monitor the status of the JSOC export request. Alternatively, the SSW Service option causes SSW routines to be used to process the cutout.
29
Guide to SDO Data Analysis version dated February 19, 2013 Upon submission of a cutout request using the SSW Service option, the user should receive an acknowledgement containing the SSW request ID that can be used to monitor the status of the SSW cutout request. Underneath the Service drop-down menu is a checkbox labeled Tracking. Checking this box will cause the eld of view of the cutout to follow the mean rotation rate of the latitude of the center of the cutout. The reference time of the cutout (to which the cutout width, height, and center apply) is the same as the reference time of the image displayed, which is usually around the 23h mark of the start date. (The sample image displayed is the same as on the archived version of The Sun Today corresponding to the start date.) In the last four elds of the request form, users should input their name, e-mail address, a brief description of the data request, and the desired cadence (image frequency). Once the dataset is ready, an email will be sent with instructions describing how to download the data. If the SSW Service option was chosen, the e-mail will contain a link to the service-request output page. There, the user can view a summary movie using the JobWWW link, or download the data by going into SSWIDL and using the command in the Get_Data eld to download the data to the currently active directory. If, instead, the JSOC Service option was chosen, the e-mail will contain a link to the (temporary) location online at the JSOC from which the data can be downloaded. The data are presented as a set of links to individual les, and as mentioned earlier, browser add-ons (such as for Firefox) come in handy for downloading such lists of links in a few mouse-clicks. If an error occurs in the request or if the request exceeds our currently tested capacity, it will stay in the queue and we shall try to process it in due time, or otherwise we may contact you describing how to reformulate your request. Users who have questions about their cutout request should send an e-mail containing the Request ID(s) to the address listed on this webpage. The online form will adjust the spatial bounds upward so that the x and y dimensions of the cutout are both multiples of 64 pixels.
TECHNICAL NOTE :
4.4 The Virtual Solar Observatory

The Virtual Solar Observatory (VSO) has been described as one-stop shopping for (nearly) all of your solar physics needs. It allows users to query a multitude of data providers (e.g., SDO, Hinode, and other observatories) using standardized interfaces: either via the web or via SolarSoft. Here, in this section, we will demonstrate how to retrieve SDO data using the web-based interface, whereas the SolarSoft interface is demonstrated for SDO data in Section 5.5. Online help at any step is available either by clicking on the yellow-question-mark information icons visible on the various the web forms, or by perusing the VSO FAQ webpage or the VSO blog. To use the web interface, navigate to the VSO entry page, shown in Figure 20, where the user is presented with a list of attributes that will frame the subsequent VSO query. Note that Time is always selected. (If the date and time of interest are not already known, then the resources described in Section 3 of this guide will be of use.) For AIA or HMI, choosing the Instrument / Source / Provider and then Instrument Only options is a simple way to narrow down the source options. Choosing Observable (for HMI) or Spectral Range (for AIA) may also be of use when selecting SDO data. Pressing the Generate VSO Search Form button at the bottom of the entry page will display the search form. On the search form, shown in Figure 21, the user enters a start and end time for the query, and checks off one or more instruments. Pressing the Search button will, after a few moments, return the results of the query. At this point, the user selects which records they wish to retrieve by marking the check-boxes in the left-most column of the table. The CheckBox Tools section on the left-hand sidebar may come in handy here, as it enables users to check or uncheck large swathes of records. The columns can be sorted by checking off the boxes next to the column headings, and then clicking on the Sort/Rearrange buttons above the table. The results page with some boxes checked is shown in Figure 22. Pressing the Request Data button at the bottom of the left-hand sidebar, and selecting URL-FILE_Rice (for Rice-compressed FITS les), will bring up a set of links to the data, as shown in Figure 23. At this stage, the image data can be downloaded individually by clicking on each of the URLs. To download large numbers of images, one will probably want to either use a browser add-on (such as DownThemAll! for Firefox), or alternatively export the list of URLs to a text le by pressing the Export Links button at the bottom of the left-hand sidebar and then make use of a utility like curl to download the data. For example, suppose the list of URLs were exported from the VSO and saved in a le named vso_export.csv, then one can download all of the URLs in the list to the current working directory by entering
30
xargs -n1 curl -O --remote-header-name < vso_export.csv at the Unix command prompt. Note that one needs curl version 7.21.2 or later for the --remote-header-name option (which properly handles the content-disposition headers used by the VSO, which in turn provides sensible lenames for the downloaded data) to work properly. The command above may seem a bit unwieldy for downloading a list of URLs contained in a le, but because curl doesnt have a clean option for specifying an input le from which the URLs are read ( la the -i option for wget), the workaround is to invoke curl via xargs. One could alternatively use a command similar to curl -O --remote-header-name -K vso_export.csv (i.e., without using xargs), but this requires editing vso_export.csv to enable curl to interpret each URL as if it were a command-line option of the form --url="http://...". In contrast, using the xargs command with curl as shown above enables the use of a vso_export.csv-type le exported from the VSO without modication.
TECHNICAL NOTE :
4.5 Synoptic Data

Synoptic data streams are made on a regular basis from the JSOC pipeline. For HMI, the synoptic stream that exists for line-of-sight magnetogram data is located here. This stream contains HMI rebinned 10241024-pixel line-of-sight magnetograms sampled once per hour. Additionally, there are images and movies of both continuum and magnetogram image series located here. For AIA, there is a set of synoptic data located here. These synoptic data consist of compressed FITS les containing rebinned (10241024-pixel) Level 1.5 images in each of the AIA wavelength channels, sampled every two minutes. There is a mission-long stream as well as a near-real time stream. There is also a stream for full-resolution AIA images in the JPEG 2000 format, which allows hierarchical uncompression of these data, located here.
4.6 Uncompressing the Data

Data from SDO are typically downloaded as Rice-compressed FITS les, in order to reduce the bandwidth needed for the download. It is anticipated that users will prefer to save disk space and store the data in the compressed format, after realizing that the time needed to read in compressed les and perform the subsequent uncompression is often about the same as to the time needed to read in the (larger) uncompressed les. Of course, the precise tradeoff depends on the compression ratio, disk I/O speeds, CPU processing speeds, etc., and so (as they say) your mileage may vary. Rice-compressed les can be uncompressed (and, in many cases, viewed) using a number of freely-available image-manipulation software programs, including SAOImage DS9, and the imcopy utility and fpack package from the CFITSIO subroutine library. Also, the read_sdo.pro IDL routine, available via the ONTOLOGY SolarSoft package (see Section 5), provides a way to uncompress and read in Rice-compressed header data and images from the IDL command line. Additional information on read_sdo.pro can be found in Section 5.6. 4.6.1 Compiling and Installing imcopy Because imcopy is a useful utility to have around, especially for users who are not SolarSoft-inclined, we here demonstrate how to compile and install it. First, download the CFITSIO subroutine library and expand the archive, which should automatically create the directory cfitsio that contains all of the CFITSIO source code. On Unix systems, one creates the makele by entering ./configure at the shell prompt. Optionally, one may include a prefix argument that species the target installation directory. For example, entering ./configure --prefix=/usr/local will cause the CFITSIO library to end up in /usr/local/lib/ and include les to end up in /usr/local/include/. If no prefix argument is given, the libraries and include les are created in the current working directory. If the configure script exits without encountering any problems, it will state something similar to, Congratulations, Makele update was successful and one can
31
Guide to SDO Data Analysis version dated February 19, 2013 proceed to the next step. If the configure script encounters a problem, it should indicate what went wrong. Common problems usually involve compilers or include les either not installed or not in the current path, and the user will need to rectify these problems before compilation can occur. After creating the makele, one then creates the CFITSIO library libcfitsio.a by entering make at the Unix prompt. If all goes well, several dozen objects will get created and stored in the library. At this stage, one can optionally install the CFITSIO library in the installation path specied by prefix above by entering make install. Depending on the installation path, one may need superuser permission to do this. Lastly, to compile the stand-alone utility imcopy, one needs simply to type make imcopy. Users may also want to also enter make fpack and make funpack if using these routines to compress or uncompress FITS les is anticipated. At this stage, the executable imcopy should be present, and can be moved into the active path. Using imcopy to uncompress a Rice-compressed FITS le, such as those downloaded from the JSOC or from the VSO, can be achieved by entering either imcopy image_compressed.fits image_uncompressed.fits or imcopy image_compressed[1] image_uncompressed.fits on the Unix command line. (The rst command usually generates a FITS le with an IMAGE extension, which might not be what was desired.) The imcopy utility is extremely versatile, as it recognizes the extensive CFITSIO extended lename syntax. In particular, one can not only deal with le compression, but also extract cutouts, resample data, download les from remote servers, and convert between various image formats via a single command. Readers may refer to these additional examples from the CFITSIO documentation for additional ideas.
32
Figure 16: The Values Display tab from the lookdata tool, showing a listing of all keywords and their values for the 10 records matching the hmi.M_45s[2011-01-01_01:30_TAI/30m@3m] query illustrated in Figure 15.
33
Figure 17: The top section of the JSOC Export Data form, illustrating how to export the 10 records matching the hmi.M_45s[2011-01-01_01:30_TAI/30m@3m] query illustrated in Figure 15. After pressing the Submit Export Request button and waiting for the export to process, the lower section will appear as in Figure 18.
34
Figure 18: The lower section of JSOC Export Data form, illustrating the completed export requested in Figure 17.
35
Figure 19: The cutout service request form, showing a request for data from the AIA 171 and 304 channels and HMI line-of-sight magnetograms covering a localized region of the solar disk.
36
Figure 20: The VSO web interface entry page, with the Instrument / Source / Provider and Instrument Only options checked. Pressing the Generate VSO Search Form button brings up the query form shown in Figure 21.
37
Figure 21: The VSO search form, with the user about to request one hours worth of AIA data from 1 August 2010. Pressing Search yields the results page shown in Figure 22.
38
Figure 22: The results of the VSO query from Figure 21, showing the top portion of the listing of 2336 records. One minutes worth of AIA data have been selected for export. Pressing the Request Data button at the bottom of the left-hand sidebar results in the download page shown in Figure 23.
39
Figure 23: The VSO download page corresponding to the set of records checked off in Figure 22. The list of links can be exported to a le by clicking on the Export Links button on the left-hand sidebar.
40
5 How to Use SolarSoft with SDO Data

The SolarSoftWare (SSW) system, or SolarSoft for short, is a software distribution system that originally stemmed from the need to coordinate solar mission data analysis across several institutions, going back as far as the Yohkoh mission (which launched in 1991). It is comprised of integrated software libraries, databases, and system utilities that provide a coordinated data analysis environment for solar physics research. It is largely hardware- and site-independent, with mirroring capabilities to keep installations at various sites consistent. Although most of the software distributed via SSW is in Exelis Visual Information Solutions Interactive Data Language (IDL) the lingua franca of the solar physics community SSW can easily accommodate software in other languages. In the following sections, we will describe how to install or upgrade SSW for use with SDO data (Section 5.1), and the how to use SSW to interface with the HEK (Section 5.2) and to get HMI or AIA image data from the JSOC (Section 5.3). Also demonstrated are the SSW interfaces to the cutout service (Section 5.4) and the VSO (Section 5.5). Once data have been received, users will need to know how to uncompress these data within SSW (Section 5.6). Additional items related to AIA image processing, while almost exclusively implemented in SSW, are found in the next section (Section 6). Information regarding access to EVE data via SSW is available here. This section contains several examples of IDL code. To guide users, many of these examples are given in pairs of code blocks: the rst block contains code that users can copy and paste into their IDL session, and the second block shows the output that should result.
5.1 Installing or Upgrading SolarSoft

5.1.1 Doing a Clean Install To do a clean install of SSW, follow these steps: Navigate to the main SSW installation webpage. This is the webpage containing instructions for installing SSW on different operating systems. Next, click on the SSW INSTALLATION FORM link. The installation form allows the user to select the top-level path in which the software will reside, along with which packages are to be downloaded. For SDO-related software, select AIA, HMI, EVE, and ONTOLOGY. The ONTOLOGY package includes software that interacts with the JSOC, and the others contain software specic to the respective SDO instruments. If you think that you will be using the SSW interface to the VSO (see Section 5.5), then the VSO package should also be selected. After completing the installation form, click on the Generate Installation Script button near the bottom of the page. The website will then give you an indication of how much disk space your SSW installation will occupy, and provide you with a link to your personalized installation script or batch le. Next, save the installation script or batch le to disk by right-clicking on the link underneath the installation table. Run the script or batch le on your machine. On Windows systems, the batch le will be contained in an archive, which needs to be unzipped. Running the script or batch le will cause the SSW packages to be downloaded. At this point, SSW and its constituent packages (should) have been downloaded to your system. On Unix and Unix-like systems, a few more steps are required for IDL to have access to this software, as described on the SSW setup webpage. First, it is necessary to dene the SSW environment variable so that it points to the top-level path of the SSW tree: setenv SSW [whatever you specified on the form] SSW also needs to know which packages are available. This is achieved by dening the SSW_INSTR environment variable so that it includes the packages that were downloaded: setenv SSW_INSTR "ontology aia hmi eve vso [etc.]" Then, after sourcing the setup le, 41
source $SSW/gen/setup/setup.ssw [/loud] one simply types sswidl (an alias that was dened in the setup le) at the Unix prompt to start up IDL. IDL combined with the SolarSoft libraries and databases is frequently referred to as SSWIDL. The setenv commands above apply only to the C shell (csh) or related shells. If using Bourne or bash, then the proper syntax for setting an environment variable is export VARIABLE=value.
TECHNICAL NOTE : After the setup le setup.ssw is sourced, an alias named setssw will be dened that allows users to load specic libraries and instrument packages. This feature is useful when the SSW startup process is found to be undesireably lengthy, a situation often caused by a large number of packages being loaded when SSWIDL is starting up. In these situations, users can reduce the startup time by loading a smaller number of packages. For example, if one only needs the aia and chianti packages, then entering setssw aia chianti will cause SSWIDL to only load those two packages upon startup until either setssw is invoked again or the SSW_INSTR environment variable is changed. TECHNICAL NOTE :
5.1.2 Updating Packages for an Existing Installation If you already have an existing SSW installation and you simply want to add a new package or update an existing package, then at a SSWIDL prompt issue a command of the form: ssw_upgrade,/spawn,/loud,/package1 [,/package2] [,/...] For example, adding or updating the AIA and ONTOLOGY packages is achieved by entering the following (note that some systems also require the /PASSIVE_FTP ag): ssw_upgrade,/spawn,/loud,/aia,/ontology [,/passive_ftp] Afterward, be sure to add any new packages to your list of instruments in the SSW_INSTR environment variable. More information regarding SSW upgrades is available on the SSW upgrade webpage. 5.1.3 Using a Proxy Server with SSW A proxy server is a server located between two networks that acts as an intermediary for all network trafc routed between them. Often, a proxy server is inserted between a local network and a larger network (such as the internet) to provide anonymity, increased security, or performance enhancements. Because some of the functionality of SSW involves downloading information from remote databases or otherwise interacting with the internet, there are times that SSW meeds to be proxy-aware in order to work properly. In some cases, there may be global system settings that automatically route outgoing trafc generated by SSW to the correct proxy server (obviating the need for the user to do any additional conguration), in which case the information in this subsection may be superuous. If additional conguring is required, however, the following may be useful. One way to make SSW proxy-aware is to set the environment variables HTTP_PROXY and NO_PROXY prior to (or during) SSW startup. The HTTP_PROXY environment variable should contain the name or IP address of the proxy server, and the NO_PROXY environment variable should list domains that do not require routing through the proxy. For example, a user on the example.edu network may need to use a proxy server at proxy.example.edu:80 in order to access the internet. In this case, for SSW to work the user will need to ensure that the environment variable HTTP_PROXY is set to proxy.example.edu:80, either by setting it prior to entering SSW or by inserting a line such as set_logenv,http_proxy,http://proxy.example.edu:80
42
Guide to SDO Data Analysis version dated February 19, 2013 in the $SSW/site/setup/IDL_STARTUP le located in ones SSW installation. For domains that do not require routing through the proxy server for access (domains on the local network typically fall into this category), these domains should be given as a commadelimited list in the NO_PROXY environment variable. As before, inserting set_logenv,no_proxy,.example.edu in $SSW/site/setup/IDL_STARTUP or setting this environment variable prior to SSW startup will cause SSW to bypass the proxy server in order to access any URL ending in .example.edu. Note that the setup le at $SSW/site/setup/IDL_STARTUP in ones SSW installation is not overwritten during SSW upgrades; thus, one has the ability to edit this le and not have these changes undone when SSW is upgraded.
5.2 Querying the HER

5.2.1 Basic Query Syntax Querying the HER (see Section 3.1) using SSW is achieved by using the IDL code available through the ONTOLOGY package. This package contains basic routines that interface with the HEK using the HEK API. For SSWIDL queries to the HER, SolarSoft provides two routines: ssw_her_make_query.pro, which generates a string of parameters in used in the subsequent CGI query, and ssw_her_query.pro, which issues an input verbatim query to the HEK CGI interface and translates the output to an IDL structure vector of event matches. In essence, these routines provide the translation between the SSW-style index metadata structures and the XML-style <paramater>=<value> syntax used within the HEK. The following example demonstrates how to run the query using ssw_her_make_query.pro: ; copy and paste these commands into your SSWIDL session: t0=2010-01-01 & t1=2010-12-31 query = ssw_her_make_query(t0,t1,/fl) print,query
IDL> t0=2010-01-01 & t1=2010-12-31 IDL> query=ssw_her_make_query(t0,t1,/fl) IDL> print,query event_starttime=2010-01-01T00:00:00&event_endtime=2010-12-31T00:00:00&event_coordsys=helioprojec Here, the /fl switch is set to indicate that we are querying for HER entries having the are event class, since fl is the two-letter code for are (as shown in the full list of event classes). To execute the query, the ssw_her_query.pro routine is used: ; copy and paste these commands into your SSWIDL session: t0=2010-01-01 & t1=2010-12-31 query=ssw_her_make_query(t0,t1,/fl) events=ssw_her_query(query) help,events help,events,/structures help,events.fl help,events.fl,/structures IDL> t0=2010-01-01 & t1=2010-12-31 IDL> query=ssw_her_make_query(t0,t1,/fl) 43
IDL> events=ssw_her_query(query) IDL> help,events EVENTS STRUCT = -> <Anonymous> Array[1] IDL> help,events,/structures ** Structure <1c4dc0c8>, 1 tags, length=182400, data length=182400, refs=1: FL STRUCT -> <Anonymous> Array[100] IDL> help,events.fl <Expression> STRUCT = -> <Anonymous> Array[100] IDL> help,events.fl,/structures ** Structure <2838808>, 114 tags, length=1824, data length=1824, refs=2: EVENT_PIXELUNIT STRING BOUND_CCSTARTC2 STRING EVENT_DESCRIPTION STRING EVENT_NPIXELS STRING FRM_HUMANFLAG STRING false REVISION STRING 1 EVENT_EXPIRES STRING KB_ARCHIVID STRING ivo://helio-informatics.org/FL_SECstandard_20100613_150757_2010010 [...deletia...] The output of the ssw_her_query.pro routine is an array of event structures (one mer match), in which are stored the metadata describing each event. The 100 matches reect the current default match limit. This can be changed by setting the RESULT_LIMIT keyword in the call to ssw_her_query.pro. In IDL Version 8.0, the help,events and help,events.fl IDL commands in the above examples do not work as shown here, and instead behave as if the /STRUCTURES switch has been set. This is apparently a bug and has been xed in IDL Version 8.0.1, according to this newsgroup discussion (Google login required). 5.2.2 Adding Filters to Queries Users can add lters that screen for, or screen out, events that satisfy a set of user-dened conditions by dening the SEARCH_ARRAY keyword in the call to ssw_her_make_query.pro. Using the same example as in the iSolSearch example of Section 3.2, suppose one wants to nd all ares having GOES Class M2 or greater that occurred in the year 2010. The resulting query would lter the results based on the FL_GOESCls (are GOES class) attribute, and would be constructed and executed as follows: ; copy and paste these commands into your SSWIDL session: t0=2010-01-01 & t1=2010-12-31 query=ssw_her_make_query(t0,t1,/fl,search_array=[FL_GOESCls>=M2.0]) events=ssw_her_query(query) help,events,/structures IDL> t0=2010-01-01 & t1=2010-12-31 IDL> query=ssw_her_make_query(t0,t1,/fl,search_array=[FL_GOESCls>=M2.0]) IDL> events=ssw_her_query(query) IDL> help,events,/structures ** Structure <1c48a7c8>, 1 tags, length=32832, data length=32832, refs=1: FL STRUCT -> <Anonymous> Array[18]
TECHNICAL NOTE :
44
Guide to SDO Data Analysis version dated February 19, 2013 Again, at the time of this writing, the query returned 18 events. Valid relational operators for the SEARCH_ARRAY keyword include =, >=, <=, <>, >, <, and like. The like operator searches for the occurrence of substrings, and is useful for searching through the event descriptions (or any other attribute dened as a string) for specic words or phrases. The following example demonstrates how to search for eruption events whose descriptions contain the word bubble (note that search strings such as bubble are case insensitive): ; copy and paste these commands into your SSWIDL session: t0=2010-01-01 & t1=2012-12-31 query=ssw_her_make_query(t0,t1,/er,search_array=[EVENT_DESCRIPTIONlikeBUBBLE]) events=ssw_her_query(query) help,events,/structures IDL> t0=2010-01-01 & t1=2012-12-31 IDL> query=ssw_her_make_query(t0,t1,/er,search_array=[EVENT_DESCRIPTIONlikeBUBBLE]) IDL> events=ssw_her_query(query) IDL> help,events,/structures ** Structure <1aea54d8>, 1 tags, length=99552, data length=99552, refs=1: ER STRUCT -> <Anonymous> Array[51] This query returned 51 events wherein the event description contained the string bubble. The HER is not intended to be complete (not all eruptions where a bubble is evident will be return), but it will show results that were noteworthy enough for (to use the above query as an example) a human annotator to indicate that a bubble was associated with an eruption.
TECHNICAL NOTE :
Although one cannot presently generate queries using lters containing the like operator from within the iSolSearch GUI (see Section 3.2), it is still possible to generate a valid URL that will perform such queries and display the results in iSolSearch. To do this, rst go into SSWIDL and generate the list of parameters and values corresponding to the query. These will be returned by the ssw_her_make_query.pro routine in the standard HTTP_GET syntax. Then, take these query parameters and append them to the end of a blank HER search URL, which begins https://www.lmsal.com/isolsearch?hek_query=https://www.lmsal.com/hek/her?cosec=2&cmd=search&ty In the case of the above example where the user is searching for the substring bubble in eruption event descriptions, open this link in a new browser window to see the complete URL and the results of this query in iSolSearch. The list of events satisfying this query that are displayed in the iSolSearch GUI should match the event listing returned in SSWIDL example above. Within the SSWIDL environment, the struct4event.pro routine is available for determining the current list of required and optional attributes for each event class (basically by returning a blank event entry). For example, one can view the attributes associated with are events as follows: ; copy and paste these commands into your SSWIDL session: flaretags=struct4event(FL) help,flaretags.required,/structures help,flaretags.optional,/structures IDL> flaretags=struct4event(FL) IDL> help,flaretags.required,/structures ** Structure <480d208>, 31 tags, length=392, data length=388, refs=2: EVENT_TYPE STRING FL: Flare KB_ARCHIVDATE STRING Reserved for KB archivist: KB entry date KB_ARCHIVID STRING Reserved for KB archivist: KB entry identifier KB_ARCHIVIST STRING Reserved for KB archivist: KB entry made by 45
Guide to SDO Data Analysis version dated February 19, 2013 KB_ARCHIVURL STRING Reserved for KB archivist: URL to suppl. info. EVENT_COORDSYS STRING UTC-HPC-TOPO EVENT_COORDUNIT STRING blank EVENT_ENDTIME STRING 1492-10-12 00:00:00 EVENT_STARTTIME STRING 1492-10-12 00:00:00 [...deletia...] IDL> help,flaretags.optional,/structures ** Structure <4802a08>, 55 tags, length=632, data length=586, refs=2: EVENT_PROBABILITY FLOAT Inf EVENT_EXPIRES STRING 1492-10-12 00:00:00 EVENT_COORD3 FLOAT Inf EVENT_MAPURL STRING blank EVENT_MASKURL STRING blank EVENT_CLIPPEDSPATIAL STRING blank EVENT_CLIPPEDTEMPORAL STRING blank EVENT_TESTFLAG STRING blank [...deletia...]
5.3 Retrieving Data from the JSOC

The SSWIDL routines that interface with the JSOC (see Section 4.1) all have the form ssw_jsoc*.pro, and are located in the ONTOLOGY package in the SSW installation. These routines have many more parameters and keywords than are outlined here, and we urge users to look at the source-code preamble for more detailed lists of options. The source code can be viewed either by using the xdoc.pro routine from within SSWIDL, by using the SSW code search webpage to search for a routine of interest, or by directly viewing the source code once the local path is known (say, after using the SSWIDL routine which.pro, which returns the path of any routine in a users IDL path). Additionally, readers should be familiar with terms describing the JSOC data management system, such as dataseries, record, and segment, as described in Section 4.1. There are three basic steps that enable users to download data from the JSOC directly into SSWIDL, as discussed in the following subsections. The steps illustrated here are directly analogous to the steps needed to use the lookdata tool demonstrated in Section 4.2. 5.3.1 Getting Dataseries Names The rst step is to search for the JSOC dataseries of interest using the FILTER keyword of ssw_jsoc.pro: result=ssw_jsoc(filter=[filter string]) This function returns a structure, with each element in the structure corresponding to a different dataseries. All of the dataseries names contain the substring provided in the FILTER keyword on input. The NAMES tag of this structure contains the dataseries name, the list of prime keys, and a one-line description of each dataseries. For example, to nd all JSOC dataseries containing the substring aia, enter the following commands into SSWIDL: ; copy and paste these commands into your SSWIDL session: result=ssw_jsoc(filter=aia)
46
Guide to SDO Data Analysis version dated February 19, 2013 help,result,/structures help,result.names(1),/structures IDL> help,result,/structures ** Structure <39259a8>, 3 tags, length=352, data length=340, refs=1: STATUS INT 0 NAMES STRUCT -> <Anonymous> Array[7] N INT 7 IDL> help,result.names(1),/structures ** Structure <3924268>, 3 tags, length=48, data length=48, refs=2: NAME STRING aia.lev1 PRIMEKEYS STRING T_REC, FSN NOTE STRING AIA Level 1 This result should mimic the output displayed in Step 2 of the lookdata tool (see Section 4.2.1).
TECHNICAL NOTE : If a proxy server is being used, then SSW needs to be proxy-aware in order for ssw_jsoc.pro to work properly; otherwise, ssw_jsoc.pro will be unable to query the JSOC and will eventually time out. To congure SSW to work with a proxy server, see Section 5.1.3.
When searching for valid dataseries using ssw_jsoc.pro, prepending the caret (circumex) character to the search substring in the FILTER keyword will return only those dataseries whose names that start with the search string. When a dataseries name is known, the ssw_jsoc.pro routine can be used (with the /SERIES_STRUCT switch set) to fetch information that denes the dataseries, such as the list of applicable keyword and segment names: ; copy and paste these commands into your SSWIDL session: result=ssw_jsoc(ds=aia.lev1,/series_struct) help,result,/structures help,result.keywords print,transpose(result.keywords.name) help,result.segments print,transpose(result.segments.name) IDL> result=ssw_jsoc(ds=aia.lev1,/series_struct) IDL> help,result,/structures ** Structure <50af208>, 12 tags, length=9456, data length=9444, refs=1: NOTE STRING AIA Level 1 RETENTION INT 10000 UNITSIZE INT 1 ARCHIVE INT 0 TAPEGROUP INT 10000 PRIMEKEYS STRING Array[2] PRIMEKEYSINFO STRUCT -> <Anonymous> Array[2] DBINDEX STRING Array[2] KEYWORDS STRUCT -> <Anonymous> Array[1] SEGMENTS STRUCT -> <Anonymous> Array[3] INTERVAL STRUCT -> <Anonymous> Array[1] 47
TECHNICAL NOTE :
Guide to SDO Data Analysis version dated February 19, 2013 STATUS LONG 0 IDL> help,result.keywords ** Structure <58539d8>, 3 tags, length=9024, data length=9024, refs=2: NAME STRING Array[188] TYPE STRING Array[188] NOTE STRING Array[188] IDL> print,transpose(result.keywords.name) cparms_sg000 image_lev1_bzero image_lev1_bscale cparms_sg001 bad_pixel_bzero bad_pixel_bscale cparms_sg002 spikes_bzero spikes_bscale BLD_VERS LVL_NUM T_REC [...deletia...] IDL> help,result.segments ** Structure <5866e28>, 5 tags, length=80, data length=80, refs=2: NAME STRING image_lev1 UNITS STRING dn PROTOCOL STRING fits DIMS STRING 4096x4096 NOTE STRING lev1 image fits file IDL> print,transpose(result.segments.name) image_lev1 bad_pixel spikes As shown above, the KEYWORDS and SEGMENTS tags of the structure returned by ssw_jsoc.pro (when the /SERIES_STRUCT switch is set) are themselves structures that contain information about the keywords and segments belonging to the dataseries specied by the DS keyword. These sub-structures are dened with tags named NAME, TYPE, and NOTE, which contain the name, datatype, and a brief description of the keyword or segment, respectively. This allows for an alphabetized list of the keywords applicable to, for example, the aia.lev1 dataseries, to be generated by the following commands: ; copy and paste these commands into your SSWIDL session: result=ssw_jsoc(ds=aia.lev1,/series_struct) print,transpose(result.keywords.name(sort(strupcase(result.keywords.name)))) IDL> result=ssw_jsoc(ds=aia.lev1,/series_struct) IDL> print,transpose(result.keywords.name(sort(strupcase(result.keywords.name)))) ACS_CGT ACS_ECLP
48
Guide to SDO Data Analysis version dated February 19, 2013 ACS_MODE ACS_SAFE ACS_SUNP AECDELAY AECMODE AECTYPE AGT1SVY AGT1SVZ AGT2SVY AGT2SVZ [...deletia...] Information about the keywords and segments of a dataseries retrieved as shown above should match the information displayed in the Series Content tab of the lookdata tool. For further information about HMI and AIA keywords, users can refer to either the JSOC keywords for metadata document (sourced from the All About JSOC Names page on the JSOC wiki) and/or the AIA/SDO FITS keyword list (sourced from the AIA Keywords entry on SDOdocs). 5.3.2 Selecting Records Once the dataseries name is known, the next step is to get a listing of available records from the JSOC. This is achieved using the ssw_jsoc_time2data.pro routine: ssw_jsoc_time2data,t0,t1,index,ds=[dataseries name] If images from the specied dataseries exist in the date/time range specied by the T0 and T1 arguments, the procedure will return (in the INDEX variable) a listing of these image headers. The structure tags mimic the list of keywords shown by the lookdata tool in Step 4 (see Section 4.2.2). For example: ; copy and paste these commands into your SSWIDL session: ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,ds=aia.lev1 help,index help,index,/structures IDL> ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,ds=aia.lev1 ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | -------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/30m] | ---------------------------------------------------------------------------------------------| selecting segment> image_lev1 | ------------------------------------------------------------------------------------------| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/30m]{image_lev1} | ---------------------------------------------------------------------------------49
Guide to SDO Data Analysis version dated February 19, 2013 | HEX handler QUALLEV0 | -----------------------------------------| HEX handler QUALITY | --------------------IDL> help,index INDEX STRUCT = -> <Anonymous> Array[1200] IDL> help,index(0) ** Structure <18a2b808>, 178 tags, length=1120, data length=1063, refs=1: BLD_VERS STRING V5R12X LVL_NUM FLOAT 1.00000 T_REC STRING 2011-01-01T01:00:01Z T_REC_STEP DOUBLE 1.0000000 T_REC_EPOCH STRING 1977.01.01_00:00:00_TAI T_REC_ROUND LONG 1 ORIGIN STRING SDO\/JSOC-SDP DATE STRING 2011-07-02T14:05:51Z TELESCOP STRING SDO/AIA INSTRUME STRING AIA_3 DATE__OBS STRING 2011-01-01T01:00:00.34Z T_OBS STRING 2011-01-01T01:00:01.34Z [...deletia...] The INDEX structure having 1,200 elements indicates that for the 30-minute time period of interest in this example, there exist 1,200 records. If a proxy server is being used, then SSW needs to be proxy-aware in order for ssw_jsoc_time2data.pro to work properly; otherwise, ssw_jsoc_time2data.pro will be unable to query the JSOC and will eventually time out. To congure SSW to work with a proxy server, see Section 5.1.3. For AIA data queries, a common activity is to select only particular wavelength channels. To do this, specify the channel(s) of interest in the WAVES keyword, which will be then checked against the value of the WAVELNTH keyword for each record:
TECHNICAL NOTE :
; copy and paste these commands into your SSWIDL session: ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,$ ds=aia.lev1_euv_12s,waves=[171,304] help,index IDL> ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,$ IDL> ds=aia.lev1_euv_12s,waves=[171,304] ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | -------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/30m] | ---------------------------------------------------------------------------------------------| selecting segment> image_lev1 | 50
Guide to SDO Data Analysis version dated February 19, 2013 ------------------------------------------------------------------------------------------| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/30m]{image_lev1} | ---------------------------------------------------------------------------------| HEX handler QUALLEV0 | -----------------------------------------| HEX handler QUALITY | --------------------IDL> help,index INDEX STRUCT = -> <Anonymous> Array[300] As a result of screening out all images except for those from either the 171 or 304 channels of AIA, the 1,200 records of the earlier example have now been reduced to 300. As mentioned in Section 4.2.2, there exist dataseries that contain subsets of the records (grouped by wavelength) from the aia.lev1 dataseries, named aia.lev1_euv_12s, aia.lev1_uv_24s, and aia.lev1_vis_1h. The advantages of using these series with ssw_jsoc_time2data.pro are that subselecting by wavelength channel is typically faster, and that the use of the CADENCE keyword is now more straightforward when searching for AIA data. For example, one can reduce the cadence of the above query to 3 minutes as follows: ; copy and paste these commands into your SSWIDL session: ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,$ ds=aia.lev1_euv_12s,waves=[171,304],cadence=3m help,index IDL> ssw_jsoc_time2data,1-jan-2011 01:00,1-jan-2011 01:30,index,$ IDL> ds=aia.lev1_euv_12s,waves=[171,304],cadence=3m ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | ------------------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1_euv_12s[2011-01-01T01:00:00Z/30m@3m] | --------------------------------------------------------------------------------------------------------| selecting segment> image_lev1 | -----------------------------------------------------------------------------------------------------| /rs_list, ds=aia.lev1_euv_12s[2011-01-01T01:00:00Z/30m@3m]{image_lev1} | --------------------------------------------------------------------------------------------| HEX handler QUALLEV0 | -----------------------------------------| HEX handler QUALITY | ---------------------
51
Guide to SDO Data Analysis version dated February 19, 2013 IDL> help,index INDEX STRUCT
= -> <Anonymous> Array[20]
As a result of reducing the cadence, there are now only 20 records found that satisfy the query.
TECHNICAL NOTE : As mentioned in Section 4.2.4, JSOC queries can be in either TAI or UTC time formats. By default, the ssw_jsoc_time2data.pro routine will use the time format from the series denition. Users can force TAI or UTC by setting either the /TAI or /UTC switches, respectively, in the call to ssw_jsoc_time2data.pro. Additionally, the DATE-OBS keyword discussed in Section 4.2.4 appears as DATE_OBS in index structures returned by ssw_jsoc_time2data.pro.
In IDL Version 8.0, the help,index IDL commands in the above examples do not work as shown here, and instead behave as if the /STRUCTURES switch has been set. This is apparently a bug and has been xed in IDL Version 8.0.1, according to this newsgroup discussion (Google login required). 5.3.3 Exporting Data Downloading the image data (in addition to the header data) is achieved using the ssw_jsoc_time2data.pro procedure but with an additional argument: ssw_jsoc_time2data,t0,t1,index,data,ds=[dataseries name] On output, the newly added DATA argument will contain the image data. What is actually being downloaded are Rice-compressed images, but ssw_jsoc_time2data.pro automatically uncompresses the data using read_sdo.pro (as mentioned in Section 4.6) prior to exiting the procedure. There will be a one-to-one correspondence between the images in the DATA array and the header information in the INDEX array. The MAX_FILES keyword limits the number of les that are downloaded. It is often prudent to use the MAX_FILES keyword unless you know how much data you are requesting. Each full-resolution, full-disk AIA or HMI image is approximately 12MB when compressed, expanding to 64MB when uncompressed. Here is an example illustrating how one would download a sample of 4 AIA images from one minute of mission time: ; copy and paste these commands into your SSWIDL session: t0=1-jan-2011 01:00 t1=1-jan-2011 01:01 ssw_jsoc_time2data,t0,t1,index,data,ds=aia.lev1,waves=[171,304],max_files=4 IDL> t0=1-jan-2011 01:00 IDL> t1=1-jan-2011 01:01 IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=aia.lev1,waves=[171,304],max_files=4 ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | -------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/60s] | ---------------------------------------------------------------------------------------------| selecting segment> image_lev1 | -------------------------------------------------------------------------------------------
TECHNICAL NOTE :
52
Guide to SDO Data Analysis version dated February 19, 2013 | /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} | ---------------------------------------------------------------------------------| HEX handler QUALLEV0 | -----------------------------------------| HEX handler QUALITY | --------------------FILTER>> ssnew=where(gt_tagval(index(ss),/wavelnth) eq 171 OR gt_tagval(index(ss),/wavelnth) eq -----------------------------------------------------------| /export, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} | -----------------------------------------------------------------------------| segment subselect | -------------------------------------------| Throtteling to MAX_FILES | -----------------------------------------------------------------------------------------------------| Getting> http://jsoc.stanford.edu//SUM103/D190077569/S00000/image_lev1.fits | --------------------------------------------------------------------------------------------------------------------------------------------------------| Getting> http://jsoc.stanford.edu//SUM103/D190077576/S00000/image_lev1.fits | --------------------------------------------------------------------------------------------------------------------------------------------------------| Getting> http://jsoc.stanford.edu//SUM103/D190077579/S00000/image_lev1.fits | --------------------------------------------------------------------------------------------------------------------------------------------------------| Getting> http://jsoc.stanford.edu//SUM103/D190077631/S00000/image_lev1.fits | -------------------------------------------------------------------------------------------------------------| Tile compressed data handling... | ---------------------------------------------------------| Uncompressing to> $HOME | --------------------------------------------------------------------------------------------------------------| $HOME//SUM103/D190077569/S00000/image_lev1.fits -> $HOME/AIA20110101_010036_0171.fits | ----------------------------------------------------------------------------------------------------------------------------------| struct2fitshead - using procedure: sxaddpar | ----------------------------------------------------------------------------------------------------------------------------------| $HOME//SUM103/D190077576/S00000/image_lev1.fits -> $HOME/AIA20110101_010044_0304.fits | -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
53
Guide to SDO Data Analysis version dated February 19, 2013 | $HOME//SUM103/D190077579/S00000/image_lev1.fits -> $HOME/AIA20110101_010048_0171.fits | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| $HOME//SUM103/D190077631/S00000/image_lev1.fits -> $HOME/AIA20110101_010056_0304.fits | --------------------------------------------------------------------------------------After downloading and uncompressing all of the requested data, the standardized AIA color tables can be loaded by using the aia_lct.pro routine (see Section 6.6.1), and a (downsampled) image can be displayed on screen. In the following example, the appropriate color table is determined from the setting of the WAVELNTH keyword from the image header: ; copy and paste these commands into your SSWIDL session: aia_lct,wave=index(0).wavelnth,/load window,0,xsiz=512,ysiz=512 tvscl,rebin(data(*,*,0),[512,512])
5.3.4 Exporting Specic Segments For dataseries with multiple segments, the default of ssw_jsoc_time2data.pro is to export the rst segment. (In the example in the previous subsection, the image_lev1 segment happens to be the rst segment.) To download any of the other segments, simply specify the segment name in the SEGMENT keyword. Be aware that only one segment name can be specied at a time, and that a series of data arrays with varying dimensions cannot be done in one pass. The following examples demonstrate how to download specic segments using ssw_jsoc_time2data.pro. As a rst example, the records of the hmi.S_720s dataseries contains 24 image segments (the four Stokes I, Q, U, and V parameters at each of six wavelengths). Because each of these segments are 40964096-pixel images, data from the same segment of multiple records can be downloaded in one pass. For example, retrieving one of the segments (in this example, the Q3 segment) for over an hour of time is achieved as follows: ; copy and paste these commands into your SSWIDL session: t0=1-jan-2011 01:00 t1=1-jan-2011 02:00 result=ssw_jsoc(ds=hmi.S_720s,/series_struct) print,result.segments.name ssw_jsoc_time2data,t0,t1,index,data,ds=hmi.S_720s,segment=Q3 IDL> t0=1-jan-2011 01:00 IDL> t1=1-jan-2011 02:00 IDL> result=ssw_jsoc(ds=hmi.S_720s,/series_struct) IDL> print,result.segments.name I0 Q0 U0 V0 I1 Q1 U1 V1 I2 Q2 U2 V2 I3 Q3 U3 V3 I4 Q4 U4 V4 I5 Q5 U5 V5 IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=hmi.S_720s,segment=Q3 ------------------------------| /SERIES_STRUCT, ds=hmi.S_720s | -----------------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60m] | ---------------------------------------------------------------------------------------------54
Guide to SDO Data Analysis version dated February 19, 2013 | selecting segment> Q3 | ----------------------[...deletia...] This can be done in one pass because all Q3 segments are 40964096-pixel images. In contrast, retrieving multiple segments from the same record (even if they are all dimensioned alike) must be done via multiple calls to ssw_jsoc_time2data.pro: ; copy and paste these commands into your SSWIDL session: t0=1-jan-2011 01:00 t1=1-jan-2011 01:01 result=ssw_jsoc(ds=hmi.S_720s,/series_struct) print,result.segments.name allq=where(strpos(result.segments.name,Q) eq 0) ; get desired segment indices ssw_jsoc_time2data,t0,t1,index,data,ds=hmi.S_720s,$ segment=result.segments(allq(0)).name ; do first one for i=1,5 do begin $ ssw_jsoc_time2data,t0,t1,itemp,dtemp,ds=hmi.S_720s,$ segment=result.segments(allq(i)).name & $ ; do the others data=[[[data]],[[dtemp]]] & endfor ; concatenate help,index,data IDL> t0=1-jan-2011 01:00 IDL> t1=1-jan-2011 01:01 IDL> result=ssw_jsoc(ds=hmi.S_720s,/series_struct) IDL> print,result.segments.name I0 Q0 U0 V0 I1 Q1 U1 V1 I2 Q2 U2 V2 I3 Q3 U3 V3 I4 Q4 U4 V4 I5 Q5 U5 V5 IDL> allq=where(strpos(result.segments.name,Q) eq 0) ; get desired segment indices IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=hmi.S_720s,$ IDL> segment=result.segments(allq(0)).name ; do first one ------------------------------| /SERIES_STRUCT, ds=hmi.S_720s | -----------------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60s] | ---------------------------------------------------------------------------------------------| selecting segment> Q0 | ----------------------[...deletia...] IDL> for i=1,5 do begin $ IDL> ssw_jsoc_time2data,t0,t1,itemp,dtemp,ds=hmi.S_720s,$ IDL> segment=result.segments(allq(i)).name & $ ; do the others IDL> data=[[[data]],[[dtemp]]] & endfor ; concatenate ------------------------------| /SERIES_STRUCT, ds=hmi.S_720s | ------------------------------55
Guide to SDO Data Analysis version dated February 19, 2013 -----------------------------------------------------------------------| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60s] | ---------------------------------------------------------------------------------------------| selecting segment> Q1 | ----------------------[...deletia...] IDL> help,index,data INDEX STRUCT = -> MS_034130194003 Array[1] DATA FLOAT = Array[4096, 4096, 6] Here, the Q0Q5 segments from the same record are being read one at a time, and concatenated into a single DATA array. Recall that because each of these segments are all part of the same record, the INDEX structure (which contains the header information for this record) will be the same for each segment, and thus there is no need to concatenate these. Retrieving segments from a set of records that are not dimensioned alike must also be done via multiple calls. In the following example, we use the spikes segment from the aia.lev1 dataseries, which contains the list of pixels that have been despiked. (For more information on despiking, see Section 6.3.) Because the number of despiked pixels varies from image to image, the amount of data stored in the spikes segment varies from record to record. One way to store this data within IDL is by using pointers, which enables the dynamic allocation of arrays: ; copy and paste these commands into your SSWIDL session: t0=1-jan-2011 01:00 t1=1-jan-2011 01:01 ssw_jsoc_time2data,t0,t1,index,ds=aia.lev1,waves=[171] ; get 5 records data=ptrarr(n_elements(index)) ; set up a pointer array for i=0,n_elements(index)-1 do begin & $ ssw_jsoc_time2data,index(i).t_obs,index(i).t_obs,ds=aia.lev1, $ segment=spikes,itemp,dtemp & $ ; get spikes segment data(i)=ptr_new(dtemp) & endfor for i=0,n_elements(index)-1 do help,*data(i) ; look at the data IDL> t0=1-jan-2011 01:00 IDL> t1=1-jan-2011 01:01 IDL> ssw_jsoc_time2data,t0,t1,index,ds=aia.lev1,waves=[171] ; get 5 records ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | -------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/60s] | ---------------------------------------------------------------------------------------------| selecting segment> image_lev1 | ------------------------------------------------------------------------------------------| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} | -------------------------------------------------------------
56
FILTER>> ssnew=where(gt_tagval(index(ss),/wavelnth) eq 171,sscnt) FILTER>> ssnew=where(strmatch(gt_tagval(index(ss),/img_type),LIGHT,fold_case=fold_case) eq 1,s IDL> data=ptrarr(n_elements(index)) ; set up a pointer array IDL> for i=0,n_elements(index)-1 do begin & $ IDL> ssw_jsoc_time2data,index(i).t_obs,index(i).t_obs,ds=aia.lev1, $ IDL> segment=spikes,itemp,dtemp & $ ; get spikes segment IDL> data(i)=ptr_new(dtemp) & endfor ----------------------------| /SERIES_STRUCT, ds=aia.lev1 | ------------------------------------------------------------------------------------------| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:01Z/0s] | ----------------------------------------------------------------------------------------| selecting segment> spikes | --------------------------[...deletia...] IDL> for i=0,n_elements(index)-1 do help,*data(i) ; look at the data <PtrHeapVar44> LONG = Array[4995, 3] <PtrHeapVar85> LONG = Array[5002, 3] <PtrHeapVar126> LONG = Array[4775, 3] <PtrHeapVar167> LONG = Array[4856, 3] <PtrHeapVar208> LONG = Array[5052, 3]
5.4 Requesting a Cutout Using SSW

The online cutout service described in Section 4.3 can also be accessed from SSWIDL using the ssw_cutout_service.pro procedure: ssw_cutout_service,t0,t1,ref_helio=[centroid coordinates],fovx=[width],fovy=[height],$ waves=[wavelengths],instr=[instrument],cadence=[cadence string],$ max_frames=[maximum frames],email=xxx@yyy.zzz To use this routine, the user should specify position (REF_HELIO is the latitude and longitude of the center of the cutout), the eld of view (FOVX and FOVY are the sizes of the eld of view, with arcseconds as the default unit), and the wavelength channels of interest (via the WAVES keyword). The MAX_FRAMES keyword should be set to the desired number of frames in the time series; setting this keyword to a value that is lower than that implied by full cadence will result in the time series being subsampled in time. Region-tracking is automatically turned on, unless the /NOTRACK keyword is set. A more comprehensive list of keywords and their default values can be found at this webpage which lists the API for the cutout service. As an example, suppose a user wants a tracked cutout from the 171 and 304 channels of AIA, sampled every four minutes over the course of a two-hour time span. The request might look similar to this sequence: t0=01-jan-2011 01:00 t1=01-jan-2011 03:00 ssw_cutout_service,t0,t1,fovx=256,fovy=256,ref_helio=N35W20,$ waves=[171,304],max_frames=30, instrument=aia,email=xxx@yyy.zzz
57
Guide to SDO Data Analysis version dated February 19, 2013 In this example, setting the MAX_FRAMES keyword to 30 in combination with the two-hour time window as specied in the T0 and T1 variables results in the desired four-minute image cadence. As another example, ssw_cutout_service.pro can be used to spatially downsample full-disk images to a more manageable size via use of the /FULL_DISK switch (which supersedes the REF_HELIO, FOVX, and FOVY keywords), with a query similar to the following: t0=01-jan-2011 01:00 t1=01-jan-2011 03:00 ssw_cutout_service,t0,t1,fovx=256,fovy=256,/full_disk,$ waves=[171,304],max_frames=30, instrument=aia,email=xxx@yyy.zzz In both cases, when a cutout request is nished, an e-mail containing a link to the data will be sent to the e-mail address provided in the EMAIL keyword.
5.5 Using the SSW Interface to the VSO

Queries to the VSO can either be submitted online (see Section 4.4) or through SSWIDL. Using SSWIDL, the basic sequence of queryselectdownload that users undergo when using the online portal is implemented using IDL routines in the VSO package from SSW, as is described in the following subsections. 5.5.1 Querying the VSO To run a VSO query, use the vso_search.pro routine. The routine has several keywords that may be useful for constructing queries, including PROVIDER, INSTRUMENT, PHYSOBS, and WAVE, that match the checkboxes on the online VSO entry page (see Figure 20 of Section 4.4). For example, in SSWIDL a query similar to that of Figure 21 for one hours worth of AIA data can be achieved as follows: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia) help,result help,result,/structures IDL> result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia) Records Returned : JSOC : 2362/2362 Records Returned : JSOC : 0/0 IDL> help,result RESULT STRUCT = -> VSORECORD Array[2362] IDL> help,result,/structures ** Structure VSORECORD, 13 tags, length=256, data length=252: TIME STRUCT -> VSOTIME Array[1] EXTENT STRUCT -> VSOEXTENT Array[1] WAVE STRUCT -> VSOWAVE Array[1] DETECTOR STRING INSTRUMENT STRING AIA SOURCE STRING SDO PROVIDER STRING JSOC INFO STRING AIA level 1, 4096x4096 [2.000 exposure] [100.00 percentd] PHYSOBS STRING intensity 58
Guide to SDO Data Analysis version dated February 19, 2013 FILEID SIZE EXPTIME DARK ECLIPSE PERCENTD URL GETINFO STRING FLOAT FLOAT LONG LONG FLOAT STRING STRING aia__lev1:171:1072918835 66200.0 1.99961 0 0 100.000
The above search returns 2362 records corresponding to images from all of the AIA channels. Some details about each record are stored in the array of structures returned by the vso_search.pro routine, with each element in the array corresponding to a valid record. To narrow this list down by wavelength channel, use the WAVE keyword: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia,wave=171) help,result IDL> result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia,wave=171) Records Returned : JSOC : 296/296 Records Returned : JSOC : 0/0 IDL> help,result RESULT STRUCT = -> VSORECORD Array[296] This query results in a more manageable 296 records. Additionally, this query can be further rened by specifying a sampling cadence (in seconds) in the SAMPLE keyword: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia,wave=171,sample=60) help,result IDL> result=vso_search(2011-01-01 1:00,2011-01-01 1:59,inst=aia,wave=171,sample=60) Records Returned : JSOC : 60/60 Records Returned : JSOC : 0/0 IDL> help,result RESULT STRUCT = -> VSORECORD Array[60] The above query returns 60 AIA records from the 171 channel from the requested hour. SDO data stored on the various VSO servers may be subject to expiration. When this happens, the result of vso_search.pro will be an empty string. In these cases, users will have to request the data directly from the JSOC, as demonstrated in Sections 4.2 and 5.3.
TECHNICAL NOTE : In IDL Version 8.0, the help,result IDL commands in the above examples do not work as shown here, and instead behave as if the /STRUCTURES switch has been set. This is apparently a bug and has been xed in IDL Version 8.0.1, according to this newsgroup discussion (Google login required). TECHNICAL NOTE :
59
Guide to SDO Data Analysis version dated February 19, 2013 5.5.2 Downloading Data from the VSO To retrieve records, one simply passes an array of structures returned from vso_search.pro to the vso_get.pro routine, as demonstrated in this two-image example: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result,out_dir=data,filenames=fnames,/rice) print,transpose(fnames)
IDL> result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) Records Returned : JSOC : 2/2 Records Returned : JSOC : 0/0 IDL> log=vso_get(result,out_dir=data,filenames=fnames,/rice) % VSO_GET: This will download 2 file(s) 2 : http://vso.tuc.noao.edu/cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;rec % RDWRT_BUFF: Please wait. Downloading... % File: /cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=171_1072918895% Size: 12343680 bytes % From: vso.tuc.noao.edu % To: data % HTTP::COPY: 12343680 bytes of 12343680 total bytes copied in 3.24 seconds % HTTP::COPY: Wrote 12343680 bytes to file data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1 1 : http://vso.tuc.noao.edu/cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;rec % RDWRT_BUFF: Please wait. Downloading... % File: /cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=171_1072918835% Size: 12343680 bytes % From: vso.tuc.noao.edu % To: data % HTTP::COPY: 12343680 bytes of 12343680 total bytes copied in 3.62 seconds % HTTP::COPY: Wrote 12343680 bytes to file data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1 Downloading completed IDL> print,transpose(fnames) data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits In this example, the OUT_DIR keyword is used to specify an (already existing) output directory, the FILENAMES keyword contains the list of les downloaded, and the /RICE switch is used to indicate that the data are to be downloaded in a Rice-compressed format (which, compared with data in an uncompressed format, signicantly reduces the download time). Unlike the ssw_jsoc_time2data.pro routine described in the previous section, the vso_get.pro does not load the data into an IDL variable; it instead only downloads the requested FITS les to the specied directory. To uncompress Rice-compressed les, one of the utility programs mentioned in Section 4.6 or the read_sdo.pro routine (see Section 5.6 below) must be used. If a proxy server is being used, then SSW needs to be proxy-aware in order for vso_get.pro to work properly; otherwise, vso_get.pro will appear to hang. To congure SSW to work with a proxy server, see Section 5.1.3. When SSW is properly congured to use a proxy server, the output to the vso_get.pro routine from the above example will be slightly different than what is shown.
TECHNICAL NOTE :
60
5.6 Uncompressing the Image Data

5.6.1 Using read_sdo.pro The SSWIDL routine read_sdo.pro can be used to uncompress and read both the header information and the data into IDL variables: read_sdo,fnames,index,data,/nodata,/noshell The input argument FNAMES contains a list of lenames to be uncompressed and read into IDL. On output, the index (header) information and image data will appear in the variables INDEX and DATA, respectively. Either setting the /NODATA keyword or omitting the DATA argument will result in only the header information being read in. Setting /NOSHELL may result in a slight speedup (if it works) when the underlying imcopy utility (from the CFITSIO subroutine library) is called during the uncompression step. The examples below assume that one has rst downloaded from the VSO the small (two-image) AIA Level 1.0 dataset demonstrated in Section 5.5.2: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result,out_dir=data,filenames=fnames,/rice) After doing so, one can use read_sdo.pro to uncompress the data: ; copy and paste these commands into your SSWIDL session: read_sdo,fnames,index,data help,index,data
IDL> read_sdo,fnames,index,data -----------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T08_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_080000_0171.fit --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T08_01_00.34Z.image_lev1.fits -> /tmp/AIA20110101_080100_0171.fit ----------------------------------------------------------------------------------------------IDL> help,index,data INDEX STRUCT = -> MS_033956771002 Array[2] DATA INT = Array[4096, 4096, 2] At this point, the data have been uncompressed and are now stored in the DATA variable. One can load the standardized AIA color tables via the aia_lct.pro routine (see Section 6.6.1) prior to displaying an image on screen. In the following example, the rst image in the DATA array is displayed using the appropriate color table, which is determined from the WAVELNTH keyword from the image header: ; copy and paste these commands into your SSWIDL session: aia_lct,wave=index(0).wavelnth,/load window,0,xsiz=512,ysiz=512 tvscl,rebin(data(*,*,0),[512,512])
61
Guide to SDO Data Analysis version dated February 19, 2013 The above example shows how to uncompress and load full-resolution AIA images into IDL, and in order to be able to display the image onscreen, the IDL function REBIN was used to downsize the image. However, the downsizing step can also be performed within read_sdo.pro by setting the OUTSIZE keyword equal to the desired output image size: ; copy and paste these commands into your SSWIDL session: read_sdo,fnames,index,data,outsize=[700,700] help,index,data
IDL> read_sdo,fnames,index,data,outsize=[700,700] -----------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010000_0171.fit --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010100_0171.fit ----------------------------------------------------------------------------------------------IDL> help,index,data INDEX STRUCT = -> <Anonymous> Array[2] DATA INT = Array[700, 700, 2] The dimensions in the OUTSIZE keyword need not be the same. Users should also take note of the fact that, when specifying the OUTSIZE keyword, the downsizing is performed in read_sdo.pro by using IDLs CONGRID function, which has a different behavior than the REBIN function used in the earlier example of this section: by default, when shrinking an array, CONGRID uses nearestneighbor resampling while REBIN averages multiple points, and the result is generally not the same. The read_sdo.pro routine accepts additional arguments LLX, LLY, NX, and NY that result in a cutout being returned. The cutout is specied by providing its lower-left corner in pixel coordinates in LLX and LLY, along with its dimensions in the NX and NY arguments, as follows: ; copy and paste these commands into your SSWIDL session: read_sdo,fnames,index,data,1024,2048,1200,600 help,index,data
IDL> read_sdo,fnames,index,data,1024,2048,1200,600 -----------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010000_0171.fit --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010100_0171.fit ----------------------------------------------------------------------------------------------IDL> help,index,data INDEX STRUCT = -> <Anonymous> Array[2] DATA INT = Array[1200, 600, 2]
62
Guide to SDO Data Analysis version dated February 19, 2013 Here, a rectangular 1200600-pixel cutout starting at pixel location (1024,2048) is returned by read_sdo.pro. Please note that specifying both the cutout information and the OUTSIZE keyword is likely to give unexpected results, and is not recommended.
TECHNICAL NOTE : The FITS world coordinate system standard uses the convention that the lower-left pixel of an image is numbered 1 (and not 0, as in IDL). Because HMI and AIA data conform to this standard, the user is urged to take care in determining the values passed to read_sdo.pro in the LLX and LLY arguments when specifying a cutout. For more information on world coordinates, see Wells et al. (1981) or Greisen & Calabretta (2002) or, for solar-specic information, Thompson (2006).
5.6.2 Using read_sdo.pro with a Shared Library The default method discussed above in Section 5.6.1 works by spawning the imcopy utility from within SSWIDL. The imcopy utility is part of the CFITSIO subroutine library, and is used to read in a compressed FITS le and write out an uncompressed FITS le to a temporary disk location, only to be read in by read_sdo.pro moments later. This process contains an extra I/O sequence, and consequently often results in a noticeable performance reduction, especially when large volumes of data are being processed. To mitigate this problem, the read_sdo.pro procedure also accepts a /USE_SHARED_LIB switch that, when set, will cause read_sdo.pro to attempt to use an external, pre-compiled shared library to perform the uncompression and return the result directly to IDL in memory. The shared library is accessed using IDLs CALL_EXTERNAL interface, and so needs to be compatible with the machine architecture and the version (32-bit or 64-bit) of IDL being used. Unfortunately, pre-compiled libraries do not exist for all platforms. If read_sdo.pro is called from a platform for which there is no library available, read_sdo.pro will default to using imcopy to perform the uncompression as in Section 5.6.1, although we encourage users familiar with compiling C code to attempt to create the shared library (see below). In this example using the /USE_SHARED_LIB switch, assume that one has rst downloaded from the VSO the same small (twoimage) AIA Level 1.0 dataset demonstrated above in Section 5.6.1: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result,out_dir=data,filenames=fnames,/rice) Then, uncompress the data using the shared library as follows: ; copy and paste these commands into your SSWIDL session: read_sdo,fnames,index,data,/use_shared_lib help,index,data IDL> read_sdo,fnames,index,data,/use_shared_lib -------------------------------------------------------------------| reading data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits | --------------------------------------------------------------------------------------------------------------------------------------| reading data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits | -------------------------------------------------------------------IDL> help,index,data INDEX STRUCT = -> <Anonymous> Array[2] DATA INT = Array[4096, 4096, 2] If a pre-compiled shared library is not available for the machine architecture on which you are running SSWIDL, you will see the following warning message when calling read_sdo.pro with the /USE_SHARED_LIB switch:
63
-------------------------------------------------------------------------------| fitsio/shared object request but not available for this OS/ARCH - using imcopy | -------------------------------------------------------------------------------When the /USE_SHARED_LIB switch is set, optional I/O keywords (such as OUTDIR, PARENT_OUT, ONLY_UNCOMPRESS, or COMP_DELETE) that can be passed to read_sdo.pro are not relevant and thus will be ignored. However, the image manipulation capabilities of read_sdo.pro, such as those invoked by setting the OUTSIZE keyword or by providing the LLX, LLY, NX, and NY arguments (as demonstrated above in Section 5.6.1), should still work with the /USE_SHARED_LIB switch. If both the cutout information and the OUTSIZE keyword are specied, and /USE_SHARED_LIB is set, then read_sdo.pro will rst extract the cutout and then resize the image. (However, as stated in Section 5.6.1 above, if /USE_SHARED_LIB is not set, then specifying both a cutout and OUTSIZE may give unexpected results, and is not recommended.) The FITS world coordinate system standard uses the convention that the lower-left pixel of an image is numbered 1 (and not 0, as in IDL). Because HMI and AIA data conform to this standard, the user is urged to take care in determining the values passed to read_sdo.pro in the LLX and LLY arguments when specifying a cutout. For more information on world coordinates, see Wells et al. (1981) or Greisen & Calabretta (2002) or, for solar-specic information, Thompson (2006). At present, shared libraries exist for only a few operating systems. If the value of the IDL variable OSARC = !version.os +_+ !version.arch is not any of darwin_x86_64, linux_x86, or linux_x86_64, then there is currently no precompiled shared library compatible with read_sdo.pro. However, if this is the case, we encourage users to attempt to create a working shared library and contribute such libraries for broader distribution by SSW. Libraries for machine architectures not listed here can be contributed by sending the library in an e-mail attachment to the address listed on this webpage. To create the shared library, a C compiler is needed (in the example below gcc has been chosen). The rst step is to compile the CFITSIO subroutine library, and take note of the location of libcfitsio.a (information in Section 4.6.1 may be useful here). Next, compile the source code for the shared library. To do this, locate the two les named get_info.c and get_data.c, available in the $SSW_ONTOLOGY/binaries/sources directory (or Windows equivalent) of your SSWIDL distribution. Also locate the header le idl_export.h that is distributed with IDL (usually found in $IDL_DIR/external/include). The compilation step is typically done using commands like gcc -I/usr/include -I$IDL_DIR/external/include -c get_info.c get_data.c to create the compiled objects get_info.o and get_data.o. If this step fails, read through the error output to see what les or objects are missing, and ensure that the compiler knows the correct locations of the various include les needed for compilation. Once the objects have been created, issue a command like gcc -shared -o fitsio.so get_info.o get_data.o libcfitsio.a to create a shared library named fitsio.so. Note that Mac OS X uses -bundle instead of -shared to indicate to the compiler and linker that a shared library is being created. Once the fitsio.so shared library has been created, users can call read_sdo.pro with the /USE_SHARED_LIB keyword by providing its path via the SHARED_LIB_PATH keyword. It is likely that the compiler may require additional switches to be set before a usable shared library can be created, and as a result some consultation with compiler manuals may be necessary. Be forewarned that using read_sdo.pro with incompatible shared library les may give incorrect or unexpected results at best, and may cause IDL to hang or crash at worst.
TECHNICAL NOTE :
64
6 How to Process AIA Data

In the following sections, we will describe the more common data-processing tasks related to the preparation of AIA data for scientic use, including information outlining how to convert AIA data from Level 1.0 to Level 1.5 (Section 6.1), how to align HMI Level 1.5 data with AIA Level 1.5 images (Section 6.2), how to nd out which pixels have been affected during the despiking process (Section 6.3), how to PSF-deconvolve AIA Level 1.0 images (Section 6.4), how to determine the lter-response proles for the AIA telescopes (Section 6.5), as well as some miscellaneous tasks (Section 6.6). These tasks by and large require IDL, along with an up-to-date SSW installation with the AIA package (see Section 5 on how to use SSW and install or update the AIA package). In some cases, the ONTOLOGY package also needs to be installed. As with Section 5, this section contains several examples of SSWIDL code. To guide users, many of these examples are given in pairs of code blocks: the rst block contains code that users can copy and paste into their IDL session, and the second block shows the output that should result.
6.1 Converting AIA Data to Level 1.5

AIA data at Level 1.0 have already been at-elded, and processed to remove bad pixels and spikes. However, AIA data at Level 1.0 are not co-aligned and likely have slightly different plate scales and roll angles across the various AIA wavelength channels, making multi-wavelength analyses problematic. Furthermore, the values of the keywords CDELT1, CRPIX1, CDELT2, CRPIX2, and CROTA2 that specify the orientation and scale for AIA Level 1.0 data may be out of date, as these values are likely to have been determined from the pointing information in effect at the time of the creation of the archived image le. Because the pointing information may undergo modest changes over time, and is occasionally subject to major revisions (such as in September 2012 when the more accurate telescope pointings acquired during the transit of Venus in June 2012 were implemented), one should not assume that the image headers contain the most accurate pointing information. To remedy these issues, the data should be converted to Level 1.5, which can be done using the aia_prep.pro routine available through SSW. Converting AIA data to Level 1.5 entails co-aligning images from all of the AIA channels, rescaling the images to a common 0.6 plate scale, and rotating the images so that the solar east-west and north-south axes are aligned with the horizontal and vertical axes of the image, respectively. The aia_prep.pro routine will seek the currently best-known pointing information before performing these manipulations. Please note that data at Level 1.5 are not exposure-time corrected. In its most basic usage, the aia_prep.pro routine is called as follows: aia_prep,fnames,indices,out_index,out_data The required FNAMES and INDICES arguments respectively contain either the index and data arrays from an already-loaded dataset (such as returned from read_sdo.pro) or a list of lenames (as a string array) and their corresponding index structures. In the latter case, aia_prep.pro will use read_sdo.pro to read the data into SSWIDL prior to performing the conversion to Level 1.5. Also in the latter case, setting the INDICES argument to a value of -1 indicates that all les in FNAMES are to be processed. In both cases, the resulting Level 1.5 index structures and data are returned to IDL in the OUT_INDEX and OUT_DATA arguments. The resulting Level 1.5 index structures and image data can be written to disk as follows: aia_prep,fnames,indices,out_index,out_data,outdir=[output directory],/do_write_fits Here, the /DO_WRITE_FITS switch indicates that the resulting Level 1.5 data are to be written to disk in the output directory specied by the OUTDIR keyword. The OUT_INDEX and OUT_DATA arguments, which are optional, can be omitted in this case. For full-disk images, the data will be translated so that sun center is located at the center of each image. For cutouts, specifying the /CUTOUT switch is needed, which indicates to aia_prep.pro that the image-center point should be taken from a reference frame (instead of using sun-center as the alignment point). This reference frame is assumed to be the rst frame in the image sequence, but can also be set to a different frame by specifying the INDEX_REF keyword. After running aia_prep.pro, the keywords CDELT1, CRPIX1, CDELT2, and CRPIX2 should all be updated to reect the new plate scale and image centering. Other keywords affected include LVL_NUM, R_SUN, CROTA1, and CROTA2, as well as the statistical 65
Guide to SDO Data Analysis version dated February 19, 2013 keywords (e.g., DATAMIN, DATAMAX, DATAMEDN, DATAMEAN, DATARMS, DATASKEW, DATAKURT, etc.). Processing data using aia_prep.pro should preserve any binning or cropping that has been done prior to export. To determine whether AIA data are at Level 1.0 or Level 1.5, users can check the LVL_NUM keyword in the image headers. Much of what is served, including AIA data requested directly from the JSOC (as described in Sections 4.2 and 5.3) and from the VSO (as described in Sections 4.4 and 5.5), are at Level 1.0. AIA data exported using the SSW cutout service (as described in Sections 4.3 and 5.4), can be at either Level 1.0 or Level 1.5. If the aia_prep.pro routine detects via the LVL_NUM keyword that the data have already been processed to Level 1.5, it will simply return the data unchanged. As an example of how to use aia_prep.pro, rst download the small (two-image) AIA Level 1.0 dataset used as an example in Section 5.5.2: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result,out_dir=data,filenames=fnames,/rice) Converting these Level 1.0 data to Level 1.5 is performed as follows: ; copy and paste these commands into your SSWIDL session: print,transpose(fnames) aia_prep,fnames,-1,out_index,out_data help,out_index,out_data
IDL> print,transpose(fnames) data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits IDL> aia_prep,fnames,-1,out_index,out_data Running AIA_PREP.PRO V4.13 -----------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010100_0171.fit ----------------------------------------------------------------------------------------------------------------------------------------| Removing uncompressed versions on request | ---------------------------------------------------------------------------------| /MIXED_COMP set but already homogenous | --------------------------------------------------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010100_0171.fit ----------------------------------------------------------------------------------------------------------------------------------------| Removing uncompressed versions on request | ------------------------------------------66
---------------------------------------| /MIXED_COMP set but already homogenous | --------------------------------------------------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010000_0171.fit ----------------------------------------------------------------------------------------------------------------------------------------| Removing uncompressed versions on request | ------------------------------------------IDL> help,out_index,out_data OUT_INDEX STRUCT = -> MS_033967876001 Array[2] OUT_DATA INT = Array[4096, 4096, 2]
TECHNICAL NOTE : If it is necessary to revert AIA data to their state prior to the despiking process, as discussed in Section 6.3, and/or to perform a PSF-deconvolution, as in Section 6.4, these should be performed prior to converting AIA data to Level 1.5. These operations are not common, but may be necessary in some cases. TECHNICAL NOTE : The FITS world coordinate system keywords that specify the orientation and scale for SDO images are CDELT1, CRPIX1, CDELT2, CRPIX2, and CROTA2. The CRPIX1 and CRPIX2 keywords specify the x and y offset in pixels from the lower-left pixel of the image to sun center, the CDELT1 and CDELT2 keywords specify the plate scale in arcseconds per pixel, and the CROTA2 keyword species the image roll in degrees relative to solar north (and is positive is the solar north pole is rotated clockwise with respect to the image). The FITS world coordinate system standard species (among many other things) that the CRPIX1 and CRPIX2 keywords use the convention that the lower-left pixel is numbered 1 (and not 0, as in IDL). Consequently, a full-resolution 40964096-pixel AIA Level 1.5 image will have these keywords set to 2048.5, because a 4096-pixel array has pixels numbered from 1 to 4096, for which the image center lies in between pixels 2048 and 2049 (hence 2048.5). For more information on world coordinates, see Wells et al. (1981) or Greisen & Calabretta (2002) or, for solar-specic information, Thompson (2006).
The default behavior for aia_prep.pro is to use the best-available values of the plate scale, pointing, and roll angle for each image. These values are not necessarily those given in the image headers, though they are usually fairly close. The best-avilable pointing parameters are derived from the master pointing table (MPT) in effect at the time of the call to aia_prep.pro, which is based upon pointing solutions calculated on a weekly basis and stored in a JSOC series. In cases where, for example, a user wishes to reproduce the results of an earlier study, it may be necessary to use a different pointing solution than the current default, because the pointing-dependent keyword values used in the original analysis may be different than those associated with the current MPT. In these cases, a call to aia_prep.pro may specify either a particular MPT record (via the MPT_REC keyword) or a particular reference date (via the T_REF keyword) to force the use of an earlier pointing solution. One can alternatively force the use of the pointing-related keyword values in the header by setting the switch /USE_HDR_PNT, though this is not recommended. Instances where some awareness of MPT matters may be useful include the analysis of nearreal-time data (where the pointing is based on extrpolated MPT records and thus is not nal) or after major MPT updates (such as in September 2012 when the calibration data acquired during the transit of Venus in June 2012 was used to better dene the telescope plate scale).
TECHNICAL NOTE :
67
6.2 Co-Aligning HMI Data with AIA Data

Because HMI images have a different plate scale than AIA images (HMI images are slightly more zoomed-in than AIA images), the ability to overlay data from HMI onto data from AIA requires the extra step of remapping data from one instrument so that it has the same plate scale as the other. Any roll angle differences must also be taken into account. To facilitate such alignment, the aia_prep.pro routine discussed in Section 6.1 can be used to process HMI Level 1.5 data so that it is co-aligned with AIA Level 1.5 data. To demonstrate this, the following example will download a concurrent AIA image and HMI line-of-sight magnetogram from the VSO, and demonstrate how to use aia_prep.pro to regrid them so that they have the same plate scale and orientation. ; copy and paste these commands into your SSWIDL session: result1=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) result2=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=hmi,physobs=los_magnetic_field) log=vso_get(result1(0),out_dir=data,filenames=fnames1,/rice) read_sdo,fnames1,index_171,data_171 log=vso_get(result2(0),out_dir=data,filenames=fnames2,/rice) read_sdo,fnames2,index_Blos,data_Blos aia_prep,index_171,data_171,out_index_171,out_data_171 aia_prep,index_Blos,data_Blos,out_index_Blos,out_data_Blos
IDL> result1=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) Records Returned : JSOC : 2/2 Records Returned : JSOC : 0/0 IDL> result2=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=hmi,physobs=los_magnetic_f Records Returned : JSOC : 1/1 IDL> log=vso_get(result1(0),out_dir=data,filenames=fnames1,/rice) [...deletia...] IDL> read_sdo,fnames1,index_171,data_171 -----------------------| Uncompressing to> /tmp | ---------------------------------------------------------------------------------------------------------------------| data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_010000_0171.fit ----------------------------------------------------------------------------------------------IDL> log=vso_get(result2(0),out_dir=data,filenames=fnames2,/rice) [...deletia...] IDL> read_sdo,fnames2,index_Blos,data_Blos -----------------------| Uncompressing to> /tmp | --------------------------------------------------------------------------------------------------------------------| data/hmi.m_45s.2011.01.01_01_01_30_TAI.magnetogram.fits -> /tmp/HMI20110101_010025_6173.fits | ---------------------------------------------------------------------------------------------IDL> aia_prep,index_171,data_171,out_index_171,out_data_171 Running AIA_PREP.PRO V4.13 IDL> aia_prep,index_Blos,data_Blos,out_index_Blos,out_data_Blos Running AIA_PREP.PRO V4.13
68
Figure 24: Sample HMI magnetogram images before and after aia_prep.pro. Note that a roll angle of approximately 180 degrees has been accounted for.
Comparing the images stored in the variables DATA_171 and DATA_BLOS, which contain the data before being run through aia_prep.pro with those contained in OUT_DATA_171 and OUT_DATA_BLOS, will reveal that the latter set of images have been rescaled and derotated to a common plate scale. The following set of commands should open two IDL graphics windows containing images similar to those seen in Figures 24 and 25: ; copy and paste these commands into your SSWIDL session: window,0 & !p.multi=[0,2,1] plot_image,data_blos<300>(-300),title=BEFORE AIA_PREP plot_image,out_data_blos<300>(-300),title=AFTER AIA_PREP window,1 & !p.multi=[0,2,1] plot_image,alog(data_171>10),title=BEFORE AIA_PREP plot_image,alog(out_data_171>10),title=AFTER AIA_PREP
6.3 Despiking/Respiking AIA Data

High-energy particles (e.g., cosmic rays) interacting with the detectors on AIA result in data values in the image data that are abnormally high. The sites at which such high data values occur are usually spatially localized, and consequently are referred to as spikes in the 69
Figure 25: Sample AIA 171-channel images before and after aia_prep.pro. The right-hand image is only slightly different than the left-hand image, but is now co-aligned with the right-hand magnetogram image in Figure 24.
data. The standard processing scheme used to produce AIA Level 1.0 data despikes (removes the spikes from) each image using a fast algorithm that searches for single pixels (or small clumps of pixels) that appear to have usually high data numbers relative to a selection of neighboring pixels. The algorithm ags such pixels, and determines a replacement value based on the values of those neighboring pixels that are deemed valid. The current despiking algorithm is generally very good for many, if not most, scientic applications, but it is not completely foolproof. For example, are kernels can often be erroneously agged (since they too t the general prole of a spike), and so some users may want to respike (undo the despiking of) their data and apply their own despiking algorithm. In this section, we will describe how to get the listing of pixels that have been affected by the despiking algorithm, and how to restore these pixels with their original data numbers. Respiking the data is performed using the aia_respike.pro routine available through SSW: aia_respike,in_index,in_data,out_index,out_data Here, the IN_INDEX and IN_DATA arguments contain the input Level 1.0 index structure and data array. The OUT_INDEX and OUT_DATA will contain the resulting respiked index and data. At present, the aia_respike.pro routine will operate on only one image record at a time. As an example, lets assume that one rst downloads and uncompresses the rst image in the AIA Level 1.0 dataset used in Section 5.5.2 to demonstrate the SSW interface to the VSO: 70
; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result(0),out_dir=data,filenames=fnames,/rice) read_sdo,fnames,index,data Respiking the image is performed as follows: ; copy and paste these commands into your SSWIDL session: print,index.nspikes ; NSPIKES keyword contains number of spikes aia_respike,index,data,out_index,out_data print,total(data ne out_data) IDL> print,index.nspikes ; NSPIKES keyword contains number of spikes 4995 IDL> aia_respike,index,data,out_index,out_data ---------------------| HEX handler QUALLEV0 | -----------------------------------------| HEX handler QUALITY | -------------------------------------------------------------------------------------------| Copying> http://jsoc.stanford.edu/SUM100/D121871420/S00000/spikes.fits | -----------------------------------------------------------------------MRDFITS: Image array (4995,3) Type=Int*4 IDL> print,total(data ne out_data) 4995.00 As is evident from this example, the number of despiked pixels is given by the NSPIKES keyword in the image header. The last statement conrms that the aia_respike.pro routine changed the correct number of pixels (4995 for this particular image) that were despiked during the original Level 1.0 processing. For each image record, the data in the spikes segment is a NSPIKES3 array containing the pixel index, prior data value, and new data value for each spike. When aia_respike.pro is called, the routine exports the spikes segment from the JSOC for the input record, and then uses this information to reconstruct the image prior to despiking. Users who have already downloaded a set of image records and wish to know which pixels in each image have been despiked should simply export (download) the associated spikes segment from the JSOC for each record (either by using the methods discussed in Sections 4.2 or 5.3, or by using the helper routine aia_jsoc_getcaldata.pro distributed via SSW), and then view the contents of these segments by using read_sdo.pro or any of the other compressed-image readers listed in Section 4.6. Using read_sdo.pro to read in multiple spikes segments in one call is not feasible at this time, due to the fact that read_sdo.pro cannot handle multiple les containing data arrays that have different dimensions. Users will need to read in multiple spikes segments one at a time in this case.
TECHNICAL NOTE :
6.4 Point Spread Function Deconvolution

Extreme ultraviolet radiation entering the AIA telescopes is diffracted and scattered by various components in the light path, including the wire meshes that support both the entrance and focal-plane lters, and the primary and secondary mirrors. The diffraction and 71
Guide to SDO Data Analysis version dated February 19, 2013 scattering caused by these components distributes incoming light from a point source across the image plane in a pattern called the point spread function (PSF). By considering each of the rays incident on the imaging system as a collection of point sources, the images recorded by AIA can be described mathematically as a convolution of the PSF with the image formed by the incoming radiation at the source (object) plane. Consequently, the act of removing the effect of the PSF from AIA images requires deconvolving the PSF from these images. Note that deconvolution is a non-standard step in processing the AIA data. Besides being time-consuming, it can create artifacts by amplifying noise occurring on small spatial scales. Therefore, unless users are specically interested in removing stray light due to diffraction and scattering by the telescope optics and/or sharpening the image, this step should probably be skipped. If the deconvolution of AIA images is desired, it should occur after respiking (as described in Section 6.3) but prior to other image manipulations, such as converting AIA images from Level 1.0 to Level 1.5 (as described in Section 6.1) or otherwise calibrating the data. Before performing a deconvolution, one needs to calculate a model PSF, which depends on the wavelength channel. This is achieved by using the aia_calc_psf.pro function in SSW: psf=aia_calc_psf(wavelength) The WAVELENGTH argument species the AIA channel of interest, and needs to be a string, with the routine accepting a value of either 94, 131, 171, 193, 211, 304, or 335. Computing the PSF takes a bit of time, but only needs to be done once per wavelength channel. The PSFs that are assumed to apply to each of the AIA channels are described in the AIA PSF and Deconvolution Notes and in the AIA calibration paper (Boerner et al. 2012; appearing in the SDO issue of Solar Physics). Once a PSF is calculated, the deconvolution step is performed by the aia_deconvolve_richardsonlucy.pro routine, which (as the name implies) uses a Richardson-Lucy iterative procedure for recovering the source image: image_deconvolved=aia_deconvolve_richardsonlucy(image,psf,/verbose) This function takes as input the IMAGE argument, containing the (oating-point) AIA image to be deconvolved, and the PSF argument, containing the model PSF computed using aia_calc_psf.pro. On output, the IMAGE_DECONVOLVED variable will contain the resulting deconvolved image. The /VERBOSE switch causes the aia_deconvolve_richardsonlucy.pro routine to optionally display informational updates as the deconvultion progresses. For example, after using SSWIDL to download and read the rst image from the example AIA Level 1.0 dataset in Section 5.5.2 via ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result(0),out_dir=data,filenames=fnames,/rice) read_sdo,fnames,index,data one can deconvolve this image as follows: ; copy and paste these commands into your SSWIDL session: psf=aia_calc_psf(strcompress(index.wavelnth,/remove)) image_deconv=aia_deconvolve_richardsonlucy(float(data),psf,/verbose) IDL> psf=aia_calc_psf(strcompress(index.wavelnth,/remove)) wx= 4.50000wy= 4.50000 First pass wide angle -100 First pass wide angle -99 First pass wide angle -98 First pass wide angle -97 First pass wide angle -96 72
Guide to SDO Data Analysis version dated February 19, 2013 [...deletia...] IDL> image_deconv=aia_deconvolve_richardsonlucy(float(data),psf,/verbose) Richardson-Lucy deconvolution (ver:v2011-Jul-13) of a 4096x4096 image and a 4096x4096 PSF. Performing 2D iteration 1 of 25 Performing 2D iteration 2 of 25 Performing 2D iteration 3 of 25 Performing 2D iteration 4 of 25 Performing 2D iteration 5 of 25 [...deletia...] At this stage, the IMAGE_DECONV variable contains the deconvolved image. The aia_deconvolve_richardsonlucy.pro routine only accepts one image at a time, but the same PSF can be used for multiple images from the same AIA wavelength channel. As detailed in the AIA PSF and Deconvolution Notes, the PSF calculated by aia_calc_psf.pro includes a component to model the large-scale diffraction pattern caused by the entrance and focal-plane lter meshes, and a Gaussian component to model the core of the PSF. By default, a PSF based upon on-orbit data is generated, and this PSF is the recommended model for removing the diffraction pattern and other large-scale stray light effects from AIA images, while simultaneously limiting the sharpening of noise and other image artifacts. However, the aia_calc_psf.pro routine also accepts a /USE_PREFLIGHTCORE switch that, if set, produces a PSF for which the Gaussian core is based solely upon pre-ight measurements of the optical system. (The diffraction pattern component is based upon the same analysis as for the default PSF.) The core of this pre-ight model is broader than that of the (default) on-orbit model, and while using this PSF produces a more aggressive deconvolution of AIA images, it may also result in an excessive amount of sharpening of noise artifacts.
TECHNICAL NOTE :
6.5 Getting AIA Filter Response Data

The lter responses for the EUV channels can be loaded using the aia_get_response.pro procedure. In the most recent version of the package (dated 13 February 2012; see Section 5.1.2 for instructions on how to update the AIA package), the following modications were made: The emissivity model has been updated to use CHIANTI version 7 to cover a broader wavelength range, and to x a bug in the calculation of the continuum emissivity. There are new options to apply corrections to the wavelength or temperature response for time-dependence and cross-calibration with EVE. An empirical x for missing emission lines in the AIA 94 and 131 channels is now available. The wavelength response (effective area) has not changed. The default behavior of the get_aia_response.pro routine has changed. The routine now requires one to specify either /DN or /PHOT to explicitly include or exclude the DN-per-photon correction. To return the response curves as a function of wavelength in units of photons cm2 : ; copy and paste these commands into your SSWIDL session: result=aia_get_response(/phot) help,result,/structures IDL> result=aia_get_response(/phot) IDL> help,result,/structures 73
Guide to SDO Data Analysis version dated February 19, 2013 ** Structure <2850c08>, 14 tags, length=352536, data length=352504, refs=1: NAME STRING AIA DATE STRING 20100804_232510 CHANNELS STRING Array[7] WAVE FLOAT Array[4001] ALL FLOAT Array[7, 4001] PLATESCALE FLOAT 8.46158e-12 UNITS STRING cm2 A94 STRUCT -> <Anonymous> Array[1] A131 STRUCT -> <Anonymous> Array[1] A171 STRUCT -> <Anonymous> Array[1] A193 STRUCT -> <Anonymous> Array[1] A211 STRUCT -> <Anonymous> Array[1] A304 STRUCT -> <Anonymous> Array[1] A335 STRUCT -> <Anonymous> Array[1] CORRECTIONS STRUCT -> <Anonymous> Array[1] When the /DN switch is set, the effective area function also includes the data-number-to-photon conversion: ; copy and paste these commands into your SSWIDL session: result=aia_get_response(/dn) help,result,/structures IDL> result=aia_get_response(/dn) IDL> help,result,/structures ** Structure <5803008>, 14 tags, length=352536, data length=352504, refs=1: NAME STRING AIA DATE STRING 20100804_232510 CHANNELS STRING Array[7] WAVE FLOAT Array[4001] ALL FLOAT Array[7, 4001] PLATESCALE FLOAT 8.46158e-12 UNITS STRING cm^2 DN phot^-1 A94 STRUCT -> <Anonymous> Array[1] A131 STRUCT -> <Anonymous> Array[1] A171 STRUCT -> <Anonymous> Array[1] A193 STRUCT -> <Anonymous> Array[1] A211 STRUCT -> <Anonymous> Array[1] A304 STRUCT -> <Anonymous> Array[1] A335 STRUCT -> <Anonymous> Array[1] CORRECTIONS STRUCT -> <Anonymous> Array[1] The response curves can also be computed as a function of (log) temperature, as shown in Figure 1, by setting the /TEMP switch: ; copy and paste these commands into your SSWIDL session: result=aia_get_response(/phot,/temp) help,result,/structures
74
IDL> result=aia_get_response(/phot,/temp) ------------------------------------------------------| Generating temperature response function from | | /ssw/sdo/aia/response/aia_preflight_all_fullinst.genx | | /ssw/sdo/aia/response/aia_preflight_fullemiss.genx | ------------------------------------------------------IDL> help,result,/structures ** Structure <19fa008>, 12 tags, length=14944, data length=14912, refs=1: NAME STRING AIA LOGTE FLOAT Array[101] CHANNELS STRING Array[7] UNITS STRING phot cm^5 s^-1 pix^-1 ALL DOUBLE Array[101, 7] A94 STRUCT -> <Anonymous> Array[1] A131 STRUCT -> <Anonymous> Array[1] A171 STRUCT -> <Anonymous> Array[1] A193 STRUCT -> <Anonymous> Array[1] A211 STRUCT -> <Anonymous> Array[1] A304 STRUCT -> <Anonymous> Array[1] A335 STRUCT -> <Anonymous> Array[1] CORRECTIONS STRUCT -> <Anonymous> Array[1] There are a number of additional options that can be specied: /UV: Returns response for the AIA 1600 1700 and 4500 channels instead of the EUV channels. /ALL: Includes functions for both thin and thick focal plane lters /FULL: Includes full detail on the components (mirrors, lters, etc.). Note that the resulting structure is formatted differently than the simple effective-area structure. /NOBLEND: Ignores potential crosstalk between two EUV channels sharing telescope (applies to the AIA 131and 335 channels and to the AIA 94 and 304 channels). VERSION: Allows the user to specify a version. Currently only two versions are available. Setting VERSION=1 uses the preight effective area, and setting VERSION=2 is the current release. There is no difference in the wavelength response functions. There are also some optional corrections to the effective area that can be applied by specifying the following keywords: /EVENORM: Normalizes the response to agree with observations from EVE on 1 May 2010. TIMEDEPEND_DATE: Applies a time-dependent correction to account for on-orbit degradation. This keyword can be set either as a switch, in which case the current date is used, or as a string specifying the time for which the correction is desired. More examples using aia_get_response.pro can be found in the text les included in the SSW telescope response tree, located in $SSW/sdo/aia/response/, or online here.
75
6.6 Miscellaneous AIA Tasks

6.6.1 Using AIA Standard Scaling and Colors The images displayed on many data-browsing webpages, such as those mentioned in Section 2, are often processed using a standardized color table and image scaling. The contrast levels of these images were chosen to bring out much of the detail in the image data, while the distinct color schemes allow fast identication of the AIA wavelength channel while browsing. The standardized scaling can be applied using the aia_colors.pro function from the AIA package from SSWIDL: result=aia_intscale(image,wave=171,exptime=exptime) After completion, the RESULT variable will contain the scaled image for the particular wavelength channel specied in the WAVE keyword (here the 171 channel is used as an example). If exposure-time normalization is needed, one may provide the exposure time of the image in the EXPTIME keyword. Note that this routine is not parallelized. The standardized AIA color tables can be accessed via the aia_lct.pro routine is also distributed via SSWIDL. Again using the 171 channel as an example, the color table can be loaded into variables as follows: aia_lct,wave=171,red,green,blue and/or made active using the /LOAD switch: aia_lct,wave=171,/load For both aia_intscale.pro and aia_lct.pro, valid arguments for the WAVE keyword are the integers 94, 131, 171, 193, 211, 304, 335, 1600, 1700, or 4500. As an example, rst download an image from the AIA 171 channel and apply aia_prep.pro: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00:00,2011-01-01 1:00:11,inst=aia,wave=171) log=vso_get(result,out_dir=data,filenames=fnames,/rice) aia_prep,fnames,-1,out_index,out_data Next, apply the standardized scaling, load in the color table for this image, and display this image on screen: ; copy and paste these commands into your SSWIDL session: scaled_im=aia_intscale(out_data,wave=out_index.wavelnth,exptime=out_index.exptime) aia_lct,wave=out_index.wavelnth,/load window,0 & plot_image,scaled_im
6.6.2 Creating AIA Tri-Color Images Tri-color images, such as those provided on The Sun Now and The Sun Today browse webpages (see Sections 2.1 and 2.2, respectively), have proved popular with research scientists and the general public alike. These types of images are created by slotting 8-bit grayscale images from three separate AIA wavelength channels into the red, green, and blue planes of a 24-bit (true-color) image. To make tri-color images, one rst co-aligns images from the different AIA channels, usually using aia_prep.pro (see Section 6.1) and then passes these data to the aia_colors.pro routine available through SSWIDL. The folling example illustrates how to make the EUV triplet from the 171, 193, and 211 channels of AIA. First, download a co-temporal set of images from these channels and run them through aia_prep.pro:
76
; copy and paste these commands into your SSWIDL session: result171=vso_search(2011-01-01 1:00:00,2011-01-01 1:00:11,inst=aia,wave=171) result193=vso_search(2011-01-01 1:00:00,2011-01-01 1:00:11,inst=aia,wave=193) result211=vso_search(2011-01-01 1:00:00,2011-01-01 1:00:11,inst=aia,wave=211) log=vso_get([result171,result193,result211],out_dir=data,filenames=fnames,/rice) aia_prep,fnames,-1,out_index,out_data Then, issue the call to aia_colors.pro to create the tri-color image: aia_colors,out_index,out_data,data_euv=data_euv,/do_jpeg After being provided an input index and data array, the routine will output a true-color image to the IDL variable specied in the DATA_EUV variable and/or to disk in the form of a JPEG image if the /DO_JPEG switch is set. There are three varieties of tri-color images using images from different combinations of AIA channels. The example above is the EUV triplet comprised of the 211, 193, and 171 channels and is output in the DATA_EUV keyword. Additionally, a high-temperature triplet comprised of the 94, 335, and 193 channels can be output in the DATA_HI_T keyword and a chromosphere/corona triplet comprised of the 304, 211, and 171 channels is output in the DATA_LOHI keyword. In fact, if one provides a co-temporal set of images from all AIA wavelength channels, the aia_colors.pro routine can output all three varieties of tri-color images as well as a data cube of images (via the DATA_CUBE_OUT keyword) to which the standardized scaling from the aia_intscale.pro function discussed in Section 6.6.1 above has been applied: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00:00,2011-01-01 1:00:11,inst=aia) log=vso_get(result,out_dir=data,filenames=fnames,/rice) aia_prep,fnames,-1,out_index,out_data aia_colors,out_index,out_data,data_cube_out=data_cube_out,$ data_euv=data_euv,data_hi_t=data_hi_t,data_lohi=data_lohi,/do_jpeg The end result here is the downloading of eight AIA images, and processing them to create a 409640968-pixel data cube in the DATA_CUBE_OUT variable and the three varieties of tri-color images, both contained in the DATA_EUV, DATA_HI_T, and DATA_LOHI variables and written to disk as JPEG images. 6.6.3 Plotting an AIA Light Curve Because the image headers are segregated from the images in the JSOC, any quantity that can be derived directly from the keywords can be computed quickly. For example, to plot an exposure-time-corrected AIA light curve as a function of time, one can use data contained in the DATAMEAN and T_OBS keywords from the image headers: ; copy and paste these commands into your SSWIDL session: ssw_jsoc_time2data,2011-02-20,2011-03-06,index,ds=aia.lev1_euv_12s,$ waves=335,key=t_obs,wavelnth,datamean,exptime,cadence=1h utplot,index.t_obs,index.datamean/index.exptime,psym=1 In this example, a light curve from the 335 channel for two weeks in early 2011 is plotted using the SSW procedure utplot.pro. (Incidentally, the jump in the light curve occurring late in the day of 24 February 2011 is a result of a bakeout of AIA Telescope #1 on this day, an event listed on the SDO Joint Science Operations Calendar.)
77
Guide to SDO Data Analysis version dated February 19, 2013 6.6.4 Interfacing with plot_map.pro (Dom Zarros Mapping Software) Many solar physicists make use of the mapping software developed by Dom Zarro and written in IDL that enables quick and easy registration and alignment of images from different sources. The software, which is included in every SolarSoft installation, works by converting images to more generic map structures that contain both the image data and their associated coordinate and scaling properties. Once in the map structures, data can then be plotted, rotated, stretched, and resized. The documentation linked to above contains several such examples. HMI and AIA data read into SSWIDL via read_sdo.pro can be readily converted to the map structure format used by the mapping software. To demonstrate this, rst acquire the two-image AIA Level 1.0 dataset used in Section 5.5.2 and convert them to Level 1.5 (as in Section 6.1): ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=aia,wave=171,sample=60) log=vso_get(result,out_dir=data,filenames=fnames,/rice) aia_prep,fnames,-1,out_index,out_data The following two commands convert the two images to a map structure, and then plots the rst one: index2map,out_index,out_data,map plot_map,map,/log,drange=[1e2,1e4] At this point, one should see an IDL graphics window open with the rst image. Axes showing the pixel coordinates in arcseconds are plotted, and the data are scaled to a range of 100 and 10,000 and plotted using logarithmic scaling. Note that the plot_map.pro will display images using the current roll angle dened in the map structure. This comes into play when working with HMI observables such as continuum intensity and line-of-sight magnetogram data, which often have a default roll angle of about 180 degrees. For example, if one were to create and plot the map structure from the HMI continuum intensity image shown in the example below, it would be plotted upside-down: ; copy and paste these commands into your SSWIDL session: result=vso_search(2011-01-01 1:00,2011-01-01 1:01,inst=hmi,physobs=intensity) log=vso_get(result,out_dir=data,filenames=fnames,/rice) read_sdo,fnames,index,data index2map,index,data,map plot_map,map To plot the map with solar north at the top of the image (that is, with a roll angle of 0 degrees), the rot_map.pro routine can be used: ; copy and paste these commands into your SSWIDL session: rotmap=rot_map(map,roll=0) ; derotated map plot_map,rotmap
6.6.5 Checking the QUALITY Keyword On occasion, AIA image headers and data are affected by operational items (e.g., off-pointing or defocusing operations associated with various calibration maneuvers) or are missing signicant data (e.g., partial images during eclipse seasons). Such operational items affecting the regular AIA data stream are usually listed in the SDO Joint Science Operations Calendar, and in addition the QUALITY keyword will be set to a non-zero value. As a result, it is worth checking either the calendar and/or the QUALITY keyword before performing any kind of analysis using AIA Level 1.0 images. 78
Guide to SDO Data Analysis version dated February 19, 2013 The QUALITY keyword is a 32-bit integer that encodes various ags in a bitwise fashion. The meanings of the individual bits are listed in Table 2. Of particular interest are bits 811 (indicating missing image data), bits 1215 (indicating pointing issues), and bit 18 (indicating that a calibration maneuver is taking place). The following example shows how to examine the QUALITY keyword from the headers of three images (one is eclipsed, one is nominal, and one was taken during a calibration maneuver) in order to determine whether the image can be used for analysis: ; copy and paste these commands into your SSWIDL session: ssw_jsoc_time2data,2011-01-20 15:30,2011-01-20 15:31,in1,ds=aia.lev1,key=quality ssw_jsoc_time2data,2011-02-20 10:30,2011-02-20 10:31,in2,ds=aia.lev1,key=quality ssw_jsoc_time2data,2011-03-20 06:30,2011-03-20 06:31,in3,ds=aia.lev1,key=quality for i=0,31 do if ishft(in1(0).quality,-i) then print,i for i=0,31 do if ishft(in2(0).quality,-i) then print,i for i=0,31 do if ishft(in3(0).quality,-i) then print,i IDL> ssw_jsoc_time2data,2011-01-20 15:30,2011-01-20 15:31,in1,ds=aia.lev1,key=quality [...deletia...] IDL> ssw_jsoc_time2data,2011-02-20 10:30,2011-02-20 10:31,in2,ds=aia.lev1,key=quality [...deletia...] IDL> ssw_jsoc_time2data,2011-03-20 06:30,2011-03-20 06:31,in3,ds=aia.lev1,key=quality [...deletia...] IDL> for i=0,31 do if ishft(in1(0).quality,-i) then print,i 17 18 IDL> for i=0,31 do if ishft(in2(0).quality,-i) then print,i IDL> for i=0,31 do if ishft(in3(0).quality,-i) then print,i 12 13 14 17 21 Here, the last three IDL command-line statements loop through each of the bits comprising the QUALITY keyword for the rst image in each of the three datasets, and print out which of these bits are set. By taking advantage of the ability to download only the header information from the JSOC (as shown here), one can examine the QUALITY keyword and screen out images that are likely undesirable without rst downloading the image. In the above example, the rst image (from 20 January 2011) was taken during an AIA calibration maneuver, and as a result bit 18 is set. Bit 17 is also set for this image. No output occurred for the second image (from 20 February 2011), indicating that QUALITY is zero and that the image is presumably clean. The third image (from 20 March 2011) was taken during the spring eclipse season for 2011, resulting in bits 1214 being set. Additionally, bits 17 and 21 are set for this image.
79
QUALITY bit(s) 0 1 2 3 4 57 8 9 10 11 12 13 14 15 16 17 18 19 20 21 2229 30 31
Meaning, if set Flateld data are not available Orbit data are not available Ancillary science data are not available Master pointing data are not available Limb-t data are not available (generally not applicable to AIA images) (empty) Value of MISSVALS keyword is nonzero Value of MISSVALS keyword is more than 1% of the value of the TOTVALS keyword Value of MISSVALS keyword is more than 5% of the value of the TOTVALS keyword Value of MISSVALS keyword is more than 25% of the value of the TOTVALS keyword Spacecraft is not in science pointing mode (coincides with ACS_MODE keyword set to a value other than SCIENCE) Spacecraft eclipse ag is set Spacecraft sun presence ag is not set Spacecraft safemode ag is set Dark image ag is set (coincides with IMG_TYPE keyword set to DARK) Image Stabilization System (ISS) loop is open Calibration image (empty) Focus is out of range Register ag is set (empty) Quicklook image Image is not available
Table 2: Listing of the meaning of the bits encoded by the QUALITY keyword for AIA Level 1.0 data. This table is reproduced from Appendix 3 of the AIA/SDO FITS keyword list.
80
7 Frequently Asked Questions

Questions, comments, and feedback:
Is there someone I can contact with problems or issues related to nding, getting, or analyzing SDO data? If, after looking through the Table of Contents as well as this list of frequent questions, your issue does not seem to be addressed, please send an e-mail to the address listed on this webpage.
Browsing and nding data:
How do I browse through the data? You can easily browse and download summary images and movies at any of the places mentioned in Section 2. How do I get a more detailed look instead of just a browse? One option is Helioviewer (see Section 2.4), which enables users to overlay images in multiple layers from multiple data sources. Another option is to query the HCR (see Section 3.1) to see whether data exists for the feature or event of interest. How do I nd data for a specic feature or event, but I havent determined (or dont know) the dates/times? The HEK (see Section 3.1) contains a fully cataloged and readily searchable database of many different events and features, such as active regions, lament eruptions, and coronal holes seen in SDO data. The database can be queried using iSolSearch (see Section 3.2), and the resulting event descriptions contain links to the associated data.
Accessing the data:
How do I get science-grade data for HMI and/or AIA? There are several options. Keep in mind that full-disk, full-resolution data adds up quickly. (If you would be satised with a subeld, or are willing to subsample the data either in space or time, see also the next two questions.) Once you have determined that your data volume constitutes a manageable size, HMI and AIA data are available through the JSOC (see Sections 4.2 or 5.3), the cutout service (see Sections 4.3 or 5.4), or the VSO (see Sections 4.4 or 5.5). Do I really have to learn about the inner workings of the JSOC in order to get AIA or HMI data? No, but it helps. Refer to the brief description in Section 4.1 for an accessible primer. More detailed information on the JSOC can be found by visiting the links found in that section. How do I request a subeld (localized patch) of the full-resolution image data? For AIA data, the easiest way to do this is to use the cutout service (see Section 4.3). If your desired subeld is related to that of a HEK event (say, as found using iSolSearch [see Section 3.2]), be aware that each HEK event entry has a link to the cutout service, as well as links to related observations that may point to relevant datasets that have already been created and thus are readily available. How do I subsample the data in either space or time? One option for subsampling in time is to use the VSO, which provides a great deal of exibility in accessing solar data at various temporal cadences (see Sections 4.4 or 5.5). Another option is to use the online cutout service form (see Section 4.3), which allows users to readily subsample in both space and/or time. A third option for subsampling in time is to specify a cadence when querying the JSOC, either using the lookdata tool (see Section 4.2.2) or using the SolarSoft routine ssw_jsoc_time2data.pro (see Section 5.3.2).
81
Guide to SDO Data Analysis version dated February 19, 2013 Ive heard that not all AIA data are available all of the time. How can I nd out what exists and what doesnt? At the time of this writing, AIA data from 13 May 2010 onward have been processed to Level 1.0, are online, and are available for export from the JSOC directly (see Sections 4.2 or 5.3) or for processing by the cutout service (see Sections 4.3 or 5.4). The exception is that there is usually about a 4-day lag for the most recent AIA data to be processed in the pipeline to Level 1.0. Changes to the data-processing status will be reected on the AIA Data webpage. The caveat is that, due to limitations in storage resources, it is possible that some AIA Level 1.0 data may be taken ofine at some point in the future. (Regarding the VSO, it is likely that the data online at the caches located at the various VSO locations will be different than what is online at the JSOC.) What about EVE data? EVE data are available from the EVE data access webpage. Access to EVE data via SolarSoft is currently under development.
Analyzing the data:
Do I need to prepare the data in any way before analyzing it (a la eit_prep.pro or trace_prep.pro)? For all published AIA Level 1.0 data, the standard and best-available gain (at-elding), lter, vignette, and bad-pixel/cosmicray corrections will have already been applied. Removing stray light, scattering and diffraction patterns from AIA Level 1.0 images is described in Section 6.4. If you have AIA Level 1.5 data, the images will additionally have already been centered, rescaled to a standard 0.6 plate scale, and rotated so that solar north is up in the image. If you have AIA Level 1.0 data, then the aia_prep.pro routine will need to be run to make these adjustments (see Section 6.1). For HMI Level 1.5 data based on ltergrams from the Doppler camera (including Dopplergrams, continuum intensity images, and line-of-sight magnetograms), such always-needed corrections will have already been applied. However, note that many HMI Level 1.5 images have a roll angle (contained in the CROTA2 keyword) of near 180 degrees (and so may appear upside-down). How do I know what data-processing level my data are at? The LVL_NUM keyword in the image header contains the data-processing level. Why are my HMI images upside-down? HMI images typically have a roll angle (contained in the CROTA2 keyword) of near 180 degrees, and it is likely that this roll angle has not been accounted for. How can I co-align AIA data with HMI data? Co-aligning data from HMI and AIA can be done using the aia_prep.pro procedure (see Section 6.2), which aligns, derotates, and rescales images from both instruments to a common 0.6 plate scale. Co-alignment can also be performed using the mapping software mentioned in Section 6.6.4. Where can I get AIA lter response data? For a summary plot, see Figure 1. For the actual numbers, see Section 6.5. Are the AIA color tables that are used on the browsing webpages available? Yes. See Section 6.6.1. How do I make the tri-color images available from the browsing webpages? For some SSWIDL commands, see Section 6.6.2. Where can I get a list of HMI or AIA keywords? Each instrument team has produced a document listing the various keywords. Because many keywords are shared between HMI and AIA, there is probably much overlap between the JSOC keywords for metadata document (sourced from the All About JSOC Names page from the JSOC wiki) and the AIA/SDO FITS keyword list (sourced from the AIA Keywords entry on SDOdocs). 82
Operations and pipelines:
Is there ops information somewhere? Yes, there is an SDO Joint Science Operations Calendar that lists instrument calibration maneuvers and other known issues that affect the standard operation of the various instruments on SDO. When are the eclipse seasons? The eclipse seasons occur annually at about the time of the vernal and autumnal equinoxes. For specic dates, consult the SDO Joint Science Operations Calendar. What is the status of the AIA and HMI processing pipeline? Check the JSOC processing status page.
83
8 Lists of Useful Links

Observatory and instrument links: SDO homepage at Goddard SDO Joint Science Operations Calendar JSOC processing status page AIA homepage at LMSAL SDOdocs (HEK and AIA documentation, mostly) EVE homepage at CU/LASP HMI homepage at Stanford HMI magnetic data splash page Data links: Sungate VSO entry page and search form JSOC data portal and the lookdata tool EVE science data access NOAA SWPC data center SolarSoft links: SSW installation webpage SSW upgrade webpage SSW codesearch form (for fast doc-header searches of entire SolarSoft library) SSW cutout service API Journal articles appearing in the SDO volume of Solar Physics (volume 275, Jan. 2012), all of which are available for download for free from the publisher via Open Access: Fisher (2012) (preface to the dedicated volume) Pesnell et al. (2012) (SDO overview) Lemen et al. (2012) (AIA instrument paper) Boerner et al. (2012) (AIA calibration paper) Hurlburt et al. (2012) (HEK paper) Martens et al. (2012) (AIA feature-nding consortium paper) Woods et al. (2012) (EVE instrument paper) Hock et al. (2012) (EVE/MEGS calibration paper) Didkovsky et al. (2012) (EVE/ESP calibration paper) Scherrer et al. (2012) (HMI instrument paper) Schou et al. (2012) (HMI design paper) Wachter et al. (2012) (HMI calibration paper #1) 84
Guide to SDO Data Analysis version dated February 19, 2013 Couvidat et al. (2012) (HMI calibration paper #2) Schou et al. (2012) (HMI calibration paper #3) Couvidat et al. (2012) (HMI time-distance pipeline paper #1) Zhao et al. (2012) (HMI time-distance pipeline paper #2) Drobnes et al. (2012) (SDO E/PO paper)
85
9 Version History of this Guide

Below are links to earlier versions of the online version of this guide, along with brief descriptions of major changes made from the previous version. This version: Included in Section 1.5 some links to the magnetic eld data products for HMI. Updated Table 1 (list of JSOC dataseries) of Section 4.1 to include the hmi.B_*, hmi.ME_* and hmi.[Ms]harp_* dataseries for HMI. Removed mention of the leap second added to UTC on 30 June 2012 in Section 4.2.4. Added Section 4.5 containing brief descriptions of and links to the HMI and AIA synoptic data streams. Corrected an error in the imcopy syntax presented in Section 4.6.1. Split Section 5.2 into two subsections, and added an example of how to search for substrings in event descriptions when performing a HER query via SolarSoft in Section 5.2.2. Removed from the examples in Sections 5.3.3 and 5.3.4 the /UNCOMP_DELETE and /COMP_DELETE switches, as these are now specied by default and are thus no longer needed. Added to Section 5.6.1 (formerly Section 5.6) information on how to use read_sdo.pro to downsample or extract cutouts from HMI or AIA images. Added Section 5.6.2, containing information on how to call read_sdo.pro using a external library to uncompress AIA and HMI data. The use of the external library often reduces the time needed to read in full-resolution AIA or HMI images (but is not yet available on all computing platforms), compared with the more I/O-intensive default process described in Section 5.6.1. Changed the title of Section 6.1 to better reect its contents, edited the text to be more clear regarding the treatment of cutouts, and added a technical note regarding the ability of aia_prep.pro to make use of historical master pointing tables. Expanded Section 6.6.1 to include information on how to apply the standardized scaling to AIA images (such as those available from the various data-browsing websites). Added Section 6.6.2 that illustrates how to create tri-color images from AIA channel triplets, and added a corresponding question in the FAQ (Section 7). Version dated 5 March 2012: Added navigational options to HTML version of the guide, namely links to sub-subsections in the sidebar and previous/up/next links in the headers and footers on each node. Updated Table 1 (list of JSOC dataseries) of Section 4.1 to include the slotted aia.lev1_* dataseries for AIA. Added examples to Section 4.2.2 demonstrating how to construct JSOC queries using the slotted aia.lev1_* dataseries for AIA. Updated the description of the cutout service in Section 4.3 as well as Figure 19 to match a more recent version of the online form. Added text to Section 4.4 discussing how to export a list of links from the VSO and download them using curl. Added Section 4.6.1 providing some pointers on how one might compile and install the imcopy utility. Added a technical note to Section 5.1.1 that discusses uses of the setssw alias. Added Section 5.1.3 describing how to congure SolarSoft so that it works with a proxy server. 86
Guide to SDO Data Analysis version dated February 19, 2013 Added an example to Section 5.3.2 to illustrate the use of the CADENCE keyword with ssw_jsoc_time2data.pro. Added Section 6.4 discussing how to perform a point spread function deconvolution on AIA images. Updated Section 6.5 to be consistent with the updated version of the AIA calibration routines in SSW. Replaced the light-curve example in Section 6.6.3 with a new one that uses the aia.lev1_euv_12s dataseries instead of the deprecated aia_test.synoptic2 dataseries. Added Section 6.6.4 describing how to use HMI and AIA image data with Dom Zarros mapping software. Updated the output of the SSWIDL examples throughout Sections 5 and 6 to reect more recent versions of the software. Also changed the vso_get.pro examples so that the FILENAMES keyword is employed to determine the list of downloaded les. Added questions to the FAQ (Section 7) addressing the HMI roll angle and the co-alignment of AIA and HMI data, and updated the answers to the questions about temporal subselection of data and AIA data availability. Added to Section 8 links to the articles appearing in the recently published (and freely available) volume of Solar Physics dedicated to SDO. Version dated 5 July 2011: Added mention of the AIA feature-nding consortium to Sections 3.1 and 8. Added Section 4.2.4, detailing the differences between some of the various timing keywords found in AIA and HMI image headers. In Sections 5 and 6, updated the outputs to several of the examples that were out of date. Version dated 20 May 2011: Added Section 6.2, describing how to co-align HMI data with AIA data. Added a contact e-mail in Section 1.2 and in the FAQ (Section 7). Version dated 28 April 2011: Added Section 6.6.5, containing information on the AIA QUALITY keyword Made minor changes to the formatting of the FAQ list in Section 7. Added links to the SDO Joint Science Operations Calendar in Sections 7 and 8. Version dated 31 March 2011: Updated Fig. 1 with a new version of the gure, which corrected a minor typo in the y-axis label. Updated Sections 4.3 and 5.4 to provide more accurate information about the cutout service. Added Section 5.6 to provide a more detailed description of how to use read_sdo.pro. Added more details to Section 6.1 regarding the particular keywords affected when converting AIA data from Level 1.0 to Level 1.5. Version dated 26 January 2011: Added Table 1 (list of dataseries published by the JSOC) to Section 4.1. Replaced references to the deprecated hmi_test.* and aia_test.* dataseries in Sections 4.1 and 4.2, in Figures 14 18, and in the SSWIDL examples in Section 5.3, with references to the nal production dataseries hmi.* and aia.*, respectively.
87
Guide to SDO Data Analysis version dated February 19, 2013 Added text to Section 5.3 describing how to get keyword and segment informational data. Added text to Section 4.2, and a new subsection in 5.3, describing how to export specic segments of JSOC dataseries. Updated the demo in Section 5.5 to use a more recent dataset, and added information describing how to use read_sdo.pro to read Rice-compressed data into IDL. Created Section 6 on AIA data processing, which includes sections discussing aia_prep.pro (Section 6.1) as well as aia_respike.pro (Section 6.3), and moved the Miscellaneous SSW Items section from Section 5 to this new section, where they comprise Sections 6.5 and 6.6. Revised the Do I need to prepare the data ... question in the FAQ (Section 7) to reect the existence of aia_prep.pro. Also made minor editions to some of the other questions. Added Section 9 (this section). Moved the link to the PDF version of the guide so that it appears (more prominently) after the link to the Contents section in the sidebar. Version dated 10 December 2010: Partitioned Section 5.3 into three subsections that directly correspond to the three subsections of Section 4.2. Made minor editions to the text in Sections 4 and 5. Version dated 30 November 2010: Earliest version of the guide in its current form.
88
10 Contributors to this Guide

Editors: Marc DeRosa, Greg Slater Browse webpages: Dean Pesnell and Kevin Addison (The Sun Now), Greg Slater and Linus Chang (The Sun Today), Peter Gallagher (SolarMonitor), Jack Ireland and Daniel Mueller (Helioviewer and jHelioviewer) AIA instrumental insight: Paul Boerner, Dick Shine, Mark Weber, Yingna Su, Paolo Grigis JSOC information and demos: Phil Scherrer, John Seran, Yang Liu, Keh-Cheng Chu, Rock Bush, Jesper Schou HEK information and demos: Mark Cheung, Neal Hurlburt, Linus Chang, Ryan Timmons, Ankur Somani, Scott Green SSW information and demos: Sam Freeland VSO information and demos: Joe Hourcl Sage advice: Karel Schrijver, Barbara Thompson, Karin Muglach, and many others...
89

SDOD0060 P Guide To SDO Data Analysis Sdoguide

Uploaded by

Copyright:

Available Formats

You might also like

SDOD0060 P Guide To SDO Data Analysis Sdoguide

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SDOD0060 P Guide To SDO Data Analysis Sdoguide

Uploaded by

Copyright:

Available Formats

Guide to SDO Data Analysis

edited by Marc DeRosa & Greg Slater February 19, 2013

10 Contributors to this Guide

PDF version of this guide

Guide to SDO Data Analysis version dated February 19, 2013

1.2 About this Guide

1.3 Data Products from AIA

1.4 Data Products from EVE

1.5 Data Products from HMI

Guide to SDO Data Analysis version dated February 19, 2013

AIA Telescopes, Wavelengths, and Thermal Response

1600 C IV + Cont. 1700 UV Cont. 4500 White Light

Looking at the AIA from the Sun

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

2 How to Browse SDO Data

2.1 The Sun Now

2.2 The Sun Today

2.4 The Helioviewer Project

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Figure 7: The SolarMonitor.org front page for 3 August 2010.

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

3 How to Find SDO Data

3.1 Heliophysics Events Knowledgebase

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Figure 11: A schematic illustration of the operation of the HER.

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

4 How to Get AIA and HMI Data

4.1 How Data are Organized at the JSOC

4.2 The lookdata Tool

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

4.3 The Cutout Service

4.4 The Virtual Solar Observatory

Guide to SDO Data Analysis version dated February 19, 2013

4.5 Synoptic Data

4.6 Uncompressing the Data

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

Guide to SDO Data Analysis version dated February 19, 2013

5 How to Use SolarSoft with SDO Data

5.1 Installing or Upgrading SolarSoft