The NCAR Command

Language (NCL)

Data Analysis Lecture

Matthew Janiga
2/11/2014
Where Can I Learn More?
The NCL Website

http://www.ncl.ucar.edu/index.shtml
 Where Can I Learn More?
Manual, Lectures, and NCL-Talk

Reference Manuals
http://www.ncl.ucar.edu/Document/Manuals/

Lectures from the NCL Workshop

http://www.ncl.ucar.edu/Training/Workshops/lectures.shtml

NCL Email Lists

https://www.ncl.ucar.edu/Support/email_lists.shtml
 Talk Overview

1) Introduction.
2) Language Basics.
3) File Input / Output.
4) Data Analysis.
5) Graphics.
6) Tips and Tricks.
1) Introduction
 What is NCL?
• NCL is a complete language with types, variables,
operators, expressions, conditional statements,
loops, and functions and procedures.
• NCL can read many data formats: NetCDF, HDF,
GRIB, ascii and csv, binary, WRF output, and
more.
• NCL like MATLAB is an array based language.
Arrays can be sent to functions (100s and
growing) which perform computations, plotting
functions (dozens and growing), or files (typically
NetCDF but also ascii and binary).
 NCVIEW
http://meteora.ucsd.edu/~pierce/ncview_home_page.html

NCVIEW is a simple GUI that lets

you explore a NetCDF file:
examine slices, time series,
animate, and more. Look at the
data you’ve generated before
spending time to make a plot!

NCVIEW is one of MANY

programs that do this.
http://www.unidata.ucar.edu/software/netcdf/software.html
 The Example Page

Documentation! The NCL webpage

has 100s of examples each
containing a picture of the plot and
the code required to make it.

http://www.ncl.ucar.edu/Applications/
Function Documentation

Every function has an

exhaustive documentation
page showing how to use it,
describing how the
calculation is performed, and
providing examples.
 Advantages of NCL
• Free, stable, and extremely well documented.
• NCL is a fully fledged programming language not calls to
compiled programs.
• NCL allows you to read in large amounts of data into memory
(64-bit vars > 2GB) and perform computations there instead of
constantly reading and writing to disk.
• Lots of meteorology and climate specific functions for data
analysis and plotting.
• NCL is very flexible, it can analyze any array of any dimension,
type, with any coordinate description.
– Store and process: trajectories, tracks, hovmollers, time
series, polar orbit swaths and other non-uniform projections,
radar data, geostationary satellite data, surface obs, profiler
data, radiosondes, and pretty much anything you can think
of.
 Disadvantages of NCL
• NCL has a bit of a learning curve. Learning how to fully
utilize it and code using arrays takes time.
• While there are lots of mathematical functions and
procedures MATLAB and Python have more types and
variants. Similarly, R has more statistical functions.
• Like all interpreted languages loops are slow. NCL is
designed to use the pre-built array-processing functions.
– The Fortran and C interfaces allow you to make your
own fast pre-built functions.
NCL Eye Candy
And more…..
2) Language Basics
 netCDF / NCL Relationship

• NCL makes GRIB, HDF, HDF-EOS

look like netCDF files
 netCDF Conventions
Convention: set of accepted rules for file contents
●
make data comparison easier
●
facilitate automatic use of viewing (eg: ncview)

COARDS (1995; frozen)

●
Cooperative Ocean/Atmosphere Research Data Service
●
http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_prof
ile.html
CF (2005/2006; continues to evolve)
●
Climate and Forecast Metadata Convention (1.0 -> 1.6)
●
generalize and extend the COARDS convention
 Parts of netCDF file
ncdump –h foo.nc (or ncl_filedump foo.nc)
DIMENSION SIZES VARIABLES:
& NAMES Names , Types, Attributes,
dimensions: Coordinate Variables
lat = 64 variables:
lon = 128 float lat(lat)
lat:long_name = "latitude"
time = 12 lat:units = "degrees_north"
float lon(lon)
time=UNLIMITED (12 currently) lon:long_name = "longitude"
lon:units = "degrees_east"
FILE ATTRIBUTES double time(time)
global attributes:
title = “Temp: 1999” time:long_name = "time"
source = “NCAR” time:units = ”hours_since …"
Conventions = ”CF-1.0” float T(time, lat, lon)
T:long_name = “Temperature”
exercise: T:units = “degC"
T:missing_value = 1.e+20f
ncl_filedump FOO.nc | less T:_FillValue = 1.e+20f
ncl_filedump FOO.grb | less
 NCL/netCDF Variable Semantics
double T(time, lat, lon)
T: long_name = “Temperature”
T: units = “degC"
T: _FillValue = 1.e+20f
variable type – double (float, int, short,..)
variable name – T
named dimensions – time, lat, lon
attributes – long_name, units, _FillValue

float prr(time, y, x)
prr:_FillValue = -9999.f ; CF
prr:missing_value = -9999.f ; COARDS
prr:long_name = "Liquid Precipitation" ; CF, COARDS
prr:grid_mapping = "Lambert_Conformal" ;
prr:units = "kg m-2 s-1" ; CF, COARDS

prr:height = "surface" ;
prr:coordinates = "lon lat" ; CF
 netCDF/NCL variable
• array [could be of length 1 (scalar)]
• (may) have additional information
4.35 4.39 0.27 -3.35 -6.90
4.36 4.66 3.77 -1.66 4.06
9.73 -5.84 0.89 8.46 10.39
17 3.68 5.08 0.14 -5.63
-0.63 -4.12 -2.51 1.76 -1.43
-4.29 0.07 5.85 0.87 8.65

name: x
type: float [real]
shape: 2-dimensions
size: 6 (rows) x 5 (columns)
values: x(2,3) = 8.46 [row major, 0-based indexing]
long_name: “Temperature”
units: “degC” Meta data
named dimensions: x(time,lat)
lat: (/ -60, -30 ,0, 30, 60 /)
time: (/2000, 2001, 2002, 2003, 2004, 2005, 2006 /)
 Detailed Look netCDF Variable (NCL)
ncl <return> ; interactive mode
ncl 0 > f = addfile ("UV300.nc", "r") ; open file (nc, grb, hdf, hdfeos)
ncl 1 > u = f->U ; import STRUCTURE
ncl 2 > printVarSummary (u) ; overview of variable

Variable: u Classic netCDF

Type: float
Total Size: 65536 bytes Variable Model
16384 values
Number of Dimensions: 3
Dimensions and Sizes: [time|2] x [lat | 64] x [lon | 128] NCL
Coordinates: syntax/funcs
time: [ 1 .. 7 ]
lat: [ -87.8638 .. 87.8638 ] query
lon: [ 0 .. 357.185] use
Number of Attributes: 5 modify
_FillValue : 1e36 [CF ]
units : m/s [COARDS, CF] add
long_name : Zonal Wind [COARDS, CF] any aspect of
short_name : U
missing_value : 1e36 [COARDS; CF-1.6 ] data object
 netCDF [NCL] Variable model
X f = addfile(“foo.nc”,”r”) ; grb/hdf
Scalar
or
x = f->X
Array
NCL reads the scalar/array,
attributes, and coordinate
attributes
long_name variables as one object (structure)
_FillValue
X
units
add_offset accessed via @ accessed via &
scale_factor
etc.
values attributes
long_name coord var
coordinates Scalar _FillValue time
time lev
or units
lev add_offset lat
lat Array lon
scale_factor
lon etc. etc.
etc.
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl" Graphics Libraries
f = addfile("erai_1989-2009.mon.msl_psl.nc","r") ; open file [hdf,grib]
p = f->SLP ; (time,lat,lon)
; ( 252,121,240)

printVarSummary(p) ; netCDF variable model

wks = gsn_open_wks("ps","parvis_1") ; open a PS file

plot = gsn_csm_contour_map(wks,p(0,:,:),False) ; default plot
; uses attributes, coordinates

p
NetCDF [NCL] Variable model accessed via @ accessed via &

p = f->SLP values attributes coords

scalar _FillValue time
NCL reads
or long_name latitude
array
• data values (scalar or array) missing_value longitude

• attributes @ units etc

• coordinate variables & etc.

as a single data object.

 p
NetCDF [NCL] Variable model accessed via @ accessed via &

p = f->SLP values attributes coords

scalar _FillValue time
NCL reads as a single data or long_name latitude
object... array
missing_value longitude
• data values units etc
• attributes etc.
• coordinate arrays
Variable: p “printVarSummary(p)” output
Type: float
Total Size: 29272320 bytes
7318080 values
Number of Dimensions: 3
Dimensions and sizes: [time | 252] x [latitude | 121] x [longitude | 240]
Coordinates:
time: [780168..963504]
latitude: [90..-90]
longitude: [ 0..358.5]
Number Of Attributes: 4
_FillValue : 1e+20
units : hPa
long_name : Mean sea level pressure
missing_value :1e+20
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_code.ncl"
load "$NCARG_ROOT/lib/ncarg/nclscripts/csm/gsn_csm.ncl"

f = addfile("erai_1989-2009.mon.msl_psl.nc","r") ; open file [hdf,grib]

p = f->SLP ; (time,lat,lon)
; ( 252,121,240)

printVarSummary(p) ; netCDF variable model

wks = gsn_open_wks("ps","parvis_1") ; open a PS file

plot = gsn_csm_contour_map(wks,p(0,:,:),False) ; default B&W plot
 NCL Syntax Characters (subset)
• = - assignment
• := - reassignment (v6.1.2)
• ; - comment [can appear anywhere; text to right ; ignored]
• -> - use to (im/ex)port variables via addfile(s) function(s)
• @ - access/create attributes
• ! - access/create named dimension
• & - access/create coordinate variable
• {…} - coordinate subscripting
• $...$ - enclose strings when (im/ex)port variables via addfile(s)
• (/../) - array construction (variable); remove meta data
• [/../] - list construction;
• [:] - all elements of a list
• : - array syntax
• | - separator for named dimensions
• \ - continue character [statement to span multiple lines]
• :: - syntax for external shared objects (eg, fortran/C)
 Data Types
numeric (classic netCDF3) non-numeric
• double (64 bit) • string
• float (32 bit) • character
• long (64 bit; signed +/-) • graphic
• integer (32 bit; signed +/-) • file
• short (16 bit; signed +/-) • logical
• byte ( 8 bit, signed +/-) • list
• complex NOT supported

enumeric (netCDF4; HDF5)

• int64 (64 bit; signed +/-)
• uint64 (64 bit; unsigned ) snumeric
• uint (32 bit; unsigned ) [numeric , enumeric]
• ulong (32 bit; unsigned )
• ushort (16 bit; unsigned )
• ubyte ( 8 bit, unsigned)
 Simple Variable Creation
a_int =1
a_float = 2.0
a_double = 3.2d
a_string = "a”
a_logical = True [False]
• array constructor characters (/…/)
●
a_integer = (/1, 2, 3/)
●
a_float = (/2.0, 5, 8.0/)
●
a_double = (/12, 2d0, 3.2 /)
●
a_string = (/"abcd", "e", "Hello, World”/)
●
a_logical = (/True, False, True/)
●
a_2darray = (/ (/1,2,3/), (/4,5,6/), (/7,8,9/) /)

http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/Ncl
DataTypes.shtml#BasicNumericTypes
 Variable Creation and Deletion
a = 2.0
pi = 4.0*atan(1.0)
s = (/ “Melbourne”, “Sydney”, “Toulouse”, “Boulder” /)
r = f->precip ; (time,lat,lon)
R = random_normal(20,7, (/N,M/) ) ; R(N,M)
q = new ( (/ntim, klev, nlat, mlon/), “double” )
; free memory; Generally, do not need to do this
; delete each variable individually
delete(a)
delete(pi)
delete(s)
delete(r)
delete(R)
; delete multiple variables in one line
delete( [/ a, pi, s, r, R, q /] ) ; [/…/] list syntax
 Conversion between data types
●
NCL is a ‘strongly typed’ language
●
constraints on mixing data types
●
coercion
●
implicit conversion of one type to another
●
automatic coercion when no info is lost
●
let i be integer and x be float or double
●
fortran: x=i and i=x
●
NCL: x=i and i=toint(x)
●
many functions to perform conversions
 Variable Reassignment
• NCL = will not allow the following
k = (/ 1, 3, 4, 9 /) ; 1d array, type integer
… later in code …
k = (/17.5, 21.4/) ; different size and type
●
Two approaches
●
Up to version 6.1.1, 2 steps required
delete(k) ; delete existing variable
k = (/17.5, 21.4/) ; new assignment
Version 6.1.2
k := (/17.5, 21.4/) ; delete previous variable
; and reassign ‘k’
 Meta Data
• information associated with variable or file
●
attributes: @ (numeric, text)
●
named dimensions: ! (text)
●
coordinates: & (numeric)
• @, !, & can be used to create/assign,
retrieve
• most frequently, meta data is read from files

• Meta Data is your friend! Helps you

remember what you did to generate your
data and share it with others.
 Attributes [ @ ]
●
info associated with a variable or file
●
attributes can be any data type except file or list
●
scalar, multi dimensional array (string, numeric)
• assign/access with @ character
T = (/ 10, 25, 39 /)
T@units = “degC”
T@long_name = “Temperature”
T@wgts = (/ 0.25, 0.5, 0.25 /)
T@x2d = (/ (/1,2,3/), (/4,5,6/), (/7,8,9/) /)
T@_FillValue = -999
title = T@long_name
• attribute functions [isatt, getfilevaratts]
if (isatt(T,"units")) then .... end if
atts = getfilevaratts (fin, "T")
• delete an attribute: delete(T@wgts)
 _FillValue attribute
• Unidata & NCL reserved attribute; CF compliant

• netCDF Operators [NCO] & CDO: _FillValue attribute

• ncview: recognizes missing_value attribute (COARDS)
●
best to create netCDF files with both

●
NCL functions recognize _FillValue
●
most functions will ignore for computations (eg, “avg”)
●
use built-in function “ismissing” to check for _FillValue
●
if (any (ismissing(T) )) then … end if
●
NOTE: if (any(T.eq.T@_FillValue)) will not work

• NCL: best to not use zero as a _FillValue

●
OK except when contouring [random bug]
 Arrays and Dimension Numbering
• row major
 left dimension varies slowest; right dim varies fastest
 dimension numbering left to right [0,1,..]
• subscripts
 0-based [ entire range for N index values: 0,N-1 ]

Consider T(:,:,:) T (0,1,2)

left dimension is 0 : varies slowest
middle dimension is 1
right dimension is 2 : varies fastest
Different language/tool ordering:
• NCL/C/C++ : 0-based; left (slowest) - right (fastest)
• fortran, Matlab, R: 1-based; left (fastest) - right(slowest)
• IDL : 0-based; left (fastest) - right(slowest)
 NCL (netCDF): Named Dimensions [!]
• may be “named”: x(time,level,lat,lon)
• dimensions are named on netCDF files
• alternative way to reference subscripts
• assigned with ! character {let T(:,:,:) -> T(0,1,2)}
T!0 = "time" ; leftmost [slowest varying] dim
T!1 = "lat“
T!2 = "lon" ; rightmost [fastest varying] dim
• dim names may be renamed, retrieved
T!1 = ”latitude" … dName = T!2
• delete can eliminate: delete (T!2)
• named dimensions used to reshape
T(lat|:, lon|:, time|:)
Create, Assign Coordinate Variables [&]
• create 1D array
time = (/ 1980, 1981, 1982 /)
time@units = “yyyy”
lon = ispan(0, 355, 5)
lon@units = “degrees_E”
• assign dimension name [same as variable name]
time!0 = “time”
lon!0 = “lon”
• let x(:,:) … dimension numbers x(0,1)
• name dimensions
x!0 = “time” … x!1 = “lon”
• assign coordinate variables to x
x&time = time … x&lon = lon
 netCDF/NCL: Coordinate Variable (CV)
• CV: Coordinate Variable ( Unidata definition )
●
one dimensional variable in which dimension name is
the same as the variable name
●
must be monotonically increasing or decreasing
• Examples of CV
●
lat(lat), longitude(longitude), plevel(plevel), time(time)
• CV allow ‘natural’ coordinates via {…} syntax
●
Q(time,plevel,lat,longitude)
●
CV: Q(:, {925:400}, {-20:60}, {130:280} )
●
Index: Q(:, 3:10, 24:40, 42:75)
The following is not a coordinate variable:
float xlat(nlat, mlon) ; two-dimensions
xlat:units = "degrees_north”
It is an array that contains coordinate information.
Requires use of standard integer index values
 Meta Data Syntax Review:
Access/Change/Create/Delete
• @ attribute
u@long_name = "U"
longName = u@long_name
• ! named dimensions
u!0 = "TIME"
uName = u!0
• & coordinate variable
u&lat = (/ -90., -85, .... , 85., 90. /)
latitude = u&lat
• $ substitute string
x = fin->$variable(n)$ … x = fin->$"T: p"$
 Variable Subscripting (1 of 3)
Standard Array Subscripting (Indexing)
• ranges: start/end and [optional] stride
• index values separated by :
• omitting start/end index implies default begin/end

Consider T(time,lat,lon)
T  entire array [ don't use T(:,:,:) ]
T(0,:,::5)  1st time index, all lat, every 5th lon
T(:3, ::-1, :50)  1st 4 time indices, reverse, 1st 51 lon

T(7:12,45,10:20)  6 time indices, 46th value of lat,

10-20 indices of lon

Good programming: Use variables not hard wired #

T(tstrt:tlast, : , ln1:ln2 )  time index tstrt:tlast, all lat :,
longitude index values ln1:ln2
 Variable Subscripting (2 of 3)

Coordinate Variable Subscripting

• only applies to coordinate variables (1D, mono)
• same rules apply for ranges, strides, defaults
• use curly brackets {…}
• standard and coordinate subs can be mixed
[if no reorder]
T(2:7,{-30:30},:)  six times, all lon, lat -30° to +30°
(inclusive)
T(0,{-20},{-180:35:3})  1st time, lat nearest -20°, every
3rd lon between -180° and 35°
T(::12,{latS:latN},:)  all times/lon, lat latS to latN
(inclusive)
T(8,{latS},{lonL:lonR:3}) 9th time, lat nearest latS, every
3rd lon between latL and lonR
 Variable Subscripting (3 of 3)

Named Dimensions
• only used for dimension reordering
• indicated by |
• dim names must be used for each subscript
• named/coordinate subscripting can be mixed

Consider T(time,lat,lon)
t = T(lat|:, lon|:, time|:)  makes t(lat,lon,time)
t = T(time|:,{lon|90:120},{lat|-20:20})  all times,
90-120° lon, -20-20° lat
 Subscripting: Index, CV
Latitude coordinate variable (1D)

Standard:
T(9:13,1:8)

Coordinate:
T({-10:20},{-170:-110})

Combined:
Longitude coordinate variable (1D) T({-10:20}, 1:8)
 “printing”
• PrintVarSummary
gives gross overview of a variable
• Print
includes same info as printVarSummary but
prints each value

• write_matrix
●
print to standard out or a file
●
format control of numerical output
●
can write to file also
 printVarSummary
• Print overview of variable contents
●
type
●
dimension information
●
coordinate information (if present)
●
attributes (if present)
• printVarSummary (u)
Variable: u
Type: double
Total Size: 1179648 bytes
147456 values
Number of Dimensions: 4
Dimensions / Sizes: [time | 1] x [lev | 18] x [lat | 64] x [lon | 128]
Coordinates:
time: [4046..4046]
lev: [4.809 .. 992.5282]
lat: [-87.86379 .. 87.86379]
lon: [ 0. 0 .. 357.1875]
Number of Attributes: 2
long_name: zonal wind component
units: m/s
 print (1 of 3)
• Prints out all variable information including
●
All meta data, values
●
T(lat,lon): print (T)
Variable: T
Type: float
Total Size: 32768 bytes
8192 values
Number of Dimensions: 2
Dimensions / Sizes: [lat | 64] x [lon | 128]
Coordinates:
lat: [-87.86379 .. 87.86379]
lon: [ 0. 0 .. 357.1875]
Number of Attributes: 2
long_name: Temperature
units: degC
(0,0) -31.7
(0,1) -31.4
(0,2) -32.3
(0,3) -33.4
(0,4) -31.3 etc. [entire T array will be printed]
 print (2 of 3)
• can be used to print a subset of array
●
meta data, values
●
T(lat,lon): print( T(:,103) ) or print( T(:,{110}) )
Variable: T (subsection)
Type: float
Total Size: 256 bytes
64 values
Number of Dimensions: 1
Dimensions / Sizes: [lat | 64]
Coordinates:
lat: [-87.86379 .. 87.86379]
Number of Attributes: 3
long_name: Temperature
units: degC
lon: 109.6875 [ added ]
(0) -40.7
(1) -33.0
• -25.1
• -20.0
(4) -15.3 etc.
 print (3 of 3)
• print with embedded strings
●
no meta data
●
print ( ”T: min="+min(T)+" max="+max(T) )
(0) T: min=-53.8125 max=25.9736

• sprintf and sprinti provide formatting

- often used in graphic labels
- print ( "min(T) = "+ sprintf("%5.2f ", min(T)) )
(0) min(T) = -53.81

• sprinti can left fill with zeros (ex: let n=3)

- fnam = "h" + sprinti ("%0.5i", n) + ".nc"
- print("file name = "+fnam)
(0) file name = h00003.nc
 write_matrix(x[*][*], fmt, opt)
•pretty-print 2D array (table) to standard out
●
integer, float, double
●
user format control (fmt)
●
T(N,M), N=7, M=5: write_matrix (T, “5f7.2”, False)

4.35 4.39 0.27 -3.35 -6.90

4.36 4.66 3.77 -1.66 4.06
9.73 -5.84 0.89 8.46 10.39
4.91 4.59 -3.09 7.55 4.56
17 3.68 5.08 0.14 -5.63
-0.63 -4.12 -2.51 1.76 -1.43
-4.29 0.07 5.85 0.87 8.65

• can also create an ASCII file

opt = True
opt@fout = “foo.ascii” ; file name
write_matrix (T, “5f7.2”, opt)
 Debugging, Error Messages, Help
• NCL does not have a built-in debugger
• use print /printVarSummary ; examine output!
• nmsg = num( ismissing(x) ) ; count # _FillValue
• print(“x: min=“+min(x) +” max=“+max(x) )

• Error messages; Warning or Fatal

• Look at the message; often describe problem/issue
• eg: Fatal: left and right side have different sizes
• printVarSummary of variables before Fatal

Help: question(s): ncl-talk@ucar.edu

- include enough info to facilitate answering
- do *not* attach large files (> 1.5 Mb)
- they will be rejected, use ftp/web
- do *not* 'dump' a messy script to ncl-talk
- Our time is valuable too!
3) File Input / Output
 I/O formats
●
Supported formats [ contain the structure of file]
●
netCDF-3/4 [network Common Data Form]
●
HDF4/H5 [Hierarchical Data Format (Scientific Data Set only) ]
●
HDF-EOS [Earth Observing System; HDF4 and HDF5]
●
GRIB-1/2 [Grid in Binary; WMO standard; NCEP, ECMWF]
●
CCMHT [CCM History Tape; COS blocked only; ccm2nc]
●
Shapefile [ESRI: geospatial vector data format GIS]
●
6.1.0  near complete netCDF4, HDF5

●
Binary
Can be a little tricky.
https://www.ncl.ucar.edu/Applications/r-binary.shtml

●
ASCII
Fairly easy if the file is just a matrix.
https://www.ncl.ucar.edu/Applications/read_ascii.shtml
 ncl_filedump
http://www.ncl.ucar.edu/Document/Tools/ncl_filedump.shtml

• ncl_filedump [-c] [-v var1[,…]] [–h] file_name

− command line utility with options
− provides textual overview of any supported file’s contents
− behavior analogous to Unidata’s ncdump -h
− file_name must have a file type suffix on command line

− .nc .grb .hdf .hdfeos .he5 .h5 .ccm .shp [case insensitive]
− suffix used as identifier only, actual file need not have

• ncl_filedump file_name.[grb/nc/hdf/hdfeos]
- output can be sent to file or viewer via Unix redirection/ pipe
ncl_filedump foo.grb > foo.txt [send to file]
ncl_filedump foo.hdf | less [send to viewer]
 ncl_convert2nc
http://www.ncl.ucar.edu/Document/Tools/
 ncl_convert2nc gribFile(s) OPTIONS
 command line utility
 converts GRIB/HDF/SHAPE file(s) to netCDF
 output name same as input with .nc extension
 ncl_convert2nc –h
 provides usage option information
 ncl_convert2nc foo.grb
 will create foo.nc
 ncl_convert2nc foo.hdf –L –nc4c –cl 1
 -L (files> 2GB); -nc4c (netCDF4); -cl 1 (compression lvl 1)
 setfileoption
www.ncl.ucar.edu/Document/Functions/Built_in/setfileoption.shtml
 allows user to specify file-format-specific options
 netCDF, GRIB and Binary options [currently]
 sample usage of selected options
 writing netCDF
 setfileoption(f, "DefineMode" ,True)
 setfileoption("nc","Format","LargeFile")
 setfileoption("nc","Format",”netCDF4Classic")
 reading GRIB
 setfileoption("grb" ,"ThinnedGridInterpolation", "cubic")
 setfileoption("grb", "InitialTimeCoordinateType" \
, "Numeric")
 reading/writing Binary
 setfileoption("bin", "ReadByteOrder", "LittleEndian")
 setfileoption("bin", “WriteByteOrder", “BigEndian")
 addfile (1 of 3)
• Opens a supported format
• Variables look like netCDF (Grib, HDF, HDF-EOS)
• f_p = addfile (file_name.ext, status )
− file_name => any valid file name; string
− ext => extension that identifies the type of file; string
 netCDF: "nc" or "cdf" [read/write]
 HDF: "hdf" , "hdfeos”, "h5”, "he5" [read/write]
 GRIB: "grb" , "grib" [read only; GRIB1 or GRIB2]
 CCMHT: "ccm" [read only]
 SHAPE (GIS): ”shp" [read only]
 extension not required to be attached to file

− status [read/write status] "r", "c", "w"

− f_p
 reference/pointer to a single file; any valid variable name
 may have attributes (file attributes or global attributes)

http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclFormatSupport.shtml
 addfile (2 of 3)
• Examples: opening a single file
●
fin = addfile ("0005-12.nc" , "r")
●
fout = addfile ("./ncOutput.nc" , "c")
●
fio = addfile ("/tmp/shea/sample.hdf" , "w")
●
g = addfile ("/dss/dsxxx/Y12345.grb", "r" )
●
s = addfile ("foo.shp" , ”r")

• Numerous functions to query contents of supported file

−getfilevarnames
−getfilevardims
diri = "/fs/cgd/data0/shea/GRIB/"
−getfilevaratts fili = ”narr_2000121106”
−getfilevardimsizes fin = addfile(diri+fili+".grb" , " r ")
−getfilevartypes
−isfilevar
varNames = getfilevarnames (fin)
−isfilevaratt
if (isfilevarcoord(fin, "U", "lat") ) then
−isfilevardim
…
−isfilevarcoord end if
Example: query file, system commands
;----------------------------------------------------------------------------
; open file, create array of variable names, # of names
;----------------------------------------------------------------------------

fin = addfile ("./in.nc", "r")

vars = (/"U", "V", "T" /) ; manual specification
nvars = dimsizes (vars) ; nvars = 3
;----------------------------------------------------------------------------
; use system to remove output file before creation
;----------------------------------------------------------------------------
fname = “out.nc”
system("/bin/rm –f ”+fname)
fout = addfile(fname, "c")
;----------------------------------------------------------------------------
; loop, query if variable on the file, then output to netCDF
;----------------------------------------------------------------------------
do n=0,nvars-1
if (isfilevar(fin, vars(n))) then
fout->$vars(n)$ = fin->$vars(n)$
end if
end do
ncrcat/ncks –v U,V,T in.nc out.nc
 Simple netCDF [hdf] Creation
fout = addfile (“foo.nc", "c")
fout@title = "Simple Example”
fout@creation_date = systemfunc(“date”)
; if ‘time’
filedimdef (fout, "time", -1, True) ; create ud

fout->U =u
fout->T = Temp
●
commonly used
●
writes all variable components [data object ;-) ]
●
may be inefficient (possibly, very inefficient)
●
use for file with few variables/records
 Efficient netCDF Creation
• requires ‘a priori’ definition of file contents
− must be done in other languages/tools also [F, C, IDL, ..]
• NCL functions to predefine a netCDF/HDF file:
− setfileoption: specify entering define mode
− filevardef: define name(s) of one or more variables
− filevarattdef: copy attributes from a variable to one
or more file variables
− filedimdef: defines dimensions including unlimited
− fileattdef: copy attributes from a variable to a file
as global attributes

• Necessary when making files with variables too big for

memory.
Example: Efficient netCDF Creation
T = …..
fout = addfile("out.nc" , "c")
setfileoption (fout, "DefineMode",True)) ; enter define mode

; create global attributes

fileAtt = True
fileAtt@creation_date = systemfunc("date")
fileattdef (fout, fileAtt)
; predefine coordinate variables
dimNames = (/ "time", "lat", "lon"/)
dimSizes = (/ -1 , nlat, mlon/) ; -1 means unknown
dimUnlim = (/ True , False, False/)
filedimdef (fout, dimNames, dimSizes, dimUnlim)
; predefine variable names, type, and dimensions
filevardef (fout, "time", typeof(time), getvardims(time))
filevardef (fout, "lat" , typeof(lat) , getvardims(lat) )
filevardef (fout, "lon" , typeof(lon) , getvardims(lon) )
filevardef (fout,”TMP” , typeof(T) , getvardims( T ) )
; create var attributes for each variable
filevarattdef (fout, "TMP", T)

; output data values only [use (/… /) to strip meta data]

fout->time = (/ time /)
fout->lat = (/ lat /)
fout->lon = (/ lon /)
fout->TMP = (/ T /) ; note the different name on file
 Creating Compressed netCDF
• compression (netCDF-4) reduces file size
− climate models create many files; saving more variables
− very high resolution forecast models

• creating compressed netCDF (2 steps)

− setfileoption("nc","Format","NetCDF4Classic")
− setfileoption("nc","CompressionLevel",1)
− 9 levels of compression possible
− compression performance similar to NCO
− cache/chunk size, variable structure, …
− recommendation: use compression level 1
− CPU (read + write) increases significantly with
compression
− File size reduce by up to 90%…
− SST will have significant reduction (land)
− random number will have almost no reduction
 Algebraic Operators
Algebraic expression operators
- Negation ^ Exponentiation
* Multiply / Divide
% Modulus [integers only] # Matrix Multiply
+ Plus - Minus
> Greater than selection < Less than selection
●
Use (…) to circumvent precedence rules
●
All support scalar and array operations [like f90]
●
+ is overloaded operator
●
algebraic operator: 5.3 + 7.95  13.25
●
concatenate string: “pine” + “apple”  “pineapple”
●
algebraic operator and string concatenator
●
"alpha” + ”_" + (5.3 + 7)  "alpha_12.3”
Logical Expressions

Logical expressions formed

by relational operators
.le. (less-than-or-equal)
.lt. (less-than)
.ge. (greater-than-or-equal)
.gt. (greater-than)
.ne. (not-equal)
.eq. (equal)
.and. (and)
.xor. (exclusive-or)
.or. (or)
.not. (not)
 Array Dimension Rank Reduction
●
subtle point: singleton dimensions eliminated
●
let T(12,64,128)
●
Tjan = T(0, :, :)  Tjan(64,128)
●
Tjan automatically becomes 2D: Tjan(64,128)
●
array rank reduced; ‘degenerate’ dimension
●
all applicable meta data copied
●
can override dimension rank reduction
●
Tjan = T(0:0,:,:)  Tjan(1,64,128)
●
TJAN = new( (/1,64,128/), typeof(T), T@_FillValue)
●
TJAN(0,:,:) = T(0,:,:)
 Array Syntax/Operators
●
similar to f90/f95, Matlab, IDL
●
arrays must be same size and shape: conform
●
let A and B be (10,30,64,128) <= conform
●
C = A+B
●
D = A-B
●
E = A*B
●
C, D, E automatically created if they did not previously exist
●
let T and P be (10,30,64,128)
●
theta = T*(1000/P)^0.286  theta(10,30,64,128)
●
let SST be (100,72,144) and SICE = -1.8 (scalar)
●
SST = SST > SICE [f90: where (sst.lt.sice) sst = sice]
●
the operation performed by < and > is (sometimes) called clipping
●
use built-in functions whenever possible
●
Let T be (30,30,64,128) and P be (30) then
●
theta = T*(1000/conform(T,P,1))^0.286
●
all array operations automatically ignore _FillValue
 if blocks (1)

●
if-then-end if (note: end if has space)
if ( all(a.gt.0.) ) then
…statements
end if

●
if-then-else-end if
if ( any(ismissing(a)) ) then
…statements
else
…statements
end if

●
lazy expression evaluation [left-to-right]
if ( any(b.lt.0.) .and. all(a.gt.0.) ) then
…statements
end if
 loops
●
do loop (traditional structure; end do has space)
●
do i=scalar_start_exp, scalar_end_exp [, scalar_skip_exp]
do n = nStrt, nLast [,stride]
… statements
end do ; 'end do' has a space
●
if start > end
●
identifier 'n' is decremented by a positive stride
●
stride must always be present when start>end
●
do while loop
do while (x .gt. 100)
… statements
end do
●
break: loop to abort [f90: exit]
●
continue: proceed to next iteration [f90: cycle]
●
minimize loop usage in any interpreted language
●
use array syntax, built-in functions, procedures
●
use Fortran/C codes when efficient looping is required
 Built-in Functions and Procedures(1 of 2)
●
use whenever possible
●
learn and use utility functions
●
all, any, conform, ind, ind_resolve, dimsizes
●
fspan, ispan, ndtooned, onedtond,
●
mask, ismissing, where
●
system, systemfunc [use local system]
●
functions may require dimension reordering
●
*must* use named dimensions to reorder
; compute zonal and time averages of variable
; T(time,lev,lat,lon)  T(0,1,2,3)
; (zonal average requires rectilinear grid)

; no meta data transfered

Tzon = dim_avg_n( T, 3) ; Tzon(ntim,klev,nlat)
Tavg = dim_avg_n( T, 0) ; Tavg(klev,nlat,mlon)
 dimsizes(x)
• returns the dimension sizes of a variable
• will return 1D array of integers if the array queried
is multi-dimensional.
Variable: dimt
Type: integer
Total Size: 16 bytes
fin = addfile(“in.nc”,”r”) 4 values
t = fin->T Number of dimensions: 1
dimt = dimsizes(t) Dimensions and sizes:(4)
print(dimt) (0) 12
(1) 25
rank = dimsizes(dimt) (2) 116
print ("rank="+rank) (3) 100
(0) rank=4
 ismissing
•
must be used to check for _FillValue attribute
•
if ( x .eq. x@_FillValue ) will not work
●
x = (/ 1,2, -99, 4, -99, -99, 7 /) ; x@_FillValue = -99
●
xmsg = ismissing(x) ; print(xmsg)
●
(/ False, False, True, False, True, True, False /)
•
often used in combination with array functions
•
if (all( ismissing(x) )) then … [else …] end if
•
nFill = num( ismissing(x) )
•
nVal = num( .not. ismissing(x) )
●
if (any( ismissing(xOrig) )) then
●
….
●
else
●
….
●
 where
•
performs array assignments based upon a conditional array
•
function where(conditional_expression \
•
, true_value(s) \
•
, false_value(s) )
•
similar to f90 “where” statement
•
components evaluated separately via array operations

; q is an array; q<0 => q=q+256

; f90: where(q.lt.0) q=q+256
q = where (q.lt.0, q+256, q)
salinity = where (sst.lt.5 .and. ice.gt.icemax \
, salinity*0.9, salinity)
 conform, conform_dims
Array operations require that arrays conform
•
function conform( x, r, ndim )
•
function conform_dims( dims, r, ndim )
•
expand array (r) to match (x) or dimensions sizes (dims)
•
ndim: scalar or array indicating which dimension(s) of x or
dims match the dimensions of r
•
array r is ‘broadcast’ (replicated) to array sizes of x
x(ntim,klev,nlat,mlon), w(nlat) ; x( 0 , 1 , 2 , 3 )
wx = conform (x, w, 2) ; wx(ntim,klev,nlat,mlon)
q = x*wx ; q = x* conform (x, w, 2)
wx = conform_dims ( (/ntim,klev,nlat,mlon/) , w, 2)
wx = conform_dims ( dimsizes(x), w, 2)
 ind
●
ind operates on 1D array only
●
returns indices of elements that evaluate to True
●
generically similar to IDL “where” and Matlab “find” [returns indices]

; let x[*], y[*], z[*] [z@_FillValue]

; create triplet with only ‘good’ values
iGood = ind (.not. ismissing(z) )
xGood = x(iGood)
yGood = y(iGood)
zGood = z(iGood)
; let a[*], return subscripts can be on lhs
ii = ind (a.gt.500 )
a(ii) = 3*a(ii) +2
●
Should check the returned subscript to see if it is missing
●
if (any(ismissing(ii))) then …. end if
 ind, ndtooned, onedtond
●
ind operates on 1D array only
●
if nD … use with ndtooned; reconstruct with onedtond, dimsizes

; let q and x be nD arrays

q1D = ndtooned (q)
x1D = ndtooned (x)
ii = ind(q1D.gt.0. .and. q1D.lt.5)
jj = ind(q1D.gt.25)
kk = ind(q1D.lt. -50)
x1D(ii) = sqrt( q1D(ii) )
x1D(jj) = 72
x1D(kk) = -x1D(kk)*3.14159
x = onedtond(x1D, dimsizes(x))
 date: cd_calendar, cd_inv_calendar
●
Date/time functions:
●
http://www.ncl.ucar.edu/Document/Functions/date.shtml
●
cd_calendar, cd_inv_calendar

●
time = (/ 17522904, 17522928, 17522952/)
●
time@units = “hours since 1-1-1 00:00:0.0”
●
date = cd_calendar(time,-2)
●
date = cd_calendar(time, 0)
●
print(date)
●
print(date)
Variable: date
Variable: date Type: integer
Type: float Total Size: 12 bytes 3 values
Total Size: 72 bytes 18 values Number of Dimensions: 1
Number of Dimensions: 2 Dimensions and sizes: [3]
Dimensions and sizes: [3] x [6] (0) 20000101
(0,0:5) 2000 1 1 0 0 0 (1) 20000102
(1,0:5) 2000 1 2 0 0 0 (2) 20000103
(2,0:5) 2000 1 3 0 0 0
●
TIME = cd_inv_calendar (iyr, imo, iday, ihr, imin, sec \
●
,“hours since 1-1-1 00:00:0.0” ,0)
 system, systemfunc (1 of 2)

●
system passes to the shell a command to perform an action
●
NCL executes the Bourne shell (can be changed)
●
create a directory if it does not exist (Bourne shell syntax)
●
DIR = “/ptmp/shea/SAMPLE”
●
system (“if ! test –d “+DIR+” ; then mkdir “+DIR+” ; fi”)
●
same but force the C-shell (csh) to be used
●
the single quotes (‘) prevent the Bourne shell from interpreting csh syntax
●
system ( “csh –c ‘ if (! –d “+DIR+”) then ; mkdir “+DIR+” ; endif ’
”)
●
execute some local command
●
system (“convert foo.eps foo.png ; /bin/rm foo.eps ”)
●
system (“ncrcat -v T,Q foo*.nc FOO.nc ”)
●
system (“/bin/rm –f “ + file_name)
 system, systemfunc (1 of 2)
• systemfunc returns to NCL information from the system
• NCL executes the Bourne shell (can be changed)

●
UTC = systemfunc(“date”) ; *nix date
●
Date = systemfunc(“date ‘+%a %m%d%y %H%M’ ”) ; single quote
●
fils = systemfunc (“cd /some/directory ; ls foo*nc”) ; multiple cmds
●
city = systemfunc ("cut -c100-108 " + fname)
 User-built Functions and Procedures(1 of 2)
●
two ways to load existing files w functions/proc
●
load "/path/my_script.ncl"
●
use environment variable: NCL_DEFAULT_SCRIPTS_DIR
●
must be loaded prior to use
●
unlike in compiled language
●
avoid function conflict (undef)

undef ("mult") undef ("mult")

function mult(x1,x2,x3,x4) function mult(x1,x2,x3,x4)
begin begin
return ( x1*x2*x3*x4) return ( x1*x2*x3*x4)
end end

load “/some/path/myLIB.ncl“ begin

begin x = mult(4.7, 34, 567, 2)
x = mult(4.7, 34, 567, 2) end
end
User-Built Functions and Procedures(2 of 2)
• Development process similar to Fortran/C/IDL
• General Structure:

undef ("function_name") ; optional

function function_name (declaration_list)
local local_identifier_list ; optional
begin ; required
statements
return (return_value) ; required
end ; required
undef ("procedure_name") ; optional
procedure procedure_name (declaration_list)
local local_identifier_list ; optional
begin ; required
statements
end ; required
 Contents of a well written netCDF variable

• Variables Consider: T(:)

T@long_name = "Temperature"
●
long_name*
T@units = ”degC"
●
units*
T@_FillValue = 1e20
●
_FillValue [if applicable]
T@missing_value = T@_FillValue
●
missing_value [ “ ] T!0 = "time"
●
named dimensions T&time = time
●
coordinate variable(s) Result: T(time)

* Follow COARDS and CF conventions so you

can use NCVIEW or IDV.
* Make up your own attributes for more info!
 addfiles (1 of 2)

• span multiple supported files

• q = addfiles (fNames, "r")
− fNames is a 1D array of file names (strings)
− can be used for any supported format
− technically, "q" is a variable of type list

T = q[:]->T ; [:] read all files

− read T [with meta data] from each file in list ‘q’
− T must exist in each file and be same shape [rank]
− a list is used to sequence results of addfiles
− normal file variable selection is used with "[…]"

lat = q[0]->lat ; [0] read from first file

Z = q[2:6:2]->Z ; extract Z only from files 2,4,6
 addfiles (2 of 2)

• 2 options on variable merging

− ListSetType (a, "cat") [default; "cat" => concatenation]
− ListSetType (a, "join")

• when to use "cat" and "join" [rule of thumb]

− cat: continuous record
− join: creating ensembles
 a record dimension will be added

netCDF Operator (NCO): cat ncrcat join  ncecat

4) Data Analysis
 Algebraic Operators
Algebraic expression operators
- Negation ^ Exponentiation
* Multiply / Divide
% Modulus [integers only] # Matrix Multiply
+ Plus - Minus
> Greater than selection < Less than selection

●
Use (…) to circumvent precedence rules
●
All support scalar and array operations [like f90]
●
+ is overloaded operator
●
algebraic operator: 5.3 + 7.95  13.25
●
concatenate string: “pine” + “apple”  “pineapple”
●
algebraic operator and string concatenator
●
"alpha” + ”_" + (5.3 + 7)  "alpha_12.3”
Logical Expressions

Logical expressions formed

by relational operators
.le. (less-than-or-equal)
.lt. (less-than)
.ge. (greater-than-or-equal)
.gt. (greater-than)
.ne. (not-equal)
.eq. (equal)
.and. (and)
.xor. (exclusive-or)
.or. (or)
.not. (not)
 Array Dimension Rank Reduction
●
subtle point: singleton dimensions eliminated
●
let T(12,64,128)
●
Tjan = T(0, :, :)  Tjan(64,128)
●
Tjan automatically becomes 2D: Tjan(64,128)
●
array rank reduced; ‘degenerate’ dimension
●
all applicable meta data copied
●
can override dimension rank reduction
●
Tjan = T(0:0,:,:)  Tjan(1,64,128)
●
TJAN = new( (/1,64,128/), typeof(T), T@_FillValue)
●
TJAN(0,:,:) = T(0,:,:)
 Array Syntax/Operators
●
similar to f90/f95, Matlab, IDL
●
arrays must be same size and shape: conform
●
let A and B be (10,30,64,128) <= conform
●
C = A+B
●
D = A-B
●
E = A*B
●
C, D, E automatically created if they did not previously exist
●
let T and P be (10,30,64,128)
●
theta = T*(1000/P)^0.286  theta(10,30,64,128)
●
let SST be (100,72,144) and SICE = -1.8 (scalar)
●
SST = SST > SICE [f90: where (sst.lt.sice) sst = sice]
●
the operation performed by < and > is (sometimes) called clipping
●
use built-in functions whenever possible
●
Let T be (30,30,64,128) and P be (30) then
●
theta = T*(1000/conform(T,P,1))^0.286
●
all array operations automatically ignore _FillValue
 if blocks (1)

●
if-then-end if (note: end if has space)
if ( all(a.gt.0.) ) then
…statements
end if

●
if-then-else-end if
if ( any(ismissing(a)) ) then
…statements
else
…statements
end if

●
lazy expression evaluation [left-to-right]
if ( any(b.lt.0.) .and. all(a.gt.0.) ) then
…statements
end if
 loops
●
do loop (traditional structure; end do has space)
●
do i=scalar_start_exp, scalar_end_exp [, scalar_skip_exp]
do n = nStrt, nLast [,stride]
… statements
end do ; 'end do' has a space
●
if start > end
●
identifier 'n' is decremented by a positive stride
●
stride must always be present when start>end
●
do while loop
do while (x .gt. 100)
… statements
end do
●
break: loop to abort [f90: exit]
●
continue: proceed to next iteration [f90: cycle]
●
minimize loop usage in any interpreted language
●
use array syntax, built-in functions, procedures
●
use Fortran/C codes when efficient looping is required
 Built-in Functions and Procedures(1 of 2)
●
use whenever possible
●
learn and use utility functions
●
all, any, conform, ind, ind_resolve, dimsizes
●
fspan, ispan, ndtooned, onedtond,
●
mask, ismissing, where
●
system, systemfunc [use local system]
●
functions may require dimension reordering
●
*must* use named dimensions to reorder

; compute zonal and time averages of variable

; T(time,lev,lat,lon)  T(0,1,2,3)
; (zonal average requires rectilinear grid)

; no meta data transfered

Tzon = dim_avg_n( T, 3) ; Tzon(ntim,klev,nlat)
Tavg = dim_avg_n( T, 0) ; Tavg(klev,nlat,mlon)
 dimsizes(x)
• returns the dimension sizes of a variable
• will return 1D array of integers if the array queried
is multi-dimensional.

fin = addfile(“in.nc”,”r”) Variable: dimt

t = fin->T Type: integer
dimt = dimsizes(t) Total Size: 16 bytes
print(dimt) 4 values
Number of dimensions: 1
rank = dimsizes(dimt) Dimensions and sizes:(4)
print ("rank="+rank) (0) 12
(1) 25
(2) 116
(3) 100
(0) rank=4
 ismissing
●
must be used to check for _FillValue attribute
●
if ( x .eq. x@_FillValue ) will not work
●
x = (/ 1,2, -99, 4, -99, -99, 7 /) ; x@_FillValue = -99
●
xmsg = ismissing(x) ; print(xmsg)
●
(/ False, False, True, False, True, True, False /)
●
often used in combination with array functions
●
if (all( ismissing(x) )) then … [else …] end if
●
nFill = num( ismissing(x) )
●
nVal = num( .not. ismissing(x) )
●
if (any( ismissing(xOrig) )) then
●
….
●
else
●
….
●
end if
 where
•
performs array assignments based upon a conditional array
•
function where(conditional_expression \
•
, true_value(s) \
•
, false_value(s) )
•
similar to f90 “where” statement
•
components evaluated separately via array operations
; q is an array; q<0 => q=q+256
; f90: where(q.lt.0) q=q+256
q = where (q.lt.0, q+256, q)
salinity = where (sst.lt.5 .and. ice.gt.icemax \
, salinity*0.9, salinity)
 conform, conform_dims
Array operations require that arrays conform
•
function conform( x, r, ndim )
•
function conform_dims( dims, r, ndim )
•
expand array (r) to match (x) or dimensions sizes (dims)
•
ndim: scalar or array indicating which dimension(s) of x or
dims match the dimensions of r
•
array r is ‘broadcast’ (replicated) to array sizes of x
x(ntim,klev,nlat,mlon), w(nlat) ; x( 0 , 1 , 2 , 3 )
wx = conform (x, w, 2) ; wx(ntim,klev,nlat,mlon)
q = x*wx ; q = x* conform (x, w, 2)
wx = conform_dims ( (/ntim,klev,nlat,mlon/) , w, 2)
wx = conform_dims ( dimsizes(x), w, 2)
 ind
●
ind operates on 1D array only
●
returns indices of elements that evaluate to True
●
generically similar to IDL “where” and Matlab “find” [returns indices]

; let x[*], y[*], z[*] [z@_FillValue]

; create triplet with only ‘good’ values
iGood = ind (.not. ismissing(z) )
xGood = x(iGood)
yGood = y(iGood)
zGood = z(iGood)
; let a[*], return subscripts can be on lhs
ii = ind (a.gt.500 )
a(ii) = 3*a(ii) +2
●
Should check the returned subscript to see if it is missing
●
if (any(ismissing(ii))) then …. end if
 ind, ndtooned, onedtond
●
ind operates on 1D array only
●
if nD … use with ndtooned; reconstruct with onedtond, dimsizes

; let q and x be nD arrays

q1D = ndtooned (q)
x1D = ndtooned (x)
ii = ind(q1D.gt.0. .and. q1D.lt.5)
jj = ind(q1D.gt.25)
kk = ind(q1D.lt. -50)
x1D(ii) = sqrt( q1D(ii) )
x1D(jj) = 72
x1D(kk) = -x1D(kk)*3.14159
x = onedtond(x1D, dimsizes(x))
 date: cd_calendar, cd_inv_calendar
●
Date/time functions:
●
http://www.ncl.ucar.edu/Document/Functions/date.shtml
●
cd_calendar, cd_inv_calendar

●
time = (/ 17522904, 17522928, 17522952/)
●
time@units = “hours since 1-1-1 00:00:0.0”
●
date = cd_calendar(time,-2)
●
date = cd_calendar(time, 0)
●
print(date)
●
print(date)
Variable: date
Variable: date Type: integer
Type: float Total Size: 12 bytes 3 values
Total Size: 72 bytes 18 values Number of Dimensions: 1
Number of Dimensions: 2 Dimensions and sizes: [3]
Dimensions and sizes: [3] x [6] (0) 20000101
(0,0:5) 2000 1 1 0 0 0 (1) 20000102
(1,0:5) 2000 1 2 0 0 0 (2) 20000103
(2,0:5) 2000 1 3 0 0 0
●
TIME = cd_inv_calendar (iyr, imo, iday, ihr, imin, sec \
●
,“hours since 1-1-1 00:00:0.0” ,0)
 system, systemfunc (1 of 2)

●
system passes to the shell a command to perform an action
●
NCL executes the Bourne shell (can be changed)
●
create a directory if it does not exist (Bourne shell syntax)
●
DIR = “/ptmp/shea/SAMPLE”
●
system (“if ! test –d “+DIR+” ; then mkdir “+DIR+” ; fi”)
●
same but force the C-shell (csh) to be used
●
the single quotes (‘) prevent the Bourne shell from interpreting csh syntax
●
system ( “csh –c ‘ if (! –d “+DIR+”) then ; mkdir “+DIR+” ; endif ’
”)
●
execute some local command
●
system (“convert foo.eps foo.png ; /bin/rm foo.eps ”)
●
system (“ncrcat -v T,Q foo*.nc FOO.nc ”)
●
system (“/bin/rm –f “ + file_name)
 system, systemfunc (1 of 2)
• systemfunc returns to NCL information from the system
• NCL executes the Bourne shell (can be changed)

●
UTC = systemfunc(“date”) ; *nix date
●
Date = systemfunc(“date ‘+%a %m%d%y %H%M’ ”) ; single quote
●
fils = systemfunc (“cd /some/directory ; ls foo*nc”) ; multiple cmds
●
city = systemfunc ("cut -c100-108 " + fname)
 User-built Functions and Procedures(1 of 4)
●
two ways to load existing files w
functions/proc
●
load "/path/my_script.ncl"
●
use environment variable:
NCL_DEFAULT_SCRIPTS_DIR
●
must be loaded prior to use
●
unlike in compiled language
●
avoid function conflict (undef)
undef ("mult")
function mult(x1,x2,x3,x4)undef ("mult")
begin function mult(x1,x2,x3,x4)
return ( x1*x2*x3*x4) begin
end return ( x1*x2*x3*x4)
end
load “/some/path/myLIB.ncl“
begin begin
x = mult(4.7, 34, 567, 2) x = mult(4.7, 34, 567, 2)
end end
User-Built Functions and Procedures(2 of 4)
• Development process similar to Fortran/C/IDL
• General Structure:

undef ("function_name") ; optional

function function_name (declaration_list)
local local_identifier_list ; optional
begin ; required
statements
return (return_value) ; required
end ; required
undef ("procedure_name") ; optional
procedure procedure_name (declaration_list)
local local_identifier_list ; optional
begin ; required
statements
end ; required
5) Graphics
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Introduction to NCL Graphics
 Scatter and line plot
Based on a
visualization
of Joel Norris (Scripps)
using dummy data

Introduction to NCL Graphics

 MJO graphic
Courtesy of Dennis
Shea/CGD
Introduction to NCL Graphics
Introduction to NCL Graphics
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Introduction to NCL Graphics
Contours (line and filled) over a map

Introduction to NCL Graphics

Introduction to NCL Graphics
 Evans plot - Created by Jason Evans of UNSW.
An Evans plot is a way to visualize spatially, two variables of
interest, one of which provides some measure of "importance".
Introduction to NCL Graphics
16 map projections (8 here)

Introduction to NCL Graphics

Introduction to NCL Graphics
Transparency and image overlay– new in V6.1.0!

Imran Nadeem, BOKU Vienna

Import existing
JPEG images
new in V6.1.0!
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Introduction to NCL Graphics
masked lambert
conformal plot
Introduction to NCL Graphics
Courtesy Dave Brown, NCAR/CISL
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Multiple overlays (contours and vectors)

Introduction to NCL Graphics

 Streamlines
over contours

Introduction to NCL Graphics

 CCSM4 data
Six fields overlaid:

Ice thickness
(filled contours)

Sea surface temperature

(filled contours)

Topo map
(filled contours)

Sea level pressur

(line contours)

UV winds

Vertically-integrated
clouds (partially
transparent
filled contours)
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Introduction to NCL Graphics
Filled polygons with text

Introduction to NCL Graphics

Introduction to NCL Graphics
Graphic by Jonathan Vigh, NCAR/ASP
Types of graphics you can create with NCL
• Over 40 plotting templates
• XY
• Contour
• Vector
• Streamline
• Overlays
− Contours over maps, vectors over contours, etc.
• Primitives
− markers, polylines, polygons, text
• Specialized plots
− bar charts, skew-T, wind roses, taylor diagrams
Special Templates and Scripts
Polygons

skew-t.ncl
Wind Rose
Histograms

Skew T
Introduction to NCL Graphics
More Special Templates and Scripts

Tables
Pie Charts

Bar plots

Blank plots

Introduction to NCL Graphics

 WRF plot templates
wrf_contour • wrf_map • wrf_vector •
wrf_map_overlays • wrf_overlays
http://www.mmm.ucar.edu/wrf/OnLineTutorial/Graphics/NCL/N
CL_examples.htm

Introduction to NCL Graphics

 Taylor diagram
Courtesy of
Dennis Shea and
Adam Phillips,
CGD

Introduction to NCL Graphics

The ESRI Shapefile is a popular
geospatial vector data format for GIS
software.
Numerous (and free) shapefiles can
be found by googling on the web.

Introduction to NCL Graphics

 The three types of shapefiles
supported by NCL:
Point – locations of cities,
population data
Line – rivers, roads, trails
Polygon – counties, lakes

Introduction to NCL Graphics

Panel plots –
multiple plots
on a page

Introduction to NCL Graphics

 OLR file
from
Bob
Setzenfand

Introduction to NCL Graphics

From John Ertl, FNMOC

Introduction to NCL Graphics

Courtesy Adam Phillips, NCAR CGD
Introduction to NCL Graphics
John Fasullo, NCAR/CGD

Introduction to NCL Graphics

 ESMF
Regridding

Introduction to NCL Graphics

Five steps to create an NCL graphic

1. Load the “gsn” scripts

2. Open a workstation
3. Define a color map (optional)
4. Set plot options (resources)
5. Call the appropriate plotting
function
 Step 1: Load the “gsn” scripts
• gsn_csm.ncl contains interfaces that look for
“CESM” conventions in your file
• gsn_csm.ncl uses functions in gsn_code.ncl;
load order is important
• gsn_code.ncl contains “generic” interfaces and
supplemental routines

No reason not to load every library in

existence at the begniging of each script.
CESM conventions recognized by gsn_csm scripts

• _FillValue attribute recognized as missing value

(“missing_value” is NOT)
• Data attributes such as “long_name” and
“units” may be used for plot titles
• Coordinate arrays used for axes values
• If data has 1D coordinate arrays and you are
plotting over a map, then “units” attribute of
“degrees_east” or “degrees_north” expected
 “basic” interface: gsn_xxxx “metadata aware” interface: gsn_csm_xxxx

automatic subtitles

contour line labels

tickmarks out &
information label Introductionlat/lon
to NCL Graphics
labels automatic labelbar
 Step 2: Open a workstation
• “Workstation” is where to send the graphics
• Can be:
– PostScript (“ps”) (“eps” - only one image)
– PDF (“pdf”)
– X11 window (“x11” – good for debugging)
– PNG (“png”)
– NCGM (“ncgm”) – rarely used

• Has a default color map associated with it

(which was changed in V6.1.0!)
• A “frame” means a “page”
 Step 2: Open a workstation
• “Workstation” is where to send the graphics
• Can be:
– PostScript (“ps”) (“eps” - only one image)
– PDF (“pdf”)
– X11 window (“x11” – good for debugging)
– PNG (“png”)
– NCGM (“ncgm”) – rarely used

• Has a default color map associated with it

(which was changed in V6.1.0!)
• A “frame” means a “page”
 Pick a different
color table or
create your own

http://www.ncl.ucar.edu/Document/Graphics/color_table_gallery.shtml
http://www.ncl.ucar.edu/Document/Graphics/create_color_table.shtml
Introduction to NCL Graphics
Step 3: Define a color map (optional)
• A default color map (256 colors) is provided
• If you want to change it, do so before
drawing any graphics to a page:
gsn_define_colormap(wks, “rainbow”)
• If you use the same color map a lot, can put it
in the “.hluresfile” file (more later)
• Can use one of the other 90+ color maps, or
create your own.
 Step 4: Set plot options (resources)
• Resources are the heart of your NCL
graphics code. There are over 1,400
resources!
• Use resources to change the look of a plot
• They are grouped by object type.
• There are 11 “graphical” objects: contours,
labelbars, legends, maps, primitives,
streamlines, text strings, tickmarks, titles,
vectors, XY plots
http://www.ncl.ucar.edu/Document/Graphics/Resources/
 Step 5: Select a plotting function

• These functions receive the workstation,

data, and resources.
• There are many different functions for
each type of plot.
• Each function returns a variable of type
graphic which can be plotted or
paneled.
7) Tips and Tricks
Editor Enhancements
https://www.ncl.ucar.edu/Applications/editor.shtml

● Color-coded code is
much easier to read!

● Some enhancements
will auto-complete or
provide documentation
on a function or
procedure.

● My personal favorite is
Kate.
http://kate-editor.org/
 Fortran / C Interface
https://www.ncl.ucar.edu/Document/Tools/WRAPIT.shtml

● Use this to make your own “built-in” functions.

● Can read or process data.

● Best for non-standard computations with lots of looping.

● Things I've used this for...

● Gridding / processing satellite and radar data.

● Computing CAPE and other thermodynamic quantities.

● Computing trajectories.

● Detecting objects or blobs in clouds.

 Making Large or Complicated
Programs
1) Split up your program... Use functions and procedures.

2) Organize your functions and procedures into libraries.

Send them to NCL they might get incorporated into the next
version!

3) Save the output from each step to a NetCDF file.

4) When making a complicated plot split that program up

too...Use functions to return the graphic from each panel or
array from each function.

5) Don't include plotting programs at the end of programs

that take a long time to run. Changing the plot then takes
forever. Save the arrays necessary to make the plot.