Input Output Files Format

Input Output Files Format

• At their lowest level, all files written to your
computer's hard drive are a series of binary bits.

• However, in LabVIEW, four of the most common

techniques for storing data are the ASCII file format,
direct binary storage, TDM file format and LVM file
formats.

• Each of these formats has advantages and some

formats work better for storing certain data types than
others.
 ASCII File Format
Text format files store data using the ASCII character
set. To store data in text format, use the String
functions to convert all data to text strings. Text files
can contain information of different data types stored
as strings. Use text format files when:
 ASCII File Format
• Text files typically take up more memory than binary and
datalog files if the data is not originally in text form, such as
graph or chart data, because the ASCII representation of data
is usually larger than the data itself.

• Additionally, it is difficult to randomly access numeric data in

text files. Although each character in a string takes up exactly
1 byte of space, the space required to express a number as text
typically is not fixed. You might lose precision if you store
numeric data in text files.
 Binary Files
• Use binary files to save numeric data and to access specific
numbers from a file or randomly access numbers from a file.

• Binary files are machine readable only, unlike text files, which
are human readable.

• They are the most compact and fastest format for storing data.

• Binary files are more efficient because they use less disk space
and because you do not need to convert data to and from a text
representation when you store and retrieve data.

• Storing binary data, such as an integer, uses a fixed number of

bytes on disk.
 Binary Files
• A binary file can represent 256 values in 1 byte of
disk space.
• Often, binary files contain a byte-for-byte image of
the data as it was stored in memory, except for cases
like extended and complex numeric values.
• When the file contains a byte-for-byte image of the
data, as it was stored in memory, reading the file is
faster because conversion is not necessary.
• For example, storing any number from 0 to 4 billion
in binary format, such as 1, 1,000, or 1,000,000, takes
up 4 bytes for each number.
 Binary Files
• The following are some of the common functions that interact
with binary files.
Binary Files
 Writing to Binary Files
• Following is an example that writes an array of
doubles to a binary file. Refer to the Arrays
section for more information about the
Prepend array or string size? option.
 Boolean values Representation
• LabVIEW represents Boolean values as 8-bit values
in a binary file.
• A value of all zeroes represents False.
• Any other value represents True.
• This divides files into byte-sized chunks and
simplifies reading and processing files.
• To efficiently store Boolean values, convert a series
of Boolean values into an integer using the Boolean
Array To Number function. Two methods for writing
six Boolean values to a binary file are displayed.
Boolean values Representation
 8-bit integers Representation
• Unsigned 8-bit integers (U8s) directly correspond to
bytes written to the file.
• When you must write values of various types to a
binary file, convert each type into an array of U8s
using the Boolean Array To Number, String to Byte
Array, Split Number, and Type Cast functions. Then,
you can concatenate the various arrays of U8s and
write the resulting array to a file.
• This process is unnecessary when you write a binary
file that contains only one type of data.
8-bit integers Representation
 Strings Representation
• Strings are stored as a series of unsigned 8-bit integers, each of
which is a value in the ASCII Character Code Equivalents
Table.
• This means that there is no difference between writing strings
with the Binary File Functions and writing them with the Text
File Functions.
 Arrays Representation
• Arrays are represented as a sequential list of each of their
elements.
• The actual representation of each element depends upon the
element type.
• When you store an array to a file, you have the option of
preceding the array with a header.
• Following is an example of writing a 2-dimensional array of 8-
bit integers to a file with a header.
• The prepend array or string size? terminal of the Write Binary
File function enables the header.
• Notice that the default value of this terminal is True.
Therefore, headers are added to all binary files by default.
Arrays Representation
 Reading from Binary Files
• When reading a binary file, there are two methods of accessing
data.
• The first is to read each item in order, starting at the beginning
of a file. This is called sequential access and works similar to
reading an ASCII file.
• The second is to access data at an arbitrary point within the
file. For example, if you know that a binary file contains a 1-D
array of 32-bit integers that was written with a header and you
want to access the tenth item in the array, you could calculate
the offset in bytes of that element in the file and then read only
that element. In this example, the element has an offset of 4
(the header) + 10 (the array index) * 4 (the number of bytes in
an I32) = 44. Accessing data in this way is known as random
access.
 Sequential Access
• To sequentially access all of the data in a file, you can call the Get
File Size function and use the result to calculate the number of items
in the file, based upon the size of each item and the layout of the
file.

• You can then wire the number of items to the count terminal of the
Read Binary function.

• Each read operation updates the position within the file so that you
read a new item each time the read function is called.

• When using this technique to access data you can check for the End
of File error after calling Read Binary or calculate the number of
reads necessary to reach the end of the file by using Get File Size.
Sequential Access
 Random Access
• To randomly access a binary file, use the Set Position
VI to set the read offset to the point in the file you
want to begin reading.
• Notice that the offset is in bytes. Therefore, you must
calculate the offset based upon the layout of the file.
In the following example, the VI returns the array
item with the index specified, assuming that the file
was written as a binary array of double precision
numerics with no header.
Random Access
 Datalog Files
• A specific type of binary file, a datalog file is the
easiest method for logging cluster data to file.
• Datalog files store arrays of clusters in a binary
representation.
• Datalog files provide efficient storage and random
access, however, the storage format for datalog files
is complex, and therefore they are difficult to access
in any environment except LabVIEW.
• In order to access the contents of a datalog file, you
must know the contents of the cluster type stored in
the file.
 Datalog Files
• Datalog files store a list of records to a file.
• Each record is represented by a cluster, and can contain
multiple pieces of data of any data type. Datalog VIs allow
you to read and write arrays of clusters to and from datalog
files.
• When you open a datalog file, you must specify the record
type used by the file.
• To do this, wire a cluster of the appropriate type to the
Open/Create/Replace Datalog function.
• After the file is open, you program datalog files like any other
binary file.
• Random access is available, although offsets are specified in
records instead of bytes.
Datalog Files
 Datalog Files
• The following figure depicts an example of
randomly accessing a datalog file by setting
the read position with the Set Datalog Position
function.

• To sequentially access data in a datalog file,

wire the output of the Get Number of Records
function to the count terminal of the Read
Datalog function.
Datalog Files
 TDM Files
• Test Data Exchange Format (TDM) is a hybrid file format that
combines Binary storage and XML formatted ASCII data.
• In a TDM file, the raw numerical data is stored in binary
format. This provides the advantages of binary, such as
efficient space usage and fast write times.
• In addition to this data, an XML format stores the structure of
the data and information about the data.
• This allows the information in the file to be easily accessible
and searchable. Typically, the binary data and XML data are
separated into two files, a .tdm file for the XML data and a
.tdx file for the binary data.
TDM Files
 TDM Files
• The following are benefits of using TDM
files.
 TDM Files
• TDM files each contain a File object and can
contain as many Channel Group and Channel
objects as you want. Each of the objects in a
file has properties associated with it, which
creates three levels of properties that you can
use to store data. For example, test conditions
are stored at the file level, and UUT
information is stored at the channel or channel
group level.
TDM Files
 Methods for Creating TDM Files
• In LabVIEW, you can create TDM files in two
ways.
 Using the Express VIs
• The Write to Measurement File and Read from Measurement
File Express VIs allow you to quickly save and retrieve data
from the TDM format.

• The configuration dialog box for the Write to Measurement

File Express VI is displayed.

• Notice that you can choose to create a LVM or a TDM file

type.

• However, these Express VIs give you little control over your
data grouping and properties and do not allow you to use the
features that make TDM files useful, such as searching for
data based upon conditions.
 Using the Data Storage API VIs
• To gain access to the full capabilities of TDM files,
use the Data Storage API.

• The Data Storage API is a set of VIs that can write

multiple file formats, however, they write TDM files
by default.

• An example of a simple program that logs channel

properties and numeric data to a TDM File using the
Data Storage API is displayed
Using the Data Storage API VIs
 Data Storage API
The Open Data Storage VI opens a reference to a TDM file. You can hard
code a file path by using the configuration dialog or determine the path at
runtime by using the block diagram terminal. The Configure Open Data
Storage dialog box is displayed
 Data Storage API
• The Write Data VI allows you to create channels
and channel groups within your file.
It also allows you to write properties
and data for the item you create.
The configuration dialog for this VI
allows you to select which properties
have block diagram terminals and specify
how the VI behaves if you attempt to store
two channels in a file with the same name.
 Data Storage API
• The Read Data VI allows you to search for channels or channel groups
based upon conditions you specify.
• The configuration dialog for this VI allows you to specify the conditions
for the search, as well as the type of data returned when the search result is
a channel. This VI can return the actual data signal from a channel, but in
order to access other properties of a channel or channel group, pass the
references returned from this VI to the Get Properties VI.
• Notice that because a query can have multiple results, this function returns
all of its results, including refnums, in arrays
 Data Storage API
• The Set Properties VI allows you
to set properties on a channel, channel
group, or file. Because the Write Data
VI allows you to set the properties of a
channel or channel group, this VI is
most often used to set file properties.

• The configuration dialog allows you to

select the type of object to set properties
for, and the properties to set. You can
also use this VI to set the data signal of a
channel.
 Data Storage API
• The Close Data Storage VI closes a reference to a TDM File.
Notice that you only must close the file reference; any
references that you acquire to channels and channel groups
close automatically when you close the file reference.
 Constructing Queries
• Constructing queries helps read data for viewing or analysis.
Perform basic queries using only the Read Data VI. A simple
query that graphs the signal data from all Channels in the file
with a maximum greater than or equal to 5 is displayed.
 Constructing Queries
• To access properties other than the data signal, use the Get Properties VI
with the result references from Read Data. Notice that because Read Data
returns an array, you must use a For Loop to index it before calling Get
Properties. Rather than displaying the signal data, the following example
displays channel properties. The example returns the Name, Description,
Minimum and Maximum of all channels with a Maximum greater than or
equal to 5.
 Constructing Queries
• When your TDM file contains channel groups, you often want to search for
channels only in a particular group. You can do this by using two Read
Data VIs, one to search for the appropriate channel group and a second to
search for channels within that group. Wiring the reference of the channel
group to the Read Data VI allows you to constrain the channel search to
channels within that group .
 Constructing Queries
• To construct complex TDM queries, use the Merge Queries VI. This VI
allows you to combine the results of two Read Data VIs. When you use
Merge Queries, you can return results that are in the first query and the
second query or combine the results of both queries. After calling Merge
Queries, you often call Get Properties to access the signals or properties of
the query results.
 Grouping Data
• Carefully consider the best way to group your data as the data grouping can
have a significant impact on both the execution speed and implementation
complexity of writes and queries. Consider the original format of your data
and how you want to search or view the data when choosing a grouping
scheme. One grouping technique is to group data by the type of data.
 TDMS Files
• TDMS (Technical Data Management - Streaming)
which is are binary files in an internal LabVIEW file
format. The TDMS format gives more effective and
accurate data storage than the LVM format. TDMS
files may be opened in LabVIEW, of course, and in
NI DIAdem which is a software tool for for
managing, analyzing, and reporting data in logfiles.
TDMS files can save data in an organized way using
a number of Groups and a number of Channels within
a given Group. (The older TDM file format is still
supported.)
Writing data to a TDMS file
 Writing data to a TDMS file
• The TDSM Write function writes the data continuously to the file. (It
contains a data buffer so that the While loop is not unnecessarily delayed
by the file writing.)
• The Case structure is used to start, and stop, the file writing. (In the figure
the True case is shown. The False case does not contain any code except
the file path wire from the input tunnel to the output tunnel.)
• Three signals - t, u, y - are collected with a Merge Signals function (which
is on the Express / Signal Manipulation palette) before they are fed into
the the TDMS Write function.
• The channel group name and the channel names are written to the logfile
together with the data. This is convenient for later use of the data, e.g.
when opening the data in the DIAdem tool.
• In this example the points of time generated by the Elapsed Time function
are saved as data together with the values of u and y. (If the data to be
saved are Waveform data the time information is contained in the data, and
then it is not necessary to define the time signal explicitely as in the
example above.)
Reading data from a TDMS file
 Reading data from a TDMS file
• If you run the prevoius VI the TDMS Viewer is also opened, in addition to
the Front panel, see the figure below. The TDMS Viewer is a graph tool
for quickly plotting the channels in the logfile. Note that you have to click
the Quit button in this window to quit the TDMS Viewer, and to stop the
VI.
 Reading data from a TDMS file
• The VI does not contain any While loop since the read from file operation is done
once. (This is the typical situation.)
• The TDMS Read function (on the File I/O palette) reads data from the logfile.
Note that this function has outputs containing the group name and the channel(s)
name(s).
• The data that are read from the file are converted to a ordnary 2-dimensional array
containing the three columns of data (t, u, y). The convertion is made by the
Convert from Dynamic Data function which is on the Express / Signal
Manipulation palette. This conversion function can be configured (i.e. selecting
the correct data type) by double-clicking it, cf. the dialog window shown in the
figure below.
• The three Index Array functions are used to extract each of the columns from the
data and create 1 dimensional arrays, labeled Array t (1D), Array u (1D) and Array
y (1D).
• The Block diagram also contains code for displaying the array in an array indicator
and to display the size of the array on the Front panel.
• The TDMS Viewer function is a graph tool for quickly plotting the channels in the
logfile.
Dialog window of the Convert from Dynamic Data function
 LVM
• LVM (LabVIEW Measurement) which are text files that
contain data that can be read by a human being because
numbers are represented as text. For example, the number
0.231 is stored as the text (or string) "0.231". A large benfit of
storing numerical data in the text format is that the file can be
opened and displayed in any tool that supports text files, e.g.
MS Word, Notepad, Excel, Matlab, Web browsers. (It may be
necessary to change the file name extension from lvm to e.g.
txt or dat before opening the file in such tools). Thus, text files
provides great portability. However, the text files will be larger
than if the data were written in a binary format, and the data
are stored with less accuracy, but these issues are not
important on ordinary PC's, but may be important on dedicated
computers with limited storage.
 LVM
• As a general rule I suggest that you use the
binary file format (TDMS) unless you really
need the text format (LVM).

• Both TDMS (binary) and LVM (text) files

writing and reading are described in the
subsequent sections.
Writing data to a LVM file
 Writing data to a LVM file
• In this example the data are writting to a file with the Write
To Measurement File function, which is an Express VI. (The
Write to Spreadsheet File function could also have been
used.)
• The boolean (switch) Start_stop_file_writing is used to start
(enable) and stop the continuous file writing. This boolean
terminal (control) is connected to the Enable input to the
Write To Measurement File function.
• Two signals, u and y, are collected with a Merge Signals
function (which is on the Express / Signal Manipulation
palette) before they are fed into the the Write To
Measurement File function. Regarding writing the time
signal, cf. the comments about the X Value Columns in the
list of comments below.
 Writing data to a LVM file
• By double-clicking the Write To Measurement File function in the block
diagram a dialog window opens, see the figure below.
 Writing data to a LVM file
• File Name box: Here you can select the file name, but if you have wired a file name control (or
constant) to the File Name input to this Express VI, the wired name is used instead (this is the
case in the example above).
• File Format radio buttons: Here you select the data format: LVM or TDMS. (The TDM file format
also available is a simpler and not so flexible data format. TDMS is a newer format.) It seems it is
not possible to change the LVM file name extension into e.g. TXT (but you can of course do this
change manually using Windows Explorer).
• Segment Headers: By selecting No headers the files will contain just plain data, which is
convenient if you plan to import your data into tools as Excel, Matlab, etc. If you choose to have
header(s), you will have to remove the header (perhaps manually) from the data file before you
can load the file into tools mentioned above.
• X Value Columns: The X values are actually the time stamps. By selecting One column only the
file will contain one left column containing the time stamps. These time stamps are recorded and
written to the file automatically, so you do not have to create any time signal yourself.
• Delimiter: The delimiter is the character that separates columns.
• File Description: The text you write here will appear in the header of the file, unless you have
selected the No headers options mentioned above.
• Note: There is no way to define the group name or the channels names for the data in the Write
to Measurement File Express VI.
• You may wonder why I did not use the Write To Measurement File function for writing the data in
TDMS format (cf. the previous section). In stead I used the TDMS Write function. The reason is
that the channel group name and the channel names can not be defined in the Write To
Measurement File function, but if this is not an issue, you may of course use the Write To
Measurement File function for writing TDMS data.
 Writing data to a LVM file
• In our example, the result of running previous VI is a text file named
logfile1.lvm. The figure below shows the file opened in Notepad from one
experiment. (The cycle time of the While loop was set to 0.1 s, cf.the block
diagram shown above. As you see, the actual cycle time varies a little, but
it is very close to the specified cycle time of 0.1 s.)

logfile1.lvm file opened in Notepad

Reading data from a LVM file
• The VI does not contain any While loop since the read from file operation is done
once. (This is the typical situation.)

• The Read From Measurement File function reads data from the logfile. (By
double-clicking the function a dialog window is opened. The parameters in this
window should be self-explanatory.

• The data that are read from the file are converted to a ordnary 2-dimensional array
containing the three columns of data, cf. the logfile1.lvm shown above. The
conversion is made by the Convert from Dynamic Data function which is on the
Express / Signal Manipulation palette. Note that you can configure the Convert
from Dynamic Data function (i.e. selecting the correct data type) by double-
clicking it.

• The three Index Array functions are used to extract each of the columns from the
data and create 1 dimensional arrays, labeled Array t (1D), Array u (1D) and Array
y (1D).

• The Signal indicator is a Graph indicator which is created by right-clicking on the

signal line out from the Expresss VI. The x-axis shows the time-index (integer).
You can configure the Graph via its Property dialog window so that the x-axis
shows time in secons (or minutes etc.).

• The Block diagram also contains code for displaying the array in an array indicator
and to display the size of the array on the Front panel.