Professional Documents
Culture Documents
NI TestStand 2 Framework Development Course Manual (National Instruments
NI TestStand 2 Framework Development Course Manual (National Instruments
Student Guide
A. NI Certification .....................................................................................................vii
B. Course Description ...............................................................................................vii
C. What You Need to Get Started .............................................................................viii
D. Installing the Course Software..............................................................................viii
E. TestStand Directories............................................................................................ix
F. Course Goals.........................................................................................................ix
Lesson 1
Test Frameworks
A. Purpose of the Test Framework ............................................................................1-2
B. Components of a Framework................................................................................1-3
C. Framework Requirements.....................................................................................1-4
Lesson 2
TestStand API
A. Introduction to the TestStand API ........................................................................2-2
B. TestStand API Organization .................................................................................2-3
C. Calling the TestStand API ....................................................................................2-13
D. Common TestStand API Uses ..............................................................................2-20
Lesson 3
Custom Steps
A. Custom Step Types ...............................................................................................3-2
B. Step Templates......................................................................................................3-17
Lesson 4
Process Models
A. Process Model Structure .......................................................................................4-2
B. Customizing a Process Model...............................................................................4-5
C. Common Process Model Modifications ...............................................................4-6
D. Creating Custom Result Processing Plug-ins .......................................................4-16
Lesson 5
User Interfaces
A. Available User Interfaces......................................................................................5-2
B. TestStand User Interface (UI) Controls ................................................................5-4
C. User Interface Messages (UIMessages)................................................................5-10
D. Front-End Callbacks .............................................................................................5-12
© National Instruments | v
Contents
Lesson 6
Design Considerations
A. Modularity ............................................................................................................6-2
B. Choosing Where To Implement Functions ...........................................................6-2
C. Data Management .................................................................................................6-5
D. Error Handling ......................................................................................................6-6
E. XML Style Sheets .................................................................................................6-8
F. Framework Deployment .......................................................................................6-11
Appendix A
Customizing Database Interaction
A. Relational Databases.............................................................................................A-2
B. Structured Query Language ..................................................................................A-3
C. Customizing Database Interaction ........................................................................A-4
D. Modifying Schemas ..............................................................................................A-4
E. Database Step Types .............................................................................................A-9
Appendix B
Additional Information and Resources
vi | ni.com
Student Guide
Thank you for purchasing the NI TestStand 2: Framework Development course kit. This course
manual and the accompanying software are used in the two-day, hands-on NI TestStand 2:
Framework Development course.
You can apply the full purchase of this course kit toward the corresponding course registration fee
if you register within 90 days of purchasing the kit. Visit ni.com/training for online course
schedules, syllabi, training centers, and class registration.
Note For course manual updates and corrections, refer to ni.com/info and enter the
Info Code rdtce2.
A. NI Certification
The TestStand 2: Framework Development course is part of a series of courses designed to build
your proficiency with NI TestStand and help you prepare for exams to become an NI Certified
TestStand Developer and NI Certified TestStand Architect. Refer to ni.com/training for more
information about NI Certification.
B. Course Description
This course covers advanced topics in TestStand, as well as how to customize TestStand to further
meet your needs. You will learn some system design scenarios that are aimed to give you some
direction and forethought before you start designing your test system. This course assumes that you
have completed the NI TestStand 1: Test Development course or have a good working knowledge
of TestStand programming fundamentals. A background knowledge of test executive software and
a familiarity with either LabVIEW or C programming is also recommended.
The course is divided into lessons that teach a topic or set of topics. Each lesson consists of the
following:
• An introduction that describes the lesson objectives and the topics discussed.
• A discussion of the topics.
• A set of hands-on exercises that reinforce the concepts presented in the discussion. Some
of the exercises are duplicated for LabVIEW and LabWindows™/CVI™ programming
environments. For duplicate exercises, select the exercise that uses your preferred
programming environment.
• A summary that highlights the concepts covered in the lesson.
• A quiz that allows you to test your mastery of the concepts covered in the lesson.
Note You can adapt the information in the lessons to more closely integrate with your
testing requirements.
Note Folder names in angle brackets, such as <Exercises>, refer to folders on the
root directory of your computer.
viii | ni.com
TestStand 2 Course Manual
E. TestStand Directories
TestStand installs three directories to store all necessary components and support files. The
TestStand documentation refers to these directories in the following ways:
• <TestStand>—Located by default at C:\Program Files\National Instruments\
TestStand on Windows 32-bit systems and at C:\Program Files (x86)\National
Instruments\TestStand on Windows 64-bit systems.
• <TestStand Public>—Located by default at C:\Users\Public\Documents\
National Instruments\TestStand on Windows 7/Vista and at C:\Documents and
Settings\All Users\Documents\National Instruments\TestStand on
Windows XP.
• <TestStand Application Data>—Hidden by default and located at C:\ProgramData\
National Instruments\TestStand on Windows 7/Vista and at C:\Documents and
Settings\All Users\Application Data\National Instruments\TestStand on
Windows XP.
• <TestStand Local Application Data>—Hidden by default and located at <User
Directory>\AppData\Local\National Instruments\TestStand on Windows 7/
Vista and at <User Directory>\Local Settings\Application Data\National
Instruments\TestStand on Windows XP.
F. Course Goals
This TestStand course teaches you the more advanced features and customizations of the TestStand
environment. The course first discusses the TestStand Application Programming Interface (API),
and then more advanced features such as multithreading and multi-UUT test systems. The course
also explains many of the customizable components within the TestStand environment. The course
concludes by presenting ideas on possible system designs and implementations.
© National Instruments | ix
Test Frameworks
1
In this lesson you learn the purpose, components, and requirements of a test framework.
Topics
A. Purpose of the Test Framework
B. Components of a Framework
C. Framework Requirements
The process model is the main component of the framework and provides the code that controls
how the test system executes tests and handles communication with the rest of the framework. The
framework also includes user interfaces, which users use to interact with the framework and with
test programs. Some frameworks might contain additional applications, such as user management
tools, automatic code generation programs, and/or data or limit management tools. The framework
is responsible for collecting and storing test data. In this capacity, the framework is the link
between the operator and the eventual consumer of the data.
TestStand includes a fully functional test framework. However, many test systems need customized
frameworks to meet their requirements. A framework developer customizes the framework and
then distributes it to test developers. After the test developers use the framework to construct test
programs, the test or framework developer creates a test system distribution that includes the test
framework and specific test programs for one or more UUTs. The developer then deploys this
distribution to each test station.
1-2 | ni.com
TestStand 2 Course Manual
B. Components of a Framework
A test framework may include the following components:
• Process Model—Controls the way tests execute, including locating the test program,
executing the test program, providing prompts for each UUT, and looping through multiple
UUTs. The process model also handles test data, such as collecting results, generating reports,
and logging data to a database. The process model also handles communication among the
TestStand Engine, user interfaces, and test programs.
• User Interfaces—Provide way for various users to interact with the test framework and test
programs. A TestStand User Interface is an application you deploy to a development system or
a production station to provide a custom GUI for executing, debugging, or editing sequences.
Simple user interfaces might only support running sequences, and custom sequence editors
might support editing, running, and debugging sequences. User interfaces can be customized
to any extent, but they generally fall into one of two categories: Operator Mode and Editor
Mode. With the user interfaces in Operator Mode, you can start multiple concurrent executions,
set breakpoints, and single-step through sequences. With the user interfaces in Editor Mode,
you can modify sequences and display sequence variables, sequence parameters, step
properties, and so on.
• Utility Tools—Frameworks can include utility tools to assist in managing the framework or
developing test programs. One example is a data management tool, such as a tool that updates
the results of previous runs in the database. Another example is a limit or setting management
tool, such as a tool that imports test limits from an external source. You can also use user
management utilities to interactively modify users or import a user list from another source.
Frameworks can also include utility tools that automatically generate sequences or other
TestStand components to serve as starting points for test developers.
• Test Building Blocks—Although test frameworks are not stand-alone components, many test
frameworks include developer resources designed to assist the test developer or give them
access to special features of the framework. Developer resources can include custom step
types, custom data types, and templates.
C. Framework Requirements
Framework requirements are separate from test program requirements. Framework requirements
include things the test system must do in addition to running the test sequence. Examples of
framework requirements include the following:
• User interface requirements, such as custom displays, buttons, or menus.
• UUT tracking requirements, such as looping or parallel execution.
• Result delivery requirements, such as custom data logging or report layouts.
• Error handling requirements, such as the ability to collect errors to a log file.
Describe, implement, test, and track framework requirements just as you do test program
requirements. Developing a test framework to meet a set of framework requirements is similar to
developing a software application to meet a set of software requirements.
1-4 | ni.com
TestStand 2 Course Manual
Self-Review: Quiz
1. Which of the following should you perform in the test framework? (Multiple answers)
a. Database logging
b. Test-specific data analysis
c. Display execution progress
d. Test-specific communication
Notes
1-8 | ni.com
TestStand API
2
In this lesson you learn about the TestStand application programming interface (API). Use the
TestStand API to read and write data from TestStand properties, sequences, and steps. You also use
the TestStand API to create, edit, run, and debug sequences. You can access the TestStand API from
any programming environment that supports access to ActiveX servers.
Topics
A. Introduction to the TestStand API
B. TestStand API Organization
C. Calling the TestStand API
D. Common TestStand API Uses
Use the TestStand application programming interface (API) to interact directly with the TestStand
Engine. Through the API, you can manipulate or use the various components that the engine
manages, as shown in Figure 2-1.
Figure 2-1. TestStand API Connecting Framework Components and TestStand Engine
The TestStand API uses an object-oriented architecture. An object represents a group of related
items. Each object can include properties (data items) and methods (functions). You can perform
API operations by setting or getting properties of objects or by calling methods on objects.
You can access the TestStand API through the TestStand ActiveX/COM server registered with the
operating system. Because the server is registered with the operating system, the operating system
can provide information about the TestStand API, such as names, properties, and methods of each
class object.
2-2 | ni.com
TestStand 2 Course Manual
Another useful resource is the NI TestStand API Reference Poster, located in the <TestStand>\
Doc\Manuals directory. Use the poster as a quick reference for most of the available classes,
properties, and methods in the TestStand API.
Objects
An object is a specific component or data item within TestStand. Each object has properties and
methods. Properties represent pieces of data for an object. Each object has a value for each of its
properties. Methods are actions that can be performed on objects. Methods execute blocks of code
much like functions, subVIs, or subsequences.
Objects are key to understanding object-oriented technology. Consider real-world objects such as
dogs, desks, and bicycles. These objects have two characteristics:
• States (properties)
• Behavior (methods)
As a real-world example, a dog named Rayne is an object. Rayne has specific states or properties,
such as blue and brown eyes and a height of 18 inches. Rayne also has behaviors or methods that
he can act on, such as the ability to sit or lay down.
Figure 2-2 illustrates the object Rayne and all its states and behaviors. By abstracting these states
(color, eye color, height, length, width) and behaviors (sit, lay down, shake, come), we can create
a class, Dog, to encompass properties and methods of all dogs. Refer to the Classes section for
more information about classes.
As a TestStand example, a class called Sequence, as listed in Table 2-1, has properties (Name,
Type, etc.) and methods (InsertStep, DeleteStep, etc.). A specific instance or object of the
Sequence class is an object with the unique Name value, MainSequence. Because the
MainSequence object is a member of the Sequence class, it has the method InsertStep available
to it. You can call this method to programmatically insert a new step into the sequence.
Classes
Classes define groups of objects. The class is a description of a type of object, similar to a blueprint
for the object. The class also defines the properties and methods each object has. Properties
represent data about the state of the object. Methods are actions that can be performed on objects.
The class itself does not have values for each of the properties, but the class might have default
values.
The TestStand API includes general classes and classes for common TestStand components, such
as sequences, executions, the user manager, and workspaces. The following are examples of
classes in TestStand.
• Sequence
• SequenceFile
• Step
• PropertyObject
2-4 | ni.com
TestStand 2 Course Manual
Class Description
Engine Main class, represents the engine
Execution Represents an execution
Module Represents the configuration of a code module
PropertyObject Parent class for most other classes
PropertyObjectFile Represents any file
Report Represents a report
SequenceFile Represents a sequence file
Sequence Represents a sequence
SequenceContext Contains the current state of a thread
Step Represents a step
StepType Represents a step type
Thread Represents a thread in an execution
User Represents a user
UserFile Represents the file in which users are stored (users.ini)
Workspace File Represents a workspace file
Workspace Object Represents a node in the workspace tree
Of these classes, Engine is the only class for which you can directly create an object. The Engine
class has methods for creating instances of many of the other classes.
Figure 2-3 is an example of the type of objects based on the SequenceContext class. A
SequenceContext object contains complete information about an execution at a particular point
during the execution. You can use the sequence context to access all the objects, variables, and
properties in the execution. From the sequence context, you also can obtain references to all the
steps in the current sequence, the sequence contexts for the calling sequences, the Process Model
entry point sequence, and the MainSequence in the client sequence file. You can pass the current
sequence context or subproperties to code modules you call from steps.
Refer to the TestStand API Reference book of the NI TestStand Help for a detailed description and
a list of the properties and methods for each class.
Inheritance
Inheritance is the relation between classes in which one class is the parent class and the other class
is a child class. A parent class is a more general version of a child class. For example, the
PropertyObjectFile class is a more generic class for referring to a file than the
2-6 | ni.com
TestStand 2 Course Manual
SequenceFile class because the ProjectObjectFile class can handle multiple file types, but
the SequenceFile child class can handle only sequence file types.
A parent class defines properties and methods that can be used by any object of the parent class or
by any object of a child class of that parent class. For example, the PropertyObjectFile class
defines the Lock method that the PropertyObjectFile, SequenceFile, and other child class
objects of PropertyObjectFile can use.
Parent classes can also have parents. You can convert a child class to any class from which it
inherits properties and methods. You can use methods, such as the AsPropertyObject method,
of the child class to get a reference to the parent class.
The PropertyObject class is a parent or grandparent class for almost every class in the
TestStand API, as shown in Figure 2-4. You can convert any class, except for the Engine class to
a PropertyObject class. Refer to PropertyObject Class section for more information about the
PropertyObject class.
Figure 2-4. Relationship Between Parent and Child Classes
It is important to understand that as you move up the inheritance hierarchy, methods and properties
become increasingly generic. As you navigate deeper into a specific class, the methods and
properties become more specific. Figure 2-5 shows an example of the inheritance hierarchy for the
SequenceFile class.
Figure 2-5. Inheritance Hierarchy for the SequenceFile Class
Each sequence file object is an instance of the SequenceFile class. If you have a reference to an
open SequenceFile object, you can access properties, such as the NumSequences property, and
call methods, such as the Save method to save a sequence file.
Although it is good practice to increment the change count when you make changes to a file to
indicate to the sequence editor or user interface that the file was modified, you cannot complete
this task with the SequenceFile reference because the SequenceFile class does not include a
method to increment the change count. However, SequenceFile is a child class of the
PropertyObjectFile class, which includes an IncChangeCount method. The
PropertyObjectFile class includes the IncChangeCount method because the method is not specific
to a file type. Therefore all PropertyObjectFile objects can use the method. To use the
IncChangeCount method, you need to use the SequenceFile.AsPropertyObjectFile method to get a
reference to the parent class. You can now access the IncChangeCount method from the returned
PropertyObjectFile reference.
The PropertyObjectFile class is a child class of the PropertyObject class. You can convert a
PropertyObjectFile class or a SequenceFile class directly to a PropertyObject class object to use
additional methods, such as the Clone method.
2-8 | ni.com
TestStand 2 Course Manual
Containment
In the TestStand API many objects contain other types of objects. For example, a SequenceFile
object contains Sequence objects, and Sequence objects contain Step objects. Containment is
different than inheritance. The Sequence class is not a child class of the SequenceFile class,
but the SequenceFile class does contain one or more sequences, as shown in Figure 2-6.
Figure 2-6. Examples of Objects that Contain Other Object Types
You can sometimes access contained classes through properties or methods. For example, you can
get a reference to a sequence contained in a SequenceFile object using the GetSequence or
GetSequenceByName methods. With the returned reference, you can now access properties and
methods in the Sequence class.
Refer to the API Containment topic in the NI TestStand Help for a diagram that shows the most
important containment relationships in the TestStand API.
References
When working with objects in TestStand, you must have a way to access them. To access a
particular object in memory, use a reference, which is a data item or variable that refers to a specific
object. When you store the location of a specific object in a reference, you can use the reference to
invoke methods and to get or set properties of the object.
You must acquire a reference to an object from somewhere. You can get a reference to an object,
such as the TestStand Engine, by creating the reference. Other object references are returned by
properties or methods.
For example, the GetStep method can return a reference to a Step object. First, you define a local
variable with an Object Reference data type. You can now store a reference to any object in the
variable. At run time, you acquire a reference to a specific Step object by calling the GetStep
method of a Sequence object. You can store this reference in the variable. Now that the reference
refers to a specific object in memory, you can call properties and methods of the Step class on the
object or you can pass the variable as an argument to other properties or methods that expect a
reference to a Step object.
PropertyObject Class
A PropertyObject class represents an item in TestStand that holds data. This definition is
generic enough that nearly all classes in the API inherit from PropertyObject. The only
significant exception is the Engine class. You can use the PropertyObject class to write very
general programs. The PropertyObject class also has generic properties and methods, such as
the Clone method, that you can use to create a copy of any object in memory.
A number of objects in TestStand do not use any specific class and are represented directly as
PropertyObjects. An example of objects that do not have a specific child class are variables.
TestStand represents all parameters and variables, such as containers and arrays, as
PropertyObjects in the TestStand API.
PropertyObjects are also the only way to create and access dynamic and custom properties.
Because these properties do not always exist or are not a part of the original TestStand data layout,
it is impossible to have specific properties or methods to access them. For example, if you add a
custom data item to a step by using a step type or a TestStand API call, there is no way that the Step
class can know of the data item and have a predefined property to access the data item through a
Step reference. Instead, you must convert the Step reference to a PropertyObject reference and
use the generic methods from the PropertyObject class to access the data item you created.
2-10 | ni.com
TestStand 2 Course Manual
In this case, there are two station global variables: MyContainer and NumericGlobal. The
MyContainer variable is a container with two fields. The NumericGlobal variable is a station
global with a numeric data type. The StationGlobals container, the two global variables, and the
two fields within MyContainer are represented as PropertyObjects in the TestStand API. The
StationGlobals container contains the two variables as subproperties. The MyContainer
PropertyObject contains two fields as subproperties.
Lookup Strings
Use lookup strings to specify the dynamic property on which a method operates. You pass lookup
string that define a complete path from the object on which you call the method to the specific
property you want to access.
Without lookup strings you must access an object through a containment hierarchy, by starting
from a top-level object, acquiring a reference to the next level of object that you want, and
proceeding until you reach the object you want to access. For example, if you are trying to access
the first step in the Main step group of the first sequence in a sequence file, you could acquire the
SequenceFile reference from the engine, the Sequence reference from a method, and the Step
reference from another method. This approach requires multiple steps and management of multiple
references.
Alternately, you can access the step directly by treating all of the objects as PropertyObjects
and using a lookup string to locate the object you want. A lookup string describes how to traverse
a containment hierarchy by describing the location of a contained item in reference to one of the
items that contains that item.
In cases where there is an array of subproperties, such as the Sequences in a SequenceFile object
or the Steps in a Sequence, the index of the PropertyObject is enclosed in square brackets. You
can also use brackets to enclose a string with the PropertyObject name rather than using the
index. An object referencing itself uses an empty string for the lookup string.
If you are looking for a particular step in a sequence file, you can convert the SequenceFile
reference to a PropertyObject reference and then call a single method: GetPropertyObject.
The GetPropertyObject method has a lookupString parameter that accepts a lookup string.
Passing "Data.Seq[\"MainSequence\"].Main[0]" as the lookup string gives you a
reference to the first step in the Main step group of MainSequence.
2-12 | ni.com
TestStand 2 Course Manual
To view a similar solution for this example that uses a FileGlobals object instead of a
StationGlobals object, open <Exercises>\TestStand 2\Demonstrations\
API_Lookup_String.seq.
The <TestStand>\API directory includes libraries, headers, VIs, and other files required to
interface with the API from each supported programming language.
The Engine class is the only class in the core TestStand API for which you can create an object
directly. Creating an Engine object means you can create an object in memory and attain a
reference to it. Using methods and properties of the Engine object, you can create or acquire
references to any other object in the TestStand API.
Use the following procedure to determine the number of sequences in a sequence file:
1. Use an ActiveX step to call the Engine.GetSequenceFileEx method, as show in
Figure 2-10. Create a new reference to the engine and store it in a local variable. Store the
reference to the sequence file in another local variable.
Figure 2-10. Using ActiveX to Call the TestStand API from TestStand
Normally you do not have to release API references when calling the API from TestStand.
However, when you use the GetSequenceFileEx method, you must explicitly call the
ReleaseSequenceFileEx method in addition to any normal reference cleanup you perform.
2-14 | ni.com
TestStand 2 Course Manual
Some TestStand API resources require a specific close method to clean up resources and are not
automatically released. Resources that require a specific close method always list the close method
in the description of the open method in the TestStand API Reference book of the NI TestStand
Help. For example, the Engine.GetSequenceFileEx method requires you to call the
Engine.ReleaseSequenceFileEx method to release the SequenceFile object.
Note TestStand maintains a set of references to open API objects, which prevents you
from having to create an Engine object and call methods or properties to get the object
you need.
When you call code modules, you can pass an existing, stored API reference as a parameter. You
do not need to close the reference from within the code module, but you must close any additional
references the code module used properties or methods to acquire.
TestStand maintains open references to a number of API objects in the RunState property. For
example, you do not need to create an Engine object when you call the TestStand API from
TestStand or from a code module TestStand calls because the RunState.Engine property
contains an open reference to the TestStand Engine. In many cases, you can bypass the engine
entirely and work directly with other open API objects, such as RunState.Sequence, which
maintains a reference to the current sequence.
RunState properties are context-sensitive, meaning that a property refers to different API objects
depending on where you use the property. The properties always refer to the current object, so if
you use a RunState property in the arguments or expressions for a step, the RunState.Step
property refers to the step you are currently editing, the RunState.Sequence property refers to
the sequence that contains the step, and the RunState.SequenceFile property refers to the
sequence file that contains the step.
You can reduce the complexity of the previous example by using a RunState property. Replace
the engine calls to get and retrieve the sequence by using the RunState.SequenceFile
reference instead to access the NumSequences property, as shown in Figure 2-11.
Using the RunState.SequenceFile reference does not have the flexibility of using the engine
calls, which can return the number of sequences in any sequence file. The context-sensitive nature
of the RunState reference also means that this example can return the number of sequences only
in the current sequence file.
Calling the TestStand API from an expression is advantageous because this technique does not
require an extra step and requires little configuration. However, complex API calls that involve
traversing multiple objects can be difficult to understand. For this reason, use expressions for
simple API calls that involve only one or two objects. To improve readability, use steps to perform
complex or nested calls.
2-16 | ni.com
TestStand 2 Course Manual
Figure 2-12 shows an example of using expressions to call the TestStand API. To view the solution
for this example, open <Exercises>\TestStand 2\Demonstrations\
API_Expressions.seq.
In addition to the user interface VIs, the palette contains the Get Property Value VI, shown in
Figure 2-14 and the Set Property Value VI to read or write the value of any property object using
its lookup string. To use the Get Property Value and Set Property Value VIs, you must obtain a
reference to a TestStand SequenceContext. You generally use these VIs in a code module called
from TestStand and pass a reference to the current SequenceContext as a parameter to the code
module.
Figure 2-14. TestStand - Get Property Value VI
2-18 | ni.com
TestStand 2 Course Manual
2-20 | ni.com
TestStand 2 Course Manual
Self-Review: Quiz
1. A sequence file stores references to sequences in the file. This is an example of:
a. Inheritance
b. Containment
c. Child classes
d. RunState references
2. What object is the parent class for almost all other TestStand objects?
a. SequenceFiles
b. PropertyObject
3. You must release references to any RunState API references you create in TEstStand and pass
to a code module.
a. True
b. False
4. From a sequence running in the TestStand Sequence Editor, how can you call the TestStand
API? (multiple answers)
a. ActiveX Adapter
b. .NET Adapter
c. Expression statement
d. You cannot call the TestStand API from the sequence editor
2. What object is the parent class for almost all other TestStand objects?
a. SequenceFiles
b. PropertyObject
3. You must release references to any RunState API references you create in TEstStand and pass
to a code module.
a. True
b. False
4. From a sequence running in the TestStand Sequence Editor, how can you call the TestStand
API? (multiple answers)
a. ActiveX Adapter
b. .NET Adapter
c. Expression statement
d. You cannot call the TestStand API from the sequence editor
Notes
2-24 | ni.com
Custom Steps
3
In this lesson you will learn how to create custom steps and learn about the differences between
custom step types and step templates.
Topics
A. Custom Step Types
B. Step Templates
Custom step types can define custom configuration dialog boxes in which test developers can
configure the settings of the step. You can modify the step description, menu location, icon, and
other appearance settings for a custom step type.
Custom step types can include predefined settings for each of the standard step properties. You can
disable built-in properties so test developers cannot modify the properties. You can also define
code templates for a custom step type, so test developers can use a standard starting point when
creating code modules.
3-2 | ni.com
TestStand 2 Course Manual
Tip When you create new step types, use unique names to avoid conflicts with the
default names TestStand uses. For example, begin new custom step type names with a
unique ID, such as a company prefix.
4. Right-click the new step and select Properties from the context menu to launch the Step Type
Properties dialog box.
5. Click the Menu tab and specify the menu item name for the new step.
6. Click the General tab and specify the default name for new steps you create from the new type
and specify the description expression for those steps.
7. Click the Substeps tab and select an adapter.
8. Click Add and select the type of step to create substeps. Use the Substep Info section on the
Substeps tab to specify the menu item name of Edit steps.
Note Step Settings panes are not available for custom step types you create. You must
use dialog boxes to set custom elements of the step types you create.
You can use an existing step type as a starting point for a custom step type by copying a step type
from one of the built-in type palette files and pasting the step type in your custom type palette file.
You must rename the step type before you edit it to avoid conflicts with the original step type.
If you want to create a new type based on an existing type, copy and paste the type within the type
palette that already references the type so that TestStand pastes a uniquely named copy of the type.
Then cut the uniquely named copy of the type and paste it in your custom type palette. Rename the
type in your custom type palette and customize the type.
Reserve the Result container for important results of the step that you always intend to include in
the report or database. Add properties to the other containers for internal data storage. For example,
you can create properties outside the Result container to store the following data:
• Temporary data items to store data from one substep to the next
• Configuration settings to be set by an Edit substep
• Minor results of the step that you might not want to include in the report or the database
You can programmatically transfer properties into the Result container to include optional results
in the report or database. Refer to Lesson 4, Process Models, for more information about results
and adding items to reports.
3-4 | ni.com
TestStand 2 Course Manual
Use the step type settings to specify the menu location, the icon, and step description. You can also
configure code module settings, define substeps, and create templates. You can use the step type
settings to set default values for any of the standard step properties. You can also disable any of the
standard properties so that a test developer cannot override the default values.
After you create a step, you can change the values of the properties for a step type instance, but
these changes do not propagate to other step type instances. Also, updating the default settings of
a step type definition do not propagate to the step type instances. For example, changing the default
module in the step type definition will not update the module called in step type instances. If you
want to propagate module call changes from the step type definition to step type instances, use a
Pre-Step or Post-Step substep instead of specifying a default module. Refer to the Pre-Step and
Post-Step Substeps section for more information about using these substeps.
Note National Insruments recommends that you store custom step types in a common
type palette file such as MyTypes.ini whenever possible.
When you use a type from a type palette, the sequence file automatically maintains a copy of the
type in case the type palette is missing or the type is removed from the palette. The version settings
resolve any differences between the sequence file version of the type and the type palette version
of the type.
3-6 | ni.com
TestStand 2 Course Manual
When you modify a type, TestStand enables the built-in Modified property for the type, as shown
in Figure 3-5.
Figure 3-5. Enabling the Modified Property for a Custom Step Type
Type Versioning
You can use only one version of a type at a time. TestStand uses the version number of a type to
determine which version of a type to use when opening a file that has a different version of the type
than the version currently in memory. Typically, the version of the type with the latest version
number is the version that TestStand automatically uses. The version number is located on the
Version tab of the Step Type Properties dialog box, as shown in Figure 3-6.
Figure 3-6. Custom Step Type Version
The Modified property for both types must be disabled for automatic type conflict resolution to
occur. If the type is in a type palette file, the latest version of the type must also be the version in
the type palette file and the earlier version of the type must be in a file other than a type palette file
for automatic type conflict resolution to occur. This behavior avoids unwanted type propagation.
You can modify the behavior to be more or less strict by changing the value of the Allow
Automatic Type Conflict Resolution option on the File tab of the Station Options dialog box.
If TestStand cannot automatically determine which version of the type to use, or if an execution is
running and the version of the type that TestStand wants to use is located in the file being loaded,
3-8 | ni.com
TestStand 2 Course Manual
TestStand informs you of the conflict through the Type Conflict in File dialog box. You can resolve
the conflict using the Type Conflict in File dialog box.
When you complete modifications to a type, disable the Modified property by incrementing the
version of the type in the Step Type Properties dialog box or in the Type Properties dialog box.
TestStand cannot automatically resolve type conflicts until you disable the Modified property. If
you do not disable the Modified property, TestStand forces you to resolve any type conflicts
manually by using the Type Conflict in File dialog box.
If you load a file that contains a type definition and another type definition of the same name
already exists in memory, TestStand verifies that the two type definitions are identical. When
TestStand compares two types with the same name, TestStand also compares all the built-in and
custom subproperties in the types. If the types are not identical, TestStand attempts to resolve the
conflict.
Refer to the NI TestStand Help for more information about type conflicts.
Use the Modified Types Warning dialog box to do one of the following when saving the modified
type(s):
• Increment the type version (will automatically disable the Modified property).
• Do not increment the type version and disable the Modified property.
• Do not increment the type version and leave the Modified property enabled.
Tip You can avoid this dialog box by manually incrementing the type version after you
modify a type. You can also disable the dialog box by permitting TestStand to
automatically increment versions by enabling the save option in this dialog box.
3-10 | ni.com
TestStand 2 Course Manual
TestStand uses this dialog box to inform you of the conflict when any of the following conditions
exist:
• The type definitions are not identical and the type versions are identical.
• At least one type is marked as modified.
• The Allow Automatic Type Conflict Resolution option on the File tab of the Station Options
dialog box restricts automatic type conflict resolution in the current situation.
This dialog box displays the conflicting type information in the file. Use this dialog box to select
one of the conflicting definitions to replace the other or to rename one of types so the types can
coexist in memory.
The Type Conflict in File dialog box contains the following options:
• Use Currently Loaded Type—Converts the instances of the type in the file to use the
currently loaded type. The newly loaded file is marked as modified.
• Use Type From <Sequence File>—Converts the instances of the type in memory to use the
type from the specified file. All previously loaded files that contain the type and instances of
the type are marked as modified.
• Rename Currently Loaded Type—Renames the type in memory and converts instances of
the type in memory to use the new name. All previously loaded files that contain the type and
instance of the type are marked as modified.
• Rename Type in <Sequence File>—Renames the type in the file and converts instances of the
type in the file to use the new name. The newly loaded file is marked as modified.
• Apply Option to All Conflicts in File—Applies the selected option to all conflicts in the
sequence file.
Using Substeps
You can use a substep to call a code module when certain events occur for an instance of the custom
step type. TestStand has four types of substeps, as shown in Figure 3-10.
• Pre-Step substeps—Influences the run-time behavior of the step before the main module
executes.
• Post-Step substeps—Influences the run-time behavior of the step after the main module
executes.
• Edit substeps—Defines configuration options for the step.
• Custom substeps—TestStand calls custom substeps when you create a new step or when you
specify a special event from a user interface.
Figure 3-10. The Four Types of Substeps in TestStand
3-12 | ni.com
TestStand 2 Course Manual
Substeps cannot use a relative path to a sequence file to locate a code module because you can use
the step type can be used in any sequence file. Therefore, place code modules for substeps into a
search directory, such as the <TestStand Public>\Components\StepTypes directory,
which is automatically configured as a search directory for supporting files for custom step types.
Substeps can call only code modules and cannot use the Sequence Adapter. Therefore, you must
write substeps in external languages. You can reuse code for existing substeps by locating the
appropriate code module in the <TestStand>\Components\StepTypes folder.
TestStand calls the Pre-Step substeps before calling the step module. You can implement a Pre-Step
substep to retrieve and store measurement configuration parameters into custom step properties the
step module can access. TestStand calls the Post-Step substeps after calling the step module. You
can implement a Post-Step substep to compare results of the code module to configured limits,
determine the step status, add additional results to the results container, or otherwise modify the
results of the step.
Refer to the Execution book in the NI TestStand Help for more information about how TestStand
runs sequences.
Edit Substeps
Edit substeps add an item to the context menu for the step type and a button to the Step Settings
pane. When the user clicks the button or selects the menu item, TestStand executes the code module
you configure. Edit substeps call code modules that usually launch a dialog box to configure
custom properties of the step. A single custom step type can have multiple Edit substeps to change
different categories of settings or to perform other operations. Each Edit substep creates a button
and a context menu item.
Custom Substeps
TestStand typically does not directly call custom step types. Use a custom step type to specify a
code module that you can call by executing the Step.ExecuteSubstep method of the TestStand API.
Use custom substeps to update a step based on changes in a user interface. For example, use a
custom substep to relate controls on the user interface to configuration properties in the custom step
type. Other entities, such as the process model or another code module, can also call custom
substeps.
The OnNewStep substep is a special type of custom substep called directly by TestStand. When
you create a custom substep named OnNewStep, TestStand calls the code module you associate
with the step each time you create a new instance of the step type. Use an OnNewStep substep to
check for and/or create any variables or other components the custom step type expects. You can
also use OnNewStep substeps to automatically create beginning and end steps, similar to the Flow
Control step types, which include a configured step and an End step to indicate the end of a block
of steps.
3-14 | ni.com
TestStand 2 Course Manual
Figure 3-12. The Code Templates Tab of the Step Type Properties Dialog Box
Combining Steps
You can combine two step types to create a new step type by right-clicking a step type and selecting
Combine With Step Type to open the step menu in which you select the second step type. The
step type you right-click is the initiating step. TestStand resolves any conflicts using the settings
from the initiating step. When you combine step types, TestStand performs the following actions:
• Copies all step properties from both steps into the new step.
• Copies all substeps from both steps into the new step.
• Resolves property or value conflicts using the settings from the initiating step.
• Uses default step settings from the initiating step.
• Uses default module settings from the initiating step.
Combine steps when you want to create sets of custom test step types. For example, if you want
custom versions of the step types in the Tests group, you do not need to create a step type for each
different test, such as Numeric Limit, String Value, and Pass/Fail. Instead, create a basic version of
the custom step that corresponds to the action you want the step to perform. Combine the custom
step you created with each step in the Tests group to create steps that have the custom settings, edit
steps, and properties of the original Tests step combined with the actions you defined in the custom
step. For example, to create a step type that handles errors that occur during a step, create a basic
version of the step type to handle errors in action steps, then combine the basic version of the
custom step with each of the Tests step types to create step types such as Numeric Limit Test with
Error Handling and String Value Test with Error Handling.
Combining step types can save development time, but it is not a complete solution. In most cases,
you must edit the new step type to achieve the results you want. For example, because the
combined step inherits step settings only from the initiating step, you must manually apply any step
settings you want from the non-initiating step. You also must edit the description, name, and menu
location of combined steps.
3-16 | ni.com
TestStand 2 Course Manual
TestStand Utility functions in CVI. LabVIEW includes several settings that control the
appearance of a VI called from TestStand. Other programming languages include internal
methods for accomplishing the same behavior.
For more information, refer to the NI TestStand Advanced Architecture series on ni.com, Best
Practices for Custom Step Type Development.
B. Step Templates
Before you create a custom step type, evaluate whether a step template is a more appropriate
solution. If you are customizing a step type by using the standard step settings, by using
expressions, or by modifying the code module for the step, a step template is an alternative
solution.
Step templates can be easier to create than custom step types because step templates avoid the
challenges of managing a type version and step templates do not create any reliance on external
files. However, step templates allow less customizing than a custom step type. Steps created from
a template are not linked to the template, which means that if you want to change all steps that were
created from a template, you must update each step manually.
Self-Review: Quiz
1. Which of the following customizations could you implement using a step template instead of
a custom step type? (multiple answers)
a. You want all the steps in the sequence to loop until they pass.
b. You want to modify the description of the steps.
c. You want all steps to pass only if execution times are under a globally defined limit.
d. You want to define custom icons for the steps related to the instruments.
e. You want to define custom configuration dialog boxes that the test developer can use to
configure the settings of the step.
3. Which of the following is a custom substep that is executed whenever a new instance of its
associated step type is created?
a. Pre-Step
b. OnNewStep
c. Post-Step
3. Which of the following is a custom substep that is executed whenever a new instance of its
associated step type is created?
a. Pre-Step
b. OnNewStep
c. Post-Step
Notes
3-22 | ni.com
Process Models
4
In this lesson, you learn about the TestStand process model. You learn how to customize and
modify common process models. You also, learn how to create custom result-processing plug-ins.
Topics
A. Process Model Structure
B. Customizing a Process Model
C. Common Process Model Modifications
D. Creating Custom Result Processing Plug-ins
Although the TestStand Engine handles the actual creation and execution of thread objects and
execution of steps, the process model controls the additional operations that occur during an
execution. When you call an entry point, you execute a sequence in the process model.
The process model is a special type of sequence file. Most custom process models are based on the
built-in TestStand process models, as shown in Figure 4-1. The sequences within a process model
include more options than sequences in a normal sequence file. Using the Advanced tab of the
Sequence File Properties dialog box to change any sequence file to a process model.
Figure 4-1. Process Model Structure in TestStand
The process model interacts with the client sequence files and code modules through the TestStand
API. A modular test architecture organizes operations common to many or all tests inside the
process model. Isolating common operations can make the test system easier to maintain because
you can make changes only to the process model instead of to each test the system contains.
4-2 | ni.com
TestStand 2 Course Manual
Sequence Types
A normal sequence file can include callbacks and subsequences. A process model file can also
contain Execution entry points and Configuration entry points. Use the icon colors in the Sequences
pane, shown in Figure 4-2, to differentiate among the sequence types in the process model.
Figure 4-2. Sequence Types
Use the Model tab of the Sequence Properties dialog box to change any sequence in a process
model to a different type.
Entry Points
A process model defines a set of entry points, and each entry point is a sequence in the process
model file that invokes a test sequence file. Defining multiple entry points in a process model gives
the user different ways to invoke a Main sequence or configure the process model. The sequence
for a process model entry point can contain calls to DLLs, subsequences, callbacks, Goto steps, and
so on. You can specify two types of entry points—Execution entry points and Configuration entry
points.
Execution entry points are the main sequences in a process model. Each execution entry point
represents one way to execute the test system. For each client sequence file that uses the process
model, TestStand adds an item to the Execute menu for each Execution entry point in the process
model file. For example, the Test UUTs Execution entry point in the Sequential process model
prompts users for a serial number, performs a test, and prompts users to test additional UUTs or
display the report. In contrast, the Single Pass Execution entry point executes the test once without
prompting for a serial number, and displays the report.
Creating a complete custom Execution entry point is a large and complex task. Copying an existing
Execution entry point and modifying the entry point can be more convenient and efficient.
Use Configuration entry points to set configuration options for the process model. For example,
select Configure»Result Processing in the Sequential process model to configure how the process
model stores data and generates and displays test reports. Each Configuration entry point adds an
item to the Configure menu.
A configuration entry point executes a sequence and can perform any type of operation. Most
configuration entry points launch a dialog box in which you can configure some aspect of the
process model. The Configuration entry point stores the configuration so that steps in the
Execution entry points can use the configuration. You can use the following two common
techniques for storing the configuration of a process model:
• Create station global variables—Station global variables are easy to create and retain their
values between executions and test sessions. However, any sequence can access station global
variables, which allows little protection for the configuration data.
• Store the configuration data in a file—The built-in TestStand process models store
configuration data in configuration (.ini) files from their configuration entry points, and load
the data from steps in each Execution entry point.
Subsequences
Process models can contain normal subsequences, that entry points or callbacks can call.
Subsequences in a process model are no different than subsequences in a normal sequence file. The
client sequence file cannot override these subsequences.
Complete the following steps to create and use a Process Model callback.
1. Create a new sequence.
2. Use the Sequence File Properties dialog box to change the sequence to a callback.
3. Call the sequence from a Sequence Call step in one or more entry points.
You can add default steps to the callback for TestStand to execute if the client sequence file does
not override the callback. Use the sequence settings of the callback sequence to determine if
TestStand copies these default steps into the client sequence file when it overrides a callback.
Engine Callbacks
Process models can also override Engine callbacks. Engine callbacks in the process model work
similarly to Engine callbacks in a client sequence file. An Engine callback defines an action to take
when certain pre-defined events occur in the engine. The only difference is that process models
have two versions of most Engine callbacks—a sequence file version and a process model version.
4-4 | ni.com
TestStand 2 Course Manual
Process model Engine callbacks define Engine callbacks for the client sequence file. Therefore, the
ProcessModelPostStep callback executes after each step in the client sequence file or any sequence
the client sequence file calls. If the client sequence file overrides the SequenceFilePostStep
callback directly, both callbacks execute after a step in the client sequence.
Some common modifications for a custom process model include creating custom configuration
and Execution entry points or additional callbacks for client sequence files to override. When you
customize an entry point, include the customization in a callback instead of using a subsequence
or inserting steps directly into the entry point so client sequence files can use a callback to override
the customization if necessary.
Prompting Mechanisms
One role of a test system is to guide the operator through the testing process. You can get
information from or display information for the operator by using the user interface, pop-up dialog
boxes, or prompts. Refer to Lesson 5, User Interfaces, for more information about user interfaces
and how you can update them from the process model.
Use prompts instead of the user interface to acquire or display critical information because prompts
launch no matter the interface in which the sequence executes, which can make debugging easier
because you can effectively run the sequence in the sequence editor.
Consider using prompts when you need the operator to enter important information, such as a serial
number or test type. Prompts function better than the user interface for this purpose because the
test automatically pauses and waits for a response from the operator, and because prompts do not
require synchronization with a user interface control.
Use prompts when the operator must manually alter the UUT or test equipment, such as connecting
the UUT to a different instrument or connecting a new UUT. You can also use prompts to notify
the operator to manipulate the UUT, such as powering on the UUT, pressing buttons, or other
interactions.
You can also use prompts to display only important information to the operator, such as
notifications of critical errors or the overall status of the UUT. Use the user interface to display
other types of information on the user interface. Avoid overusing prompts to display information
because users must click each prompt to clear it, which might significantly increase test time.
For each prompt, determine whether the prompt is specific to one type of UUT or if it applies to a
category of UUTs. For example, in cell phone testing, prompts to navigate the menu and enable a
particular feature are most likely specific to a single type of cell phone. However, prompts to power
on the cell phone or press the 9 button can apply to all cell phones. Include UUT-specific prompts
in the client sequence file. Include general prompts in a custom process model.
To include a prompt in the process model, define a callback and add the callback to an Execution
entry point. Using a callback is a better choice than adding the prompt directly to the entry point
or using a subsequence because client sequence files can use a callback to override the default
prompt in case a particular UUT is an exception to the rule. Considering the cell phone example, a
specific model of cell phone might power on automatically when you connect the cell phone to the
test equipment. In this case, the client sequence for that cell phone model must override the
callback for the power on prompt to leave the callback blank.
4-6 | ni.com
TestStand 2 Course Manual
Result Collection
The process model stores execution data in local variables as the process model executes, including
the configuration settings for the report and database, the ResultList, and information about the test
station, such as the operator’s name. You can pass many of these variables as parameters to
callbacks and subsequences. You can also customize the process model to modify the data or add
additional items. You can perform modifications statically by directly editing the local variables or
dynamically by calling the TestStand API.
The recommended way to modify process model variables is to pass the variables into a callback
as parameters and then modify the values in the callback so client sequence files can override the
settings with custom behavior.
By default, TestStand collects the results of certain step properties in memory and generates a
report, as shown in Figure 4-4.
Figure 4-4. Result Collection Process
ATML reports by modifying the style sheets they use. Refer to Lesson 6, Design
Considerations, of this course for more information about XML and ATML reports and style
sheets.
TestStand automatically collects the results of each step. Each sequence includes a ResultList local
variable, as shown in Figure 4-5, that is an initially empty array of containers. TestStand appends
a new container property called the Step Result to the end of the ResultList array before a step
executes. After the step executes, TestStand automatically copies the contents of the Result
subproperty for the step (Step.Result) into the appropriate step result container of the ResultList
array.
Figure 4-5. Result Collection
4-8 | ni.com
TestStand 2 Course Manual
ResultList
Each step type can include a different set of subproperties under its Step.Result property.
Therefore, the step result varies according to the step type. TestStand can append step results that
contain Result properties from different step types to the same ResultList array. When
TestStand copies the Result property for a step to the Step Result container, TestStand adds the
step name and position in the sequence, as shown in Figure 4-6.
Figure 4-6. Creating the ResultList Array
Subsequence ResultList
When a step calls a subsequence or generates a call to a callback sequence, TestStand creates a
special step result subproperty to store the result of the subsequence, shown in Figure 4-8, unless
the callback or sequence disables results.
Figure 4-8. Storing the Result of a Subsequence
The subproperty name varies depending on the type of step that called the subsequence or
generated a call to a callback sequence. For example, TS.SequenceCall is the subproperty name
for a Sequence Call step type, and TS.PostAction is the subproperty name for a Post Action
callback.
TestStand adds the following properties to the subproperty it creates for each subsequence called:
• SequenceFile—Absolute path of the sequence file that contains the subsequence.
• Sequence—Name of the subsequence the step called.
• Status—Status of the subsequence the step called.
• ResultList—Value of Locals.ResultList for the subsequence the step called. This
property contains the results for the steps in the subsequence.
The ResultList property of a subsequence is also stored in the ResultList entry for the Sequence
Call step that called the subsequence.
4-10 | ni.com
TestStand 2 Course Manual
Additional Results
You can use the Additional Results built-in step type when you want to log data outside the context
of one particular step in the sequence. The Additional Results step specifies values to add to the
result list of a step. An additional result is a value TestStand adds to the result list of a step when
the step executes. An additional result can be a module parameter or a custom additional result in
which you specify the name and value of the result. You can configure TestStand to automatically
include additional results in reports and when logging results to a database. Use the Additional
Results edit tab, shown in Figure 4-9, in the TestStand Sequence Editor and the Additional Results
dialog box in a TestStand User Interface to add and configure additional results.
Figure 4-9. Additional Result Built-in Step Type
You can also use the Additional Results panel on the Properties tab of any step to add and configure
additional results, as shown in Figure 4-10. Additional Results are values TestStand adds to the
result list of a step when the step executes. An additional result can be a module parameter or a
custom additional result in which you specify the name and value of the result. You can configure
TestStand to automatically include additional results in reports and when logging results to a
database. The default TestStand report generation sequences and XML style sheets do not display
additional results for skipped steps.
Figure 4-10. Additional Results on Properties Tab of a Step
You can place the additional properties you want to include in result collection in a custom step outside
the Step.Result container and then call the Execution.AddExtraResult method from an
Execution entry point in the process model. You can place a precondition on the Execution.
AddExtraResult method to specify whether to include the properties in the report for the execution.
Recall that extra results can also be collected from the Properties tab in the step settings. You can
also use the Additional Results panel on the Properties tab of the Step Settings pane to log extra
parameters.
You must select the appropriate options in the Configuration entry points for report generation and
database logging to include limits and comparison types. The built-in process models include the
limits and operator outside the Step.Result container. Add these data items to report generation
and database logging by using the Execution.AddExtraResult method from a Setup Result
Settings sequence that the built-in entry points call, as shown in Figure 4-11. Preconditions on the
AddExtraResult steps cause the steps to execute only when you include limits or operators in the
report or database.
4-12 | ni.com
TestStand 2 Course Manual
Insert Subproperties
The Step.Result container and the Locals.ResultList array are PropertyObjects.
Therefore, you can use the TestStand API to directly modify the containers. Use the
PropertyObject.InsertSubProperty method to copy existing properties into the
ResultList. Use the series of SetVal methods, such as PropertyObject.SetValString,
to create new custom items with the InsertIfMissing option.
You can add or modify items directly in the Step.Result container by using a custom step
substep that calls the TestStand API. You can also use the TestStand API from a code module or
process model to directly insert properties into the ResultList. Take this approach if you want
to copy a local variable into the results of a step without using additional results.
Report Generation
Modifying the result collection is one part of generating a custom report. However, you must also
modify the report generation settings or routines to change the report. Use the following methods
to modify the report:
• Report Options—Set the report format and location and the items to include in the report.
Refer to the NI TestStand 1: Test Development Course Manual for more information about
report options.
• Add items to the report—Add items to the report by formatting the item into a string and
storing the string in the Step.Result.ReportText property. You can also include a
property in the ResultList and set the IncludeInReport flag on that property.
• Modify the report generation routines—Modify some or all the report by overriding the
report generation callbacks in the process model. You can also open the source code TestStand
uses to create the report and modify the code directly.
You must format the data for the ReportText property only as text. Additionally, each step
includes only one ReportText property, so you must limit the property to one data item or
carefully format data items within the string to avoid confusion.
TestStand step types enable the IncludeInReport flag on a subset of the step result properties to
specify that these properties automatically appear in the report. TestStand step types also enable
the IsMeasurementValue and IsLimit flags on the properties that hold output values or limit values.
The report generator uses these flags to selectively exclude limits or output values according to the
option you select in the Report Options dialog box. If an array or container property enables one
of these flags, the report generator considers the flag enabled for all array elements or subproperties
within the containing object.
You can set these flags manually or programmatically. Complete the following steps to manually
set the flags of a step property:
4-14 | ni.com
TestStand 2 Course Manual
1. Right-click the property in the step type definition and select Properties.
2. Click Advanced in the Step Properties dialog box.
3. Set the flags using the Edit Flags dialog box.
You call the PropertyObject.SetFlags method on the property for which you want to set
flags programmatically.
If you use the DLL option for generating the report body, modify the C_report.c source file and
rebuild modelsupport2.dll.
If you use the Sequence option for generating the report body, customize the sequence files in
reportgen_txt.seq or reportgen_html.seq.
Note Before making changes to these report generation components, copy the
<TestStand>\Components\Models\TestStandModels directory to the
<TestStand Public>\Components\Models\TestStandModels directory, then
edit the copy of TestStandModel.seq. By default, TestStand searches for the
TestStandModel.seq file in the <TestStand Public>\Components\Models
directory before searching the <TestStand>\Components\Models directory. Select
Configure»Search Directories to configure the search directories.
TestStand provides a method to generate a plug-in template which creates the sequences and data
types needed for the plug-in to work with the default process models. By creating a new plug-in,
you can implement your custom code without needing to make any modifications to the default
process models, preventing you from needing to maintain a custom model. Also, custom plug-ins
integrate seamlessly into the Result Processing window, shown in Figure 4-14, and appear much
like the built-in plug-ins.
4-16 | ni.com
TestStand 2 Course Manual
Refer to the Creating Process Model Plug-ins topic in the NI TestStand Help for a more detailed
tutorial on creating a new model plug-in sequence.
To add an instance of the new plug-in to a result processing configuration, complete the steps
below. The process model uses this result processing configuration at run time to determine the
plug-ins to invoke.
1. Select Configure»Result Processing to launch the Result Processing dialog box.
2. Click the Add button and select the plug-in name from the drop-down list.
1 2
Refer to the Model Plug-in Entry Points topic in the NI TestStand Help for more information on
each plug-in entry point.
To gain a better understanding of where each plug-in entry point is called, inspect the process
model execution entry points (Single Pass or Test UUTs), and note the sequence calls which begin
with the text Model Plug-ins. Also, refer to the Model Plug-in Entry Points topic in the
NI TestStand Help for more information on each plug-in entry point.
By default, each plug-in entry point contains a number of parameters which contain information
about the current state of the execution and configuration information. These parameters are
automatically populated when the plug-in entry point is called by the process model.
4-18 | ni.com
TestStand 2 Course Manual
Plug-in—Properties
Each plug-in has a number of configuration properties stored in the ModelPlugin container. This
container is accessible to all plug-in entry points as a parameter. Properties that are shared by all
plug-ins, such as the Enabled or NewThread property are contained in the ModelPlugin.Base
container.
Properties unique to the plug-in are contained in the ModelPlugin.PluginSpecific container. This
container is defined by custom data types that are generated with the plug in:
• <plug-in name>Options
• <plug-in name>AdditionalOptions
When creating a custom plug-in, configuration properties should be added to these containers as
needed. The plug-in entry points for configuration (such as Model Plugin - Configure Standard
Options) can be used to expose these properties to the user. Property values for custom plug-ins are
automatically stored in a configuration file along with settings for the built-in result processing
plug-ins.
For each example, a sequence is provided to install the plug-in to the correct directory. Once
installed, the example plug-ins will appear in the result processing dialog.
Self-Review: Quiz
1. Client sequence files can override normal sequences defined in the process model.
a. True
b. False
2. Configuration entry points for the built-in process models store information in
a. File global variables
b. Station global variables
c. Parameters that are passed into Execution entry points
d. Configuration files on disk
3. If you override the ProcessModelPostStep Engine callback in a process model and the
SequenceFilePostStep Engine callback in a client sequence file, what happens after a step in
the client sequence file executes?
a. TestStand executes only the callback from the process model
b. TestStand executes only the callback from the sequence file
c. TestStand executes both callbacks
4. Which of the following techniques can you use to change the contents of the
Locals.ResultList property?
a. Use additional results to include extra information from any step
b. Use a custom step type to add additional properties to the Step.Results container
c. Use the Execution.AddExtraResult method
d. Directly insert subproperties into the Step.Result container using the TestStand API
2. Configuration entry points for the built-in process models store information in
a. File global variables
b. Station global variables
c. Parameters that are passed into Execution entry points
d. Configuration files on disk
3. If you override the ProcessModelPostStep Engine callback in a process model and the
SequenceFilePostStep Engine callback in a client sequence file, what happens after a step in
the client sequence file executes?
a. TestStand executes only the callback from the process model
b. TestStand executes only the callback from the sequence file
c. TestStand executes both callbacks
4. Which of the following techniques can you use to change the contents of the
Locals.ResultList property?
a. Use additional results to include extra information from any step
b. Use a custom step type to add additional properties to the Step.Results container
c. Use the Execution.AddExtraResult method
d. Directly insert subproperties into the Step.Result container using the TestStand
API
Notes
4-24 | ni.com
User Interfaces
5
In this lesson you learn about the TestStand user interface and front-end callbacks.
Topics
A. Available User Interfaces
B. TestStand User Interface (UI) Controls
C. User Interface Messages (UIMessages)
D. Front-End Callbacks
The simple user interfaces provide minimal, operator-only functionality with easy-to-edit code.
The full-featured user interfaces provide most of the functionality of the sequence editor with
significantly more complex code.
TestStand provides source code in each supported language for each version of the built-in user
interfaces, which you can use as a starting point to create custom user interfaces.
5-2 | ni.com
TestStand 2 Course Manual
With the full-featured user interface, you can open, run, debug, and create sequences as well as
configure TestStand options. The full-featured interface includes most of the functionality of the
sequence editor except for workspaces, the user manager, and some other functions. The
full-featured user interface is a large, complex application.
National Instruments recommends that you use the TestStand UI Controls to develop custom user
interface applications. However, you can also create an application by directly calling the
TestStand API. Refer to the Writing an Application with the TestStand Engine API book of the
NI TestStand Help for more information about writing an application by directly calling the
TestStand Engine API.
5-4 | ni.com
TestStand 2 Course Manual
On the block diagram, use the ActiveX reference from the control terminal to access properties,
call methods, and register events, as shown in Figure 5-5. You do not need to close the ActiveX
reference at the end of the application.
Figure 5-5. Using ActiveX TestStand UI Sequence Control Reference in LabVIEW
To interact with the controls, first use the GetObjHandleFromActiveXCtrl method to return a
handle to the control to pass to subsequent function, as shown in Figure 5-7. Call functions from
the NI TestStand UI Controls library to access properties, methods, and events.
Figure 5-7. Using the GetObjHandleFromActiveXCtrl Method
5-6 | ni.com
TestStand 2 Course Manual
1 2 3
1 2
Manager Controls
The manager controls call the TestStand API to perform tasks, such as loading files, launching
executions, and retrieving sequence information. Manager controls also notify you when
application events occur, such as when a user logs in, when an execution reaches a breakpoint, or
when a user changes the file or sequence that they are viewing.
There are three types of manager controls—the Application Manager control, the
SequenceFileView Manager control, and the ExecutionView Manager control. These controls are
visible at design time but invisible at run time.
You can call methods from the manager controls to create connections between manager controls
and visible controls. When you connect a manager control to a visible control, the manager control
automatically interacts with the TestStand Engine to provide functionality for the visible control.
TestStand includes the following four types of connections:
• List connections
• View connections
• Command connections
• Information source connections
5-8 | ni.com
TestStand 2 Course Manual
The type of connection you need depends on the manager control and visible control that you are
connecting.
List Connections
Use list connections to connect visible TestStand UI Controls for lists to a manager control that
contains information about the sequence or execution. The SequenceFileView Manager control
contains sequence files, sequences, and step groups. The ExecutionView Manager control contains
executions, threads, and stack frames. Depending on the method you use to connect the manager
control to the visible TestStand UI Control, the visible control shows the contents of the TestStand
list or stack. For example, if you use the SequenceFileViewMgr.ConnectSequenceList
method, TestStand displays a list of the sequences in the visible TestStand UI Control. TestStand
automatically updates the list in the visible TestStand UI Control if the selected items change.
View Connections
You can connect manager controls to the SequenceView and ReportView controls to display the
current sequence, execution, or report in the following ways:
• Connect a SequenceFileView Manager control to a SequenceView control to display the steps
in the selected step group of the selected sequence in the selected file.
• Connect an ExecutionView Manager control to a SequenceView control to display the steps in
the currently executing sequence of an execution.
• Connect an ExecutionView Manager control to a ReportView control to display the report for
the execution.
Command Connections
TestStand applications execute commands through menus, buttons, and other controls. The
TestStand UI Controls provide a set of common commands, such as OpenSequenceFile,
ExecuteEntryPoint, RunSelectedSteps, Break, Resume, Terminate, and Exit. Each manager control
contains a set of defined commands. The CommandKinds enumeration includes the complete list
of all defined commands. Refer to the NI TestStand Help for more information about the
commands in the CommandKinds enumeration.
You can connect the commands to TestStand buttons or application menu items. When you connect
a command to a button or menu item, the button or menu item automatically executes the
command. You do not need an event handler to implement the command. The commands also
determine the button text or menu item to display and automatically disable or enable buttons or
menu items according to the state of the application.
You can use TestStand UI Controls API methods to connect to and execute TestStand commands.
The ConnectCommand method of each of the manager controls connects a command to a button
control. The GetCommand of each of the manager controls and
ApplicationMgr.NewCommands methods return a reference to a command API object. Use the
Command.Execute method to execute a command programmatically instead of initiating the
command from a visible TestStand UI Control.
5-10 | ni.com
TestStand 2 Course Manual
Posting
Before user interfaces can capture event messages, you must first post the message from a sequence
file or process model. To post a UIMessage, call the Thread.PostUIMessageEx method. When
you post a UIMessage, you must specify an event code (numeric) that helps the event message
handler in the user interface determine which message you are posting.
Event Codes
The event code identifies each posted event message. TestStand includes a list of predefined
UIMessages with event code values less than 10000. These UIMessage enumerations identify
fundamental events to TestStand. For example, TestStand sends the UIMsg_AbortingExecution
(Value: 6) UIMessage just before aborting an execution so that the user interface can handle events
before an abort.
User messages are custom, user-defined UIMessages with an event code value of 10000 or greater.
Use UIMsg_UserMessageBase as the minimum event code value for user messages you define.
Refer to the UIMessageCodes Enumeration topic in the NI TestStand Help for a list of predefined
UIMessage event codes.
Handling
Configure an ActiveX event handler to respond to a UIMessage and parse the event code and data
sent by the Thread.PostUIMessageEx method.
To handle any type of UIMessage, register an event callback to handle the UIMessageEvent event.
When you register UIMessageEvent, the event callback handles predefined and custom messages.
This technique might cause a decrease in performance because this callback executes for every
UIMessage posted.
To handle only custom, user-defined messages, register the callback to handle the UserMessage
event. This technique handles UIMessages with an event code equal to or above the value of the
UIMsg_UserMessageBase (Value: 10000) constant.
Modularity
National Instruments recommends using user interface messages for communicating between the
process model and user interface, or sequence file and user interface because of modularity. When
a message is posted and the user interface does not capture the event, an error will not occur and
the execution will continue. Likewise, if the user interface is configured to capture a UIMessage
or UserMessage event, an error will not occur if the execution fails to post that message. This
behavior guarantees independence between the sequence file and the user interface and makes the
test system more maintainable because an event message change in the sequence file does not
render the user interface useless.
D. Front-End Callbacks
Front-End callbacks are sequences in the FrontEndCallbacks.seq file multiple user interface
applications can call to share the same implementation for a specific operation. The
FrontEndCallback.seq file TestStand installs in the <TestStand>\Components\
Callbacks\FrontEnd directory contains one LoginLogout Front-End callback sequence. The
TestStand Sequence Editor and default user interfaces call the LoginLogout callback.
Use Front-End callback sequences to implement operations so you can modify a Front-End
callback without modifying the source code for the user interfaces or rebuilding the executables for
the user interfaces. For example, to change how various user interfaces perform the login
procedure, modify only the LoginLogout sequence in FrontEndCallbacks.seq.
To modify the default implementation of the Front-End callback or to create new Front-End
callbacks, copy the FrontEndCallbacks.seq file from the <TestStand>\Components\
Callbacks\FrontEnd directory to the <TestStand Public>\Components\
Callbacks\FrontEnd directory and make any changes to copy of the file. When you copy
installed files to modify, rename the files after you modify them if you want to create a separate
custom component. You do not have to rename the files after you modify them if you only want to
5-12 | ni.com
TestStand 2 Course Manual
modify the behavior of an existing component. If you do not rename the files and you use the files
in a future version of TestStand, changes National Instruments makes to the component might not
be compatible with the modified version of the component. Storing new and customized files in the
<TestStand Public> directory ensures that new installations of the same version of TestStand
do not overwrite the customizations and ensures that uninstalling TestStand does not remove the
files you customize. You can use functions in the TestStand API to invoke the modified Front-End
callback sequence file from each user interface application you create. However, because you
cannot edit the source for the sequence editor, you cannot make the sequence editor call new
Front-End callbacks you create.
Self-Review: Quiz
1. You can use a user interface created in LabWindows/CVI to run a sequence developed with
LabVIEW code modules.
a. True
b. False
2. Visible TestStand UI Controls must have a connection to a manager control to display data.
a. True
b. False
4. Which of the following enables the process model to communicate with the user interface?
a. UIMessages
b. Front-End callbacks
2. Visible TestStand UI Controls must have a connection to a manager control to display data.
a. True
b. False
4. Which of the following enables the process model to communicate with the user interface?
a. UIMessages
b. Front-End callbacks
Notes
5-18 | ni.com
Design Considerations
6
In this lesson you learn the design considerations that affect the long-term effectiveness of your
system.
Topics
A. Modularity
B. Choosing Where To Implement Functions
C. Data Management
D. Error Handling
E. XML Style Sheets
F. Framework Deployment
A. Modularity
As you create a test framework, you must make a number of important design decisions that affect
the long-term effectiveness of the test system.
Following a modular design pattern means separating unrelated portions of code into distinct
modules. Each individual module completes its own task without any dependence on other
modules. As a result, each module is more reusable, which leads to more organized code and a
more scalable and easier-to-maintain application.
Developers should strive to create code modules, sequences, process models, and user interfaces
that are not tied to any specific implementation and that can work with any other element. For
example, a modular user interface can work with any sequence as opposed to working only with
the sequence for which it was originally developed.
Client Sequence
Use the client sequence file only for code that is specific to a particular type of UUT. Additional
code in the client sequence file makes the code difficult to reuse among other sequences and
reduces the readability of the client sequence file.
Use client sequence files for UUT-specific test programs. The test program can include prompts to
the operator, but the prompts should include only instructions or information specific to the
particular type of UUT. Many test programs require local data, which you should store in the client
sequence file. You can manage test limits in a number of ways, and one acceptable technique is to
include the test limits in the client sequence file and use the TestStand Import and Export properties
to manage the test limits. A particular type of UUT might require an exception to the process
model, in which you can override callbacks in the client sequence file to implement the exception.
You can also use client sequence files to store error handling code for specific error conditions that
steps in the test program might return.
6-2 | ni.com
TestStand 2 Course Manual
Process Model
Use the process model as a central place to implement operations. Because the process model
interacts extensively with the client sequence file and user interface, operations you implement in
the process model can easily communicate with other components. For example, the process model
can access the results of client sequence steps using engine callbacks and can send UIMessages to
the user interface.
When you implement functions in the process model, you can use callbacks to give the operations
additional flexibility. You can use callbacks to implement an operation that is the same for most
types of UUTs but make exceptions as needed.
Some of the more common operations you can implement in the process model include the
following:
• UUT Management—The process model maintains information about the current UUT, such
as the serial number. When you test multiple UUTs at the same time, the process model
maintains information about each UUT and associates results with the correct UUT.
• Operator Prompts—The process model provides instructions and information to the operator
as the test executes. Use process models for general prompts that apply to multiple types of
UUTs. Include information or instructions for operating a specific type of UUT in the client
sequence file.
• Result Collection and Processing—The process model collects the results from the client
sequence and uses the results to generate reports and log data to databases.
• Execution and Thread Management—TestStand assigns an initial execution and thread
when the operator initiates an entry point. However, the process model creates and manages
any additional executions and threads, such as those needed to implement a multithreaded
process model.
• General Error Handling—The process model defines the actions to take in the event of a
critical error and logs or displays error messages.
• Configuration—The process model manages the configuration of all the components the
process model manages, such as multithreading, result collection, report generation, and
database logging.
The most basic use of a custom step type is as a development tool to add new operations or make
sequence creation easier through configuration, automatic code generation, and/or code module
templates. You can also use custom step types to add run-time operations, such as specific error
handling, that perform before or after a step.
When included with a framework, custom step types can communicate with the process model and
user interfaces. This technique expands custom step type capability so you can use custom step
types to display custom data or use the process model configuration.
User Interface
You can implement operations in the user interface, but use care when implementing any critical
operations because critical operations might limit the test system to that user interface, which can
make development and testing difficult and limits the flexibility of the test system.
The user interface must implement core functionality to execute sequences and view sequences,
executions, and reports. The user interface can also implement some or all the other capabilities of
the sequence editor, such as configuration, user management, debugging, and sequence editing. In
addition to the core functionality, the primary modification to the user interface is the ability to
display additional data. Additional data is usually transmitted in UIMessages from the process
model or elsewhere in the framework, which keeps the framework flexible. User interfaces that do
not require the UIMessages and additional data that you add ignore these elements and execute in
the test framework without modification. The user interface can also initiate custom commands,
operations, or configurations. However, avoid implementing custom commands directly in the user
interface. The user interface should provide a way for the operator to execute custom commands,
then notify another component to execute the operation using an entry point, custom substep, or
front-end callback.
6-4 | ni.com
TestStand 2 Course Manual
C. Data Management
Storage Locations
Another important consideration is the storage location of data items. Often, the location where you
implement operations dictates the location where you must store data. In other situations, you must
transfer data among multiple components. However, in some situations, you can choose the
location for the data storage. In situations where you can choose the data location, consider the
following guidelines:
• Use sequence files to store limits and local data. Because sequence files must not include
non-UUT-specific steps, storing too much data in the sequence file makes it difficult for the rest
of the framework to access the data. In addition, you must include non-UUT-specific steps in
the client sequence file to transfer the data to the user interface, place the data in a report, or
log the data to a database.
• Use the process model to store global configuration settings. Store any user-configurable
setting that is not specific to an instance of a step in the process model as well. The process
model must eventually gather all test results for display in the report or logging to the database.
However, in most cases, custom step types, instead of the process model, define the results
themselves.
• Use custom step types to define test results that eventually must be logged to a database or
displayed on a report or user interface. Custom step types can also store step-specific
configuration settings. Store any configuration that might vary between different instances of
the same step type within the step rather than in the process model.
• Avoid using user interfaces for data storage because different user interfaces can execute the
rest of the test framework. In addition, communicating data from the user interface back to the
test framework is difficult.
In LabVIEW, this can be accomplished using Data Value References. Refer to the New Data Value
Reference Function and Delete Data Value Reference Function topics in the LabVIEW Help for
more information on Data Value References. With .NET/C/C++, this is accomplished using Object
References and Pointers.
Result Collection
When collecting results, categorize the results into required results and optional results. Required
results are always included in the database, the report, or both. The step results container includes
most results that are specific to individual steps. The easiest way to add data to the results container
is to use a custom step type. To add results that are general to the UUT, use a Process Model
callback, such as the ModifyReportHeader callback or ModifyReportFooter callback.
Add or remove optional results to or from the report or database with user-controlled configuration
settings. Define how the results are configured using a configuration entry point in the process
model or an Edit substep of a custom step type.
Process model configuration is appropriate for results that are general to the UUT as a whole and
for enabling or disabling the results for an entire class of steps. If you use a process model setting,
use TestStand API calls, such as the Execution.AddExtraResult or
PropertyObject.SetFlags methods, from callbacks in the process model to add the results
when the configuration option is enabled.
Configuring the option using a Step substep is appropriate for results that might display for one
instance of a step type but be hidden for another instance. Use TestStand API calls in a Post-Step
substep to add the results when the configuration option is enabled.
D. Error Handling
A robust test system responds appropriately to unexpected or undesired conditions. Errors occur in
all test systems. Even a well-written test system cannot account for some situations, such as a faulty
or unplugged piece of test equipment, a full hard disk, operator error, or a lost packet in network
communication. Carefully consider error handling techniques to make the process of detecting,
responding to, and reporting an error as simple for the operator as possible. Use the following basic
guidelines for responding to errors:
• Avoid unnecessary error prompts—Errors that drivers or other communication programs
return might not be a serious problem for the testing system. Warnings can indicate non-critical
issues, such as a particular test failed, non-critical data was not logged, or a communication
problem occurred and was later resolved. It is not necessary to launch a dialog box to report the
warning to the operator. The best solution is to attempt to solve these types of errors
programmatically using corrective error handling. If the error cannot be resolved, you can
ignore the error entirely or write the error to a log file so that a technician or framework
developer can analyze the error at a later date. Launch dialog boxes only for errors that require
the immediate attention of an operator.
• Avoid system shutdown for non-critical errors—Errors that pose a serious problem for the
test system, such as a disconnected piece of test equipment, prevent the test system from
completing the test. However, these types of errors do not require you to terminate the test or
shut down the test system, which is the default action most programs take when encountering
an error. A better solution is to display a prompt to the operator to correct the problem and retry
the operations after the operator indicates that the problem was resolved. Always give the
operator an option to stop the test if the operator cannot resolve the problem.
• Provide meaningful error messages to the operator—Use error messages to describe the
component that reported the error, suggest methods for correcting the error, and, when
necessary, provide contact information for the correct person to debug or analyze the error.
6-6 | ni.com
TestStand 2 Course Manual
General error handling refers to the actions you take to respond to any error. General error handling
includes reporting the error using a dialog box, logging the error to a file, and/or initiating a system
shutdown. General error handling does not check for individual error codes, but might take
different actions based upon the classification of the error. Specific error handling refers to the
actions you take to respond to a specific error code.
By default, TestStand treats all errors the same. When you write custom error handlers, you can
differentiate among types of errors, such as warnings, normal errors, and critical errors. You might
log a warning to a file, launch a dialog box to respond to a normal error, and immediately initiate
a shutdown in the event of a critical error. Implement error classification by creating custom error
containers or by using the fields in the error container in different ways. For example, one common
way to implement a warning is to set the Code element of the error container to the error code but
leave the Occurred element set to False.
You can implement general error handling in the client sequence file. However, because general
error handling is not dependent on specific error codes, it is more effective to implement general
error handling in the process model. Multiple client sequence files can then use the same error
handling techniques. The SequenceFilePostStepRuntimeError and
ProcessModelPostStepRuntimeError callbacks are good locations to implement general error
handling.
It can be time consuming to predict and respond to all the errors that might occur in a test system,
but developing error handling for the most common errors can lead to a much more reliable and
robust test system. Specific error handling requires detailed knowledge of the operations the
system performs to determine what errors the operations might cause. Implement specific error
handling in a client sequence file or in a custom step type.
When you detect a specific error, you can take the following actions:
• Ignore specific errors—Code modules or other operations can return expected errors that are
not relevant to the test system. For these errors, you can clear out the error container and
continue with the test.
• Correct errors—When you know the cause of a specific error, there are many actions you can
take to resolve the error programmatically. For example, if a file routine returns an error
indicating that the file routine cannot write a file because of a lack of disk space, you might
programmatically clear out caches or other temporary files from the disk to create enough disk
space to continue. Other examples of corrective error handling include prompting the operator
to fix the error, retrying the error until the error resolves itself, or closing and reopening a lost
connection. After you correct an error, clear the error to prevent general error handling routines
from responding to the error.
• Classify errors—You can respond to a specific error by classifying the error so the general
error handling routine can handle the error appropriately. Typically, specific error handling
routines execute before the general error handling routines, and any errors that the specific
error handler does not clear are passed to the general error handler. For example, if an error is
too important to ignore but does not require notifying the operator or taking immediate action,
you can use a specific error handler to change the value of the Occurred field of the error
container to False and pass the error to a general error handler. The general error handler
recognizes that the error is a warning and logs the error to a file rather than taking more serious
action.
The <xml-stylesheet> node of an XML report contains the URL path of the style sheet to apply
to the XML file when you view the file in a browser or other viewer. The following example uses
the TestStand horizontal style sheet:
You can use the Contents tab of the Report Options dialog box to select the style sheet to apply to
an XML report. You can also customize the style sheet directly.
6-8 | ni.com
TestStand 2 Course Manual
• report.xsl—Displays reports in a vertical, tabular, indented format with data for each step in
multiple rows, as shown in Figure 6-2. Customizable, color-coded rows visually separate
relevant information, such as step group and status.
Figure 6-2. Report.xsl Report Style Sheet
6-10 | ni.com
TestStand 2 Course Manual
Refer to the XML Report Style Sheets topic in the NI TestStand Help for more information about
XML style sheets.
F. Framework Deployment
For test developers to use a framework, you must package the framework and install the framework
on test developer development computers. You can use the TestStand Deployment Utility to create
an installer for the framework, similar to the installers you use to deploy a test system to a test
station. However, instead of deploying the TestStand Engine and files to run a stand-alone test
system, you can assume that the development computers have TestStand installed and deploy only
the components of the framework. Install the framework to the appropriate subdirectories of the
<TestStand> directory. Do not include run-time engines or drivers with the framework unless the
run-time engines or drivers are for components that the test developers are not likely to have
installed on their development computers.
Self-Review: Quiz
1. Match each storage location to the type of data that you should store there:
2. Match each operation with the location that is best suited to performing that action:
2. Match each operation with the location that is best suited to performing that action:
Notes
6-16 | ni.com
Customizing Database Interaction
A
This appendix contains the following sections of useful information for TestStand users:
Topics
A. Relational Databases
B. Structured Query Language
C. Customizing Database Interaction
D. Modifying Schemas
E. Database Step Types
A. Relational Databases
A relational database contains tables, similar to spreadsheets, that use rows and columns for storing
data. The tables in a relational database store references to rows in other tables to create
relationships among the tables. The relationships are possible because each row in each table is
unique. Two rows cannot store exactly the same data. A relational database uses the relationship
among tables to handle much more complex data relationships than a standard spreadsheet. For
example, a relational database can handle multi-dimensional relationships, whereas a spreadsheet
can handle only two dimensions.
A-2 | ni.com
TestStand 2 Course Manual
STEP_RESULT table stores the primary key of a single record of the UUT_RESULT table in
the UUT_RESULT column. Multiple records in the STEP_RESULT table can refer to a single
record in the UUT_RESULT table because testing a UUT requires the result of multiple steps.
• Statement—A command executed on the database. Use statements to create and manage
tables, insert or remove data, and query the database to return stored data. For example, you
can execute a SELECT statement on the STEP_RESULT table to access all the steps that failed
in the last hour.
• Recordset—A set of multiple rows within a table. Some statements operate on recordsets. For
example, a SELECT statement can return multiple rows that match the query. An UPDATE
statement can change the value of multiple existing rows. For these types of statements, a
recordset contains the column data for each row that you transfer or receive from the database.
You can consider a recordset as a temporary table. For example, if you queried the
STEP_RESULT table for all steps that failed in the last hour, more than one step might meet
the criteria. Therefore, the SELECT statement returns a recordset, which is a table that contains
only the rows that matched the query. You can parse the recordset one row at a time and extract
the column data.
When you log data to a database, TestStand automatically handles the parsing of data, construction
of recordsets, and insertion of data into the database. You need no knowledge of SQL to enable
database logging in TestStand. However, to customize database logging, you need a basic
understanding of SQL for the following reasons:
• When you log data, basic SQL statements determine which data TestStand reads and updates
from a given table. To add a new table, you must write an SQL command for that table.
• If you use the Database step types to communicate with the database directly, you must create
and parse SQL commands to define the operations you want to perform.
Basic Commands
You can use several basic commands in SQL to construct and manage the tables that define the
database. Call these commands when you first configure the database or when you must change
how the database stores data. Other commands operate on records. Call these commands each time
you want to add, remove, change, or access data in the database. Refer to the SQL Commands topic
of the NI TestStand Help for the syntax of these commands.
D. Modifying Schemas
Use the Database Options dialog box to configure a schema, which defines the data TestStand logs to
a database. A schema also determines when, and in what format, TestStand logs the data. Each schema
requires a matching database layout. For example, if the schema logs data to a UUT_RESULT table,
a UUT_RESULT table must exist in the database and must have at least the same number of columns
expected by the schema. When you create a new schema or modify an existing schema, you can write
SQL commands to create or modify the tables in the database. You can also instruct the schema to
auto-generate SQL statements to create the objects the schema expects.
Figure A-1. Modifying the Schema in the Database Options Dialog
A-4 | ni.com
TestStand 2 Course Manual
Note The auto-generated SQL statements might not always work on every database
and might require modification for more complex schemas.
Review—Default Schema
To log results, TestStand uses a schema that defines which results to log to the database and into
which table and field to log each result. The default TestStand database schema defines and
requires the following database tables. You can also define a custom own schema to control how
TestStand logs data.
Figure A-2. Default TestStand Database Schema
• UUT_RESULT—The default TestStand schema logs station, batch, UUT, and execution
information to this table.
• STEP_RESULT—The default TestStand schema logs common subproperties of all steps to
this table.
• STEP_SEQCALL—For Sequence Call steps, the default TestStand schema logs additional
subproperties of Sequence Call steps to this table.
• PROP_RESULT—The default TestStand schema logs information about the properties in a
step result to this table for steps. If necessary, the default TestStand schema logs additional
information about specific property data types in the other tables named with a PROP prefix.
• PROP_NUMERICLIMIT—If necessary, the default TestStand schema logs the limit
information for a Numeric Limit Test to this table.
• PROP_BINARY—If necessary, the default TestStand schema logs array data, such as numeric
arrays, to this table.
Statements
A schema is composed of statements that can perform almost any operation on a database. Because
statements execute for step or UUT results, statements are almost always used to insert data into a
table. In the default TestStand schema and most similar schemas, each statement represents a table
in the database.
Figure A-3. Modifying Statements in the Database Options Dialog
A-6 | ni.com
TestStand 2 Course Manual
Statement Types
You can use the following three types of statements with TestStand:
• Recordset—Executes a query that returns a recordset. TestStand then uses the recordset to hold
the data to log and inserts the recordset back into the database. To create a recordset statement,
specify command text that includes a SELECT statement. The default TestStand schema is
based on recordset statements, and recordset statements are the most common way to log data
from TestStand.
• Command—Executes a standard SQL command. Use command statements to directly call an
INSERT command instead of using a recordset statement.
• Stored Procedures—Executes and passes data to a stored procedure, which are special
features of some databases that implement code within the database itself. You can call the code
using SQL commands.
Columns
The Columns/Parameters tab of the Database Options dialog box defines the data items TestStand
places in each column of the table for each recordset that TestStand logs. Each column has a value
from the Logging property and can have an optional precondition. If the precondition prevents
TestStand from inserting data into a column for a recordset, TestStand inserts a NULL value into
that column instead. The Columns tab refers to parameters instead of columns when you use the
stored procedure statement type. The configuration and functionality are basically the same.
Logging Property
Database logging routines are called from the process model, much like report generation. When
TestStand initiates database logging, TestStand compiles all the information required to check or
log to the database into a Logging container. The compiled information includes the configured
database settings, some information from the process model, such as the start and end times for the
UUT, and the ResultList. Within statement and column configuration pages, you can use items
from the Logging container as preconditions or data items to insert into the database.
Figure A-5. Logging Container
A-8 | ni.com
TestStand 2 Course Manual
Each database step type returns or accepts one or more references, which have a numeric data type.
Complete the following steps to execute a command on the database:
1. Execute an Open Database step, which returns a database reference. Store the database
reference in a local variable.
2. Execute an Open SQL Statement step, which returns a reference to an SQL statement. The
Open SQL Statement expects a database reference and a string representing the SQL command
to execute. Store the statement reference in a local variable. The Open SQL Statement also
returns the number of results in the recordset after executing the command. Store the number
of results in a separate variable.
3. For each result, execute a Data Operation step. The Data Operation step expects a database and
statement reference. You can use the Data Operation step to parse a recordset to insert or read
values.
4. Execute a Close SQL Statement step to close the statement reference.
5. Execute a Close Database step to close the database reference.
Open Database
Use the Open Database step to open a database for TestStand to use. The Open Database step also
returns a database handle (number) you can use to open SQL statements in an Open SQL Statement
step. The Data Link tab of the Edit Open database dialog box is similar to the Data Link tab of the
Database Options dialog box. The Edit Open Database dialog box includes the Database Handle
control to specify the name of a variable or property of type Number to which you assign the
returned database handle.
Figure A-7. Edit Open Database Dialog Box
A-10 | ni.com
TestStand 2 Course Manual
Data Operation
The Data Operation step type performs operations on the recordset returned by a preceding Open
SQL Statement step. Use the Data Operation step to fetch new records, retrieve values from a
record, modify existing records, create new records, and delete records.
Use the Edit Data Operation dialog box to configure the step. Use the Record/Operation tab to
specify a reference to selected records on which you want to operate. You can also specify how to
reference these records while performing operations and what operations to perform.
Figure A-10. Edit Data Operation Dialog Box
If the operation includes Get or Set, use the Column/Parameter Values tab to map TestStand
properties and variables to the database field. The Column/Parameter Values tab indicates which
TestStand property stores the values of a database field and vice versa. After you select the
statement with which you selected the set of records, TestStand automatically populates the
Column Name/Number control. You can then select columns and enter the corresponding
TestStand property, variable, or literal value to map to the column.
Refer to the Edit Data Operation Dialog Box topic of the NI TestStand Help for more information
about the options in the Edit Data Operation dialog box.
A-12 | ni.com
TestStand 2 Course Manual
Statement Handle option specifies the name of the variable or property of type Number that
contains the statement handle to close. After closing the SQL statement handle, the step assigns a
value of zero to the variable or property.
In addition to the common custom properties, the Close SQL Statement step type defines the
Step.StatementHandle step property, which specifies the name of the variable or property of
type Number that contains the SQL statement handle to close.
Note TestStand does not automatically close SQL statement handles. You must call a
Close SQL Statement to close open handles when you finish with the handles. If you
abort an execution, you must exit the application process that loaded the TestStand
Engine to ensure that TestStand frees all database handles. Selecting File»Unload All
Modules does not close the handles.
Close Database
Use the Close Database step to close the database handle you obtain from an Open Database step.
National Instruments recommends placing Close Database steps in the Cleanup step group. Use the
Edit Close Database dialog box to configure the step. The Database Handle option specifies the
name of the variable or property of type Number that contains the database handle to close. After
closing the database handle, the step assigns a value of zero to the variable or property.
In addition to the common custom properties, the Close Database step type defines the
Step.DatabaseHandle step property, which specifies the name of the variable or property of
type Number that contains the open database handle to close.
Note TestStand does not automatically close open database handles. You must call a
Close Database step to close open handles when you finish with the handles. If you abort
an execution, you must exit the application process that loaded the TestStand Engine to
ensure that TestStand frees all database handles. Selecting File»Unload All Modules
does not close the handles.
A-14 | ni.com
TestStand 2 Course Manual
Self-Review: Quiz
1. A single query can return multiple results.
a. True
b. False
Self-Review: Quiz
1. A single query can return multiple results.
a. True
b. False
Notes
A-18 | ni.com
Additional Information and
B
Resources
This appendix contains additional information about National Instruments technical support
options and TestStand resources.
You also can visit the Worldwide Offices section of ni.com/niglobal to access the branch
office Web sites, which provide up-to-date contact information, support phone numbers, email
addresses, and current events.
TestStand Resources
Remove or modify this section depending on the resources available for the product. This section
does not include index markers. Create index markers as necessary for information you add to this
section. The following list outlines possible product resources to include.
• Newsletters
• Books
• Listserves
• Web sites
B-2 | ni.com