Professional Documents
Culture Documents
The File Collector Historian
The File Collector Historian
The File Collector Historian
The information contained in this manual is believed to be accurate and reliable. However, GE Intelligent Platforms,
Inc. assumes no responsibilities for any errors, omissions or inaccuracies whatsoever. Without limiting the foregoing,
GE Intelligent Platforms, Inc. disclaims any and all warranties, expressed or implied, including the warranty of mer-
chantability and fitness for a particular purpose, with respect to the information contained in this manual and the
equipment or software described herein. The entire risk as to the quality and performance of such information, equip-
ment and software, is upon the buyer or user. GE Intelligent Platforms, Inc. shall not be liable for any damages, includ-
ing special or consequential damages, arising out of the use of such information, equipment and software, even if GE
Intelligent Platforms, Inc. has been advised in advance of the possibility of such damages. The use of the information
contained in the manual and the software described herein is subject to GE Intelligent Platforms, Inc. standard
license agreement, which must be accepted by the buyer or user before the use of such information, equipment or
software.
Trademark Notices
GE, the GE Monogram, and Predix are either registered trademarks or trademarks of General Electric Company.
All other product names and marks identified throughout this book are trademarks or registered trademarks of their
respective companies. They are used throughout this book in editorial fashion only. No such use, or the use of any
trade name, is intended to convey endorsement or affiliation.
No part of this publication may be reproduced in any form, or stored in a database or retrieval system, or transmitted
or distributed in any form by any means, electronic, mechanical photocopying, recording or otherwise, without the
prior written permission of GE Intelligent Platforms. Information contained herein is subject to change without notice.
We want to hear from you. If you have any comments, questions, or suggestions about our documentation, send them
to the following email address:
doc@ge.com
Table of Contents
Format 8
For Data: 8
For Messages: 8
Index 21
You can install the File Collector using the instructions in the Data Collector - General e-book. Optionally,
you can use the silent install feature, which is also described in the Data Collector - General e-book.
Use the File Collector to import CSV and XML text files into Historian. Since the files can contain data, tags
and tag properties, as well as messages, the File Collector is a very useful tool for bringing 3rd party data
into Historian.
The File Collector uses the ImportFiles folder for its operations, which can be found in the Historian pro-
gram folder. The ImportFiles folder contains the following subdirectories:
Directory Function
Error If a CSV or XML file contains errors, the file collector will stop pro-
cessing it and place the file in this folder.
Incoming Files that are to be processed by the file collector are placed here.
Processed Files that have been successfully imported by the file collector are
placed here.
Working Files are placed in this folder while the file collector is importing
their contents.
NOTE: The ImportFiles folder is not created until you run the File Collector.
To import a file, place it in the Incoming folder. At the beginning of the next cycle (one of the parameters
entered by the user through the Historian Administrator), the system initiates the file import operation, pro-
cesses the data, stores the result in an archive, and moves the file from the Incoming folder to the Pro-
cessed folder. During the processing operation, the file moves to the Working folder and the filename
changes to YMDHMS-Data.csv or .xml file (for example, 010810103246-data.csv).
When the number of days you specify in the Processed File Purge parameter have passed, the system
deletes the imported file from the Processed folder. The Error directory is never cleared.
NOTES
l Since a File Collector is really an import function rather than a data collection operation, standard col-
lector features such as compression, buffering, browsing, start/stop collection do not apply.
l If you are adding tags using the File Collector, ensure that you specify the Collector Source Address and
the Tag Source Address fields in the CSV file. If you do not, the Collector and Source Names are not added
to the tags created by the File Collector.
where Command = [Tags], [Data], [Messages], or [Alarms] and Keywords = (see the following list). Please
note that the order of the data must match the order in the header. For example, if you are importing a
tagname, timestamp, value, and quality you would use the following syntax:
[Data]
Tagname,TimeStamp,Value,DataQuality
TIGER.IMPORT_TAG1.F_CV,7/20/01 11:07,1,Good
While importing an Alarm and Event only the following headers are considered.
l Acked
l Acktime
l Actor
l AlarmID
l Condition
l DataSource
l Enabled
l EndTime
l EventCategory
l ItemID
l Message
l Quality
l Severity
l Source
l StartTime
l SubCondition
l Tagname
l Timestamp
Tagname Description
EngineeringUnits Comment
DataType StringLength
StoreMilliseconds CollectorName
CollectorType SourceAddress
CollectionType CollectionInterval
CollectionOffset CollectionDisabled
LoadBalancing TimeStamp
Type TimeZoneBias
HiEngineeringUnits LoEngineeringUnits
InputScaling HiScale
LoScale CollectorCompression
CollectorDeadbandPercentRange CollectorCompressionTimeout
ArchiveCompression ArchiveDeadbandPercentRange
ArchiveCompressionTimeout Timeout
CollectorGeneral1 CollectorGeneral2
CollectorGeneral3 CollectorGeneral4
CollectorGeneral5 ReadSecurityGroup
WriteSecurityGroup AdministratorSecurityGroup
Calculation CalculationDependencies
CalculationExecutionTime StepValue
Acked Condition
SubCondition EventCategory
Message Source
Severity StartTime
EndTime TimestampType
SpikeLogic SpikeLogicOverride
InterfaceAbsoluteDeadband InterfaceAbsoluteDeadbanding
LastModified LastModifiedUser
ArchiveAbsoluteDeadband ArchiveAbsoluteDeadbanding
StepValue Value
NumberOfElements CalcType
NOTES:
l StringLength = 2 times the number of characters (ASCII, Single Byte). For example, "ABC" = 6 StringLength.
l TimeStamp: Timestamp resolution is in milliseconds for this collector. Do not try to import microsecond
timestamps.
You cannot import Last Modified User, Last Modified Date, and Calculation Execution Time fields on a tag.
Example of CSV File that Imports Tags
* This is a comment
*
[Tags]
Tagname,Description,DataType,HiEngineeringUnits,LoEngineeringUnits
TIGER.IMPORT_TAG1.F_CV,Import Tag 1,SingleFloat,35000,0
TIGER.IMPORT_TAG2.F_CV,Import Tag 2,SingleFloat,35000,0
TIGER.IMPORT_TAG3.F_CV,Import Tag 3,SingleFloat,35000,0
TIGER.IMPORT_TAG4.F_CV,Import Tag 4,SingleFloat,35000,0
TIGER.IMPORT_TAG5.F_CV,Import Tag 5,SingleFloat,35000,0
TIGER.IMPORT_TAG6.F_CV,Import Tag 6,SingleFloat,35000,0
TIGER.IMPORT_TAG7.F_CV,Import Tag 7,SingleFloat,35000,0
TIGER.IMPORT_TAG8.F_CV,Import Tag 8,SingleFloat,35000,0
TIGER.IMPORT_TAG9.F_CV,Import Tag 9,SingleFloat,35000,0
TIGER.IMPORT_TAG10.F_CV,Import Tag 10,SingleFloat,35000,0
Example of a CSV File that Imports Data and Data Quality
* This is a comment
*
[Data]
Tagname,TimeStamp,Value,DataQuality
TIGER.IMPORT_TAG1.F_CV,7/20/01 11:07,1,Good
TIGER.IMPORT_TAG1.F_CV,7/20/01 11:08,2,Good
TIGER.IMPORT_TAG1.F_CV,7/20/01 11:09,3,Good
TIGER.IMPORT_TAG1.F_CV,7/20/01 11:10,4,Good
NOTE: Before importing MultiField tag data in this format, you should add the User Defined Type using Historian
Administrator and associate that type to the MultiField tag.
NOTES:
For more information on these tags, refer to the Python Expression Tags ebook.
Format
The format for a XML file is as follows:
NOTE: Any tag property can be modified this way. See CSV File Formats for a list of properties.
For Data:
<Import>
<Datalist>
<Tag>
<Data>
<Timestamp>..... </Timestamp>
<Value>.... </Value>
</Data>
</Tag>
</Datalist>
</Import>
For Messages:
<Import>
<MessageList>
<Data>
<Timestamp>..... </Timestamp>
<Topic>.... </Topic>
<Username>..... </Username>
<MessageNumber>.... </MessageNumber>
<MessageString>..... </MessageString>
<Substitutions>.... </Substitutions>
</Data>
</MessageList>
</Import>
NOTES:
For more information on these tags, refer to the Python Expression Tags ebook.
Polled Collection No
Unsolicited Collection No
Timestamp Resolution 1 ms
Binary Data No
Collector compression No
Start/stop collection from Historian Admin- No, scanned at user-specified interval. Set to 0 to stop.
istrator
To display the Configuration tab for a File Collector, select a File Collector from the list on the left and click
the Configuration Tab. The screen shown in the following figure appears.
Scan Interval The rate at which the input directory, Historian\ImportFiles, is scanned for any
new files. To change it, enter a new value. The scan interval cannot exceed 65
seconds.
NOTE: Changes to the Scan Interval do not take effect until the File Collector is restar-
ted
CSV File Spe- The file extension for the CSV file being imported. You can specify more than
cifications one extension type, such as: csv,txt,dat.
XML File Spe- The file extension for the XML file being imported.
cification
Purge Pro- The number of days a file can reside in the Processed Files directory before
cessed Files being deleted automatically.
(days)
Purge Error The number of days an error file is retained before being deleted automatically.
Files (days) The default is 10.
If you are experiencing any problems with the File Collector, use the .LOG file (in the \LogFiles folder) to
troubleshoot. The .LOG file sometimes logs errors that do not get processed to the Historian Admin-
istrator. For example, if you have no archives in your system and you attempt to import a .CSV file with
formatting errors, the file is not processed and no alerts are sent to the Historian Administrator (if there
are no archives created, the message database has not been created). But this error does appear in the
.LOG file.
The following table lists typical Error messages received with the File Collector and tips for troubleshoot-
ing those messages.
12-Aug-01 13:38:00 - Import Line Error: There is an extra line in the file. Open the file in Note-
Input past end of file pad and remove the extra line.
10-Aug-01 15:47:02 - Import Line Error: Two fields were rolled into one due to a missing
Type mismatch comma. Open the file in Notepad and add the
comma.
10-Aug-01 15:47:02 - Error Occurred On
Line 2: General Format Error
12-Aug-01 14:18:12 - Invalid Import Field: If an import field is invalid, (in this case Time res-
TimeResolution olution should be StoreMilliseconds) the import will
abort.
12-Aug-01 14:18:12 - zPro-
cessFragment>> Aborted Import Due To
Formatting Errors
the line number in the log is produced by the SDK, not the File Collector. This means that the line number is
counted relative to the header or field list for each section of the file, ignoring comments and blank lines.
CSV File Imports
If a .CSV file has an extra line in it, it may not successfully import using the File Collector. If a .CSV file has extra
commas on the data line, it may not import completely.
Typically, a .CSV file must be less than 10 MB and an ideal file should be 1 or 2 MB. If the file size is greater,
then the File Collector may not respond.
You cannot import CSV or XML data that goes back beyond the Archive Active Hours setting (1 month by
default). Adjust your Archive Active Hours setting and re-import the data.
Enter values for the File Collector-specific field parameters through the File Collector Maintenance - Con-
figuration Tab of the Historian Administrator.
Scan Interval Collector initiates import operation at beginning of the scan interval spe-
(seconds) cified in this field.
NOTE: Changes to the Scan Interval do not take effect until the File Collector is
restarted
CSV File Specification The file extension for a .CSV file to be imported.
XML File Specification The file extension for an XML file to be imported.
Purge Processed Files The contents of the Processed Files folder is automatically purged after the
After (days) number of days specified in this field.
Purge Error Files The contents of the Error Files folder is automatically purged after the num-
After (days) ber of days specified in this field.
collector-specific fields 20
collector features 14
collectors
File 2
configuration 20
features 14
File Collector
collector-specific fields 20
features 14
overview 1
troubleshooting 18
file formats 8
.CSV 2
.XML 7
formats 8
.CSV file 2
.XML file 7
summary of features 13
troubleshooting 18