Configuring for Custom Queries

Learn how to configure schematic dataset to run standard SQL queries to a table or set of tables and
then use the results to create nodes, links and connectivity on diagrams. See other resources for
ArcGIS Schematics.
http://video.arcgis.com/watch/1540/configuring-for-custom-queries

Video Transcription
00:06 Welcome to the Configuring for Custom Queries video.
00:10 This video is part five of the configuration videos from the...
00:13 ...ArcGIS Schematics Video Series.
00:16 The video is based on data that is available in the tutorials available from ArcTutor.
00:21 So to follow along, you will need to have the schematics tutorials installed.
00:26 You will also need to have the extension licensed and available.
00:30 This picture shows the three main components...
00:33 ...to understanding how schematics works.
00:36 Builders are the initial part of the system that must understand how to derive...
00:40 ...connectivity from the incoming data.
00:42 After the builder is able to get the connectivity...
00:45 ...then rules and algorithms can optionally be applied to produce the final diagram.
00:51 So what is custom query capability?
00:54 Well, this capability allows someone to configure an application that runs standard...
00:59 ...SQL, Structured Query Language, type of queries...
01:02 ...to a table, or a set of tables, and then use the results of those queries...
01:07 ...to create nodes, links, and the connectivity on a schematic diagram.
01:12 Custom query capability is unique in that it spans across...
01:16 ...all the normal schematic builders.
01:19 That said, however, normally, one should use the Standard Builder when creating...
01:23 ...an application configuration that requires custom query capability.

01:29 To be able to create custom query configurations...

01:32 ...you must understand some basic SQL, again, Structured Query Language, syntax...
01:38 ...you must understand your data model and you must also understand what kind...
01:43 ...or what kinds of diagrams, you are trying to create.
01:47 From a configuration perspective this is our most difficult type of configuration.
01:53 There are typically four different kinds of custom query configurations.
01:58 The first two are just dealing with the number of tables involved.
02:02 We often see cases where there is only one table being used and the data in that table...
02:07 ...needs to be visualized in a diagram as both nodes and links.
02:11 A good example of this is an organizational chart.
02:15 Often, this is only one table where each person is a record...
02:19 ...and there is an attribute showing who each person reports to.
02:23 So we get the data from the table one time to create a node on the diagram for each person.
02:29 Then we get the data from the table again to draw links from each person...
02:33 ...to the person that they report to.
02:36 The next case is more common, where there are a series of tables holding information...
02:40 ...about things that are typically not spatial in nature.
02:43 They are inside of something that might be visible on the map, though.
02:47 In these cases, there are typically one or more tables holding things...
02:51 ...that will be visualized as nodes...
02:53 ...and there are one or more tables holding things that will also be visualized as links.
02:58 The link tables typically have fields describing the connectivity information...
03:03 ...such as "from node" and "to node."
03:07 The third kind of configuration is dealing with either retrieving all the data...
03:11 ...or needing to filter the data.
03:13 So in the case of an organizational chart, we probably want to get all the data and...
03:18 ...visualize it so the user just needs to generate a diagram.
03:22 They don't need to pick something first to help us filter the data.

03:26 In the second case, the internals, we will most likely need the user to pick something.
03:32 If our tables have all the information for all the things that have internals...
03:36 ...we likely don't want to see it all at once.
03:39 We want the user to tell us which item they are interested in, then we can use that...
03:43 ...to filter the data in the tables down to what we really want to have on our diagram.
03:49 The final type of configuration can really be a combination of all the above concepts...
03:54 ...but at least one table, or more tables...
03:57 ...is spatial in nature instead of everything being non-spatial.
04:01 So in these cases, we can use the spatial information for the initial placement or...
04:06 ...positioning of the nodes in the diagram, but still, perhaps, get the logical link...
04:11 ...connectivity from the same table or maybe from another table in the database.
04:18 So now let's get into an actual configuration.
04:21 To get started, we will look at an example that has multiple tables...
04:25 ...and will require some filtering.
04:28 The data for this example is in the tutorial installation location...
04:32 ...and then under Schematics\Schematics_ Configuration\Mixed_Data.
04:42 Keep in mind that this example is utilities based.
04:45 But the concepts that you will learn will work for any data requiring custom query capability.
04:52 Inside this database, you will notice three tables that have a prefix of ISP.
04:58 That stands for "Inside Plant."
05:01 So the records inside of these tables are things that are inside of a gas plant.
05:06 And those gas plants are actually part of a geometric network that is visible on a map.
05:11 So these are non-spatial tables, no-shape field, no XY information.
05:18 To create custom query diagrams, you must understand how your data is organized...
05:23 ...what makes records unique across the database...
05:26 ...how to filter it and how to derive the connectivity between things.
05:32 So let's start with the ISP_EQUIPMENTS table.
05:36 This table has a field called NUMBER with an underscore...

05:41 ...that has a unique range of numbers in the 200 to 500 range.
05:45 So that can be used as a unique identifier for each row when visualized on a diagram.
05:51 This is critical to understand as when it comes time to have a link connect...
05:55 ...from a node to another node...
05:57 ...there can't be any ambiguity or schematics won't know how to connect things properly.
06:03 Also notice that there is a field for PLANT_NUMBER.
06:08 You can see here that we can use this to filter out only records that belong to...
06:13 ...a cetain plant and avoid seeing all records for all plants at the same time.
06:18 That is all we really need to know for the nodes.
06:21 Now let's take a quick look at the ISP_VALVES table...
06:25 ...and you will notice that, just like the ISP_EQUIPMENTS table, it has a field...
06:30 ...called NUMBER_, and it also has the PLANT_NUMBER.
06:37 Valves all happen to be in the 100 range in their NUMBER_ field.
06:43 So again, this allows us to use that field as a unique field.
06:48 Now for the final table, and the key to the connectivity in this case...
06:52 ...let's look at the ISP_PIPES table.
06:56 Unique numbering isn't so important for data that will be links, because nothing else...
07:00 ...is trying to connect to them.
07:02 So I will show you a trick for that, in a minute, during configuration.
07:10 Notice that there are two fields in this table...
07:13 ...one is called, "FROM_NODE_NUM," and the other one is called, "TO_NODE_NUM."
07:21 That happens to be the connectivity.
07:23 The values in there will be pointing to either a record that is coming from the...
07:27 ...ISP_VALVES table or from the ISP_EQUIPMENTS table.
07:33 Finally, there is also a GAS_PLANT field that we can use for filtering.
07:39 So we have now seen how the data is organized, understands what makes things...
07:44 ...unique, how to filter out the data, and how to derive the connectivity.
07:50 So now we can jump into the actual configuration.

07:54 We will create a New Schematic Dataset for our configuration...

07:58 ...inside this same Brazil database.
08:01 I will go through this quickly, so for more information on creating and editing...
08:04 ...schematic datasets, please see the appropriate video in this series.
08:09 To create a New Schematic Dataset, just right-click the geodatabase...
08:13 ...and choose New > Schematic Dataset.
08:17 I can now rename the Schematic Dataset...
08:20 ...and then, once I'm ready, I can right-click it and choose Edit, which will open up...
08:24 ...the Schematic Dataset Editor application, where we will do our configuration.
08:41 Here we will create a new diagram template by right-clicking on my dataset...
08:46 ...and choosing New Schematic Diagram Template.
08:52 I will use the Properties tab to give my template a new name.
08:55 In this case, "MyCustomQuery."
08:58 The default Schematic Builder is already set to Standard Builder...
09:02 ...so we don't need to change anything else.
09:04 So once I set the name, I just click the OK button, and then we will see the...
09:08 ...MyCustomQuery entry show up in the tree view, letting me know that it has been set.
09:18 So far, this configuration is the same as the previous configuration videos.
09:23 But now is where we get to the harder part.
09:26 We need to manually create the schematic feature classes and the various attributes...
09:30 ...to support this application and configuration.
09:33 In the previous videos, you saw that this was all pretty much automated for you.
09:39 Again, custom query is the most difficult case.
09:42 Always remember though, this should be a one-time thing done by one person...
09:47 ...for an entire organization.
09:49 The actual end user in ArcMap simply generates diagrams against this configuration.
09:55 To begin with, let's forget for a moment that we want to filter the data that will...
09:59 ...eventually be on the diagram.

10:01 We will just configure our schematic feature classes to retrieve...

10:04 ...all the data from the three database tables.
10:07 After that, we will come back and show how to set up the filtering.
10:11 When we are done, this will give you the basis for two of the typical configuration types...
10:15 ...I mentioned previously.
10:18 So right-click the Schematic Dataset and choose New Schematic Feature Class.
10:24 We will start with a node, so let's name this one, "ISP_VALVES."
10:31 Note that you can name this anything you want.
10:33 It doesn't have to be the same name as the underlying table you will query.
10:38 Since we want this class to be represented as a node all the defaults in this dialog are okay.
10:43 So just click the OK button.
10:47 Now we need to modify the query to be used to retrieve the data.
10:52 In the Query section, change the Evaluation Mode to Query On Generate/Update...
10:59 ...then click the Query/Identifier Editor button.
11:04 Now in the Query/Identifier Editor dialog, we will deal with all the SQL stuff.
11:10 First, we need to set the Data Source to CURRENTDS.
11:15 CURRENTDS is an internal key word for the current database.
11:20 Now from the Tables list, double-click the ISP_VALVES entry.
11:26 This will fill in the basic query for us.
11:29 In the Select unique identifier section...
11:32 ...double-click the NUMBER_ field to add it to the Identifier box.
11:46 And finally, click the OK button to exit this dialog.
11:52 Now we will repeat this whole process for the ISP_EQUIPMENTS table.
11:57 Right-click the Schematic Dataset and choose New Schematic Feature Class.
12:03 We'll rename this one to be ISP_EQUIPMENTS.
12:10 Since we also want this class to be represented as a node...
12:14 ...all the defaults are okay, such just click the OK button.
12:18 Now, again, we need to modify the query that is going to be used to retrieve the data.

12:23 In the Query section, change the Evaluation Mode to Query On Generate/Update.
12:29 And then click the Query/Identifier Editor button.
12:33 Again, we need to set the Data Source to CURRENTDS.
12:37 And this time, we'll double-click the ISP_EQUIPMENTS table.
12:42 This sets up the basic query for us.
12:44 And just like last time, double-click the NUMBER_ field to add it to the Identifier...
12:50 ...and then click OK.
12:53 Finally, we need to deal with our links.
12:55 Right-click the Schematic Dataset and choose New Schematic Feature Class.
13:02 We will name this one, "ISP_PIPES."
13:07 Since we want this class to be represented as a link...
13:10 ...we need to change the Type to Link and then click the OK button.
13:18 Now we need to modify the query to be used to retrieve the data.
13:22 In the Query section, change the Evaluation Mode to Query On Generate/Update.
13:26 And click the Query Identifier/Editor button.
13:30 Change the Data Source to CURRENTDS.
13:33 Double-click the ISP_PIPES and now we've got the basic query set again.
13:39 Remember that I mentioned a trick for the link identifiers.
13:43 This time, click the button in the Identifier section for Add Identifier.
13:49 Here, just type in the actual string, "PIPES."
13:56 And then click the ENTER button.
13:59 Now, double-click the OBJECTID field to add it to the Identifier section.
14:05 And last, but not least, click the OK button to exit this dialog.
14:09 So all our pipes will be identified by the actual string "PIPES," and then an OBJECT ID.
14:16 This will make them all unique for this database.
14:19 Now all we have left to do before we can actually visualize one of these diagrams...
14:24 ...is to set up a couple of attributes.
14:26 If your nodes happen to have some X,Y information you want to use...

14:30 ...where you are actually querying a spatial feature class and want to decode...
14:34 ...the Shape field, that can all be done with a couple attributes called "initial X position"...
14:40 ...and "initial Y position" on your various node feature classes.
14:44 In this case, these tables were non-spatial.
14:47 So we don't need to do anything else for nodes at the moment.
14:51 However, for the links, we need to define two, predefined attributes...
14:55 ...for origin node and extremity node.
14:58 These two attributes will tell schematics which fields in the database...
15:02 ...it is supposed to be used to figure out the connectivity.
15:05 To do this, right-click the ISP_PIPES entry and choose New Attribute.
15:12 Choose the Name drop-down list to choose ExtremityNode.
15:17 This is your "to node" information.
15:19 So find and double-click the TO_NODE_NUM field to add it to the Identifier's list...
15:26 ...and click the OK button to continue.
15:35 Now we need to do the same thing for the OriginNode.
15:38 Right-click ISP_PIPES and choose New Attribute.
15:42 Use the Name drop-down to choose OriginNode.
15:46 And now double-click on the FROM_NODE_NUM field.
15:51 Click OK when you're done.
15:53 So now we have told schematics how to get the connectivity for this pipes feature class.
15:59 Click the Save button and close the Schematic Dataset Editor.
16:08 We are now ready to test out this initial configuration...
16:11 ...so go ahead and start up ArcMap.
16:21 Then click the Generate New Schematic Diagram button.
16:26 Use this dialog to navigate to the New Schematic Dataset that you created.
16:47 Now we can type in a name for this diagram and click the OK button.
16:59 Now, you finally have your initial diagram.
17:02 It probably looks a bit strange at first, but remember that there were no X,Y coordinates.

17:08 So everything is on top of each other at the 0,0 coordinate and the links have been...
17:12 ...slightly modified so that they don't cause an invalid geometry problem.
17:17 Using the Schematic Editor toolbar's drop-down, choose Start Editing.
17:27 Now go ahead and choose the Hierarchical- Smart Tree layout algorithm...
17:32 ...and click the Apply Layout Task button.
17:35 Here you can see several disconnected trees.
17:38 These are all the various internals...
17:40 ...for the different gas plants that have data in this database.
17:43 It looks okay here because there were only a few.
17:46 Imagine if there were hundreds or thousands; it wouldn't look so nice.
17:50 Anyway, this is the basics for creating diagrams from all the data in the tables.
17:55 And in many cases, this is actually what you might need.
17:59 Now we will go back and modify this configuration so we can allow the user...
18:03 ...to filter the data that will be displayed for each diagram.
18:07 So now we can go ahead and close ArcMap and then edit our schematic...
18:11 ...dataset again to open up the Schematic Dataset Editor application.
18:26 To allow for filtering, we need to get some information from the user...
18:30 ...when they try to generate a diagram.
18:32 To do this, we will modify the diagram template level first.
18:36 In this case, we need to get the gas plant number.
18:39 But what if the user doesn't know it by the number?
18:42 They know it by the name.
18:43 Let's work with that case now.
18:45 Select the template level and click the Query/Identifier Editor button.
18:54 Now we can double-click the pug_PUG_gas_plants table...
19:01 ...to have the system build the initial query.
19:11 Notice that this table has both the gas plant NAME as well as the PLANT_NUMBER.
19:18 What you define here as the Identifier ends up being presented...

19:21 ...to the end user in the Generate Schematic Diagram dialog.
19:25 Double-click the NAME field first and then double-click the PLANT_NUMBER field.
19:32 We're done with this Identifier so go ahead and click OK.
19:36 This will prompt the user during diagram generation with the name-plant number...
19:42 ...in a nice pick list, and they can choose which one they're interested in.
19:46 Now internally, we need to get the plant number they picked in that dialog...
19:50 ...so we can use it to dynamically change our SQL queries to the three tables.
19:55 To do that, we just need to pick up an attribute.
19:59 So right-click the template and choose New Attribute.
20:04 Here, we will go ahead and type in "PlantNum" for the name of our attribute.
20:10 And again, the attribute that we're really interested in getting is the...
20:13 ...PLANT_NUMBER coming back from this query, so we'll just double-click...
20:18 ...PLANT_NUMBER, adds it to the Identifier, and click OK.
20:22 Now when a user picks a value from the list, we will hold on to this number for later use.
20:28 Now we are ready to modify the individual schematic feature class queries.
20:33 Start by selecting the ISP_EQUIPMENTS schematic feature class...
20:37 ...and then click the Query/Identifier Editor button.
20:42 We will now modify this query to have a where clause.
20:45 We want it to say, "Where plant_number = ?"
21:00 The question mark is a special character for schematics.
21:03 It tells the system that you are going to be passing in an attribute for this value.
21:08 We will notice that a new area of the dialog opens up for specifying the parameter to use.
21:14 It is already defaulted to that PLANT_NUM attribute that we created moments ago...
21:19 ...at the diagram template level.
21:21 This is a numeric attribute though, so we will need just to uncheck the Is Text checkbox.
21:30 If you need to test, you can use this dialog to test by typing in a value and choosing...
21:35 ...the Show Results button, and then make sure you get some results in the grid.
21:40 We are done modifying this one, so click OK to exit.

21:44 So now we need to do exactly the same thing for the other two schematic feature classes.
21:50 So we will select the ISP_VALVES schematic feature class...
21:54 ...and then click the Query/Identifier Editor button.
21:58 Just like we did on the last one, now we just need to add the where clause.
22:10 The defaulted parameter is correct.
22:12 We just need to uncheck the Is Text checkbox and then click OK.
22:19 Again, but now for ISP_PIPES.
22:38 So that's it for this configuration change.
22:40 Click the Save button and then close the Schematic Dataset Editor.
22:48 Now we'll start up ArcMap again and then generate a diagram.
22:56 So now I'll click the Generate New Schematic Diagram button and use the dialog...
23:01 ...to navigate and select our Schematic Dataset.
23:06 Notice this time that you can't type in a diagram name.
23:09 You're provided with a list.
23:13 This list is coming from the Identifier setting that was done at the diagram template level...
23:18 ...configuration, where we specified name and then plant_number.
23:24 We'll keep the first item selected and click the OK button.
23:30 We see the same initial problem as the last time that we generated a diagram...
23:34 ...where everything is a 0,0, so start editing and apply the Smart-Tree.
23:49 Note that during configuration, you can set your template to automatically apply...
23:53 ...the smart trees, so the users never actually see this initial state.
23:58 After applying the Smart Tree, you will see that, this time, the data was, in fact...
24:03 ...filtered down to just the records for this ARACA gas plant, or plant number 101.
24:16 So now you have seen how to work with data and multiple tables, and either getting...
24:20 ...all the data at once or allowing for a filter of the data.
24:24 The next thing to look at is combining the filtering verse all data...
24:28 ...along with the data just coming from one table.
24:32 I will be using a preconfigured example for this part, and it is not available in the tutorials.

24:37 To start with, we'll take a look at the data here in ArcCatalog.
24:48 Notice that this database only contains one table, and it is non-spatial.
24:56 When we look at the data...
24:57 ...we see that it contains information about authors and articles.
25:01 So some authors have worked on multiple articles...
25:04 ...and some articles have been worked on by multiple authors.
25:07 Looking closer at the data...
25:09 ...we can see that there are only three unique author IDs: number 23, 24, and 25.
25:16 And there are three unique article IDs: number 12, 13, and 14.
25:21 Looking at the data, we can envision a diagram that has nodes...
25:24 ...representing the unique authors, nodes representing the unique articles...
25:29 ...and then links connecting the authors to the articles that they have worked on.
25:33 Now we will edit this preconfigured schematic dataset...
25:36 ...to take a quick look at the configuration.
25:44 Hopefully, the first thing that you notice is that there are multiple diagram templates...
25:48 ...inside this Schematic Dataset.
25:50 Remember that a schematic dataset can have many diagram templates.
25:54 Diagram templates contain all the information needed to produce that type...
25:58 ...or kind of diagram.
26:01 When looking at the All Diagram template...
26:04 ...we will notice that it doesn't have a query at the template level...
26:07 ...meaning that we won't be able to filter the data; we will be showing all the data.
26:12 Opening this up exposes the three schematic feature classes underneath it.
26:17 There is one class for authors, one for articles, and one for the links.
26:23 Notice how the queries to the articles and authors tables use a special...
26:28 ...key word in SQL called, "distinct."
26:32 This key word helps us ensure that...
26:34 ...we don't end up with duplicate objects on our diagram...

26:37 ...we only want one node for each unique article and one node for each unique author.
26:44 The ByAuthor and ByArticle templates both have queries attached to them.
26:50 This will allow the end user in ArcMap to pick which article they want to diagram for...
26:56 ...or which author they want to diagram for.
26:58 And the data would be filtered specifically for that article or that author.
27:03 The interesting thing about the schematic feature class is in that, in all three of these...
27:07 ...templates, is that they all query the same, single table in the geodatabase.
27:12 So one table, being represented as both nodes and links...
27:16 ...and within the nodes category as both articles and authors.
27:20 Let's see what this looks like for the actual end user in the map.
27:29 When I click the Generate Schematic Diagram button...
27:32 ...I will have to navigate to this database and dataset.
27:46 You will now see that in the Schematic Diagram template drop-down...
27:50 ...I can choose between the three different diagram templates that were configured.
27:54 Remember that the All template did not have a filter, so it allows me to type in...
27:59 ...a name and generate this diagram.
28:09 Now I will turn on some labeling, so we can see the three unique articles...
28:13 ...the three unique authors, and the information about who worked on what.
28:25 Now I will click on the Generate Schematic Diagram button again...
28:28 ...but this time we will choose the ByArticle template.
28:35 This time, I am prompted to choose which article I am interested in viewing...
28:39 ...so I pick one and click OK.
28:45 Now this diagram is centered around that article and who worked on it.
28:57 Finally, I will click the Generate Diagram button again, this time pick ByAuthor...
29:02 ...template and now you can see I'm prompted to pick which author I'm interested in...
29:07 ...pick one; click OK and just like last time, this one is centered around the author.
29:19 So, hopefully, you found this to be an interesting case here.
29:22 One table being represented with multiple schematic feature classes...

29:27 ...but we also have multiple diagram templates that are doing a different thing.
29:31 One shows all data and the other ones are filtering data based on some unique...
29:36 ...need that we want from our schematic diagrams.
29:40 And now for the final example: This time, we will take a look at another...
29:44 ...preconfigured example and the data is also not in the tutorial system.
29:49 This is some hydrology data from the state of Louisiana.
29:52 There is one feature class that contains polygons.
29:55 Each polygon record also has information about the next down-stream polygon.
30:01 The goal here is to get a diagram showing nodes representing each polygon...
30:05 ...and then links connecting to the downstream nodes or polygons.
30:10 So one spatial polygon feature class being represented as both a node...
30:14 ...and a link-schematic feature class.
30:18 Here is a quick look at the configuration.
30:20 Two schematic feature classes that are both querying the same table.
30:24 Notice that the node schematic feature class has the InitialX...
30:28 ...and InitialYPosition attributes configured.
30:31 These are pointing at the real feature classes shape field...
30:35 ...and then decoding the geometry to get the X or the Y position.
30:40 The links schematic feature class has the mandatory origin and extremity nodes...
30:45 ...configured appropriately for this particular data.
30:49 Back in the map, we can switch to the other data frame containing the diagram that...
30:54 ...was generated from this data.
30:56 Here in the diagram, you can see all the points and lines...
30:59 ...that come from querying this one feature class.
31:02 Because the X,Y coordinates were actually used for these nodes...
31:06 ...we can now drag and drop this diagram onto the geographic map...
31:11 ...and see the diagram directly on top of the map.
31:19 So those are the basics of custom query functionality in ArcGIS Schematics.

31:24 We have seen examples of the four main use cases...

31:27 ...and hopefully you now have a good understanding about how it works.
31:31 Thanks for watching.

Esri 2015 http://www.esri.com