Learn XQuery in 10 minutes, SQL Server 2005 Style

By Kent Tegels, Database Curriculum Lead, DevelopMentor

I always love reading the works of Dr. Michael Kay. He recently posted what I thought was a really
good introduction on XQuery over on the Stylus Studio site
(http://www.stylusstudio.com/xquery_primer.html.) But being the SQL Server 2005 fanatic that I
am, I couldn’t help but notice he didn’t give the product any love. So for those you using (or will
soon be using) SQL Server 2005 and XQuery, I’m happy to present this quick guide to XQuery,
SQL Server 2005 style.

What is XQuery for?

Like T-SQL is a query language over data stored in tables, XQuery is a query language over XML
instances. The key difference is that instead of returning a set of fields in a row, XQuery returns a
sequence of nodes. The primary use case of XQuery in SQL Server 2005 is to extract some portion
of a given number of XML nodes from stored set of XML instances. You might to do something
simple like get a list of the values in the nodes conforming to a given XPath expression, or you
might want to do something more complex, like transform a set of XML nodes into a new XML
instance. Normally, you’ll be doing these operations to return data to a client or high-tier service or
object. Of course, since we’re storing them these XML instances in a database, it is likely that we’ll
be doing Data Manipulation Language (DML) operations like inserting, updating and deleting nodes
as well. As XQuery stands today, there are not specific operations for DML. Microsoft has added
them using a new and unique to SQL Server concept called XML DML. To save time and space, I
won’t be covering XML DML in this posting.

In many ways, XSLT and XQuery give you different ways to achieve the same result with neither
always being “better.” A big difference for us as SQL Server 2005 users is that XQuery is
implemented as actual part of the Query Processing engine so it take advantage of indexing and type
inference from schemas. This provides for optimized query processing. You can do XSLT
transformations fairly easily with a SQLCLR function, but this won’t leverage indexes or schema
collections.

Playing with XQuery

Getting started with XQuery in SQL Server 2005 is fairly straight forward. One thing that some
folks have struggled with is the dependency on having the right .NET Common Language Runtime
(CLR) installed on your machine for the release of SQL Server 2005 you want to use. For this
exercise, we will be using the free (as in free beer) version of SQL Server 2005. You can download
this from http://www.microsoft.com/downloads/details.aspx?familyid=6E4AAC3A-9D85-4734-
B1FD-318FB83B0D29&displaylang=en, but be sure to pay close attention to all of the requirements
and install notes on this page. We’re still in the beta phase and not everything is perfect… yet.

As Dr. Kay notes in his work, if you have to install anything, it is going to take more than ten
minutes, so feel free to just read along if you like.

Your first XQuery

Since many folks like to start with the canonical example program, “Hello World,” so let’s have a go
at that. The first thing you’ll want to do is drop to a command shell by typing “CMD” after clicking
start and selecting RUN.

Page 1 of 8 – © 2005 by Kent Tegels, All Rights Reserved

Start by moving up to the root directory of the “C” drive. Next, let’s log into the SQL Express
instance. For our purpose, I’ll assume that you’re running as an administrator of the local computer.
At the command prompt type:

sqlcmd –S .\sqlexpress –E

And yes, the –S and –E do need to be capitalized. This will start the SQLCMD utility, an easy to use
interface that lets you submit queries to SQL Server 2005. The “-S .\sqlexpress” option tells
SQLCMD that you want login into the local instance of SQL Server 2005 Express Edition while the
–E option says you want to use Windows Integrated Security. After doing all of that you should be
greeted with “1>” prompt. At that prompt type each of the following lines in individually, pressing
enter at the end of each line.

set quoted_identifier on
declare @x xml
set @x='<?xml version="1.0" ?><p>Hello, World!</p>'
select @x.value('(/p)[1]','varchar(max)')
go

When you’re done, your screen should look something the following:

Page 2 of 8 – © 2005 by Kent Tegels, All Rights Reserved

There’s actually quite a lot that happens with that little bit of T-SQL, so let’s cover each line one at
time.

1. set quoted_identifier on: This is required for SQLCMD to properly understand line 4.
2. declare @x xml: This declares a variable, named “@x,” using the new XML data type.
3. set @x='<?xml version="1.0" ?><p>Hello, World!</p>': This populates the variable with
the XML of interest. Under the covers, SQL Server discards the document prolog and just
stores the <p> tag and the text in a fast binary form in memory.
4. select @x.value('(/p)[1]','varchar(max)'): There’s actually quite a bit of work going on
here. First, we’re executing a select T-SQL query. We have to do this because our XML is
stored in a T-SQL variable. In fact, almost all access to XML data type instances in SQL
Server requires us to access the instance via T-SQL. Why? As previously noted, SQL Server
2005 XQuery process is integrated with the T-SQL query processor. Next we invoke the
value method on variable. Unlike many other T-SQL Data Types, the XML Data Type is
primarily worked with using methods on instances. SQL Server 2005 engine processes the
queries within these method calls as XQueries. The value method takes an XQuery which
returns a scalar result and then type casts that to the second parameter to the method call. In
this case, we’re making a simple XPath query over the stored data (which is fine, XQuery is
a superset of XPath) with the “/p” part of the expression. We wrap this in parentheses to
indicate that the query returns a sequence of values. Finally we tack on the “[1]” part to
indicate that we want the first value from the sequence. The “varchar(max)” actual parameter
indicates that we want the value returned by the query returned as a string.
5. go: This simply tells SQLCMD to transmit the command to server and display and response
returned.

And I’ll agree with Michael, we’re not pushing the envelope of Rocket Science, but we have already
covered the primitives of the XQuery usage pattern: write a T-SQL query that invokes one or more
of the methods of the XML data type. We pass an XQuery into the query processor as parameters of
one of these methods.

Accessing XML Documents with XQuery

Although we’ve already done some of this, there’s a bit more to the story. Following Dr. Kay’s
example, let’s start with a slight modified version of his videos.xml file. Since that file is fairly good
sized, I’m not going to bother listing it here, but I would like you download this installation script
and then follow these instructions (note that you will need to change the file extension from .sql.txt
to .sql):

1. Download this file. It will create a custom database, a table and will populate a table with the
videos.xml file. Save this file to “C:\”
2. If you are still in the SQLCMD shell, type exit to return to the DOS prompt.
3. At the command prompt type:
sqlcmd -S .\sqlexpress -E -i xqueryin10minutes.sql -h -1

This particular version of the SQLCMD invocation uses the –i option to run the download script.
The output of this command should look like this:

Page 3 of 8 – © 2005 by Kent Tegels, All Rights Reserved

So now let’s try Dr. Kay’s first query. Again, this is really an XPath query but this time we will use
the Query method instead of the values method. The difference between these two methods is that
whereas the values method returns a type-cast scalar value, query returns a sequence of values in
something that looks like a partial XML document. Let’s have a go at it:

1. Type “sqlcmd –S .\sqlexpress –E –I –h -1” (without the quotes) at the

command line and press enter
2. Now type the following query and press enter after each line:
a. use xqueryin10minutes
b. go
c. select list.query('.//actors') from dbo.videos
d. go

Your output should look something like this:

Now if you’re comparing this along side Dr. Kay’s tutorial, you’re probably expecting me to show
you the “find all actors with a first name of Lisa query.” We can do that, but keep in mind that SQL
Server 2005 doesn’t currently support the ends-with function he uses. There are a few more
functions that are missing. The downloadable “Books On Line” talks about them. We can, however,
show a similar search using contains. For example, enter this at the SQLCMD 1> prompt:

1. select actors.actor.value('.','varchar(max)')
2. from dbo.videos
3. cross apply list.nodes('.//actor') as actors(actor)

Page 4 of 8 – © 2005 by Kent Tegels, All Rights Reserved

 4. where actors.actor.value('contains(.," Lisa")','bit') = 1

Wow, there’s a lot goes in this relatively short query! This is because we are taking a part of single
document and breaking it into rows using the node so let’s break it down line by line to see what
each part means.

1. This line uses the value method on the “actor” field in the “actors” virtual table created on
line three. The key point here is that the virtual table has exactly one row for each actor
element in our document, so we need to reference the whole virtual table to access its rows
and thus values.
2. This simply tells the query to use the videos table in the dbo schema.
3. This line does three main things. First, we use cross apply to include the results of functions
called on part to the data consumed in the from statement. The called to the nodes method
here simply creates a virtual table with one row for each node matching the XQuery
statement passed into the method call. In this case, we’re looking for any elements named
“actor”. The last part, starting with the as, defined the virtual table name and column name.
4. Finally, the where predicate filters rows out the virtual table. We’re using another XQuery
here as an actual parameter to the value method. In this case, we’re calling the Contains
function. This function simply tests to see if the text passed as the second actual parameter to
that method exists in the text value of the node indicated in the first part. In this case, we’re
looking to see if the string “ Lisa” (and yes, it’s case sensitive), exists in the text part of an
“actor” element.

XQuery FWOR Expressions

One of Dr. Kay’s uses of this query is list all of the productions that have an actor with a first name
of Lisa in them. We can do that do fairly easily as well, but the script runs a bit long. To save you the
effort of typing that query, I’ve created it as a stored procedure that gets created when you run the
installation script that creates the database. The stored procedure has a parameter of type
varchar(max). Let’s look at the “useful guts” of the stored procedure:

1. declare @n xml
2. select @n = list.query('
3. for $video in (.//video)
4. for $actor in (.//actor)
5. where $video/actorRef/text() = $actor/@id
6. and contains(($actor/text())[1],sql:variable("@actor"))
7. order by ($video/title/text())[1]
8. return <v>{$actor/text()} was in "{$video/title/text()}"</v>')
9. from dbo.videos

We start by declaring that “@n” will be an XML typed variable. Line two sets that value of the
variable to the result an XQuery. Lines three through eight represent an XQuery in what is known as
FWOR notation. As we talked about earlier, XQuery is a superset of XPath, so almost any XPath
expression is also a valid XQuery. However, XQuery has another allowed syntax, using a pattern of
F(or), W(here), O(rder) and R(eturn). It works as you might expect: each of the “for” exposes the
nodes selected in the following XPath expression for use as a variable. In this case, on lines three
and four, we will have two sequences of values to work with: $video representing each of the
“video” elements in the documents while $actor represents each actor element in the document.

Page 5 of 8 – © 2005 by Kent Tegels, All Rights Reserved

Following our “fors,” we have a “where” predicate on lines five and six. Here we start by relating
key elements (named actorRef) in the video element values using $video/actorRef/text(), with the
actors they represent by their actor element’s ID attribute. This is a natural concept for database
programmers as this is semantically the same as doing a join. As are also reusing the “contains”
function as we did in the previous example with a minor variation. Since this is a stored procedure, it
doesn’t make sense to hard code in “Lisa.” Instead, we would likely and to use a parameter instead.
To bring the value of a variable into our XQuery, we call a Microsoft-specific extension to XQuery –
the sql:variable function. This function just takes the name of variable or parameter you want
substitute into XQuery.

Another SQL-like concept that XQuery provides in FWOR queries is the ordering of results using
“order by.” The essential different with XQuery is that instead of specifying a column name, you
provide it with an XPath expression that returns a scalar value. For our example, line seven simply
selects the text value of the video title element.

The last part of our XQuery is the return on line eight. XQuery’s return is pretty easy to understand.
Anything that is not wrapped in curly-braces (e.g., { and }) is output directly (and unevaluated) into
the sequence returned by the XQuery. Anything wrapped up in curly-braces is evaluated as a
statement and the resulting value will be emitted into the resulting sequence as literal values.

So let’s see this stored procedure in action. If you still have SQLCMD open, you can simply type the
following query in:

select dbo.VideosWithActorsWithNamed ‘Lisa’

And you should get output like this:

Now, you might asking why I didn’t call them FLWOR expressions or talk about the L(et) part of
XQuery. This is one of another one of these cases where SQL Server 2005 doesn’t fully implement
the XQuery working drafts – there’s simply no support for the Let construct at all.

Page 6 of 8 – © 2005 by Kent Tegels, All Rights Reserved

Generating XML output with XQuery
One of the envisioned use cases for XQuery is an alternative to XSLT. Some users of XSLT have
said they feel that it harder to master because of its declarative template nature. XQuery allows users
to craft XML transformations using a query language. Our examples so far have shown us the
relative ease of querying data of out the XML in scope, but we’ve not seen it generating XML
output. Turns out that this is actually really simple to do! If you recall, the return part of a FWOR
pattern spits out literal values. There’s no reason then that we can’t emit XML tags. Then consider
that with the FOR XML option in SQL Server 2005, we can generate actual XML instances. So what
happens when we combine XQuery and FOR XML? Again, the code for this is a bit longer than you
should have to type by hand, so I’ve included in the install script as a stored procedure. The
interesting parts of that look like this:

1. select list.query('
2. for $video in (.//video),$actor in (.//actor)
3. where $video/actorRef/text() = $actor/@id
4. and contains(($actor/text())[1],sql:variable("@actor"))
5. order by ($video/title/text())[1]
6. return <li>{$video/title/text()}</li>')
7. from dbo.videos
8. for xml path(''),root('ul'),type

Lines one through five and seven are little changed since our last example, so I won’t detail them
here again. Line six, however, is. Here we are using XQuery’s ability to emit literal values to
generate XML tags (the <li> and </li> strings) around our data. Line eight shows using a FOR XML
extension to the T-SQL select to generate XML. In this case, we’re using the path option, but since
all our elements – the sequence generated by the XQuery – already represents full-formed elements,
we don’t need to specify any element names for it to wrap our values with. We simply ask it to emit
an empty string instead. The Root part then wraps all of the elements from the XQuery into a single
XML element named “ul.” Finally, the type part tells SQL Server that we returning an SQL Server
style XML instance. You can test this stored procedure out just by calling
dbo.VideosWithActorsWithNamedAsXML as shown following:

Show me the Database!

Dr. Kay does a good job talking about the big differences between working with XQuery against file
and database, but since we’ve been working against the database all the long, there’s really not much
more to say. And there’s no magic here either. If you’re able to program against SQL Server 2005

Page 7 of 8 – © 2005 by Kent Tegels, All Rights Reserved

using .NET, you can fully use XQuery via the standard classes in the SqlClient namespace just as
you would any other T-SQL query. If you want to use XML instances generated by the XQueries
easily, simply execute your query using the ExecuteXmlReader method of the Command class. This
will give you an XML reader over the returned values to work with.

Time Is Up!
We’ve covered a lot of ground in this article: you’ve seen how you can get started using XQuery
with SQL Server 2005 Express edition and the SQLCMD utility. The key things to remember are:

• To use XQuery within SQL Server 2005, you write T-SQL queries that call methods on
XML-typed instances on columns and/or variables.
• The commonly used methods are Query, which returns the sequence of values directly as part
of a result set; Value, which converts the returned sequence as a T-SQL scalar value type and
Nodes, which parses a single XML instance into a virtual table split on a given XPath
expression.
• XQuery is a superset of XPath. Almost all XPath expressions are also valid XQuery
expressions. XQuery in SQL Server 2005 also offers the FWOR pattern syntax – For, Where,
Order and Return – which can simply writing XML transformations.

If you’re interested in learning even more about XQuery in SQL Server 2005, there are a number of
resources I highly recommend, including:

• The SQL Server 2005 Books On Line covers many of the technical details of using XQuery
in a reference format style.
• Dr. Michael Rys’s Webcast on using XQuery at http://url123.com/dgs47
• The book A First Look at SQL Server 2005 for Developers by Beauchemin, Berglund and
Sullivan has excellent coverage of both the XML Data Type and XQuery.

Page 8 of 8 – © 2005 by Kent Tegels, All Rights Reserved