CODE 2-2023 Web

Cloud Computing, Razor, Kubernetes, Dapr, Chrome Debugging
MAR
APR
2023
codemag.com - THE LEADING INDEPENDENT DEVELOPER MAGAZINE - US $ 8.95 Can $ 11.95
Introduction
to Snowflake
Hidden Secrets of Event Systems Exploring (More)

the Chrome Debugger with Dapr EF Core Features
INCREASE
YOUR CYBER
shutterstock_ran
gizzz/Africa Stu
dio/onsight
SECURITY DEFENSES
IDENTIFY POTENTIAL SECURITY PROBLEMS
CODE Security offers application security reviews to help organizations uncover security vulnerabilities.
Let us help you identify critical vulnerabilities in complex applications and application architectures and
mitigate problems before cybercrime impacts your business.
• CODE Security offers three types of security testing for many applications and services,
including simulating a real-world attack scenario. We can test web applications, internal applications,
mobile applications and do code audits and test hardware. We can even reverse engineer software
if you no longer have the source code.
fter our security audit, our CODE Security experts provide a technical report including
•A
all identified vulnerabilities along with a severity rating according to industry standards.
ADDRESS POTENTIAL SECURITY PROBLEMS

With a list of identified vulnerabilities, what do you do next? We have two suggestions:
•G
ive your internal developer team the knowledge to address security problems in your applications.
CODE Security offers two flavors of Secure Coding for Developers courses. Each course
is three days but follow the same basic outline. One course is primarily for C# developers
and the other is for Java developers. See our training page for a list of upcoming classes.
• If you cannot spare developers to overhaul your software, hire CODE Consulting
to work with your teams and we’ll address the security vulnerabilities together.
Let 2023 be the year you mitigate cyber security risk in your applications.
CODE Security experts can help you identify and correct security vulnerabilities
in your products, services and your application’s architecture. Additionally,
CODE Training’s hands-on Secure Coding for Developers training courses educates
developers on how to write secure and robust applications.
Contact us today for a free consultation and details about our services.
codemag.com/security
832-717-4445 ext. 9 • info@codemag.com
TABLE OF CONTENTS
Features
8 Chrome Debugging Tips 60 uilding an Event-Driven
B
Oh, those other browsers are okay, but they have their limits.
Sahil takes a look at debugging HTML pages and JavaScript in Chrome.
.NET Core App with
Sahil Malik Dapr in .NET 7 Core
If you need to be able to create event-driven applications quickly
14 reate Your Own SQL Compare
C and efficiently, Joydip suggests exploring Distributed Application
Runtime (Dapr).
Utility Using GetSchema() Joydip Kanjilal
Paul shows you how to retrieve and store metadata using
the GetSchema() method. It’s surprisingly easy.
Paul D. Sheriff
69 rchitects: The Case for Software
A
Leaders
27 S ome Overlooked EF Core 7 Despite our best efforts, relatively few development projects
are a raging success. Jeffrey talks about the various roles—even
Changes and Improvements non-technical ones—that every project should have if there’s
any hope of success.
You were probably swept up in the exciting new changes to Entity
Jeffrey Palermo
Framework that came out near the end of 2022. Julie told you about
the big changes and now she’s going to show you how even the little
changes are terrific.
Julie Lerman
32 lean Shave: Razor Pages

C Columns
for Web Forms Developers
74 CODA: On Responsibility: Part II
If you’re already familiar with WebForms, you’re going to love how
easy it is to use Razor Pages instead. Shawn shows you how to bring John looks at what happened with Southwest Airlines and
your development into the mid-twenty-first century. the FAA during the 2022 holiday season and puts it in context
Shawn Wildermuth of technical debt.
John V. Petersen
38 Introduction to Snowflake
Cloud databases are both the latest thing and a great way to deal
Departments
with legacy database services. Rod introduces you to a very clever tool,
Snowflake, and shows you that it’s the next best thing.
Rod Paddock
49 Kubernetes Security for Starters 6 Editorial

You’ve been hearing about Kubernetes for a while now, but have you
thought about Security? Alexander shows you how to protect your work.
Alexander Pirker
28 Advertisers Index
73 Code Compilers
US subscriptions are US $29.99 for one year. Subscriptions outside the US pay $50.99 USD. Payments should be made in US dollars drawn on a US bank. American
Express, MasterCard, Visa, and Discover credit cards are accepted. Back issues are available. For subscription information, send e-mail to subscriptions@codemag.com
or contact Customer Service at 832-717-4445 ext. 9.
Subscribe online at www.codemag.com
CODE Component Developer Magazine (ISSN # 1547-5166) is published bimonthly by EPS Software Corporation, 6605 Cypresswood Drive, Suite 425, Spring, TX 77379 U.S.A.
POSTMASTER: Send address changes to CODE Component Developer Magazine, 6605 Cypresswood Drive, Suite 425, Spring, TX 77379 U.S.A.
4 Table of Contents codemag.com

EDITORIAL
The Way of the Coder

“Good job for finishing 80% of the project. Now go get some rest because the other 80% is waiting to
be done.” I know that you’ve had that conversation. Nearly every software project inspires the same
conversation. As soon as significant development takes place, the scope increases. There are many
reasons for this. Building the software uncovers

business process improvements and the soft-
ware shows a path of opportunity that never
existed before the new development. Simply
put, you just didn’t estimate the real scope of
the work. Or maybe the customer changes the
whole focus—you still need to include the work
you already did, but now there’s a whole new
whatzit that’s now the main selling point of the
development.
Or how many times have you heard this one. “I

have a great idea for an app and if you build
it, I’ll cut you in.” I’ve had conversations with
friends and family where, as soon as they learn
that I’m a software engineer, they profess to
possess the secret recipe for untold riches if Figure 1: Finding the bug was the real reward, but this check didn’t hurt. (https://en.wikipedia.org/
only they had someone to build it. Non-coders wiki/Knuth_reward_check)
seem to think that the simplicity they see in
their interactions with completed software
apps is all it takes to make (and market) the in the wee hours of what has become Satur- code. These checks are the prized possessions
numerous apps they consume. What they fail day morning, salvaging your weekend despite of the developers they were awarded to. Knuth
to understand is just how long these applica- sleep deprivation. Later the next week, you call did this because finding the “last bug” doesn’t
tions take to build and how difficult they can Jamie to see what the customer thought and mean there aren’t more.
be to implement. And how much real work it is get the all-too-common news. The meeting got
to make something look simple. What we do is postponed to next week. “I’ll let you know.” Being a coder is a great gig. You can work from
magic to some. an office in Hawaii (where shoes are frowned
There are always productivity and project man- upon), in a bank vault in Central Oregon, or in a
Projects of value take time to design, code test, agement tools that someone who knows little high-rise office in the London financial district.
and deploy. Building software isn’t like writing to nothing about writing code thinks will be You can listen to Metal at 100 decibels or just
a term paper in college—you can’t construct the solution to all our coding woes. Every com- code away to low-fi. It’s your choice. The com-
the entire thing in a giant code binge and be pany in the industry has said, “Wow, this new piler calls and you listen… And there’s always
content with a passing grade because you con- IDE/library/tool/gizmo/widget will be a game the next project, some interesting quirky little
vinced one person that you have a tiny cor- changer. Software development will never be thing to test your creative skills against.
ner of a clue. Software has a skadillion moving the same.” As the late Fred Brooks said in his
parts. It has to be a home run or it never makes seminal book “No Silver Bullet—Essence and Such is the way of the coder…
it out of the dugout. I learn this every time I Accident in Software Engineering,” no IDE,
come up with my own wild idea. I open the library, or tool will ever change the principal
IDE, create a project, and start hacking away rules of software development. The universal Rod Paddock
like Indiana Jones in a South American jungle. principles of time, scope, and resources control
the destiny of all projects.
It doesn’t take much time before I move on to
the next great idea. You’ve probably heard “Here, let me make this
simple cosmetic change and push it to produc-
Or how about this one? “Hey, this is Jamie in tion. No need to test. It won’t affect anything.”
the marketing department. Can you please cre- And of course, it affected everything.
ate a report on the air-speed velocity of our
new shipping carrier? The customer needs it Even when your project is up and running,
Monday.” Of course, this request comes at the there will be bugs. Or rather, there will always
last minute on Friday afternoon. You head home be bugs. Check out Figure 1. This check comes
for a late night of Red Bull and Pop-Tarts and from Donald Knuth, who paid people to find
get the feature coded, tested, and deployed legitimate bugs in the LaTeX project source
6 Editorial codemag.com
CUSTOM SOFTWARE DEVELOPMENT
STAFFING TRAINING/MENTORING SECURITY
MORE THAN JUST

A MAGAZINE!
Does your development team lack skills or time to complete all your business-critical software projects?
CODE Consulting has top-tier developers available with in-depth experience in .NET,
web development, desktop development (WPF), Blazor, Azure, mobile apps, IoT and more.
Contact us today for a complimentary one hour tech consultation. No strings. No commitment. Just CODE.
codemag.com/code
codemag.com
ONLINE QUICK ID 2303021
Chrome Debugging Tips

All right, I’m going to get a lot of hate for this, but do you know what browser I use as my default browser? Safari. There. I said it.
Not Edge, not Chrome, not Firefox. Safari. Look, I have my reasons. I’m deep into the Apple ecosystem, and I really appreciate the
performance that Safari offers over any other browser. And then there are features such as keychain synch, passkeys, Apple Pay,
etc. that only Safari offers. But when it comes to debug- article. My focus will be on some of the interesting arcane
ging stuff, I despise Safari dev tools. I only use Safari dev tricks, some of which will (hopefully) pleasantly surprise
tools when I absolutely must. The dev tools I still prefer you and be a part of your dev tricks going forward. Let’s
to use are those built into the Chrome browser. And, I get rolling.
understand, Edge dev tools are now also quite similar.
Let me put it this way, I’m happy to use either Edge or
Chrome, but my muscle memory still takes me to Chrome. Supercharge with the Command Menu
One of the biggest frustrations I have with the dev tools
In this article, I’d like to share with you some beyond- is simply how powerful they are and how many com-
Sahil Malik the-obvious tips and tricks when it comes to debugging mands and possibilities they have. Seriously, don’t get
www.winsmarts.com HTML pages and JavaScript in Chrome. me wrong, you need all those facilities to do your work,
@sahilmalik but if someone wrote an end-to-end book on every fea-
ture in dev tools, they’d probably never finish the book.
Sahil Malik is a Microsoft What Are Dev Tools? Sometimes I remember that there’s a way to do something
MVP, INETA speaker,
Although I want to focus on the less obvious but highly in command tools, but I just can’t remember where that
a .NET author, consultant,
useful tips, I must get this out of the way. The main particular command was.
and trainer.
characters in this article are the dev tools, so I must in-
Sahil loves interacting troduce the main character and how to launch it. Dev For instance, when trying to understand another coder’s
with fellow geeks in real tools are built into the Chrome browser. You can launch code, I frequently find code folding useful. I vaguely re-
time. His talks and train- the browser and simply launch them with a short cut key member that there was a feature in Chrome that’s off by
ings are full of humor and (F12 on Windows or CMD_OPT_i on a Mac). Dev tools may default, to allow me to enable code folding. To be honest,
practical nuggets. launch either docked to a side in your browser or they I’ve no idea where to toggle that feature from, but I know
can be free floating. This behavior can be toggled via the I can easily find it using the command menu. Simply press
His areas of expertise are triple dot menu on the top right hand corner of dev tools, CMD_SHIFT_P on Mac or CNTRL_SHIFT_P on Windows to
cross-platform Mobile app as shown in Figure 1. show the command menu and find “Enable code folding”
development, Microsoft from there. This can be seen in Figure 2.
anything, and security As you can see, dev tools are fairly involved. They’re
and identity.
organized into various tabs, such as Elements, Console, As soon as you enable code folding, you’ll notice that all
Sources, Network, Performance, and so much more. Within logical blocks of code shown to you can be collapsed or
each tab, there’s a specific user interface designed for expanded. For instance, if you have an if/then/else state-
that section. Each of these sections is loaded with tips ment, you can collapse the “if” part so you can focus on
and tricks. Some are obvious, like you can use the con- the “else” part. Or you can collapse a large switch state-
sole window to run arbitrary JavaScript commands in the ment, or function, or really any logical part of your code.
scope of the page. Or you can use the Network tab to
record network activity.
Load Any File Quickly with
I’ll assume that you’ve used dev tools in the past and are the Open Dialog
generally familiar with basic usage of dev tools already, Modern websites are skyscrapers built with sticks, twigs,
so I won’t focus on a 101 introduction of dev tools in this and cards stuck together with chewing gum and spit. You
Figure 1: Changing how dev tools appear Figure 2: The command menu
8 Chrome Debugging Tips codemag.com

know that is an accurate description. There are days I’m
amazed that this even works, but it does. Just visit www.
google.com, for instance, go to the sources tab, and start
looking at what it took to build that simple page with
a textbox and an image. There are literally hundreds of
files there, from all sorts of domains, with all sorts of
extensions.
The problem this causes is finding a particular file on a

large complex page can be tedious. But you can greatly
simplify your task by simply hitting CMD_P on Mac or
CTRL_P on Windows to launch the open dialog. This
launches a VS code-like open file experience, and you can
simply type the name of the file to search for what you
are looking for and easily open it. This can be seen in
Figure 3.
Prettify JavaScript Files

JavaScript is complex as it is, but before it’s shipped to a
production application, it’s minified. This means that all
variable names are munched up into as small names as Figure 3: The open file experience
possible, and all whitespace and return characters are re-
moved unless absolutely necessary to keep. The end result
is that the file ends up being obfuscated so your logic
can’t be stolen. And it’s smaller, so it loads faster. And it
runs faster because it’s smaller and more optimized. You
can pretty much assume that any production application,
including yours, will use minified JavaScript.
The one big disadvantage is that such files become im-

possible to debug. The solution is pretty simple. At the
bottom of any JavaScript file, you’ll find a curious looking
button, as shown in Figure 4.
Click on that button and voila! Just like magic, you get all
the indentation, whitespace, and tabs back. Now enable
code folding using the previous command menu tip, and
you can start understanding the logic of this complex file.
The same file after prettifying and enabling code folding
looks like Figure 5.
Your Personal CodePen

JavaScript is a weird language, and no matter how good
you are at it, you’re never really good at it. When we work
on complex JavaScript applications, we frequently want to
try out things. Typically, we use sites such as codepen.io
for this. Those sites work great, especially when you wish
to share code among friends. But did you know that there’s
a CodePen equivalent built right inside of dev tools? The
advantage is obviously that you can now store the .js snip-
pets on your disk. It’s called the “snippet editor.”
To access the snippet editor, go to the sources tab, click

the “>>” menu, and look for “Snippets”. Here, you can
manage your code snippets, as can be seen in Figure 6.
Although this is great, you may be wondering about a se-

rious limitation here. These snippets are just JavaScript,
but our websites are frequently made up of HTML, CSS,
and so much more. Well, it’s no issue. The idea is that you
can have a bunch of these helper snippets on your disk
that you can use for quick debugging helper functions,
etc. But you can also pair them with any other kind of
file you wish. Figure 4: The prettify button
codemag.com Chrome Debugging Tips 9

wish to keep an eye on the value of a variable or expres-
sion. I know what you’re thinking: You can set a “watch”
variable, right? The problem is that watch will only work
if something is in scope and, effectively, in debug mode.
There’s another way. The JavaScript console lets you liter-

ally keep an eye on an expression. Let me explain with an
example.
Let’s say you have a complex JavaScript method, as be-

low.
(Math.random() + 1).
toString(36).substring(7);
Running this generates a random set of characters every

time. And you wish to be able to keep an eye on the out-
put of this as you interact with the page, or maybe not
even interact with the page. Here’s how you do it.
You open the dev tools and navigate to the “console” tab.
Figure 5: Prettified and code folded is much easier to Here, click on the “eye” to create a live expression, as
debug. shown in Figure 8.
For instance, I went ahead and stored the above snippet to In the text box, go ahead and place your live expression
a directory called “temp”, and in the same “Sources” tab to text. This could be a variable or the output of a function.
the left of snippets, there’s another tab called “Filesystem” Now, Chrome dev tools pins this value on the top and
that allows you to add a local folder to a given site. You’ll updates it constantly. This is a great way for you to keep
be shown a warning like that in Figure 7. an eye on a value as the value changes.
The advantage now is that you can store any kind of file
on your local file system. You can edit it in VSCode as you
wish. And you can see the results directly in the produc- The Chrome tool pins the value on
tion site without deploying to the server. I’ve found this the top and updates it constantly,
tip extremely helpful in not only diagnosing issues, but
also creating a bunch of helper functions. For instance, if allowing you to keep an eye
a site has a bunch of download links, I could easily write on the value as it changes.
a helper script with UX, if necessary, to help me download
all files quickly. Or I could easily insert a debugging ren-
dering UX with charts and graphs driven completely from
the local operating system. Screenshot Large Pages
Okay, this next one is a pretty amazing tip that I’ve seen
developers install tools for or do all kinds of gymnas-
Keep an Eye on a Value tics for. Little do they know, this is built right inside of
Let’s say that you’re writing a programming a game and Chrome. Sometimes you have a long page, and no matter
are trying to narrow down on a difficult bug. Almost like how big your monitor is, it requires you to scroll. What if
you wish you had four hands as you played the game, you you want to take a screenshot of the entire page, includ-
Figure 6: The snippet editor built inside of dev tools
Figure 7: Warning when adding a filesystem folder to dev tools

Figure 8: Create a live expression.
ing the areas that aren’t visible? Like I said, I’ve literally
seen developers painfully scroll, take pictures, and stitch
them together.
But it’s built right into Chrome. Again, just open dev tools Figure 9: Various screenshot options
and press CMD_SHIFT_P (Mac) or CTRL_SHIFT_P (Win-
dows) and choose to do a “full size” screenshot, as shown
in Figure 9. Doing so creates a screenshot of the entire
page and downloads it.
Now feel free to play around with the other options as

well. I especially like the “node screenshot” option,
where you can first select an element using dev tools and
choose to take a clean screenshot of only that node and Figure 10: The preserve log checkbox
its children. This is great for documentation and emailing
and really takes the hassle out of taking clean screen-
shots of complex page elements. to hide/show information as columns. Sometimes I like to
make the domain visible to eliminate issues coming from
my site vs. others. This can be seen in Figure 11.
Play with HTTP Requests
Chrome dev tools have an interesting Network tab that lets Another interesting set of columns can be seen in Figure
you do quite a bit. At its very basic, it lets you record your 12. The size column is essential to know if your caching
HTTP traffic going back and forth. Because this occurs in strategy is working as intended. This can be a make or
the user’s context, you can even see HTTPS traffic. This, of break for both functionality and performance. Time lets
course, is great for debugging issues. Sometimes you still you see which requests are too slow. You can clearly see
need to branch out to fiddler or wire shark when you wish to that the stuff that comes from cache vs. network is hun-
capture OS-level traffic or act as reverse proxy, but 90% of dreds of times faster. This is great because you can now
the time, Chrome dev tools are enough for most of my needs. easily zero in on where your bottlenecks are and see if
you can cache them.
Hidden here, though, are bunch of really cool tricks.
And finally, the waterfall lets you identify opportunities
First is the “Preserve log” checkbox, as shown in Figure 10. for parallelizing the requests to reduce the overall time. Figure 11: The domain column
The screenshot in Figure 12 is for the google.com home
By checking this checkbox, the log doesn’t get cleared out page. As you can see, they’ve done a great job at optimiz-
every time you move page to page. I find this especially ing their home page, and why wouldn’t they? They prob-
useful when diagnosing authentication issues because a ably have a team of engineers dedicated to fine tune ev-
typical sign-in process does go through multiple pages. ery nanosecond of this operation. Given how many times
In Figure 10, you can also see the “Disable cache” and
the “Throttling” UX. Those are great to diagnose hard
client-side problems and get caching out of the picture,
and to simulate low bandwidth situations.
Below this UX and not visible in the screenshot are a

number of other tools. If you don’t see the filters, click
on the funnel icon shown in blue shown in Figure 10. For
instance, in a large network log, you can separate out the
traffic you’re concerned about by using various filtering
capabilities. For instance, you can limit the UX to just
XHRs, or just Docs, so all those CSS/JS downloads don’t
interfere with your view.
In the actual network log, you can see a number of other

interesting things. First, you can right-click and choose Figure 12: The size, time, and waterfall columns

www.google.com is served a day, just caching 100 bytes click on any request and choose to copy a number of
probably means a million dollars or something ridiculous details. A common example is when I have a REST call
for them. with an access token, and I wish to debug my API, but
don’t want to go through the UX to get a request to my
Speaking of performance, if you hover over any network API. I can easily copy its cURL and simply replay it from
request, you can see a pretty nice breakdown of where the the command line as many times as I wish until the token
time was spent, in addition to the total time. This can be expires, of course. Or, even better, I can edit the request
seen in Figure 13. slightly and replay it. This can be seen in Figure 14.
Network tools are essential not just for performance but This really simplifies my task both for debugging and
also for debugging your APIs. For instance, you can right documentation.
Finally, you can also import/export the entire network trace

as a HAR file and mail it to your friend for debugging. Be
careful though, the HAR file contains sensitive information,
like passwords. But you can open the HAR file in notepad
first and clean out all passwords, etc., if you’re concerned
before mailing it out. Again, be wary that passwords can
be removed, but tokens could be valid for some duration.
Realize that what you’re emailing contains sensitive infor-
mation. I prefer to do this only for dev environments.
Record User Journeys

Frequently, you’re trying to debug issues that require you
to traverse specific flows in the page. This may involve
login, clicking around on certain links, page refreshes, etc.
You may want to share such user journeys with the team.
Or perhaps you wish to track a hard-to-measure perfor-
mance issue that spans multiple pages or replay such a
journey under different network throttling situations.
All of this is possible via Chrome dev tools. To do so, visit

the “recorder” tab in dev tools, and choose to start a new
recording. Then simply interact with the page and the
recorder records all your actions. Once recorded, you can
rename the recording, or import/export it as a puppeteer
script, to use it for acceptance tests. Or you can choose
to replay that script with performance measurement un-
der different network conditions.
Bonus Tip: Dark Mode

This one isn’t a developer tip but more of productivity
tip. There was a time when I used to be an MS Office
and Microsoft products junkie. Well, not anymore. Over
the past few years, I’ve gradually become more and more
cross-platform both in terms of business tools and dev
tools. Look, I don’t have a horse in this race and I’ll use
whatever tool is right for the job.
The other thing that has changed is age. Although my

Figure 13: Single request performance breakdown passion for technology is still there, I now use an 120hz
OLED screen to reduce glare and flickering. And my eyes
are thankful for it. Seriously, these are things I never paid
Listing 1: Dark mode script. any attention to just 10 years ago. But life slaps you fast.
var head = document.getElementsByTagName('head')[0]; I really enjoy using dark mode on an OLED screen, it really
javascript: function addcss(css) { cuts down all eye stress.
var s = document.createElement('style');
s.setAttribute('type', 'text/css'); So it really irks me that in the year 2023, we still don’t
s.appendChild(document.createTextNode(css));
head.appendChild(s); support dark mode in Google Docs. Especially when
} addcss( working late at night when every other app supports
"html{filter: invert(1) hue-rotate(180deg)}"); dark mode, and I open a Google Doc, I feel like a vam-
addcss("img{filter: invert(1) hue-rotate(180deg)}"); pire exposed to sunlight. Heck, you can read the oper-
addcss("video{filter: invert(1) hue-rotate(180deg)}");
ating system settings and see if the user prefers dark
mode, and y’know, just render it?

Here’s a trick I use to support dark mode in Chrome, even
when the underlying product doesn’t support dark mode.
In the bookmarks bar, I create a bookmark. You can name

the title whatever you want, personally I like to have a
convenient emoji like the sun or something simple that
doesn’t take up too much space on the bookmarks bar.
In the URL, I put the code shown in Listing 1 (the line

breaks are for readability). You probably want to run it
through a minifier and turn it into a single line of text.
Or you can grab a copy and paste-able version from here
https://winsmarts.com/dark-mode-in-chrome-without-
extensions-36418f57fe34.
Now, visit any site that doesn’t support dark mode. For
instance, I visited https://developers.google.com/docs/
api/samples. Once you’re subjected to the harsh sunlight
vampire style, click the bookmark you just created. It im-
mediately turns it into a nice and readable version in dark
mode, as shown in Figure 15.
I know there are extensions that can do this for me auto-

matically, but I’m always wary of extensions. They’re the
equivalent of running JS code on all your secure pages,
which isn’t a great idea. So although I can’t autorun dark
mode, at least I can switch to it with ease and read with
comfort. Figure 14: Playing with network requests
Figure 15: Dark mode works quite well.
I use this technique quite a bit, and it’s really helped me out and solve some very hard problems. Sometimes I won-
across many products. I highly recommend you trying it, der how large the Chrome dev tools team is.
if you’re a dark mode fan like I am.
What are some of your favorite Chrome dev tool tricks?
Would you like to see more such tricks?
Summary
A developer’s life isn’t easy. We deliver complex function- Do let me know. Until next time, happy debugging.
ality through HTML and JavaScript, and we need some
tools to help us out. Chrome dev tools are very powerful Sahil Malik
and have a seemingly infinite number of features. I’m
frequently amazed at the new things I learn and discover
every day.
I could easily come up with another 10 tips. Although the

tools have grown organically, they do seem well thought

Create Your Own SQL Compare

Utility Using GetSchema()
Each database system, such as SQL Server or Oracle, maintains a set of system tables that contain the metadata about the
database objects contained therein. These system tables hold table and column names, relationship data, check constraints,
and much more data. The SQL to retrieve this metadata is very different on each database server. For example, on SQL Server,
you select all tables using the sys.tables table or the a valid connection string to the constructor. Open the
Information_Schema.Tables view. To retrieve all tables connection and invoke the GetSchema() method passing
on Oracle, you use the all_tables table. In PostgreSQL, in a string of the collection you wish to return, as shown
you can use the Information_Schema.Tables view. The in the following code snippet.
problem is that if you want code that’s consistent be-
tween different databases, you must write different SQL using SqlConnection cn = new("CONNECTION STRING");
for each database system, then create an API for develop-
ers to call. cn.Open();
Paul D. Sheriff In this article, you’ll learn how to use the GetSchema() DataTable dt = cn.GetSchema("COLLECTION NAME");
psheriff@pdsa.com. method on the DbConnection class to retrieve tables,
www.pdsa.com views, columns, index, stored procedures, and more from The “COLLECTION NAME” you pass to the GetSchema() de-
any database system. This method is implemented by termines which data you get back. For example, if you
Paul Sheriff has been in
each data provider to retrieve schema information in a pass “Tables”, you get the list of tables within the da-
the IT industry since 1985.
generic fashion. What do you do with this information? tabase specified in your connection string. If you pass
In that time, he has suc-
You can present column names to your user to let them “Columns”, you get a list of all the columns within the
cessfully assisted hundreds
of companies architect select columns to filter on for a report. You can use it to database specified in your connection string.
software applications to build your own code generator. You can even use it to cre-
ate a SQL comparison tool. Follow along with this article
solve their toughest busi-
ness problems. Paul has to see how easy it is to use GetShema() to accomplish The Same but Different
been a teacher and mentor these various tasks. Although you can retrieve columns, tables, indexes, etc.
through various media from each provider, be aware that they don’t always re-
such as video courses, turn the same column names for each call. For example,
blogs, articles, and speak- Introducing the GetSchema() Method when you request all tables from SQL Server, GetSchema()
ing engagements at user GetSchema() is a virtual method on the DbConnection returns the columns TABLE_CATALOG, TABLE_SCHEMA,
groups, and at conferences class in the .NET Framework. This method is overridden by TABLE_NAME, and TABLE_TYPE to describe the tables. In
around the world. Paul has the SQL Server, Oracle, ODBC, and OLE DB, and other data Oracle, the columns OWNER, TABLE_NAME, and TYPE are
multiple courses in the providers. Each provider uses the appropriate metadata used to describe the tables in the database. The ODBC
www.pluralsight.com library tables in their respective servers to retrieve the metadata and OLE DB providers provide different column names.
(https://bit.ly/3gvXgvj) for each call you make to the GetSchema() method. The Not all providers return the same data. For a complete list
and on Udemy.com following list is an example of some of the metadata that of the SQL Server, Oracle, ODBC, and OLE DB, check out
(https://bit.ly/3WOK8kX) you may be able to return from your database server. How the Microsoft documentation at https://bit.ly/3V3CMZv.
on topics ranging from C#, many of these items you can retrieve is dependent on the
LINQ, JavaScript, Angular,
data provider you’re using. I wish Microsoft had provided a specification for what
MVC, WPF, XML, jQuery,
was returned by each call, as that would have made it
and Bootstrap.
• Columns easier if you need to target different database systems
• Databases at your work. I guess Microsoft thinks most developers
• Data types typically only work with a single system. For those of us
• Foreign keys who must work with multiple database systems, I recom-
• Index columns mend mapping the data returned from each provider to
• Indexes your own schema classes. You can then create your own
• Procedure parameters collection of schema classes for each type of metadata
• Procedures you need to retrieve. I’ll show you how to do this later
• Reserved words in this article.
• Tables
• Users Adding Restrictions
• User-defined types Most of the calls to GetSchema() support a second param-
• Views eter called restrictions. The restrictions are a string array
• View columns with anywhere from one to five elements. For example,
let’s say you wish to restrict the columns to retrieve from
To use the GetSchema() method, create an instance of the a single table, declare a four-element array, and fill in
SqlConnection or OracleConnection, or your own provid- the second and third elements with the schema and table
er’s implementation of the DbConnection class, and pass name, as shown in the following code.
14 Create Your Own SQL Compare Utility Using GetSchema() codemag.com

string?[] restrictions = new string?[4];
restrictions[1] = "SalesLT";
restrictions[2] = "Product"; ®
DataTable dt = cn.GetSchema("Columns",
restrictions);
Each collection has its own restrictions array, but most

of the arrays use the catalog (database) name as the first
element, the owner (schema) as the second element, and Instantly Search
Terabytes
a database object name as the third element. You should
note that there’s no way to specify a wildcard search us-
ing the GetSchema() method. This means you can’t have
it return all tables that start with “Prod”, for example.
Get All Tables and Views

If you want to follow along with this article, create a new
console application in VS 2022 or in VS Code named Sche-
maInfo. I’m using .NET 6/7 for these samples, but the dtSearch’s document filters support:
GetSchema() method is also available in the later versions • popular file types
of the .NET Framework, so the code should work with some
minor changes. After creating a console application, go • emails with multilevel attachments
into the NuGet Package Manager and add the System.Data.
SqlClient package if you’re using SQL Server as I’m going
• a wide variety of databases
to be using in this article, or select the data provider for • web data
your database server. Open the Program.cs file and replace
the contents with the code shown in Listing 1.
Add two using statements to bring in the namespaces for

System.Data and System.Data.SqlClient. The GetSchema()
Over 25 search options including:
method returns an instance of the DataTable class that’s • efficient multithreaded search
contained in the System.Data namespace. The SqlConnec-
tion class is contained within the System.Data.SqlClient • easy multicolor hit-highlighting
namespace. Create a string variable named conn and set • forensics options like credit card search
the connection string to one that references one of your
SQL Server (or your specific database server) instances
and a database within that instance. I’m using the Ad-
ventureWorksLT sample database located on my local
computer. Developers:
Next, create an instance of a SqlConnection class passing
• SDKs for Windows, Linux, macOS
in the connection string within a using block. Open the • Cross-platform APIs cover C++, Java
SqlConnection object and call the GetSchema() method and recent .NET (through .NET 6)
passing in the string “Tables”. The GetSchema() method
goes out to the database and uses system tables and/or • FAQs on faceted search, granular data
views to retrieve the list of tables in that database. classification, Azure, AWS and more
Each .NET Data Provider calls different system tables/
views to retrieve the list of tables. In SQL Server, for in-
stance, the INFORMATION_SCHEMA.TABLES view is used
to retrieve the list of tables. If you’re using Oracle, ODBC,
or OLE DB, those providers use different methods to re-
trieve the list of tables. Be aware that each provider can Visit dtSearch.com for
return different column names and may not return the • hundreds of reviews and case studies
same number or names of columns.
• fully-functional enterprise and
When returning the list of tables from SQL Server, four developer evaluations
columns are returned: the catalog/database, the owner/
schema, the table, and the type of object (BASE_TABLE or
VIEW). When you pass “Tables” to the GetSchema() meth- The Smart Choice for Text
od, it returns both tables and views because a view is
really just a read-only table. If you want to retrieve views
Retrieval® since 1991
only, you can pass “Views” to the GetSchema() method.
If you want to retrieve only tables, you need to pass a dtSearch.com 1-800-IT-FINDS
second parameter to the GetSchema() method, as you will
see in the next section.
codemag.com Create Your Own SQL Compare Utility Using GetSchema() 15

The last few lines of code in Listing 1 set up a format The array you pass is different for each method but is
string with how much space to allow for each column always a string array with one to five elements in it.
in the DataTable object. Within curly braces, you specify For most calls, the first element is the catalog/database
the column number and how much space to allow. Use name, and the second parameter is the owner/schema
a minus sign (-) to specify that you want the data in name. After that, the elements you pass are dependent
the column to be left-aligned. Iterate over each row and on what you’re calling. Leave an element null if you don’t
display each column on the console. To figure out how wish to restrict the return values based on that item. For
many and which columns are returned from each call to example, if you don’t care to restrict the list of tables by
GetSchema(), check the documentation online, or call the owner/schema, set that value to null.
each one and view the columns that are returned.
In Listing 2, you see an example of a restriction array
Try It Out that’s used to filter the tables returned. There are four
Create the console application, add the System.Data. restrictions you can specify for tables: catalog/database,
SqlClient package (or the applicable package for your da- owner/schema, table name, and a string with either
tabase server), and add the code shown in Listing 1 to “BASE TABLE” or “VIEW” within it. As I mentioned previ-
the Program.cs file. Modify the connection string to point ously, the one thing missing from this interface is the
to your server and a database on that server. Run the ap- ability to use a wildcard to retrieve all tables that start
plication and you should see a list of tables displayed like with “CU”, for example. After creating the four element
Figure 1. The list of tables you get back will be different array, pass it as the second argument to the GetSchema()
from mine as it will be based on which database you are method. In Listing 2, you only get those tables back that
connected to. are in the SalesLT schema and are base tables, not views.
Try It Out
Get Tables within a Specific Schema Add the restrictions array shown in Listing 2 to the code
As mentioned previously, you can pass in an array of val- you wrote in the Program.cs and run the application to
ues to the second parameter of the GetSchema() method. see that you now only return tables within the SalesLT
schema. If you’re using a different database from Adven-
tureWorksLT, adjust the schema/owner name to what is
Listing 1: Display all tables in a database using the GetSchema() method applicable to your database.
using System.Data;
using System.Data.SqlClient;
Get Views Only
string conn = "Data Source=Localhost; There are two ways to retrieve views within a database.
Initial Catalog=AdventureWorksLT;
Integrated Security=True;"; You can set the fourth element to “VIEW” in Listing 2,
or you can call GetSchema() method using “Views” as the
using SqlConnection cn = new(conn); first argument, as shown in Listing 3. Be aware that the
columns returned when you use “Views” as the first argu-
cn.Open();
ment are different than when you use “Tables”. You still
// Get All Tables in the Database get the catalog/database, schema, and view name, but
DataTable dt = cn.GetSchema("Tables"); you also get CHECK_OPTION and IS_UPDATABLE columns
in SQL Server. In Oracle, you get many other columns as
// Display Column Names well, and you also get the SQL text for the view. That’s
string format = "{0,-20}{1,-10}{2,-35}{3,-15}";
Console.WriteLine(format, "Catalog", "Schema", something I wish Microsoft had included in this call to
"Name", "Type"); GetSchema() for SQL Server.
// Display Data Try It Out
foreach (DataRow row in dt.Rows) {
Console.WriteLine(format, Change the code you wrote in the Program.cs to match
row["TABLE_CATALOG"], row["TABLE_SCHEMA"], the code shown in Listing 3 and run the application to
row["TABLE_NAME"], row["TABLE_TYPE"]); see just the views from your database.
}
Get Columns
When you request column information, 21 discreet pieces
of information about the column are returned when using
SQL Server, but the Oracle provider only returns nine piec-
es of information. The ODBC and OLE DB providers return
21 and 33 data points for a column, respectively. The data
points in common among all providers are the catalog/da-
tabase, owner/schema, column name, data type, length,
precision, scale, and nullable. Listing 4 shows the code
you write to retrieve column data from a SQL Server.
Try It Out
Change the code you wrote in the Program.cs to match
the code shown in Listing 4 and run the application to
Figure 1: GetSchema() can return a list of all tables and views in a database. see all the columns in your database.

Listing 2: Display tables within a schema by using the restrictions array
using System.Data; // Use 'BASE TABLE' or you get views too
using System.Data.SqlClient; restrictions[3] = "BASE TABLE";
string conn = "Data Source=Localhost; // Get All Tables in the Database

Initial Catalog=AdventureWorksLT; DataTable dt = cn.GetSchema("Tables",
Integrated Security=True;"; restrictions);
using SqlConnection cn = new(conn); // Display Column Names

string format = "{0,-20}{1,-10}{2,-35}{3,-15}";
cn.Open(); Console.WriteLine(format, "Catalog", "Schema",
"Name", "Type");
// restrictions[0] = Catalog/Database
// restrictions[1] = Owner/Schema // Display Data
// restrictions[2] = Table Name foreach (DataRow row in dt.Rows) {
// restrictions[3] = Table Type Console.WriteLine(format,
string?[] restrictions = new string?[4]; row["TABLE_CATALOG"], row["TABLE_SCHEMA"],
restrictions[0] = null; row["TABLE_NAME"], row["TABLE_TYPE"]);
restrictions[1] = "SalesLT"; }
restrictions[2] = null;
Get Columns for a Single Table Listing 3: Pass "Views" to the GetSchema() method to retrieve all views in the database
The code in Listing 5 shows the restrictions you use to using System.Data;
filter the column data. The first element in the restric- using System.Data.SqlClient;
tions array is the catalog/database name. The second ele-
ment is the schema name. The third element is the table string conn = "Data Source=Localhost;
Initial Catalog=AdventureWorksLT;
name. In the code, I’m only setting the schema and table Integrated Security=True;";
name to just list those columns that make up that one
table. The fourth element can be a specific column name using SqlConnection cn = new(conn);
if you only need the data for a single column. cn.Open();
Try It Out // Get All Views in the Database

Change the code you wrote in the Program.cs to match DataTable dt = cn.GetSchema("Views");
the code shown in Listing 5 and run the application to // Display Column Names
see the columns in the Product table. If you’re using a string format = "{0,-20}{1,-10}{2,-35}{3,-15}";
different database, just substitute a different schema and Console.WriteLine(format, "Catalog", "Schema",
"Name", "Check Option");
table name into the second and third elements of the
restrictions array. // Display Data
Console.WriteLine(format,
Get Stored Procedures and Functions row["TABLE_CATALOG"], row["TABLE_SCHEMA"],
row["TABLE_NAME"], row["CHECK_OPTION"]);
The call to GetSchema() to return stored procedures is }
similar to the call to return tables. It returns stored
procedures and functions just like the tables call returns
both tables and views. The code in Listing 6 is just like
Listing 4: Retrieve column data using GetSchema() to retrieve column names and data types
the previous functions, except you pass “Procedures” to
the GetSchema() method. The columns returned include using System.Data;
the catalog/database, owner/schema, routine name, using System.Data.SqlClient;
routine type (procedure or function), the date created, string conn = "Data Source=Localhost;
and the date the routine was last altered. I’m only dis- Initial Catalog=AdventureWorksLT;
playing a few of these columns in the code provided in Integrated Security=True;";
Listing 6. using SqlConnection cn = new(conn);
Try It Out cn.Open();

Change the code you wrote in the Program.cs to match
// Get All Columns in the Database
the code shown in Listing 6 and run the application to DataTable dt = cn.GetSchema("Columns");
see all the stored procedures and functions in your da-
tabase. string format = "{0,-20}{1,-30}{2,-20}{3,-20}";
// Display Column Names

Get Stored Procedures Only Console.WriteLine(format, "Table Schema",
The problem with the above code is that it gives you both "Table Name", "Column Name", "Data Type");
stored procedures and functions. You need to pass the
// Display Data
second parameter to this call and specify “PROCEDURE” foreach (DataRow row in dt.Rows) {
or “FUNCTION” as the fourth element of the restrictions Console.WriteLine(format, row["TABLE_SCHEMA"],
array. Add the code shown below prior to calling the row["TABLE_NAME"], row["COLUMN_NAME"],
row["DATA_TYPE"]);
GetSchema() method, and add this restrictions array as }
the second parameter in the call to GetSchema().

Listing 5: The column restrictions include catalog, owner, table and column name
using System.Data; restrictions[3] = null;
// Get Columns for a Specific Table
string conn = "Data Source=Localhost; DataTable dt = cn.GetSchema("Columns",
Initial Catalog=AdventureWorksLT; restrictions);
Integrated Security=True;";
string format = "{0,-20}{1,-30}{2,-20}{3,-20}";
using SqlConnection cn = new(conn);
cn.Open(); Console.WriteLine(format, "Table Schema",
"Table Name", "Column Name", "Data Type");
// restrictions[0] = Catalog/Database
// restrictions[1] = Owner/Schema // Display Data
// restrictions[2] = Table/View Name foreach (DataRow row in dt.Rows) {
// restrictions[3] = Column Name Console.WriteLine(format, row["TABLE_SCHEMA"],
string?[] restrictions = new string?[4]; row["TABLE_NAME"], row["COLUMN_NAME"],
restrictions[0] = null; row["DATA_TYPE"]);
restrictions[1] = "SalesLT"; }
restrictions[2] = "Product";
Listing 6: The call to get procedures returns both stored procedures and functions
using System.Data; DataTable dt = cn.GetSchema("Procedures");
string conn = "Data Source=Localhost; string format = "{0,-12}{1,-40}{2,-12}{3,-30}";
Initial Catalog=AdventureWorksLT; Console.WriteLine(format, "Schema",
Integrated Security=True;"; "Procedure Name", "Type", "Created");
using SqlConnection cn = new(conn); // Display Data

cn.Open(); Console.WriteLine(format,
row["ROUTINE_SCHEMA"], row["ROUTINE_NAME"],
// Get the Stored Procedures and row["ROUTINE_TYPE"], row["CREATED"]);
// Functions in the Database }
Listing 7: Only when using SQL Server can you get the complete list of databases from Try It Out
your server instance Add the code shown above into the appropriate location
using System.Data; in the Program.cs file and run the application to see just
using System.Data.SqlClient; the stored procedures within your database.
string conn = "Data Source=Localhost;
Get a List of Databases on SQL Server
using SqlConnection cn = new(conn); In SQL Server, there’s the concept of a database within a
cn.Open(); server. In Oracle, the database is the instance of the Ora-
cle server you connect to. So, the only database provider
// Get the Databases in SQL Server you can pass the string “Databases” to the GetSchema()
DataTable dt = cn.GetSchema("Databases");
method is the SQL Client provider. Listing 7 shows the
// Display Column Names code to get the list of databases within a SQL Server in-
string format = "{0,-25}{1,-10}{2,-25}"; stance. Notice that the connection string does not require
Console.WriteLine(format, "Database Name", the Initial Catalog key/value pair; you just need to con-
"DB ID", "Created");
nect to the server with valid credentials to retrieve the
// Display Data list of databases. Only three columns are returns from this
foreach (DataRow row in dt.Rows) { call: database_name, dbid, and create_date.
row["database_name"], row["dbid"],
row["create_date"]); Try It Out
} If you’re using SQL Server, modify the code you wrote in
the Program.cs to match the code shown in Listing 7 and
run the application to see all the databases in your SQL
string?[] restrictions = new string?[4]; Server instance.
restrictions[2] = null; Get Users
restrictions[3] = "PROCEDURE"; When using SQL Server or Oracle, you can retrieve the list
of users by passing “Users” to the GetSchema() method.
// Get Stored Procedures Only On each system, you get the user name, the user ID, and
DataTable dt = cn.GetSchema("Procedures", the creation date. On SQL Server, you also get the last
restrictions); time the user information was modified. The code in

Listing 8 shows you how to retrieve all users within the Listing 8: You can retrieve user data on both SQL Server and Oracle
database AdventureWorksLT. If you eliminate the Initial
using System.Data;
Catalog key/value pair, you can retrieve all users in the using System.Data.SqlClient;
SQL Server instance.
string conn = "Data Source=Localhost;
Try It Out Initial Catalog=AdventureWorksLT;
Change the code you wrote in the Program.cs to match the
code shown in Listing 8 and run the application to see all using SqlConnection cn = new(conn);
the users in your database. Try running the code with and
cn.Open();
without the Initial Catalog to see the differences.
// Get the Users
DataTable dt = cn.GetSchema("Users");
Get Metadata about GetSchema()
Method string format = "{0,-25}{1,-10}{2,-25}";
There are a lot more objects you can retrieve from the da- Console.WriteLine(format, "User Name",
"User Id", "Created");
tabase other than tables, views, procedures, and columns.
But I think you see the pattern on how to retrieve the other // Display Data
items I listed at the beginning of this article. Let’s now look foreach (DataRow row in dt.Rows) {
at getting information about the GetSchema() method itself. row["user_name"], row["uid"],
row["createdate"]);
Because you’re passing a string as the first parameter to }
the GetSchema() method, you may be wondering how I
knew what string value to pass to get each collection. It
turns out that if you call GetSchema() with no parameters Listing 9: Call GetSchema() with no parameters to retrieve information about each
(or with the parameter “MetaDataCollections”), it returns collection name you can retrieve
a list of all of the collection names you may use, how using System.Data;
many restrictions there are for each collection, and how
many identifier parts are used. string conn = "Data Source=Localhost;
In the code shown in Listing 9, I’m using the SQL Serv- using SqlConnection cn = new(conn);
er provider, so the list of collections that are returned
(shown in Figure 2) are different if you are using an cn.Open();
Oracle, ODBC, or OLE DB provider. // Get the Metadata Collections
DataTable dt = cn.GetSchema();
Try It Out //DataTable dt =
Change the code you wrote in the Program.cs to match // cn.GetSchema("MetaDataCollections");
the code shown in Listing 9 and run the application to // Display Column Names
see all the collections you can pass to the GetSchema() string format = "{0,-25}{1,-20}{2,-20}";
method. When running SQL Server, you should see a list Console.WriteLine(format, "Collection Name",
"# of Restrictions", "# of Identifier Parts");
shown in Figure 2.
// Display Data
Identifier Part foreach (DataRow row in dt.Rows) {
The last column shown in Figure 2 is called Identifier Part. row["CollectionName"],
An identifier part refers to how many composite parts are row["NumberOfRestrictions"],
needed to fully qualify a database object. For example, in row["NumberOfIdentifierParts"]);
}
SQL Server, to identify a table, it takes three parts in the
form of DatabaseName.SchemaName.TableName. For Or-
acle, it only takes two parts: OwnerName.TableName. So
the numbers you see in Figure 2 may be different if you’re
running a provider other than SQL Server.
Get Metadata about Restrictions

Other questions you may have include how you know how
many elements and what elements you can set in the re-
strictions array. Once again, you pass an argument to the
GetSchema() method, the string “Restrictions”. Calling
GetSchema() with this argument returns each collection
name and any corresponding values you can set, as shown
in Figure 3. Methods that are self-documenting like this
are a great idea and I wish Microsoft would do this with
more of their APIs.
In the code shown in Listing 10, you get five columns re-
turned, as shown in Figure 3. The first column is the col- Figure 2: A list of all the collections that GetSchema() can retrieve when using
lection name, the second column is the restriction name the SQL Server provider

Figure 3: Retrieve a list of all restrictions for each collection name using the GetSchema() method.
Listing 10: The GetSchema() method tells you which restrictions are available for each collection
using System.Data; {2,-20}{3,-25}{4,-10}";
using System.Data.SqlClient; Console.WriteLine(format, "Collection Name",
"Restriction Name", "Parameter Name",
string conn = "Data Source=Localhost; "Default", "Number");
// Display Data
using SqlConnection cn = new(conn); foreach (DataRow row in dt.Rows) {
cn.Open(); row["CollectionName"],
row["RestrictionName"],
// Get All Restrictions row["ParameterName"],
DataTable dt = cn.GetSchema("Restrictions"); row["RestrictionDefault"],
row["RestrictionNumber"]);
// Display Column Names }
string format = "{0,-25}{1,-20}
that tells you what you would put into each element of to a development database. When you’re ready to move
the array. The third and fourth column aren’t that impor- your code to the QA team for them to check, you not only
tant, but the fifth column tells you into which element need to create a build of your application, but you also
number you place the data you want to filter upon. need to update the QA database so it matches your devel-
opment database. Once your QA process is complete, you
Try It Out need to determine the differences between your QA data-
Change the code you wrote in the Program.cs to match base and the production database. There are a few tools on
the code shown in Listing 10 and run the application to the market that you can purchase to accomplish this task.
see all the restrictions that are applicable for your data However, you have almost all the tools you need from the
provider. Depending on what provider you’re running, you code shown in this article to create your own SQL compare
might see different values from those shown in Figure 3. utility. You just need a few classes and some LINQ queries
to build this compare utility.
SQL Compare Utility Create Schema Entity Classes
Now that you ‘ve seen how to retrieve metadata about Because each data provider returns different columns in
your databases such as tables and columns, what can you their DataTable objects, it’s a good idea to create some
do with this information? As I mentioned at the begin- classes to hold the table and column information. You
ning of the article, you can present a list of fields to filter could also create classes to hold index, procedure, func-
upon for reports or build a code generator. Another idea tion, and view information as well, but let’s just build a
is to build a tool that checks to see what tables, columns, couple so you can see the design pattern. You can then
indexes, etc. are missing between a development, a QA, add additional classes as you need them.
and production databases.
Create a class named SchemaBase that holds common
Think about a typical software development lifecycle. As properties for different schema items, as shown in the
you develop your application, you make database changes following code snippet.

#nullable disable public string ColumnName { get; set; }
public int OrdinalPosition { get; set; }
namespace SchemaInfo; public string DataType { get; set; }
public class SchemaBase public override string ToString()

{ {
public string Catalog { get; set; } return $"{base.ToString()}
public string Schema { get; set; } .{ColumnName} ({OrdinalPosition})";
public string TableName { get; set; } }
}
public override string ToString()
{ Create Compare Helper Class
return $"{Schema}.{TableName}"; Create a static class named SqlServerCompareHelper in
} your console application. This class is going to have all
} static methods to perform the comparisons. Make the new
class file look like the following code snippet.
I’m not going to worry about initializing every non-
nullable string, so I’m adding the #nullable disable di- #nullable disable
rective at the top of the file. This class is going into
the namespace SchemaInfo. Add three properties to hold using System.Data;
the catalog/database: the owner/schema, and the table
name. I like overriding the ToString() method to make namespace SchemaInfo;
looking at classes easier when in a breakpoint in Visual
Studio. public static class SqlServerCompareHelper
{
Next, create a class named TableSchema that inherits }
from SchemaBase. The only other property you need for
a table, other than the three in the SchemaBase class, Add a method named GetData() to which you pass a con-
is the TableType property. Put this class into the Sche- nection string and the collection name to retrieve. This
maInfo namespace as well. method creates a new connection and opens that con-
nection to the database. It then calls the GetSchema()
#nullable disable method, passing in the collection name. The DataTable
returned from GetSchema() is returned from this method.
namespace SchemaInfo;
public static DataTable GetData(
public class TableSchema : SchemaBase string conn, string name)
{ {
public string TableType { get; set; } using SqlConnection cn = new(conn);
}
cn.Open();
Finally, create a class named ColumnSchema that also
inherits from the SchemaBase class and adds three // Get the Meta Data
more properties: ColumnName, OrdinalPosition, and return cn.GetSchema(name); Getting the Sample Code
DataType. Once again, override the ToString() method to }
You can download the sample
show information about the column when using the data code for this article by visiting
viewer in Visual Studio or when passing an instance of Map DataTable Columns to Schema Classes www.CODEMag.com under
this class to the Console.WriteLine() method. Add a TableSchemaToList() method (Listing 11) to the the issue and article, or by
SqlServerCompareHelper class. This method takes the visiting www.pdsa.com/
#nullable disable DataTable from the call to the GetSchema(“Tables”) meth- downloads. Select “Articles”
od and builds a generic list of TableSchema objects. Each from the Category drop-
namespace SchemaInfo; iteration through the rows of the DataTable creates an down. Then select “Create
instance of a TableSchema class and maps the columns Your Own SQL Compare
public class ColumnSchema : SchemaBase from the DataTable to each property in the TableSche- Utility Using GetSchema()”
{ ma class. If you’re using a data provider other than SQL from the Item drop-down.
Listing 11: Map the columns from the DataTable to create a list of TableSchema objects
private static List<TableSchema> TableName = row["TABLE_NAME"].ToString(),
TableSchemaToList(DataTable dt) TableType = row["TABLE_TYPE"].ToString()
{ };
List<TableSchema> ret = new();
ret.Add(entity);
foreach (DataRow row in dt.Rows) { }
TableSchema entity = new()
{ return ret;
Catalog = row["TABLE_CATALOG"].ToString(), }
Schema = row["TABLE_SCHEMA"].ToString(),

Server, modify the columns I’m using in the DataTable to one is to the database with the changes made during
the ones returned from the GetSchema() method of your development, the second one is to the database you wish
provider. to check to see if the changes have been made or not.
Add a method named ColumnSchemaToList() (Listing 12) Pass in two different connection strings to this method.
to the SqlServerCompareHelper class. In this method, you Create three generics lists of TableSchema classes. One is
take the results from the call to the GetSchema(“Columns”) used as the return value, the other two hold the list of
method and turn each row of the DataTable into a generic TableSchema classes returned after building the DataTa-
list of TableColumn objects. If you’re using a data pro- bles from each database by calling GetSchema(“Tables”)
vider other than SQL Server, modify the column names and turning those into the generic lists by calling the
in the DataTable. I’m using the ones returned from the TableSchemaToList() method.
GetSchema() method of your provider.
Instead of using two different servers and two different
Create Table Compare Method databases, I’m going to use the same database, but I’m
Add a method named CompareTables() (Listing 13) to going to simulate that the target database is missing a
perform the comparison of the tables between two differ- few items. I do this by removing a few items using the
ent databases on two different servers. In this method, RemoveRange() method on the targetList variable. The
you pass in two different connection strings. The first data within the sourceList and the targetList collections
are now different.
Listing 12: Map the columns from the DataTable to create a list of TableColumn objects To retrieve the list of what items are missing from the
private static List<ColumnSchema> target database, you can employ the LINQ ExceptBy()
ColumnSchemaToList(DataTable dt) method. This method tells you which items are missing
{ from one collection to another. The ExceptBy() method
List<ColumnSchema> ret = new(); is a generic method so the first generic type you supply
foreach (DataRow row in dt.Rows) { is the type of data to return, in this case that’s Table-
ColumnSchema entity = new() Schema. The second generic type to pass is the key type
{ you’re going to be using to compare the data between the
Catalog = row["TABLE_CATALOG"].ToString(), source and the target lists.
Schema = row["TABLE_SCHEMA"].ToString(),
TableName = row["TABLE_NAME"].ToString(),
ColumnName = row["COLUMN_NAME"].ToString(), The ExceptBy() method needs two lists of data, so two
arguments are passed to it. Create the first argument
OrdinalPosition = by selecting all rows from the targetList collection and
Convert.ToInt32(row["ORDINAL_POSITION"]), turning each row into an anonymous object with all four
DataType = row["DATA_TYPE"].ToString(),
}; properties. The second argument is a lambda expression
created by turning each row from the sourceList into an
ret.Add(entity); anonymous object with four properties that match the
} first list. Remember that when using comparison methods
return ret; in LINQ, unless you’ve implemented an EqualityComparer
} class for the TableSchema class, the method uses refer-
ence comparison to see if one object reference is equal
to the other. That won’t work, so you must create anony-
mous objects to compare all four properties from one col-
Listing 13: Use LINQ to compare two collections to determine what tables are missing
lection to the other.
from a database
public static List<TableSchema>
CompareTables(string connSource, Create Column Compare Method
string connTarget) Now that you’ve seen how to perform comparisons us-
{ ing the LINQ ExceptBy() method, create other comparison
List<TableSchema> ret = new(); methods to find the differences between all the objects
List<TableSchema> sourceList;
List<TableSchema> targetList; between one database and another. In Listing 14, you
see a method named CompareColumns() that compares all
sourceList = TableSchemaToList( the columns from one database to another. I’m just going
GetData(connSource, "Tables")); to write these two comparison methods for you. You now
targetList = TableSchemaToList(
GetData(connTarget, "Tables"));
have the design pattern you can follow to perform com-
parisons between other database objects such as indexes,
// Simulate missing tables stored procedures, views, etc.
targetList.RemoveRange(2, 3);
// Use ExceptBy to locate differences

Try It Out
// between the two lists Modify the Program.cs file to match the code shown be-
ret = sourceList.ExceptBy<TableSchema,object>( low and run the application to see the simulated differ-
targetList.Select(row => new {row.Catalog, ences between tables from one database to another.
row.Schema,row.TableName,row.TableType}),
row => new {row.Catalog, row.Schema,
row.TableName, row.TableType}).ToList(); using SchemaInfo;
return ret; string conn = "Data Source=Localhost;

} Initial Catalog=AdventureWorksLT;

Integrated Security=True;"; Listing 14: Use LINQ to compare two collections to determine what columns are
missing from a database
List<TableSchema> tables = public static List<ColumnSchema>
SqlServerCompareHelper CompareColumns(string connSource,
.CompareTables(conn, conn); string connTarget)
foreach (TableSchema item in tables) { {
List<ColumnSchema> ret = new();
Console.WriteLine(item);
List<ColumnSchema> sourceList;
} List<ColumnSchema> targetList;
Now, change the last few lines of code to retrieve the sourceList = ColumnSchemaToList(
differences between columns and run the application to GetData(connSource, "Columns"));
targetList = ColumnSchemaToList(
see the differences. GetData(connTarget, "Columns"));
List<ColumnSchema> columns = // Simulate missing columns

SqlServerCompareHelper targetList.RemoveRange(1, 5);
.CompareColumns(conn, conn);
// Use ExceptBy to locate
// differences between the two lists
foreach (ColumnSchema item in columns) { ret = sourceList
Console.WriteLine(item); .ExceptBy<ColumnSchema, object>(
} targetList.Select(row => new {
row.Catalog, row.Schema,
row.TableName, row.ColumnName,
Query Collections Using T-SQL row.OrdinalPosition, row.DataType }),
row => new { row.Catalog, row.Schema,
Although the GetSchema() method is a very useful tool, row.TableName, row.ColumnName,
it doesn’t provide you with all of the metadata you might row.OrdinalPosition, row.DataType })
wish to retrieve. Almost all database servers provide .ToList();
you with much more information than is returned from return ret;
GetSchema(). For example, in SQL Server, if you select all }
columns from the sys.tables system table, you get a ton
of information about the table that GetSchema() doesn’t
return. For example, you can get the creation date, last
Listing 15: When using T-SQL you can add wildcard capability to your code to display tables
modified date, the maximum IDENTITY value generated,
and many more items. If you wish to retrieve check con- public static void DisplayTables(string conn,
straints, column privileges, or if you wish to join two string? schema, string? table)
{
or more of the system tables to retrieve a table and all DataTable dt = new();
of its related tables, there’s no way to accomplish these string sql = "SELECT * FROM
tasks using the GetSchema() method. As I also mentioned INFORMATION_SCHEMA.TABLES";
previously, there’s no way to use wildcards to retrieve all sql += " WHERE TABLE_TYPE = 'BASE TABLE'";
tables or columns that match a specific criterion. if (schema != null) {
sql += $" AND TABLE_SCHEMA LIKE '{schema}%'";
}
Instead of using the GetSchema() method to return meta- if (table != null) {
data, let’s look at using T-SQL and ADO.NET to retrieve sql += $" AND TABLE_NAME LIKE '{table}%'";
the same metadata, but solve some of the issues I just }
mentioned. Add a new class named TSqlHelper to your
using SqlDataAdapter da = new(sql, conn);
console application as shown in the code snippet below.
da.Fill(dt);
using System.Data;
using System.Data.SqlClient; // Display Column Names
string format = "{0,-10}{1,-35}{2,-15}";
Console.WriteLine(format, "Schema",
namespace TSqlSchemaInfo; "Name", "Type");
public static class TSqlHelper // Display Data

{ foreach (DataRow row in dt.Rows) {
public static void DisplayTables(string conn, row["TABLE_SCHEMA"],
string? schema, string? table) row["TABLE_NAME"],
{ row["TABLE_TYPE"]);
} }
} }
Get Tables
Fill in the DisplayTables() method with the code shown parameter to see if it’s null, and if it isn’t, add the AND
in Listing 15. This method accepts three parameters: statement to check for where the TABLE_SCHEMA column
a connection string, an optional schema name, and an is like the value passed in. Add the percent sign (%) after
optional table name. For this sample, I’m using the In- the schema variable so it can find all schemas that match
formation_Schame.Tables view, but I could have used the beginning of the schema value. Feel free to modify
the sys.tables system table as well. Check the schema this wildcard to work as you see fit.

Next, check the table parameter to see if it’s null, and approach than the one used by the GetSchema() method.
if it isn’t, add the AND statement to check where the The rest of the code in this method is standard ADO.NET.
TABLE_NAME is like the value passed in. You now have Use the SqlDataAdapter to submit the SQL and retrieve a
very flexible code to check for both a schema name and DataTable filled with the metadata. Display the column
table name, just a schema name, just a table name, or headings, then iterate over the rows in the DataTable and
don’t check for either and have all tables in all schemas display the table information found.
displayed from this method. This is a much more flexible
Try It Out
Now that you have this method created in the TSqlHelper
Listing 16: Check constraints cannot be retrieved using GetSchema(), so write your class, open the Program.cs file and modify it to look like
own code to retrieve this metadata the following:
public static void DisplayCheckConstraints(
string conn, string? schema) using TSqlSchemaInfo;
{
DataTable dt = new();
string sql = "SELECT * FROM string conn = "Data Source=Localhost;
INFORMATION_SCHEMA.CHECK_CONSTRAINTS"; Initial Catalog=AdventureWorksLT;
if (schema != null) { Integrated Security=True;";
sql += $" WHERE CONSTRAINT_SCHEMA
LIKE '{schema}%'";
} // Get all Tables
TSqlHelper.DisplayTables(conn, null, null);
Run the console application and you should see a list of
da.Fill(dt);
all tables in your database. You can also pass in a schema
// Display Column Names name as the second parameter to display all tables just
string format = "{0,-10}{1,-40}{2,-40}"; within that one schema.
Console.WriteLine(format, "Schema",
"Name", "Clause");
// Get Tables in 'dbo' Schema
// Display Data TSqlHelper.DisplayTables(conn, "dbo", null);
Console.WriteLine(format, Another option is to pass in a partial table name as the
row["CONSTRAINT_SCHEMA"],
row["CONSTRAINT_NAME"],
last parameter and display all tables that start with that
row["CHECK_CLAUSE"]); prefix as shown in the following code snippet.
}
} // Get any Tables That Starts with 'Cust'
TSqlHelper.DisplayTables(conn, null, "Cust");
Figure 4: Display the foreign key name, table name, and column used in the foreign key

Listing 17: Get a list of foreign key table and column information using a custom SQL query
public static void DisplayForeignKeys( INNER JOIN sys.tables refTable
string conn, string? table) ON refTable.object_id =
{ fkc.referenced_object_id
DataTable dt = new(); INNER JOIN sys.columns refCol
string whereSQL = string.Empty; ON refCol.column_id = referenced_column_id
if (table != null) { AND refCol.object_id = refTable.object_id
whereSQL += $" WHERE parentTab.name {whereSQL}
LIKE '{table}%' "; ORDER BY TableName";
}
string sql =
@$"SELECT da.Fill(dt);
obj.name AS KeyName,
sch.name AS SchemaName, // Display Column Names
parentTab.name AS TableName, string format = "{0,-48}{1,-33}{2,-25}";
parentCol.name AS ColumnName, Console.WriteLine(format, "Key Name",
refTable.name AS ReferencedTableName, "Table Name", "Column Name");
refCol.name AS ReferencedColumnName
FROM sys.foreign_key_columns fkc // Display Column Data
INNER JOIN sys.objects obj foreach (DataRow row in dt.Rows) {
ON obj.object_id = string kName = row["KeyName"].ToString()
fkc.constraint_object_id ?? string.Empty;
INNER JOIN sys.tables parentTab string tName = row["TableName"].ToString()
ON parentTab.object_id = ?? string.Empty;
fkc.parent_object_id Console.WriteLine(format,
INNER JOIN sys.schemas sch kName[0..Math.Min(45, kName.Length)],
ON parentTab.schema_id = tName[0..Math.Min(30, tName.Length)],
sch.schema_id row["ColumnName"]);
INNER JOIN sys.columns parentCol }
ON parentCol.column_id = }
parent_column_id
AND parentCol.object_id =
parentTab.object_id
The last option is to pass both a schema name and a table Get Foreign Keys with Columns
name (or partial name) to only display tables that match Another set of data that’s impossible to retrieve using the
the name passed within the schema “dbo”. GetSchema() method is to retrieve the foreign keys with
the column(s) that make up those foreign keys. Yes, you
// Get any Tables in the "dbo" Schema can get the foreign key names and the tables to which they
// That Start with "Cust" belong, and you can get the index columns, but there’s no
TSqlHelper.DisplayTables(conn, "dbo", "Cust"); way to get this data back in a single call. However, using
the system tables in SQL Server, you can create a JOIN to
Get Check Constraints retrieve this information, as shown in Figure 4.
Check constraints is metadata that you can’t retrieve
from the GetSchema() method. Add another method to Add a new method to the TSqlHelper class named Display-
the TSqlHelper class named DisplayCheckConstraints(), ForeignKeys(), as shown in Listing 17. Create a SQL JOIN
as shown in Listing 16. In this method, use the Infor- to retrieve the key name, the parent table and column, and
mation_Schema.Check_Constraints view to retrieve all the referenced table and column for each foreign key in your
check constraints. You may also optionally pass in a sche- database. You can also pass in a table name to just retrieve
ma name to only display those constraints within that the foreign keys for that specific table. This SQL is specific
schema. If you want, feel free to add a third parameter to to SQL Server, but you should be able to create similar SQL
filter on a constraint name similar to what you did in the to retrieve the same information from any database server.
DisplayTables() method.
Try It Out
Try It Out Now that you have the foreign key method created in the
Open the Program.cs file and call this new method to TSqlHelper class, open the Program.cs file, and make the
display all the check constraints in your database. call to this method as follows:
// Get all Check Constraints // Get all Foreign Keys

TSqlHelper.DisplayCheckConstraints(conn, null); TSqlHelper.DisplayForeignKeys(conn, null);
Next try out passing in a specific schema to only retrieve To view just a single table’s foreign keys, pass in the table
those check constraints within a specific schema. name to the second parameter.
// Get all Columns in 'SalesLT' Schema // Get Foreign Keys for a Table
TSqlHelper.DisplayCheckConstraints(conn, TSqlHelper.DisplayForeignKeys(conn,
"SalesLT"); "SalesOrderDetail");

Figure 5: Download the free set of PDSC Developer Utilities to get the SQL Compare tool, plus much more.
As you can see, you have much more flexibility when you erator tool generates full CRUD classes using the Entity
use the system tables/views in your database system as Framework. It also can generate full CRUD MVC pages us-
compared to using the GetSchema() method. ing .NET 6/7. You can customize this code generator. You
may also add your own sets of classes, pages, or anything
you want.
Free SQL Compare and
Code Generation Utility
Instead of having to build your own SQL Compare tool, Summary
you can download one for free. I created a set of devel- In this article, you learned how to retrieve metadata
oper utilities (Figure 5) years ago and they are available about your database using the .NET Framework’s GetSche-
at https://github.com/PaulDSheriff/PDSC-Tools. Besides ma() method. This method provides a standard method
a SQL Server comparison tool, you also get a Computer to retrieve data about your database objects. However,
Cleaner tool that helps clean up the multiple locations be aware that the data that’s returned from each call is
where Visual Studio and .NET leave a bunch of temporary different from one data provider to another. Retrieving
files. There is also a Project Cleaner to remove folders metadata from your database can be useful to create all
and files that are unnecessary when you’re doing an ini- sorts of utilities such as a SQL comparison tool, or a code
tial check-in to source control, or you just want to zip up generator. You should take some time to understand the
the project to send to a colleague. The next utility is a system tables in your specific database server as they
Property Generator that helps you build different types can provide you with much more information than the
of properties for your classes. Included are auto-proper- GetSchema() method.
ties, full properties, and raise property changed proper-
ties. Simple text templates are available for you to modify Paul D. Sheriff
these or add your own types of property generation.
Two other tools (JSON Generator and XML Genera-

tor) let you choose a table, view, or type in your own
SQL and generate either XML or JSON from your table.
The C# Data Repo lets you choose a table, view, or type
in your SQL and generate a hard-coded C# class with data
from the SQL. This is useful if you’re building a prototype
and don’t want to mess with a database. The Code Gen-

Some Overlooked EF Core 7

Changes and Improvements
In the .NET 7 CODE Focus issue (November 2022), I wrote about the big changes that came to EF Core 7 focusing on features
that will likely have the most impact for developers, such as performance, simpler ways to update and delete data, EF6 parity,
and more. But there were so many interesting changes in EF7, way more than just those few. Just peruse the long list relayed
in the November 24, 2022 bi-weekly updates to get an back FromSql, ExecuteSql, and ExecuteSqlAsync to the
idea of the broad scope of features affected. Unless lexicon of EF Core methods, replacing the Interpolated
you’ve experienced some of the things fixed or enhanced methods. They’re identical to the methods they replace.
by these features, reading through the list doesn’t always
give you a good understanding of the change. I’ve se- int start = 2010;
lected some of the more interesting items from the list int end = 2015;
and experimented with the features to be sure that I truly var authors = _context.Authors
understand what the problem was that they were solving .FromSql($"AuthorsPublishedinYearRange
and how they work now. This meant building a lot of dem- {start}, {end}")
os, running them in EF Core 6, running them in EF Core .ToList(); Julie Lerman
7, re-reading the discussions in the GitHub issues, read- @julielerman
ing code from the repository, and, in one case, emailing The team considered removing the longer versions but left thedatafarm.com/contact
someone from the EF Core team for further information. them in place to avoid breaking changes. Their guidance
Julie Lerman is a Microsoft
is to use the new methods.
Regional director, Docker
In this article, I’ll share what I learned about these issues
Captain, and a long-time
and hope that you find them as interesting and useful as
I have. The relevant projects can be found in my GitHub Use Raw SQL to Query Scalar Values Microsoft MVP who now
counts her years as a
repository https://github.com/julielerman/CodeMagEF- Here’s a wonderful addition to the APIs that I’ve already coder in decades. She
Core72023. benefited from in my own work, as I’m building demos makes her living as a
for a new EF Core and Domain-Driven Design course coach and consultant to
for Pluralsight. A new method of DbContext.Database,
FromSQL: Back to the Future called SqlQuery, lets you pass in a raw SQL query to get
software teams around
the world. You can find
The EF Core team divided the original FromSQL method into scalar data directly from the database. You don’t need a Julie presenting on Entity
two explicit methods—FromSqlRaw and FromSqlInterpo- mapped entity to capture the results. Here’s an example Framework, Domain-Driven
lated—to overcome some issues that could occur if you where I wanted to retrieve an int value from the data- Design and other topics
were expressing your raw SQL as an interpolated string. The base that’s mapped to a private field in an entity. at user groups and confer-
original FromSQL method was removed. The same was done ences around the world.
for ExecuteSQL, which was replaced by ExecuteSQLRaw, var value = _context.Database.SqlQuery<int> Julie blogs at thedata-
ExecuteSqlInterpolated and ExecuteSqlInterpolatedAsync. (@"SELECT TOP 1 [_hasRevisedSpecSet] farm.com/blog,
FROM [ContractVersions]") is the author of the highly
Here’s an example of FromSqlRaw that takes a stored pro- acclaimed “Programming
cedure with placeholders along with parameters to popu- This method isn’t in the documented list of changes to EF Entity Framework” books,
late the placeholders: Core 7 but can be found in the section on Raw SQL. and many popular videos
on Pluralsight.com.
var authors = _context.Authors
.FromSqlRaw(
SqlClient Changed Connection
"AuthorsPublishedinYearRange {0}, {1}", String Encryption Defaults
2010, 2015) This change is notable because it’s a breaking change
.ToList(); and the change wasn’t to EF Core, but to Microsoft.Data.
SqlClient, which is a dependency of EF Core’s SQL Server
The interpolated versions of these methods accept only a provider. Version 7 of the provider depends on SqlClient
FormattableString and that string leverages interpolation 5.0 and it’s this version of SqlClient that introduced the
to supply the parameters. change.
int start = 2010; Prior to this, the default value of a SQL Server connec-
int end = 2015; tion string’s Encrypt parameter was False. This made life
var authors = _context.Authors easy during development because it meant that we didn’t
.FromSqlInterpolated have to have a valid certificate installed on our develop-
($"AuthorsPublishedinYearRange {start}, {end}") ment computer. But the team responsible for SqlClient
.ToList(); decided it was time to make SqlClient more secure. If you
take a look at the GitHub issue for this change (https://
But oh boy they are a PIA to type or discuss with your github.com/dotnet/SqlClient/pull/1210), you’ll see that
teammates! The EF Core team made a decision to bring members of the EF Core team and community shared
codemag.com Some Overlooked EF Core 7 Changes and Improvements 27

their concern about the breaking changes this would For example, let’s consider a system where you have li-
create—not just for EF Core, but even more broadly. The brary patrons who borrow books. When a patron returns
change went forward and now you really do need to be the book, it’s no longer tied to that patron, so in the book
aware of it. type, you have the nullable FK property PatronId:
Encrypt=True requires that the server is configured with a public int? PatronId { get; set;}
valid certificate and that the client trusts the certificate.
When the book is returned, your code should set PatronId
If you’re targeting a local SQL Server instance on your to null.
development computer and don’t happen to have a
development certificate installed, you can simply add If a patron with a book moves away and you delete them
Encrypt=False to your connection string. Otherwise, from your system, moving them to a different system
you’ll get a SqlException telling you that the certifi- maintaining inactive patrons, by default, EF Core sets the
cate was issued by an untrusted certificate authority. In value of that book’s PatronId to null. This doesn’t make
most other cases, it’s a good thing to have the proper sense because now the book will never get returned. The
setup that aligns with the encryption. Hopefully, you’re behavior that the librarian requested is that the book
using separate connection strings for dev than for get deleted along with the patron. To achieve that, it’s
production, but do be sure not to send that hack into possible to override the default behavior by forcing the
production. OnDelete method on the relationship in OnModelCreating
SPONSORED SIDEBAR:
to Cascade.
CODE Is Hiring! Orphaned Dependents are modelBuilder.Entity<Patron>()
CODE Staffing is accepting Protected from Inadvertent Deletion .HasMany(p => p.Books).WithOne()
resumes for various Although this is listed as a breaking change for EF Core 7, .OnDelete(DeleteBehavior.Cascade);
open positions ranging it’s something that was working in EF Core 5, got broken
from junior to senior in EF Core 6.0.0, and fixed in 6.0.3. As I spent some time Unfortunately, there was a side effect created in EF Core 6
roles. We have multiple investigating it as an EF Core 7 change, I’ll share it with that, because of that explicit cascade delete, also deleted
openings and will consider you because it’s still notable. a book if you set its PatronId to null.
candidates who seek
full-time employment or Although it’s possible to define a nullable relationship in And that’s the scenario that was fixed in 6.0.3 and is
contracting opportunities. a few ways, this is specific to the case where you have a listed as a change in 7.0.
For more information, visit
nullable foreign key property or a reference in a depen-
www.codemag.com/jobs.
dent. Now, if you have a nullable relationship that you’ve over-
ridden so that deleting the principal cascades and deletes
the dependents, simply setting the foreign key to null will
just update the FK to null in the database.
ADVERTISERS INDEX
Filtered Includes for Hidden
Navigation Properties
Advertisers Index Here’s a feature that improves the experience of using EF
Core to persist types that follow Domain-Driven Design
CODE Consulting guidance. In fact, the community member who requested
www.codemag.com/code 7 this capability called out DDD in their request (https://
github.com/dotnet/efcore/issues/27493).
CODE Legacy
www.codemag.com/legacy 76 The scenario provided was an entity with a one-to-many
CODE Security relationship where the dependents are encapsulated to
www.codemag.com/security 2, 75 protect how they are interacted with. This is an impor-
tant capability. For example, a typical entity might totally
dtSearch expose the dependents for any type of operation whether
www.dtSearch.com 15 you are adding new books, removing a book, or editing a
book. And this class doesn’t at all express the true behav-
LEAD Technologies
ior: that the books are being checked out and returned,
www.leadtools.com 5
Advertising Sales: not added and removed.
Tammy Ferguson
832-717-4445 ext 26
public class Patron
tammy@codemag.com
{
public int PatronId { get; set; }
public string? Name { get; set; }
public List<Book> Books { get; set; }
This listing is provided as a courtesy
= new List<Book>();
to our readers and advertisers. }
The publisher assumes no responsi-
bility for errors or omissions. Alternatively, you can encapsulate the Books and ensure
that the Patron class (as an aggregate root) not only con-
28 Some Overlooked EF Core 7 Changes and Improvements codemag.com

Listing 1: Hidden collection protects the books navigation property
private readonly List<Book> _books = new List<Book>(); {
public void CheckOutBook(Book bookToCheckout) if (bookToReturn.PatronId == PatronId)
{ {
if (bookToCheckout.PatronId is null) bookToReturn.PatronId = null;
{ }
bookToCheckout.PatronId = PatronId; }
} public List<string?> CheckedOutTitles =>
} _books.Select(b => b.Title).ToList();
public void ReturnBook(Book bookToReturn)
trols how the books are checked out and returned but capability didn’t originally make it into EF Core. If you were
that the Patron entity better describes what’s happening. to look at the original request in GitHub to support this in
Expressing behavior is an important part of DDD. EF Core (https://github.com/dotnet/efcore/issues/620),
you’d see that the team had to keep punting this change
There are different ways to achieve this, but one way is from one version to another. They finally enabled it in EF
shown in Listing 1, where a _books private field allows Core 7 with a new mapping method called SplitToTable.
special protected access to the books but checking in and
out is simply performed on the book in question. At the Here’s an example where I’ve added a property, Mobile-
same time, the class also makes it easy to see the list of Number, to the Patron class:
checked out titles.
public string? MobileNumber { get; set; }
In a repository or other class you may use for persistence,
it’s still possible to query for a patron and books if needed. You can use the SplitToTable method to not only specify
This is made possible by specifying a Books property in the which properties of the type go to the alternate table(s)
data model with the following mapping in the DbContext: but you can also specify a name for the column if it dif-
fers from the property name, as I’ve done here.
modelBuilder.Entity<Patron>()
.HasMany("_books").WithOne(); modelBuilder.Entity<Patron>()
.SplitToTable("PatronContactInfo",
EF Core knows to tie the _books property to the Books p => p.Property(c => c.MobileNumber)
property here because of the naming conventions I used. .HasColumnName("CellPhone"));
So now there’s a secret Books property that’s not exposed
in the Patron class but is known to EF Core. Along with this feature, the ability to specify the column
name was also added to mappings for TPT and TPC inheri-
Given that set up, it was already possible to eager load those tance where multiple tables are also involved. You can see
books using the same string specified in the mapping. examples of these mappings in this GitHub issue: https://
github.com/dotnet/efcore/issues/19811.
context.Patrons.Include("_books").ToList();
But when filtered includes (i.e., a way to sort or filter the Unidirectional Many-to-Many
dependent collection being loaded) was introduced in EF Another improvement that lends support to DDD entities is
Core 6, there was no way to apply this capability to the the ability to expose only one side of a many-to-many rela-
above pattern using the string parameter. tionship. Quite often in your domain models, you might have
such a relationship but only need to navigate in one direction.
The solution the team arrived at was to allow the use of
EF.Property in an Include method. For example, you can have categories for the library books.
Each book may have one or more category and each cat-
That means the above query can also be expressed as: egory may have one or more book. However, while getting
a list of categories for a book is common, the library told
context.Patrons.Include(p => us that it’s highly unusual to want to find every book for
EF.Property<Book>(p, "_books")) a single category. So why complicate the category class
.ToList(); with an unneeded Books property?
And the filtering or sorting methods can be appended The book class has a Categories property:
to the EF.Property, as long as you first specify that the
property is, in fact, a collection: public List<Category> Categories { get; set; }
= new List<Category>();
context.Patrons.Include(p =>
EF.Property<ICollection<Book>>(p, "_books")) Yet the category class has no property for books:
.ToList();
public class Category
{
Entity Splitting public int CategoryId { get; set; }
EF6 allowed us to split the properties of an entity across public string? Name { get; set; }
multiple tables, referred to as “entity splitting” but the }

With no additional help, EF Core assumes that this is a The key is that both entities must be explicitly mapped
one-to-many relationship. To counter this, I’ve added a to the target database table and configured as temporal.
mapping in the DbContext to ensure that the data model That’s more effort than standard table splitting mappings.
is aware that it’s a many-to-many: And for owned entities, it can get pretty cumbersome.
Here’s an example of what that would look like given two
modelBuilder.Entity<Book>() entities, Patron and ContactInfo, where ContactInfo is an
.HasMany(b => b.Categories).WithMany(); owned entity of Patron.
The database schema reflects the many-to-many and EF First, here’s the ContactInfo class:
Core respects it as well.
public class ContactInfo
In my code logic, I can add categories to a book but I can’t {
add books to categories. Yet because the many-to-many public string? MobileNumber { get; set; }
does exist in the database, I can access the data from other public string? MainEmailAddress { get; set; }
logic in my software as well as reporting solutions. }
Patron now has a nullable ContactInfo property.

Enhance Temporal Table Support
for Owned Entities public ContactInfo? ContactInfo { get; set; }
Table splitting is the opposite of entity splitting and has
been part of EF Core for a while. It allows mappings where Without temporal tables, you only need to specify that
the columns of a single table are split across multiple Patron owns ContactInfo like this:
types in your system. You can use it to force properties
of one entity to be stored in the table of another entity modelBuilder.Entity<Patron>()
using the ToTable mapping. See https://learn.microsoft. .OwnsOne(p => p.ContactInfo);
com/en-us/ef/core/modeling/table-splitting for exam-
ples of this scenario. Table splitting is also the feature To ensure that the Patrons table with the combination of
that enables the properties of owned entities to be stored columns from both types is temporal, you’ll have to configure
alongside the properties of the entity that owns them. a lot of information. In fact, with standard temporal tables,
you can rely on the default start and end column names that
When temporal table support arrived in EF Core 6, it couldn’t EF Core supplies. For this mapping, you must also explicitly
be combined with table splitting. It was explicitly blocked, map those columns for both types to ensure that they match
which I gather is due to time constraints and other priori- (Listing 2). The resulting table is shown in Figure 1.
ties. The team was able to “unblock” the limitation early on
but decided that it was too big of a change to release in a You may want to keep a copy of that mapping handy to
patch of EF Core 6, so they delayed it until EF Core 7. copy and paste but be aware that the team is working on
Figure 1: Temporal table created by EF Core with columns from an owned entity
Listing 2: Mapping an owned entity for temporal tables

modelBuilder.Entity<Patron>().ToTable("Patrons", tableBuilder =>
tableBuilder => {
{ tableBuilder.IsTemporal();
tableBuilder.IsTemporal(); tableBuilder.Property<DateTime>("PeriodStart")
tableBuilder.Property<DateTime>("PeriodStart") .HasColumnName("PeriodStart");
.HasColumnName("PeriodStart"); tableBuilder.Property<DateTime>("PeriodEnd")
tableBuilder.Property<DateTime>("PeriodEnd") .HasColumnName("PeriodEnd");
.HasColumnName("PeriodEnd"); }
}).OwnsOne(p => p.ContactInfo, ));
ownedBuilder => ownedBuilder.ToTable("Patrons",
30 Some Overlooked EF Core 7 Changes and Improvements codemag.com

a simpler way to achieve that mapping. Follow https:// With this set up, runtime sends logs to the runtimelog.txt
github.com/dotnet/efcore/issues/29303 if you want to file and migrations sends logs to the migrationlog.txt file.
see the progress.
Wrapping Up
Migration File Naming Protection It’s important and interesting to explore the EF Core docs
Sometimes when adding migrations, you may just be ex- sections on what’s new and breaking changes. Although
perimenting and it’s very tempting to name the migration the documentation is very detailed, sometimes you just
file “migration”. In some cases, this causes a circular de- need to explore those changes yourself in order to really
pendency with the Migrations class (https://github.com/ understand what’s going on. This article is a result of that
dotnet/efcore/issues/13424). experimentation and I hope that it deepens your under-
standing of these features.
Thanks to community member Kev Ritchie (https://github.
com/KevRitchie) for his pull request to avoid this problem It’s clear that the EF Core team listens and responds to
by having the migrations command warn you away from this feedback from the community. They may not always be
naming with an internal validation and a message telling able to execute on those ideas quickly due to prioritiza-
you “You cannot add a migration with the name ‘Migration’.” tion. But in addition to the high-level changes that get
the most attention (and which I covered in the earlier ar-
ticle), there are many more interesting tweaks in EF Core
New IsDesignTime Flag to Help 7 that can make a big difference to many of our projects.
with Migrations in Minimal APIs
EF Core migration commands need a runtime execution to Julie Lerman
do their job. When Minimal APIs were introduced in .NET
6, there were some scenarios where production startup
logic, such as forced migrations with the Migrate com-
mand, were being inadvertently executed. EF Core 7 intro-
duced a new flag, EF.IsDesignTime, to help avoid running
this production code. For example:
if (!EF.IsDesignTime)
{
//do something irrelevant to or in
// conflict with migrations
}
Not only can this flag help you avoid conflicts that could
arise between production code and migrations, you can
also use it for other purposes. For example, I’m having
my sample web app use Serilog, a .NET logging library
(http://serilog.net), instead of the simple logging in EF
Core. Therefore, I’m configuring Serilog and then adding
it to the web app’s services.
When I configure it, I ask my app to log to different files

depending on whether I’m running the application or if
it’s migrations that have triggered the application.
if (!EF.IsDesignTime)
{
_logfile = "logs/runtimelog.txt";
}
else {
_logfile = "logs/migrationlog.txt";
}
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.File(_logfile,
rollingInterval: RollingInterval.Day)
.CreateLogger();
Then, when I’m injecting services into the pipeline, I also

inject Serilog into ASP.NET Core’s logging pipeline.
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Logging.AddSerilog();

Clean Shave: Razor Pages for

WebForms Developers
If you’re an ASP.NET WebForms developer and you’re unsure of how to take your skills to the future of the .NET platform (e.g.,
.NET Core or .NET 6), there’s hope. Although Microsoft won’t port WebForms, you can apply your existing skillset to a similar
framework, called Razor Pages. You’ll still need to think about this new platform differently, but if you take the time, I think you’ll
find a similarly powerful and easy to use tool to create Naming is the Hardest Job in Software Engineering
web applications. Before we get started, I want to define some terms. Ra-
zor is a language for adding logic to HTML markup (and
was invented for ASP.NET MVC). Because this language
Where You’re Coming From is so useful, it’s used by a number of technologies in
Back in the early 2000s, I was a C++ developer and was the Microsoft stack and that has led to everything being
one of these “you’ll have to take my pointers out of my called Razor-something. In most cases, Razor files end
cold dead hand” guys. But once I was introduced to how in “.cshtml” or “.vbhtml”. So, let’s try to disambiguate:
garbage collection worked in .NET, I was a convert. In
Shawn Wildermuth those early days, I was writing ASP.NET (after my time • Razor View: The files associated with a View in ASP.
shawn@wildermuth.com writing components for ASP projects). NET MVC
wildermuth.com • Razor Page: The files associated with a Page in Razor
@shawnwildermuth The reality was that I didn’t understand how the web actu- Pages
ally worked, but I was tasked with creating websites and • Razor Component: A component used by the Blazor
Shawn Wildermuth has
been tinkering with web apps using ASP.NET. Microsoft came to my rescue by framework for Web Assembly-based web applications
computers and software introducing Web Forms. Nowadays, Web Forms gets quite
since he got a Vic-20 a lot of hate from many directions about how un-web-like Enough with definitions, let’s see how Razor Pages work.
back in the early ’80s. it was. But it helped people like me dip my toe in the web
As a Microsoft MVP since world without the fear that comes from something new. Let’s try to map WebForms nomenclature to Razor Pages,
2003, he’s also involved Microsoft successfully turned desktop developers into web as seen in Table 1.
with Microsoft as an ASP. developers. But it wasn’t without inherent risks.
NET Insider and ClientDev Brief Overview
Insider. He’s the author Web Forms introduced drag-n-drop designing to web de- Razor Pages is based on two fairly simple concepts:
of over twenty Pluralsight velopment. Under the covers, it was trying to hide the
courses, written eight details of the web and feel like the server-side code was • Convention-based URLs
books, an international something akin to a stateful development solution. Add • Razor Pages to produce content
conference speaker, and in ViewState and Session State, and lots of developers
one of the Wilder Minds. were able to accomplish a lot of value for their companies What I mean by this is that if you create a new project
You can reach and employers. using Razor Pages, a new piece of middleware is added to
him at his blog at handle requests:
http://wildermuth.com. But it’s now 2023. We’ve been through a world of change
He’s also making his first,
since those early days. For many Web Forms developers, it var app = builder.Build();
feature-length documen-
can be overwhelming to be asked to learn JavaScript on
tary about software
the client, separate concerns into Controllers and Views, // Configure the HTTP request pipeline.
developers today called
“Hello World: The Film.” and write code that is truly stateless. But that’s where we if (!app.Environment.IsDevelopment())
You can see more about it at are now. There isn’t a perfect upgrade path to ASP.NET {
http://helloworldfilm.com. Core for Web Forms developers. But there are some ways app.UseExceptionHandler("/Error");
to apply our existing knowledge without throwing out the }
baby with the bathwater. In comes Razor Pages.
app.UseStaticFiles();
app.UseRouting();
Introducing Razor Pages app.UseAuthorization();
As an answer to Web Pages, Microsoft introduced ASP.NET
MVC as a Model-View-Controller framework that separated app.MapRazorPages();
(and simplified testability) views and logic. This has been
the prevailing framework for many projects, although it app.Run();
never did replace Web Forms. After .NET Core was intro-
duced, Razor Pages was introduced to have a model closer MapRazorPages simply listens to requests and sees if
to a page-by-page solution instead of complete separa- there is a match to Razor Page files. If found, the middle-
tion. Now with Blazor, another solution has been added ware returns a rendered page, as seen in Figure 1.
to the quiver of tools. For this article, I’m going to fo-
cus on Razor Pages themselves as I think it’s the most How does it know if there’s a Razor Page for the request? It
straightforward migration path for Web Forms developers. uses a convention to find the files. Although the specific
32 Clean Shave: Razor Pages for WebForms Developers codemag.com

implementation might be a bit different, you can simply
think of the Pages folder as the root of the web server.
This means Razor Pages will follow the folder/file structure
to respond to requests. For example, if the request URL is
/contact, the middleware will look in the Pages folder to find
a file named contact.cshtml. If it finds it, it then renders
that page and returns it to the client, as seen in Figure 2.
The exception to this is the Index.cshtml file. This file

is used when the URL points directly to a folder instead
of an individual file name. It’s the default page name and
fallback. This means that if the URL looks like https://
localhost/, it will search for the index.cshtml.
Folders work in the same way. Any folders inside of the Figure 1: Razor Pages middleware
Pages folder map to URL fragments. For example, if you
have an URL like /Sales/ProductList, the file matches, as
seen in Figure 3. WebForms Term Razor Page Term
Web Form (.aspx) Razor Page (.cshtml/vbhtml)
Okay, now that you can see how Razor Pages are mapped,
Web Control (.ascx) Partial Page (also .cshtml/vbhtml)
let’s dig into what makes a Razor Page.
MasterPage` Layout
Anatomy of a Razor Page AJAX Just JavaScript
Although you’ll often use scaffolding to create Razor Pages, global.asax Program.cs or Startup.cs
let’s look at what makes a Razor Page a Razor Page. A Razor
Page is just a file that contains a @page declaration: Table 1: Translating WebForms terms to Razor Pages
@page
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible"
content="IE=edge">
<meta name="viewport"
content="width=device-width,
initial-scale=1.0">
<title>Document</title>
</head>
<body>
<h1>Hello from Razor</h1>
</body>
</html>
Figure 2: Looking for a Razor Page Figure 3: Folders in Razor Pages
This tells the middleware that this is a servable file. Your
Pages folder might include other files like layout files or
partial files that you don’t want to be rendered as in- var title = "CODE Magazine - Razor Pages";
dividual pages. This helps the middleware differentiate }
whether it’s actually a Razor Page or not. The “@” sign <!DOCTYPE html>
isn’t an accident. The Razor syntax uses the “@” symbol <html lang="en">
to indicate the start of a server-side code operation. For <head>
example, you can create an arbitrary code block like so: <meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible"
@page content="IE=edge">
@{ <meta name="viewport"
var title = "This is made with Razor Pages"; content="width=device-width,
} initial-scale=1.0">
<!DOCTYPE html> <link href="~/css/site.css" rel="stylesheet" />
<title>@title</title>
The curly braces (e.g. {}) are just to allow for multi-line </head>
code. This code is just plain C# (or VB.NET if you’re using <body>
.vbhtml files). With that set, you can use it with the “@” <div>
symbol to start: <h1>@title</h1>
</div>
@page </body>
@{ </html>
codemag.com Clean Shave: Razor Pages for WebForms Developers 33

With a variable created here, you can just insert the title // Index.cshtml.cs
in both places you need it. Razor also allows you to call using Microsoft.AspNetCore.Mvc.RazorPages;
methods or properties because everything after the @
sign is interpreted as the language of the file: namespace CodeRazorPages.Pages;
<p>Today is @DateTime.Now.ToShortDateString()</p> public class IndexModel : PageModel

{
In fact, you can use control-flow statements too:
public string Title { get; set; } = "Welcome";
<!DOCTYPE html>
<html lang="en"> public void OnGet()
<head> {
<meta charset="UTF-8"> }
<meta http-equiv="X-UA-Compatible" }
content="IE=edge">
<meta name="viewport"
You can then use this on the cshtml file, like so:
initial-scale=1.0">
@page
<link href="~/css/site.css" rel="stylesheet" />
@model IndexModel
<title>@title</title>
...
</head>
<h1>@Model.Title</h1>
<body>
<p>Today is @DateTime.Now.ToShortDateString()</p>
<div>
@if (string.IsNullOrEmpty(@title) == false) ...
{
<h1>@title</h1> The model declaration exposes a property called Model
} that’s an instance of the IndexModel class. You can see
</div> that you can use Razor syntax to inject the Title into the
</body> page that you defined in the PageModel class.
</html>
The PageModel class can do more complex interactions
You’re using an If statement to determine whether to too. You’ll notice that the PageModel class contains a
show the title. This works with all control-flow state- method called OnGet. This method is executed before the
ments like for, foreach, switch, etc. page is rendered when this page is retrieved via the GET
verb in HTTP. So, you could do something like this in your
If you’re coming from Web Forms, this sort of server-side syn- example:
tax should be comfortable to you, although the syntax is dif-
ferent. But using in-line code in this way could mix logic and public class IndexModel : PageModel
design, which concerns me. That’s where page models come in. {
The Page Model Class public string Title { get; set; } = "Welcome";
When you create a new Razor Page using Visual Studio, it public List<double> InvoiceTotals { get; set; }
will automatically add a .cshtml.cs file of the same name. = new List<double>();
This is called the PageModel class. For example, the index
page would look like this: public void OnGet()
{
// Index.cshtml.cs
for (var x = 0; x < 10; ++x)
using Microsoft.AspNetCore.Mvc.RazorPages;
{
InvoiceTotals.Add(
namespace CodeRazorPages.Pages;
Random.Shared.NextDouble() * 100);
}
public class IndexModel : PageModel
}
{
public void OnGet() }
{
} You can see here that you can create content (in this
} case, random invoice totals) and then use them:
This is a class that derives from the PageModel class. This <h1>Model.Title</h1>
class is wired up to the cshtml file using the @model <p>Today is @DateTime.Now.ToShortDateString()</p>
declaration on the Index Page: <h3>Invoices</h3>
@foreach (var invoice in Model.InvoiceTotals)
@page {
@model IndexModel <div>$ @invoice.ToString("0.00")</div>
}
By doing this, you have access to the PageModel in-
stance. If you were to add a property on PageModel class, Although this is a convoluted example, you could imagine
you could access it here. For example: reading from a database to show information in this same

way. You want to do this in the OnGet method instead of public void OnPost(UserSettings settings)
the constructor because you don’t want to generate the {
data if you were to POST to this page, which you’ll see next. var UserId = settings.UserId;
if (UserId is null)
Model Binding {
When creating forms (or other interactions) with Razor throw new InvalidDataException(
Pages, you can use model binding to help you take the "UserId can't be null");
data from the form and inject it into the page model }
class. Let me explain. Let’s start with a simple class that }
holds data for a user:
This works because the data on the form matches the
public class UserSettings properties in UserSettings. But you don’t always want
{ to pass in the settings, sometimes you want to bind it
public string? UserId { get; set; } directly to the page model class. In that case, you can
public string? FullName { get; set; } add a [BindProperty] attribute:
}
[BindProperty]
In the page model class, let’s add the property for the public UserSettings Settings { get; set; }
UserSettings: = new UserSettings();
public UserSettings Settings { get; set; } public void OnPost()

= new UserSettings(); {
var UserId = Settings.UserId;
public void OnGet() if (UserId is null)
{ {
Settings.UserId = "shawn@aol.com"; throw new InvalidDataException(
Settings.FullName = "Shawn Wildermuth"; "UserId can't be null");
} }
}
Then you create a form on that page. Notice that the Set-
tings is accessible from the Model: In this case, the OnPost (or OnPut, OnDelete) takes the
values and binds them to the instance on the class. In
<h1>Settings</h1> this way, it can be easier to work with large sets of data
<form method="POST"> on your Razor Page. But building the form should be eas-
<label>Email</label> ier. Let’s see how.
<input type="email"
name="UserId" TagHelpers
value="@Model.Settings.UserId" /> When building a form, you might want to more easily
set the values and bindable properties (and matching
<label>Full Name</label> names). To do this, Microsoft supports something called
<input type="text" TagHelpers. Tag helpers allow you to more easily bind to
name="FullName" the Page model. For example, you could use the asp-for
value="@Model.Settings.FullName" /> attribute to tell the form which properties you want to
bind:
<button type="submit">Send...</button>
</form> <form method="POST">
<label asp-for="Settings.UserId">Email</label>
Note that the name is the same as the property value. <input asp-for="Settings.UserId" />
You’re setting the value from the model so you can have
the current values shown. But if you want the form to be <label asp-for="Settings.FullName">
able to send the changes, you’ll need an OnPost method: Full Name
</label>
public void OnPost() <input asp-for="Settings.FullName" />
{
var UserId = Settings.UserId; <button type="submit">Send...</button>
if (UserId is null) </form>
{
throw new InvalidDataException( Although TagHelpers are usually wired up for you in a new
"UserId can't be null"); project, I think it’s important for you know where they
} come from. There’s a special file called _ViewImports.
} cshtml. It can contain any namespaces you want to be
implicit in your Razor Pages (i.e., YourProject.Pages) and
When you submit the form, the exception is being thrown. it’s where TagHelpers are included:
Why doesn’t this work? Model binding isn’t being used.
There are a couple of ways to make this happen. First, you @* _ViewImports.cshtml *@
could accept a UserSettings object in the OnPost method: @namespace CodeRazorPages.Pages

</div>
</body>
</html>
Although this example is really simple, you can imagine

that you have navigation and footers defined here too. To
use the layout, your page can just set the Layout property:
@page
@{
Layout = "_layout";
}
<h1>Razor Page</h1>
<p>Today is @DateTime.Now.ToShortDateString()</p>
Having to add this to every page is annoying, so you can

create a special file called _ViewStart.cshtml. This file is
used to specify common things for each Razor Page. In
this case, you can simply move the setting of the layout
to that file and it will apply to all Razor Pages. This file
Figure 4: Location of the Layouts Figure 5: Location of the _ViewStart.cshtml file must reside in the Pages folder (it applies to a folder and
all subfolders), as seen in Figure 5:
Source Code @addTagHelper *, Now that you have a Layout, you might find it useful
Microsoft.AspNetCore.Mvc.TagHelpers to share information with the Layout. The most common
The source code can be
scenario here is to set the title tag in the header. You can
downloaded from https://
github.com/wilder-minds/ If you’re not getting access to the tag helpers, this file is use a property bag called ViewData. If you set this on the
CodeRazorPages and from missing or incorrect. Now that you can see how individual Razor Page, you’ll be able to access it in the Layout. For
the www.CODEMag.com pages work, let’s look at how to better compose pages example, in your Razor Page:
page associated with from individual components.
this article. @page
@{
Composing Razor Pages ViewData["title"] = "Razor Pages Example";
Like WebForms, Razor Pages has the ideas around break- }
ing out individual parts of a page like UserControls and <h1>@ViewData["title"]</h1>
MasterPages. Let’s see how that looks in Razor Pages. <p>Today is @DateTime.Now.ToShortDateString()</p>
Using Layouts Notice that even though you’re setting it, you can use it
Having the entire HTML page defined in every Razor Page on your page as well. In the Layout, you can just refer to
would be a waste of time. Just like Web Forms, you can the ViewData object:
have master pages, which are called Layouts in Razor
Pages. Layouts are kept in a folder called Shared (this <head>
folder is part of the search path of several kinds of files <meta charset="UTF-8">
(e.g., partials, layouts, etc.). This way, they’re available <meta http-equiv="X-UA-Compatible"
to every page that needs them. By convention, a Layout content="IE=edge">
is called _Layout, as you can see in Figure 4. <meta name="viewport"
The Layout will usually have the HTML boilerplate that initial-scale=1.0">
you want on every page. You can call RenderBody() to tell <link href="~/css/site.css" rel="stylesheet" />
the Layout where you want the page’s content to appear: <title>
CODE Magazine - @ViewData["title"]
<!DOCTYPE html> </title>
<html lang="en"> </head>
<head>
<meta charset="UTF-8"> Notice how you can use the ViewData to insert into con-
<meta http-equiv="X-UA-Compatible" tent, not just be the entire content. In this example,
content="IE=edge"> every page’s title starts with some boilerplate, but then
<meta name="viewport" each page can specify their own title.
initial-scale=1.0"> In addition, Layouts support the concept of Sections. An
<link href="~/css/site.css" rel="stylesheet" /> area of the page that the Razor Page itself can insert into
<title>Razor Page</title> the Layout. For example:
</head>
<body> <head>
<div> <meta charset="UTF-8">
@RenderBody() <meta http-equiv="X-UA-Compatible"

content="IE=edge"> <div>
<meta name="viewport" <partial name="_ContactForm" />
content="width=device-width, </div>
initial-scale=1.0">
<link href="~/css/site.css" rel="stylesheet" /> Notice here that the name of the view is the name with-
<title> out an extension. This isn’t a path to the partial, but
CODE Magazine - @ViewData["title"] instead a name that can be looked up. When finding a
</title> partial, the system searches in several places (in order):
@RenderSection("head", false)
</head> • The same directory as the Razor Page that contains
the partial element
You can call @RenderSection to specify a section with • Any parent directory above the page’s directory
any name you want. The second (optional) parameter de- • /Shared directory
fines whether the section is required. Two common use • /Pages/Shared directory
cases for this are to allow an individual page to inject • /Views/Shared directory
page specific styling or scripts. But you could use it for
any section of a page. To use the section, you use the If the partial can’t be found here, it returns an error. If
section keyword in your .cshtml file: this happens, either the name is incorrect or it’s some-
where that Razor Pages can’t find it.
@page
@{ You can think of partial pages as being very similar to a
ViewData["title"] = "Contact Page"; regular Razor Page. This means that it has data that can
} be handed to it and can use Razor syntax to change it. In
@section head { this example, in order to fill-in the form, you need to pass
<link src="./someextracss" rel="stylesheet" /> it the UserSettings. You do this with the model attribute:
}
<h1>Contact Us</h1> @page
@model UserSettingsModel
You can see the @section near the top of the file. The <h1>Settings</h1>
name of the section is supplied to specify which section <partial name="_SettingsForm"
you want to insert into. Everything inside the curly braces model="Model.Settings" />
is injected into the Layout.
Once you do this, you need to set the Model in the partial
Partials page:
Sometimes you want to be able to re-use a common piece
of markup. Razor Pages supports this via Partials. Partials @model UserSettings
allow you to inject a shared file with Razor syntax into an <form method="POST">
existing page. You could take the form you created on the <label asp-for="UserId">Email</label>
UserSettings page earlier and move it into a partial page. <input asp-for="UserId" />
To do this, create a cshtml file prefixed by an underscore.
<label asp-for="FullName">Full Name</label>
This isn’t required but it makes it easier to see what are <input asp-for="FullName" />
actual pages and partial pages. For example, if you were
to create a _SettingsForm.cshtml file like this: <button type="submit">Send...</button>
</form>
<form method="POST">
<label asp-for="Settings.UserId">Email</label> Note that this is only necessary because you’re using model
<input asp-for="Settings.UserId" /> binding here. If this was just a snippet of HTML, you wouldn’t
need this at all. By using these different mechanisms, you
<label asp-for="Settings.FullName"> can build server-side web projects. Let’s wrap this up.
Full Name
</label>
<input asp-for="Settings.FullName" /> Where Are We?
Razor Pages is a page-centric web development program-
<button type="submit">Send...</button> ming model similar to how the WebForms framework works.
</form> Although you can apply much of your experience with Web-
Forms directly to Razor Pages, there are major differences.
Note that this new file doesn’t have an @page declara- If you’re used to working with a design surface and setting
tion; no declaration is necessary. To use this partial page, properties in a user-interface, you still have to get comfort-
you just use a partial element: able with how HTML actually works. Because there’s no no-
tion of session state, you’ll need to change how you think
@page about these. But if you’re ready to move your C# skills to
@{ .NET Core, Razor Pages are a great place to start.
ViewData["title"] = "Contact Page";
} Shawn Wildermuth
<h1>Contact Us</h1>

Introduction to Snowflake
Cloud computing is now in its second decade of existence, and those two decades of development have been nothing but
astounding. In these two decades, we’ve moved way beyond virtual machines and cloud storage. We now have tools like cloud
functions (lambdas), containerization, and too many technologies to discuss in one article. One area that’s seen a huge amount
of innovation is cloud databases. We have legacy data- performing analytics on it and a quick turnaround is para-
base services hosting familiar databases like MySQL, Post- mount. This dataset will be around 100,000,000+ records
gres, and SQL Server. There are also offerings for many and will be pushed to the usual Amazon Bucket. I know
other styles of data, such as documents, columnar data, that’s not hard to handle but there’s an exception. Isn’t
key value data, etc. there always? This dataset has PII (personally identifiable
information) in it and needs to be viewable only by the
One of the relatively new entrants to cloud computing, analytics team. Oh, and there’s one more little item. We
and the focus of this article, is called Snowflake. Snow- need to share the raw results with the client in real-time
flake is a unique offering because it provides many capa- every time we process the data. Please get back to us
Rod Paddock bilities that developers need today. Need a cloud-based with an estimate.”
rodpaddock@dashpoint.com SQL database? Need a database capable of querying JSON
data stored in a column? Need the ability to securely There are two ways to handle this request. The first way
Rod Paddock founded Dash share data with external vendors without exposing your is a classic cloud process. You add resources to the EC2
Point Software, Inc. in infrastructure to them? Snowflake handles many of these (server) and RDS (database) instances to handle the load.
2001 to develop high- concepts very well.
quality custom software
1. Create a process to download the data from S3 into
solutions. With 30+ years
In my 30+ years of software development, I’ve seen many a set of objects.
of experience, Rod’s current
products come and go. Every once and a while, I come across 2. Send that data into Postgres (or SQL Server, MySQL, etc.).
and past clients include:
Six Flags, First Premier a product that I consider a game changer. The product that 3. Create a process for querying the data masked or un-
Bank, Microsoft, Calamos I‘m about to discuss is called Snowflake and I can say without masked.
Investments, The US Coast a doubt that this is a game-changing product. 4. Create a process to send data to the client’s pre-
Guard, and US Navy. Along ferred sharing technology (downloadable file, SFTP,
with developing software, Imagine a request from the marketing department. “We’ll S3 bucket, Azure Storage, etc.).
Rod is a well-known author be receiving a huge dataset from Widgemo, Inc. We’ll be 5. Test and put the process into production.
and conference speaker.
Since 1995, Rod has given
talks, training sessions,
and keynotes in the US,
Canada, and Europe. Rod
has been Editor-in-Chief of
CODE Magazine since 2001.
38 Introduction to Snowflake codemag.com

The second way to handle the load is to use a few com- of data warehouses in Snowflake is COMPUTE not storage.
mands in Snowflake, like this: Let me reiterate: It’s COMPUTE and not a store of data.
1. Add resources. A good metaphor is that it’s like a car into which you can
add cylinders on the fly. Want to save gas ($$$)? Choose
ALTER WAREHOUSE SET WAREHOUSE_SIZE=LARGE; a small car. Want to head down the Autobahn at 200KM
an hour? Make the car an eight cylinder. Snowflake COM-
2. Import data from S3. PUTE is run in powers of 2: 1, 2, 4, 8, 16, 32, and on up
to 256 cylinders of COMPUTE power. The unique aspect of
COPY INTO FROM S3 <COMMAND OPTIONS HERE> Snowflake is that you can control how many cylinders you
use for any given transaction. You pay for them, but the
3. Control PII information. power of upgrading a single query or set of queries with
the flick of a switch is compelling.
CREATE OR REPLACE MASKING POLICY address_mask
AS (VAL string) RETURNS string -> You pay for it using Snowflake’s simple model to pay for
CASE credits based on the edition of Snowflake you create. Cred-
WHEN current_role() IN ('MARKETING') THEN VAL its are used to pay for credit hours: one credit = one credit
ELSE '*********' hour. Your warehouse size is how the system determines how
END credit hours are billed and you pay more credits for the larg-
er warehouses. All queries are billed on a per-minute basis.
4. Share data with the client. Figure 1 shows the prices per credit hour based on edition.
CREATE SHARE <COMMAND OPTIONS HERE> To better understand how this works, let’s look at an ex-
ample. The following code is simple. It creates new tables
As you can see, each of these processes can satisfy the using different warehouse sizes. The source table is called
user’s request. The bgd difference is the time from request DEMO_DATA and has 100,000,000 records in it. This code
to production. The first process could take a day or more, creates new copies of that table using different ware-
depending on the development team’s backlog. The second house sizes. The command is as follows:
process is only a few commands from start to finish. Speed
is a definite competitive advantage, and it’s the built-in ALTER WAREHOUSE SET WAREHOUSE_SIZE=SMALL;
features of Snowflake that enable it. Let’s have a look. CREATE TABLE DEMO_DATA_SMALL AS
SELECT * FROM
PRARIE_DEVCON_DATABASE
Snowflake Features .PUBLIC.PEOPLE_HUGE;
Snowflake is a strange amalgamation of many modern di-
vergent database concepts. The following list represents ALTER WAREHOUSE SET WAREHOUSE_SIZE=MEDIUM;
some of the high-level features:
CREATE TABLE DEMO_DATA_MEDIUM AS
• It’s SQL compliant, offering SELECT, INSERT, UP- SELECT * FROM
DATE, DELETE, etc., and CREATE TABLE, CREATE VIEW, PRARIE_DEVCON_DATABASE
CREATE SCHEMA, and JSON querying .PUBLIC.PEOPLE_HUGE;
• Snowflake can store and query JSON data stored in
a specialized column type. ALTER WAREHOUSE SET WAREHOUSE_SIZE=LARGE;
• It’s cloud agnostic (Azure, AWS, GCP). I call this
BYOC (bring your own cloud). You can have your CREATE TABLE DEMO_DATA_LARGE AS
Snowflake infrastructure set up on the cloud pro- SELECT * FROM
vider of your choice. PRARIE_DEVCON_DATABASE
• Embedded Python. Snowflake can embed and call .PUBLIC.PEOPLE_HUGE;
Python code from your queries and procedures.
• Secure data sharing. Snowflake can both share and ALTER WAREHOUSE SET WAREHOUSE_SIZE=XXLARGE;
consume shared data with other Snowflake instanc-
es. The cost of compute in shared environments is CREATE TABLE DEMO_DATA_XXLARGE AS
paid by the consumer of that data, not the host. SELECT * FROM
• Multiple client access. You can access your data us- PRARIE_DEVCON_DATABASE
ing the technology of your choice. There are drivers .PUBLIC.PEOPLE_HUGE;
for ODBC, Python, Node, and GO.
• Pay as you go with “infinite” scaling. With the flick of The results are as follows:
a switch (really, a simple command line), you can add
or reduce your computer power for queries performed. • SMALL: 24 seconds
• MEDIUM: 19 seconds
• LARGE: 12 seconds
“Infinite” Computing with • XXLARGE: 8 seconds
Snowflake Warehouses
When you’re dealing with Snowflake, you must understand As you can see, the performance for larger warehouse
that the concept of the warehouse isn’t what you normal- sizes is significant. In some, the tables contain over a
ly think of: organized data storage. Instead, the concept billion records. The difference is astonishing.
codemag.com Introduction to Snowflake 39

Figure 1: The price you pay for Snowflake is based on the edition and size of your warehouse.
Snowsight 3. Construct a Format in code.

The main mechanism for managing your snowflake infra- 4. Construct a Stage in code.
structure is a web-based application called Snowsight. 5. Create a COPY INTO command.
From Snowsight has many features. Figure 2, Figure 3, 6. Open a connection.
and Figure 4 highlight some of the most frequently used 7. Run a SQL command to load data.
features of Snowsight.
Understanding Worksheets Starting Your Project

The worksheets screen is where you can run scripts. Figure This example creates a small console application to fabri-
5 shows a worksheet with a snippet of code. This snippet cate some sample data and send it to Snowflake. The first
is used to create the database, table and useraccount that step is to create a console application via the Visual Studio
you’ll use in the examples in this article. interface. The code samples that follow depend on C# 11
and its new string capabilities. To enable this in your proj-
CREATE DATABASE CODE_MAGAZINE_DEMO; ect, you’ll need to add the following section to the top of
create or replace TABLE your Visual Studio project file.
CODE_MAGAZINE_DEMO.PUBLIC.DEMO_DATA (
UNIQUEID VARCHAR(200), <LangVersion>latest</LangVersion>
LASTNAME VARCHAR(200),
FIRSTNAME VARCHAR(200), Once you’ve created your project and enabled C#11, you
ADDRESS VARCHAR(200) , can go to work on the pipeline. Add the code from Listing
CITY VARCHAR(200), 1 to your application. Listing 1 is a class capable of cre-
STATEORPROVINCE VARCHAR(200) ating a specified number of sample objects. These sample
); objects are returned in a List<T> structure, which you can
CREATE USER CODE_MAGAZINE_USER then save to a CSV file that will ultimately make it into
WITH PASSWORD='PASSWORD'; your Snowflake database via an S3 bucket.
GRANT ROLE ACCOUNTADMIN Send to CSV File

TO USER CODE_MAGAZINE_USER Once you’ve created your sample data, you can save those
records to a CSV file. This example uses the CsvHelper
library, which you can install using NuGet with the fol-
Building a Data Loading Pipeline lowing command:
Before you can begin querying, analyzing, and sharing data,
you need to get that data into your Snowflake instance. In Install-Package CsvHelper -Version 30.0.1
the following section, you’ll learn how to build a simple
pipeline so you can access data via Snowflake. Building a Once you’ve installed the package, add the code from
pipeline consists of the following steps: Listing 2. This class has a function capable of writing
data to a CSV file with a specified delimiter. My prefer-
1. Create a CSV file. ence is the pipe character (“|”) because commas are more
2. Send CSV to an S3 bucket. prevalent in data than pipe characters.

Figure 2: The Data screen is used to create data elements like databases, schemas, tables, and views.
Figure 3: The Activity log is used to monitor the performance, success, and failure of commands executed against Snowflake.
Figure 4: The Admin screen is used to add Users, Groups, Warehouse, etc.

Snowflake supports multiple types of data formats that
can be used to load data into snowflake tables. Snowflake
supports CSV, JSON, XML, Parquet, and others. FORMATS
describe how the data to be loaded is represented in your
file to import. The following code represents the format of
a CSV file delimited with a pipe character and with data
quoted in double-quote characters.
file_format=(type='csv'
COMPRESSION='AUTO'
FIELD_DELIMITER='|'
RECORD_DELIMITER = '\n'
FIELD_OPTIONALLY_ENCLOSED_BY= '"'
SKIP_HEADER = 1
TRIM_SPACE = FALSE
ERROR_ON_COLUMN_COUNT_MISMATCH = TRUE
ESCAPE = 'NONE'
DATE_FORMAT = 'AUTO'
TIMESTAMP_FORMAT = 'AUTO'
NULL_IF = ('\\N','NULL')
)
The following code builds a STAGE fragment.
namespace SnowflakeDemo;
public class SnowflakeTools

{
public string GetStage(
Figure 5: The Snowsight worksheet screen is used to create, edit, execute, and analyze string bucketName,
Snowflake queries.
string itemKey,
string accessKey, string secretKey)
{
CODE Available Sending Files to S3
Once you’ve created your CSV file, you then need to send it var retval = $"""
The code associated with into an Amazon S3 bucket. This is done using the AWS SDK 's3://{bucketName}/{itemKey}'
this article is both on S3 package that can be installed via NuGet with the follow- CREDENTIALS = (AWS_KEY_ID=
the www.CODEMag.com ing command:
site and at https:// '{accessKey}'
github.com/rjpaddock/ AWS_SECRET_KEY='
Install-Package AWSSDK.S3 -Version 3.7.101.58 {secretKey}')
CodeMagIntroToSnowflake
FORCE=true
Once you’ve installed the package, add the code from """;
Listing 3. This code uses the Amazon SDK to write your return retval;
file to an S3 bucket. You will need a few elements: the }
Bucket you wish to write to, AWS Access, and the Secret }
keys that grant access to this bucket.
The following code builds a FORMAT fragment.
Sending Data to Snowflake
The final step of this process is to copy data from S3 into public string GetFormat(string delimiter)
your Snowflake databases. This is done via the Snowflake {
COPY INTO command. The COPY INTO command has two sec- var retval = $"""
tions that are required to facilitate the loading of your com- file_format=(type='csv'
mand. These are the STAGE and the FORMAT sections of the COMPRESSION='AUTO'
command. Let’s have a look. FIELD_DELIMITER='{delimiter}'
RECORD_DELIMITER = '\n'
A STAGE is a set of code that provides the information SKIP_HEADER = 1
needed to access your S3 data. In particular, you need the TRIM_SPACE = FALSE
bucket, item key, AWS access key, and AWS secret key. The ERROR_ON_COLUMN_COUNT_MISMATCH
following block of code represents a STAGE used to access = TRUE
data via an Amazon S3 bucket: ESCAPE = 'NONE'
DATE_FORMAT = 'AUTO'
's3://[[BUCKET_NAME]] TIMESTAMP_FORMAT = 'AUTO'
/[[ITEM_KEY]]' NULL_IF = ('\\N','NULL')
CREDENTIALS = )
(AWS_KEY_ID='[[AWS_ACCESS_KEY]]' """;
AWS_SECRET_KEY='[[AWS_SECRET_KEY]]') return retval;
FORCE=true }

Listing 1: Code to create sample data
public List<string> GetFirstNames()
public class RandomDataService {
{ var retval = new List<string>();
public class Person retval.Add("Mary");
{ retval.Add("Leslie");
public string UniqueId { get; set; } = retval.Add("Jane");
System.Guid.NewGuid().ToString(); retval.Add("Jessica");
public string LastName { get; set; } = ""; retval.Add("John");
public string FirstName { get; set; } = ""; retval.Add("Paul");
public string Address { get; set; } = ""; retval.Add("George");
public string City { get; set; } = ""; retval.Add("Ringo");
public string StateOrProvince { get; set; } = ""; retval.Add("Eddie");
} retval.Add("Alex");
return retval;
public List<Person> GetSamplePeople(int howMany = 100000) }
{
var lastNames = GetLastNames(); public List<string> GetStreetNames()
var firstNames = GetFirstNames(); {
var addresses = GetStreetNames(); var retval = new List<string>();
var cities = GetCities(); retval.Add("Orange");
var states = GetStatesAndProvinces(); retval.Add("Main");
retval.Add("Maple");
var retval = new List<Person>(); retval.Add("Oak");
for (int i = 0; i < howMany; i++) retval.Add("Poplar");
{ retval.Add("Chestnut");
var person = new Person(); retval.Add("Elm");
person.LastName = lastNames[Between0and9()]; retval.Add("Redwood");
person.FirstName = firstNames[Between0and9()]; retval.Add("Lincoln Blvd");
person.City = cities[Between0and9()]; retval.Add("Sepulveda Blvd");
person.StateOrProvince = states[Between0and9()]; return retval;
person.Address = }
$"{RandomAddressNumber()} " +
$"{addresses[Between0and9()]}"; public List<string> GetCities()
{
var retval = new List<string>();
retval.Add(person); retval.Add("Seattle");
} retval.Add("Austin");
return retval; retval.Add("Regina");
} retval.Add("Calgary");
public int RandomAddressNumber() retval.Add("Winnipeg");
{ retval.Add("Portland");
var random = Random.Shared.Next(1000, 99999); retval.Add("Los Angeles");
return random; retval.Add("Encino");
} retval.Add("Montreal");
public int Between0and9() retval.Add("Ottawa");
{ return retval;
}
var random = Random.Shared.Next(0, 9);
return random; public List<string> GetStatesAndProvinces()
} {
var retval = new List<string>();
public List<string> GetLastNames() retval.Add("AB");
{ retval.Add("SK");
var retval = new List<string>(); retval.Add("CA");
retval.Add("Lucas"); retval.Add("OR");
retval.Add("Smith"); retval.Add("WA");
retval.Add("Spielberg"); retval.Add("TX");
retval.Add("Gygax"); retval.Add("CO");
retval.Add("Garland"); retval.Add("NY");
retval.Add("Wolff"); retval.Add("MN");
retval.Add("West"); retval.Add("KY");
retval.Add("Kardashian"); return retval;
retval.Add("Van Halen"); }
retval.Add("Grohl");
return retval;
} }
Create a COPY INTO Command FROM {stageInfo} {formatInfo};

Once you have your STAGE and FORMAT you’re ready to cre- """;
ate your COPY INTO command. The following code can be return retval;
used to facilitate this process. }
public string GetCopyCommand( The following code shows the final COPY INTO command
string databaseName, that you can execute via a database connection within a
string tableName, Snowsight worksheet.
string stageInfo,
string formatInfo) COPY INTO
{ CODE_MAGAZINE_DEMO
var retval = $""" .PUBLIC.DEMO_DATA FROM
COPY INTO 's3://dashpoint-demo/SampleData.csv'
{databaseName}.PUBLIC.{tableName} CREDENTIALS = (AWS_KEY_ID='ACCESS_KEY'

Listing 2: Use CsvHelper to create a delimited CSV file
using System.Globalization; config.HasHeaderRecord = true;
using CsvHelper;
using CsvHelper.Configuration; //change delimiter
config.Delimiter = delimiter;
//quote delimit
public class CsvTools config.ShouldQuote = args => true;
{
public static void using (
WriteCsvFile( var writer = new StreamWriter(
dynamic dataToWrite, outputFile))
string outputFile, using (var csv =
string delimiter) new CsvWriter(writer, config))
{ {
var config = csv.WriteRecords(dataToWrite);
new CsvConfiguration( }
CultureInfo.InvariantCulture); }
}
//include header
Listing 3: Use the Amazon SDK to write the file to an S3 bucket

using Amazon; string awsSecret)
using Amazon.Runtime; {
using Amazon.S3; var retval =
using Amazon.S3.Transfer; new BasicAWSCredentials(
awsAccessKey, awsSecret);
namespace SnowflakeDemo; return retval;
}
public class AmazonTools
{ public void UploadFile(AmazonS3Client client,
public AmazonS3Client string bucketName, string fileName)
GetS3Client(BasicAWSCredentials creds, {
RegionEndpoint endpointRegion = null) var ms = FileToMemoryStream(fileName);
{ var utility = new TransferUtility(client);
var clientRegion = RegionEndpoint.USEast1; var req = new TransferUtilityUploadRequest();
if (endpointRegion != null) var fi = new FileInfo(fileName);
{ utility.Upload(ms, bucketName, fi.Name);
clientRegion = endpointRegion; }
}
public MemoryStream FileToMemoryStream(string fileName)
var client = {
new AmazonS3Client(creds, clientRegion); var fs = new FileStream(fileName,
return client; FileMode.Open, FileAccess.Read);
} var ms = new MemoryStream();
fs.CopyTo(ms);
public BasicAWSCredentials return ms;
GetBasicAwsCredentials( }
string awsAccessKey, }
AWS_SECRET_KEY='SECRET_KEY') ment is the connection string you’ll use to call Snowflake

FORCE=true via ADO.NET. The next snippet contains a method that
file_format=(type='csv' helps you create a proper connection string:
COMPRESSION='AUTO'
FIELD_DELIMITER='|' public string GetConnectionString(
RECORD_DELIMITER = '\n' string snowflakeIdentifier,
SKIP_HEADER = 1 string userName,
TRIM_SPACE = FALSE string password,
ERROR_ON_COLUMN_COUNT_MISMATCH = TRUE string databaseName,
ESCAPE = 'NONE' string tableName)
DATE_FORMAT = 'AUTO' {
TIMESTAMP_FORMAT = 'AUTO' var retval = $"""
NULL_IF = ('\\N','NULL') account={snowflakeIdentifier};
); user={userName};
password={password};
db={databaseName};
Executing Commands via ADO.NET schema=public;
Once you’ve created your COPY INTO command, you need to warehouse=COMPUTE_WH
call it to load your data into Snowflake. You can run this via """;
a worksheet or via code. Figure 6 shows the code being return retval;
executed via a worksheet. }
If you wish to execute this command from your applica- The following code should be familiar. You simply open a con-
tions, you need to add one more code fragment. This frag- nection, create a command object and call ExecuteNonQue-

ry(). This is what I was talking about earlier. Snowflake meets of different data elements. This is particularly important
you where you are and supports the tools you’re familiar with. when dealing with PII information. This feature is known
as a masking policy in Snowflake. The following com-
using (var conn = new SnowflakeDbConnection() mand shows how to secure data via masking policy. It
{ ConnectionString = connStringCode } creates a masking policy that shows data unmasked for
{ people with the MARKETING role and shows asterisks for
conn.Open(); all others. That policy is then applied to the ADDRESS
var cmd = conn.CreateCommand(); column in the DEMO_DATA table.
cmd.CommandText = copyCode;
cmd.CommandType = CommandType.Text CREATE OR REPLACE MASKING POLICY address_mask
cmd.ExecuteNonQuery(); AS (VAL string) RETURNS string ->
CASE
WHEN current_role() IN ('MARKETING') THEN VAL
ELSE '*********'
The Pipeline Code END
Now you have a nice set of pipeline code. Listing 4 shows ALTER TABLE DEMO_DATA
the result of all the code you created above. Figure 7 shows MODIFY COLUMN ADDRESS SET
the data you created in the pipeline process. MASKING POLICY address_mask;
Figure 8 shows the unmasked data.

Securing Information with Masking
One of the more compelling features of Snowflake is the Figure 9 shows the masked data that most users will
ability to control access fo who can view the contents see.
Figure 6: The COPY INTO code executed via a worksheet
Figure 7: A sample of data imported using the pipeline code

Figure 8: Unmasked data
Figure 9: A sample of data imported using the pipeline code in a masked state due to the ROLE selected via Snowsight
Sharing Data of your Snowflake instances. Inbound shares are data sourc-
Once you have data in Snowflake, you can share that data es provided to you by other Snowflake users.
with other snowflake users. The real benefit of this is that
the consumer of that data pays for the compute costs. With You have two ways to create an outbound share: via SQL
other cloud databases, the owner of the data is responsible code or via Snowsight. This article demonstrates the SQL
for its consumption costs. code way to do this. The first is to execute commands
from a worksheet like this:
You have a couple of choices when sharing data. The first
choice is to share data with a consumer in the same re- CREATE SHARE CODE_MAG_OUTBOUND ;
gion where your Snowflake instance was set up. This is by
far the simplest mechanism for sharing data and, from my GRANT USAGE ON DATABASE
experience, the most common use case. CODE_MAGAZINE_DEMO TO
SHARE CODE_MAG_OUTBOUND
The second choice is to share data with a consumer re- GRANT USAGE ON SCHEMA
siding in a different region or other cloud provider. This CODE_MAGAZINE_DEMO.PUBLIC
type of sharing uses Snowflake’s replication tools. For this TO SHARE CODE_MAG_OUTBOUND;
article, I’ll be exploring the simple use case.
GRANT SELECT ON
CODE_MAGAZINE_DEMO
Outbound Shares and Inbound Shares .PUBLIC.DEMO_DATA
There are two basic share types. Outbound shares allow you TO SHARE CODE_MAG_OUTBOUND;
to share data with other Snowflake users from one or more

Listing 4: The full pipeline code
using System.ComponentModel; @"D:\data\junk\CodeSampleData.csv";
using System.ComponentModel.DataAnnotations; var itemKey = "CodeSampleData.csv";
using System.Data;
using CsvHelper; var random = new RandomDataService();
using Snowflake.Data.Client; var csvTools = new CsvTools();
var amazonTools = new AmazonTools();
namespace SnowflakeDemo var snowflakeTools = new SnowflakeTools();
{
internal class Program
{ var data = random.GetSamplePeople(250000);
static void Main(string[] args) csvTools.WriteCsvFile(data,fileName,"|");
{ amazonTools.UploadFile(amazonTools.GetS3Client(
if (Environment amazonTools.GetBasicAwsCredentials(
.GetEnvironmentVariable("DEMO_KEY") awsAccessKey,awsSecret))
== null || ,bucketName, fileName);
Environment
.GetEnvironmentVariable("DEMO_SECRET") var stage = snowflakeTools.GetStage(
== null || bucketName
Environment ,itemKey
.GetEnvironmentVariable("DEMO_SNOWFLAKE_USER") ,awsAccessKey
== null|| , awsSecret);
Environment var format = snowflakeTools.GetFormat("|");
.GetEnvironmentVariable("DEMO_SNOWFLAKE_PASSWORD") var copyCommand = snowflakeTools.GetCopyCommand(
== null) databaseName
{ ,tableName
Console.WriteLine( ,stage
"You need the following environmental " + ,format);
"variables setup:\n" +
"DEMO_KEY\n"+ var connString =
"DEMO_SECRET\n"+ snowflakeTools.GetConnectionString(
"DEMO_SNOWFLAKE_USER\n"+ snowflakeIdentifier,
"DEMO_SNOWFLAKE_PASSWORD"); snowflakeUser,
Console.ReadKey(); snowflakePassword,
return; databaseName,
} tableName);
var awsAccessKey =
Environment //send the data
.GetEnvironmentVariable("DEMO_KEY"); using (var conn = new SnowflakeDbConnection()
var awsSecret = { ConnectionString = connString })
Environment {
.GetEnvironmentVariable("DEMO_SECRET"); conn.Open();
var snowflakeUser =
Environment var cmd = conn.CreateCommand();
.GetEnvironmentVariable("DEMO_SNOWFLAKE_USER"); cmd.CommandText = copyCommand;
var snowflakePassword = cmd.CommandType = CommandType.Text;
Environment cmd.ExecuteNonQuery();
.GetEnvironmentVariable("DEMO_SNOWFLAKE_PASSWORD"); }
var snowflakeIdentifier = "gra75419"; Console.WriteLine("Data Sent To Snowflake");
var databaseName = "CODE_MAGAZINE_DEMO"; Console.ReadKey();
var tableName = "DEMO_DATA";
}
var bucketName = "dashpoint-demo"; }
var fileName = }
Figure 11: The Get Data screen is used to create

Figure 10: The list of datasets shared with a Snowflake account databases from external shares.
CREATE SECURE VIEW V_SORTED_DEMO_DATA

AS GRANT SELECT ON VIEW
SELECT * FROM DEMO_DATA CODE_MAGAZINE_DEMO
ORDER BY LASTNAME,FIRSTNAME; .PUBLIC.V_SORTED_DEMO_DATA

Figure 12: The Snowsight data screen showing the database created from a share
Figure 13: The shared data has the masking policy in effect
TO SHARE CODE_MAG_OUTBOUND; The share name parameters for this command can be derived
from the screen in Figure 11 from the previous example.
ALTER SHARE CODE_MAG_OUTBOUND
ADD ACCOUNTS=XXXXXX; Once you’ve created a database from a share, you can access
it via Snowsight, as shown in Figure 12.
Creating Databases from Shares Figure 13 shows data shared from the demos created earlier
Once data has been shared with you from another Snow- in this article. One thing you’ll notice immediately is that
flake account, you can access that data via a database the masking policy is in effect on shared data.
that’s created from the share. There are two ways to cre-
ate a database from a share. The first is Snowsight. You I‘ve been working with Snowflake for several years now and it
do this by selecting Data-Private Sharing from Snowsight. wasn’t until 2022 that I realized that the ability to share data
Then choose “Shared with You” on the data sharing screen. in such a simple fashion is monumental. We no longer need to
This brings up a screen with datasets shared with you, as set up S3 buckets or Azure Blog Storage accounts. We simply
shown in Figure 10. grant access to a set of tables/views and the customer can ac-
cess them using the SQL commands they’re already proficient in.
Click on the share you wish to create a database for. You will
be presented with the Get Data dialog, as shown in Figure 11.
Conclusion
From this dialog, you can specify the name of your database. Snowflake is that rare product that possesses immense
You can also optionally specify which security group(s) can power yet can be used by mere mortals to access that pow-
access this database. er. This article only scratches the surface of what Snowflake
is capable of. Other areas of interest to you might include
The second way to create a database from a share is to ex- Python integration, clean room technology, and more in-
ecute a command from a Snowsight worksheet. To create a teresting use cases using another “snow” technology called
database from a share, issue this command: Snowpipe. Give them a look. You won’t be disappointed.
CREATE DATABASE ANOTHER_DEMO Rod Paddock

FROM SHARE XXXXX.YYYYY.CODE_MAG_OUTBOUND

Kubernetes Security for Starters

Kubernetes is the holy grail of modern microservices architecture. It’s software that makes the life of DevOps engineers and
developers much easier. Many, if not even all, modern back-end applications use Kubernetes in one way or another, whether
deployed in-house or hosted by a cloud provider. What is it that makes Kubernetes so valuable? Why does Kubernetes help us
that much? Kubernetes abstracts the entire infrastructure across in my daily work as a senior security consultant.
on which your services run. It doesn’t matter whether They range from broken network isolation over insecure
your services run on bare-metal servers in-house or in host path mounts to missing resource limitations, just to
a cloud environment. Developers don’t have to change name a few. Independent of your level of expertise with
anything when switching from one to the other. This pro- regard to security in Kubernetes, I’ll provide you with
vides a lot of freedom. You move your cluster around as (hopefully new) insights into Kubernetes security.
required. Scaling up your services also becomes trivial. If
you need more services of a particular type to handle the
workload—you just increase the number of Pods of that
Let’s Have a Look at What
service and you’re done. Of course, when you run out of Kubernetes Is Alexander Pirker, PhD
physical resources, you must increase your worker nodes, Before I talk about security in Kubernetes, I’ll give a brief apirker.consultant@gmail.com
but integrating them into the cluster is straightforward introduction to what Kubernetes, also known as k8s, is,
and Kubernetes takes care of scheduling new Pods to the what it does, and how it works. Alexander works as a senior
new worker nodes. security consultant. In his
What’s Kubernetes? daily work, he performs
security audits including
That’s great! Kubernetes provides a lot of flexibility to de- Kubernetes orchestrates the deployment of containers on
assessments, penetra-
velopers and DevOps engineers. Moving around the entire computers down the line. It manages the entire lifecycle
tion testing, and security
cluster, scaling up as necessary, there are so many good of all containers running in such a cluster, including their reviews. He holds secure
reasons to go for it. But what about security in this con- creation and destruction, and also inter-container com- coding workshops, and
text? Is Kubernetes secure? Or, more precisely, did you set munication. For that purpose, the Kubernetes API server also gives trainings for
up your cluster in a secure way? Are there any loopholes? manages, uses, and controls worker nodes. Such nodes Kubernetes security and
can be seen as computational resources, like servers in a provides consulting ser-
Kubernetes provides a lot of configurations to make your data center. Worker nodes just run containers and set up vices for software design
cluster secure. The crucial thing is that you need to know the network routing to enable inter-container and exter- and architecture. He has
about them. Let’s look at a basic example. Think about a nal communication. Of course, under the hood, they per- experience in designing
Kubernetes newbie who starts to configure the first de- form many more tasks, but for understanding Kubernetes, microservices and desktop
ployments and services in a cluster. Because everything it suffices to know that worker nodes run the containers. or mobile applications and
runs in a cluster, our newbie believes it’s isolated from To know which containers the operator wants to run, the also in writing or migrating
everything: the host, the network, the outside world, etc. Kubernetes API server uses a database, the famous etcd them. He received a PhD in
Our newbie couldn’t be more wrong. database. The operator of the cluster defines, in terms of Physics from the University
YAML (yet another markup language) files, which contain- of Innsbruck and holds
By default, the pods can reach the internet without re- ers, services, or other resources should run within the master’s degrees in both
strictions. They can access HTTP servers and so on. They cluster. The Kubernetes API server persists those into the Technical Mathematics and
also don’t have a read-only file system. Hence the ap- etcd database. The Kubernetes API server reads that con- Biomedical Informatics.
plication within the container of the Pod, or anyone else figuration from the database and spawns instances of the In his free time, he likes
to go to the gym and also
in the container (I’ll talk about this later in length), can required resources on worker nodes.
enjoys hiking in the Alps.
modify files. Further, Kubernetes maps, by default, the
service account token of the user running the Pod into Let’s walk through a concrete example to get more famil-
the container. That’s a very handy tool for anyone inside iar with the terms. Suppose you develop a back-end sys-
a container because it serves as an authorization token tem comprising three services, each of those correspond-
to the Kubernetes API server. Using that token, anyone ing to a single container. You have a billing service, a
inside the container could potentially start new Pods, de- shipping service, and an order service. All three together
lete existing deployments, alter ConfigMaps, and so on. make up your back-end application, as shown in Figure 1.
What a nice, insecure, cluster!
If you go with Kubernetes as your deployment infrastruc-
As you see, by default, your cluster is anything but se- ture to run the entire back-end, you probably wrap each of
cure. Depending on the application you run in the cluster, your services first into a Deployment, DaemonSet, Rep-
this poses a significant problem. Suppose your applica- licaSet, or similar resource. Those three types correspond
tion deals with medical data or credit card information. to executable, small, individual applications that run in
Having such security issues in a cluster may even prevent Kubernetes. The operator defines in a YAML file how the
admission to the market. Deployment for instance looks. Specifically, the contents
of this file tell Kubernetes where to find the container
Luckily, you found this article, because I’m here to help image(s) of the application, the ports it requires, the file
you out. I’ll cover the security baseline for a Kubernetes system mounts, etc. The Kubernetes API server, in turn,
cluster. I’ll first explain at a very high level how Kuber- runs such a Deployment as a Pod. A Pod corresponds to a
netes works, what it does, and what the purpose of it is. running instance of a Deployment, DaemonSet, or Rep-
Then I’ll elaborate on the common issues I usually come licaSet. Where do Pods run? They run on a worker node,
codemag.com Kubernetes Security for Starters 49

have their own IP address, etc. You can think of them
like having small virtual machines running on the worker
node (abstractly speaking, since there is actually a big
difference between a virtual machine and a container in
terms of how they run on the host, but it makes it easier
to understand). How does a Pod come to live?
The Kubernetes API server looks at the etcd database and

checks whether all resources the operator configured run
within the cluster. If some are missing, the Kubernetes
API server orchestrates the creation of the missing re-
sources to worker nodes. This corresponds to a control
process, where the Kubernetes API server takes the role
of both the observer and controller. The Kubernetes API
server observes the current state of the entire cluster, and
depending on that state, the server takes actions. Those
actions could be spawning new Pods on worker nodes, for
example, if there aren’t sufficient Pods running. In fact,
the Kubernetes API server also balances out the resource Figure 1: A typical back-end containing three services
load of the entire cluster on the worker nodes to use all
resources efficiently.
To better understand these processes in Kubernetes, it

helps a lot to distinguish two planes, namely the control
plane and the data plane. Kubernetes uses the control
plane, which corresponds to a network connection be-
tween the Kubernetes API server and the worker nodes,
to send control commands between the Kubernetes API
server and the worker nodes. The deployed applications
itself, in contrast, uses the data plane for communica-
tion. Of course, down the line, all of the traffic moves
through the same network(s) between the Kubernetes API
server and the worker nodes, but this distinction makes it
easier to talk and think about Kubernetes. Figure 2 shows
the situation.
What happens now when a Deployment requires three

Pods running to handle all the workload, but they can’t
run on the same worker node due to a lack of resources? Figure 2: The Kubernetes control plane and data plane
In the end, they should act like a single unit.
On top of Deployments and other Pod-creating resources You now understand the basic deployable unit in Kuber-
(there are many more than just Deployment, Daemon- netes, the Pod, a bit better. You know when they come to
Set, or ReplicaSet), operators define Services. Services live and what happens if one of them crashes. Kubernetes
group together Deployments containing a certain label, takes care of recreating your lost Pod to restore the state
and they act as a load balancer to Pods comprising a of the cluster as the operator configured. But Kubernetes
Deployment. Services provide Pods with a single net- also defines other interesting resources. For example,
work identity, one that other Pods use for network con- most back-end applications require some form of configu-
nections. For a Service, it doesn’t matter on which work- ration. Very often, operators configure services through
er node the Pods run. Kubernetes forwards the traffic to config files, like the appsettings.json file when you run
Services that ultimately distribute the workload among an ASP.NET Web API. Instead of baking an appsettings.
their Pods. In total, there are four types of Services, json file into your container image, it would be great to
including ClusterIP (only reachable within the same have it configurable. That allows you to change the con-
cluster) and LoadBalancer (reachable from outside the figuration of your service on-the-fly without rebuilding a
cluster). new Docker image.
To separate services, Kubernetes allows operators to de- For that purpose, Kubernetes provides the ConfigMaps
fine Namespaces. A Namespace in Kubernetes groups to- resource. Operators use ConfigMaps to persist configura-
gether related services and other components into a larg- tion files, or other configurations to the etcd database
er resource. In big companies that run huge back-end ap- of the Kubernetes API server. Deployments, for example,
plications, very often one team owns a single Namespace reference such ConfigMaps within their YAML file, and
and all the services within it. Namespaces allow you to Kubernetes mounts them into the Pods of the Deploy-
group together services forming a cohesive part of your ment. Yes, Kubernetes fully takes care of that. Even more,
entire back-end system. In the example from Figure 1, I Kubernetes also allows you to store your secrets, like, for
identify three Namespaces: billing, shipping, and order. instance, a database password, in Secrets. Secrets work
Figure 3 shows how the billing Namespace looks. like ConfigMaps, but instead of storing them in plaintext
50 Kubernetes Security for Starters codemag.com

like ConfigMaps, Kubernetes enables the encryption of
Secrets to protect sensitive information through the use
of cryptography.
Enough for the Kubernetes intro and how it works. Let’s

talk about why Kubernetes makes sense at all. First, Ku-
bernetes abstracts the underlying infrastructure on which
your services run. And that’s very powerful because it gets
almost trivial to move an entire back-end system from one
place to another. Second, Kubernetes keeps your back-end
application running 24/7, in case of service crashes. When
Pods crash, Kubernetes recreates them for you. So you don’t
need to worry about whether Services crash or not because
Kubernetes brings them back to life. Finally, it makes the
development of services much faster and easier. Developers
write the service applications and the CI/CD pipeline packs Figure 3: A depiction of the billing namespace
those applications into containers. That’s it. Developers
don’t need to know where the services are running, what
specs the servers have, or etc. They just write code. • Internal Attacker with control-plane access: An SPONSORED SIDEBAR:
Internal Attacker with control-plane access has ac-
But Now We Must Talk about Security My Dear… cess to the Kubernetes API server and can interact Are You Writing
I’m really a fan of Kubernetes and I use it myself. But we with it, usually through the kubectl utility. Here, it Secure C# Code?
must talk about security. Before diving deep into indi- depends a lot on the roles within your cluster what
vidual, common security pitfalls in Kubernetes, I’ll rough an attacker can do or not. In the case of very loose Hopefully you are, but
out a threat model for Kubernetes. The model potentially role-based access control (RBAC) rules, it could end “hopefully” isn’t good
enough, is it? Learn to
lacks some details, but I believe it provides a solid base up in a full cluster takeover.
develop robust and secure
to continue the security discussion here.
C# applications in our
Sometimes it helps a lot to distinguish among those online, three-day, hands-
When developing a threat model, the first step is to iden- threat actors to provide more precise statements about on, Secure Coding for C#
tify the assets you want to protect. Let’s have a look at potential attacks. Observe that depending on the threat Developers course. Register
the assets in a Kubernetes cluster. actor, the attack surface also changes totally. The Ex- today for courses starting
ternal Attacker only sees the public-facing API but the in 2023! https://www.
You have the Pods as assets, as those run your applica- attacker doesn’t know anything about the internals of the codemag.com/training
tion, right? Pods potentially depend on other resources, cluster, which services run, etc. In contrast, the Internal
like, for example, ConfigMaps or Secrets. Those could Attacker with data-plane access already sits inside the
contain sensitive information like passwords, database cluster, even though the attacker doesn’t have full access
connection strings, or similar, so they are also valuable to the configuration of the cluster (for the moment). The
and you’d better protect them. On iterating this thought, attacker could certainly destroy the container he com-
you must realize that all configurations comprising your promised, but the blast radius appears limited. Finally,
cluster correspond to assets. the Internal Attacker with control-plane access poses
a major security risk because the attacker could recon-
After identifying the assets, you identify the threat ac- figure your entire cluster, thereby destroying your entire
tors. A threat actor corresponds to an entity. Such enti- back-end.
ties could be humans, but also services or other compo-
nents or programs. A threat actor tries to obtain access to For most of the recommendations and issues I show, as-
the assets, or even destroy them, depending on the asset. sume either an Internal Attacker with data-plane ac-
There are three different types of threat actors: cess or Internal Attacker with control-plane access.
• External Attacker: An External Attacker attacks your

Kubernetes cluster from the outside. This means that Now, We Get Serious
the attacker doesn’t have direct access to the cluster Let’s get your hands dirty and talk about the cool stuff.
and the only way for him to attack is through the pub- To demonstrate the security pitfalls, I’ll use the microser-
lic API, ending up in the data plane of your cluster. vices and cluster setup as shown in Figure 4 throughout
• Internal Attacker with data-plane access: An In- the rest of the article.
ternal Attacker with data plane access is an attack-
er that manages to compromise a container. Very As shown in Figure 4, there are, in total, three
often, this happens through supply chain attacks Namespaces in the cluster: billing, shipping, and order.
in which an attacker manages to get a foot into a The billing Namespace contains the billing and invoice
container through a malicious package in the third- service, the shipping Namespace contains the shipping
party dependencies of an application. The malicious and logistics service, and the order Namespace contains
dependency opens a port and listens for incoming the order and warehouse service. In certain occasions,
commands or performs automated malicious com- those Namespaces will contain further resources, like for
mands within the container. Depending on the con- example ConfigMaps.
figuration of your cluster, the attacker could even
compromise the control plane of your cluster from You’re all set with an example. I promised to provide you
there. More details about such an escalation later. with a starter-kit for Kubernetes security, and here we go.

Pod Security Baseline sociates it with the Pod. Kubernetes also creates a new
First up, I’ll talk about Pod security. Recall that Pods ServiceAccountToken for the ServiceAccount running the
correspond to the entity in Kubernetes tying together Pod. That token enables communication with the Kuber-
several containers, turning them into a manageable re- netes API server and corresponds to a very valuable asset.
source. They form cohesive execution units. In fact, Pods By default, Kubernetes mounts the ServiceAccountToken
ultimately form your application. When Kubernetes brings into each Pod, specifically into the directory /var/run/se-
a new Pod to life on a worker node, several things hap- crets/kubernetes.io/serviceaccount. Why is Kubernetes
pen. Please note that I also use the term host to refer to doing that? Well, sometimes the containers of a Pod need
the worker node of a Pod throughout the article, unless to communicate with the Kubernetes API server to spawn
explicitly specified otherwise. new Pods or to read ConfigMaps or similar. In almost all
cases, it’s not required. Why am I telling you this?
First, Kubernetes identifies that a Pod is missing. How?
Controllers, running on the Kubernetes API server, ob-
serve that the state of the cluster doesn’t match the
configured state of the operator. Hence, a controller on Don’t auto-mount service
the Kubernetes API server searches for a worker node and account tokens unless
instructs the worker node to spawn a new Pod. For that,
the worker node requires some context information, like you really require them.
the ServiceAccount running the Pod. Yes, Pods run in the
context of a ServiceAccount that corresponds to some
form of user in Kubernetes to give Pods an execution There’s a good reason for pointing the auto-mount of
context. Furthermore, the controller also tells the worker ServiceAccountTokens out. If you don’t set up RBAC ac-
node the file system and the file system mounts of the cordingly, those tokens turn into missiles, hitting you
Pod, i.e., the external directories that the Pod requires to hard. Imagine that an attacker managed to compromise a
run. Lastly, because the Pod runs in terms of containers single container and that Kubernetes maps the Service-
on the worker node, the worker node also needs to know AccountToken into the container. The attacker can use
which host user the container process should run (yes, the ServiceAccountToken to communicate directly with
container run as processes). Of course, there’s much more the Kubernetes API server from within the container by
to write about the creation of a Pod, but for our security using, for instance, curl (if available and the Kubernetes
starter kit, it’s sufficient. So, what could go wrong here? API server is reachable). Depending on RBAC, this results
Let’s go through it step by step. in fatal consequences. Suppose that RBAC isn’t set up
properly, and that the actions allowed for a ServiceAc-
A ServiceAccount runs a Pod. For that, the Kubernetes count aren’t restricted. The attacker inside the compro-
API server creates a new ServiceAccount instance and as- mised container could spawn new Pods, delete existing
Deployments, read ConfigMaps, or alter them. The list
goes on and on, but I guess you get an idea of the dam-
age. Figure 5 shows the scenario.
Please note that in case RBAC isn’t set up properly, a

ServiceAccountToken is extremely powerful. So only map
ServiceAccountTokens if you really require them. Other-
wise, disable them by setting the automountServiceAc-
countToken flag for the ServiceAccount, Pod, or Deploy-
ment to false.
Recall that when Kubernetes creates a new Pod, it spawns a

new container. Spawning a new container involves the cre-
ation of the file system of the container. By default, Kuber-
netes makes the root file system of the container read- and
Figure 4: The cluster configuration that I use throughout the rest of this article writable. This means that the application inside the con-
tainer can read and write to the root file system within the
container. It sounds like every container requires that and
it’s absolutely normal for applications. Not necessarily. Of
course, many applications require write access to a direc-
tory, such as to cache data in a file or so. However, is it re-
ally the best choice to have a fully writable root file system?
Again, let’s assume for a second that the container got

compromised by an attacker. In case of a read- and writ-
able root file system, the attacker now has full control
over the file system within the container (given the re-
strictions of the user running the container). Sounds bad,
doesn’t it? It enables the attacker to alter all the files
accessible when the user runs the container. This could
include the application executables itself, in the worst
Figure 5: Service account tokens can be dangerous in the case of container compromise. case. Therefore, depending on the capabilities within the

container (build environment, etc.), an attacker could
even alter the application inside the container. Figure 6
summarizes the situation.
I can provide you with a solution for that issue. Luck-

ily, Kubernetes enables operators to declare the root file
system of a container as read-only by using the readOn-
lyRootFilesystem flag. Setting that flag to true turns
the root file system of your container into a read-only file
system. But where to store data then? That’s easy! Mount
a volume into your container just for the data of your
application. In case an attacker manages to compromise
your container, the only affected asset of the container
will be its data, not more.
Turn the root-file system of

your containers read-only. Figure 6: A writable file system enables an attacker to potentially override your
application or other vital files inside a container.
Finally, recall that I was talking about the user executing

the container. To better understand this statement, I want • Make the root file system of your containers read-
to explain a bit how containers on Linux work. Contain- only whenever possible.
ers correspond to small applications running on the host. • Never run Pods as root user.
How does it work in detail? Well, containers correspond
to isolated processes on the host system, which is also Of course, there are plenty more considerations, recom-
in strong contrast to virtual machines. Virtual machines mendations, and guidelines for how to make Pods se-
don’t run natively directly on the host operating system, cure, but it’s a good starting point to have a first line
but they come with their own kernel, etc. Containers, in of defense. For more information on how to configure
contrast, run directly on the host operating system. They it and how to set things up, I refer you to check out
share the same kernel with all other processes. In an ideal PodSecurityPolicies (for older clusters), the PodAdmis-
case, they run fully isolated from the host in terms of file sionController for newer versions of Kubernetes, and the
system, networking, processes, etc. So, a container, in securityContext attribute for Pods. An excerpt of a se-
the best case, can’t reach the host network or see other curityContext from a Deployment preventing many of
processes running on the same host. The container run- the issues discussed here is shown in the snippet below.
time strictly isolates the container from the host.
[...]
With that in mind, recall that Kubernetes needs to deter- spec:
mine the user running the new container. By default, the dnsPolicy: Default
user that runs the container corresponds to the default enableServiceLinks: false
user, that is, most of the time, the root user. Hence, when [...]
running a container without taking any further precau- automountServiceAccountToken: false
tions, the user executing your application is the root user. securityContext:
I believe it’s clear where this is going now. runAsUser: 1000
runAsGroup: 3000
Suppose an attacker compromised your container. In case fsGroup: 2000
the container runs as root user, it immediately grants the containers:
attacker root privileges inside the container and that’s securityContext:
a big problem. Depending on other configurations that runAsUser: 1000
I’ll discuss briefly, this could allow the attacker to even [...]
compromise the host system. In case the host isn’t reach- readOnlyRootFilesystem: true
able for the attacker, it’s still very powerful, as for non-
read-only root file systems, the attacker could still alter Networking Issues in Kubernetes
sensitive files in the container. Let’s move on a bit and have a look at Kubernetes and
networks. Pods never run fully isolated; they always com-
municate with some other services, client applications, or
even external services. Hence, a Pod hardly ever runs in
Never run Pods as root user. full isolation. Kubernetes is built for microservices, and
those sometimes require a lot of communication. How
does communication in Kubernetes really work? I don’t
The takeaway messages are the following: want to go into all the details regarding networking in
Kubernetes because that topic alone would probably fill
• Never auto-mount ServiceAccountTokens unless an entire book. However, there are certain things to un-
you really need them. derstand.

By default, Kubernetes doesn’t isolate Pods from different
namespaces. This means that Kubernetes, by default, allows
Pods from different namespaces to communicate with each
other without any restrictions. You may be tempted to think
“That’s not an issue! Why shouldn’t they be able to com-
municate?” and to be honest, I was the same. But I have my
reasons to think different now, and I’ll share them with you.
Suppose you have your billing Namespace and order

Namespace. Recall that the billing Namespace comprises
the billing and invoice Service and the order Namespace
the order and warehouse Service. As explained, by default,
Kubernetes allows for all kinds of communication inside the
cluster. This means that the order Service, for example, can
reach the invoice Service. However, imagine that the only
service that really needs to talk to the invoice Service is
the billing Service of the same Namespace. In that case,
the order Service doesn’t need to talk to the invoice Ser-
vice at all, but it can. Figure 7 shows the situation.
Now suppose that an attacker managed to compromise the

Figure 7: Without any further precaution, full communication inside the cluster is possible. order Service, maybe through a vulnerability in a third-
party dependency. Because Kubernetes doesn’t restrict the
communication pathways inside the cluster, the attacker
Recall that Pods in Kubernetes correspond essentially to can now move laterally through the cluster in terms of
(a collection of) containers. The container networking communication. The goal of the attacker is to acquire in-
interface, CNI, oversees managing the networking con- formation about the invoices of customers. Because there’s
siderations in Kubernetes. There are plenty of providers no network isolation, the attacker uses the compromised
that implement CNI, such as Calico. The security goal of Pod of the order Service to query the invoice Service.
networking in a Kubernetes cluster could be described as Most of the time, the services in a Kubernetes cluster don’t
follows: implement any authentication mechanism because authen-
tication usually gets handled by an API gateway or similar
• The cluster internal communication should be iso- in front of the cluster. Hence, the attacker can query the
lated from the host. invoice Service without any restriction and obtain all the
• The host communication should be isolated from information he wants. Nice job by the attacker, bad for us.
the cluster.
• Hosts from the outside should not be able to reach Attackers can cause even more harm than gathering in-
Pods in the cluster unless permitted. formation. Suppose that the invoice Service contains
• Pods from inside the cluster should not be able to endpoints to create new invoices for customers. In nor-
reach services outside unless permitted. mal operation, the billing Service sends a request to the
• Communications inside the cluster should be regu- invoice Service with all the details about the invoice for
lated and secured. a customer. The invoice Service generates an invoice for
the customer and maybe even automatically charges the
I know, I’m enforcing a rather strict view on Kubernetes invoice. In case an attacker reaches the invoice Service,
networking here, but I have good reasons for that. Let’s the attacker could generate invoices for arbitrary victims,
have a look at the situation in Kubernetes without any resulting in financial damage to the victims. Of course,
configuration by the operator. the attack assumes again when the back-end terminates
authentication within an API gateway in front of the clus-
ter. But yes, it’s really a problem here.
Luckily, there’s a way out. Kubernetes allows for the defini-

tion of NetworkPolicies. Such policies define the inbound
and outbound traffic of Namespaces. For that purpose, op-
erators define allowed ports and IP addresses (or even en-
tire networks), either for inbound or outbound communica-
tion. Pods in the Namespace to which the NetworkPolicy
applies obey those rules. Inbound network policies are
often referred to as ingress NetworkPolicies whereas out-
bound network policies are often referred to as egress Net-
workPolicies. In summary, NetworkPolicies allow to you
to define rules for the communication into a Namespace,
but also outside of the Namespace. Figure 8 shows the
situation of the cluster after using NetworkPolicies.
When it comes to networking in Kubernetes, there’s a fur-

Figure 8: NetworkPolicies help to regulate the allowed traffic for a cluster. ther interesting consideration you should be aware of.

NetworkPolicies regulate the traffic among namespaces. How do you prevent access to the host network interface
Recall that a container or Pod in Kubernetes, in the end, of the host? That’s easy. In Kubernetes, you can specify
runs on a worker node, that has its own network interface whether a Pod has access to the host network or not
and connects to a network, which I refer to as host net- by simply using the hostNetwork flag. The flag can be
work. Eaarlier, I talked about what happens if an attacker used directly for Pods and other executable resources like,
can reach other Pods in the same cluster, but what hap- for instance, Deployment. Alternatively, you can specify
pens in case an attacker reaches the host network? the flag as part of a PodSecurityPolicy that applies to a
multitude of Pods at the same time. That will save your
operator a lot of time on configuration. Additionally, you
should also consider putting a firewall or similar in place
Use NetworkPolicies to isolate to really make sure traffic from within the cluster can’t
the networking of namespaces. reach out to your local network of the worker nodes.
I’ll summarize the networking considerations before mov-

ing on:
This opens a completely new attack surface for the at-
tacker. Suppose an attacker managed to compromise a • NetworkPolicies allow you to define networking
Pod of the billing Service and that the container of the rules for Namespaces. For that purpose, use ingress
Pod can reach the host network. Further suppose that rules for inbound traffic and use egress rules for out-
there’s a vault running on the host network, or a data- bound traffic. You can specify those rules for ports,
base intended to be used for internal purposes of your specific IP addresses, or even entire networks.
company only. Well, the attacker inside the billing Pod • Accessing the host network from within a Pod can
can reach those services and servers suddenly. Depending be very dangerous. Keep in mind that everything
on the protection of those services, the attacker could your worker node can reach can also be reached by
run Denial-of-Service attacks, try to acquire sensitive in- a potentially compromised Pod, including network
formation, or similar. As you can see, having access to services on the same network as the worker node
the host network is extremely powerful for an attacker but also other worker nodes. Therefore, disable the
and should be circumvented in any case. Figure 9 shows access to the host network whenever possible and
the situation. put firewalls or similar in place to prevent any un-
foreseen access from a Pod to the host network.
But attacks aren‘t only limited to computers on the host
network. The worker node itself is also a very valuable Using Mounts Can’t Be that Bad, Right?
target. Worker nodes in Kubernetes run kubelets that fa- As I’ve shown before, sometimes Pods in Kubernetes re-
cilitate communication with the Kubernetes API server. quire some external files or even entire folders. For in-
Those kubelets connect to the Kubernetes API server stance, a Pod could require a ConfigMap that contains an
and receive control commands from it. Suppose a worker appsettings.json file for initializing some variables, like
node runs a compromised Pod. Further, assume that the port, TLS keys, or similar. To mount a ConfigMap into a
compromised Pod can access the host network. In that Pod, you use the volumes attribute in the Pod specifica-
situation, the Pod could try to reach kubelets of other tion. You specify the ConfigMap that Kubernetes mounts
worker nodes and cause harm to them. Therefore, in terms via the configMap attribute inside the volumes attribute.
of network compromise, the situation corresponds to a To mount the volume into the Pod now, you create a mount
real nightmare. point in the container specification of the Pod through
setting the volumeMounts attribute using the name of the
ConfigMap in the volumes section and by providing the
path for where to mount the ConfigMap. When starting the
container of the Pod, Kubernetes takes care of creating the
mount on the worker node for the container.
Kubernetes offers many more different kinds of Volumes,

including, for instance, Volumes of type hostPath. These
kinds of Volumes appear very interesting from a security
perspective. Why? They provide an operator with the capa-
bility to mount an entire directory from the host into a con-
tainer. Hence, they could be a good target for attacks, right?
Let’s get a little bit more concrete about Volumes of

type hostPath. I’ll demonstrate now the real danger of
such mounts. Suppose that a Pod requires, for mystical
reasons, the /var/run directory of the host. In normal
applications, this shouldn’t be required, at all, but I was
coming across such configurations quite often. Now, fur-
ther assume that the application in the container even
requires write access to /var/run, to write to files in that
directory. Does it still not sound bad to you? Okay, let’s
Figure 9: Containers with access to the host network have a look at what kind of files reside within /var/run
pose a security risk. on Linux to make the point.

files in the /etc directory of the worker node. Again, you
could immediately say “That doesn’t make any sense. I
can’t imagine that someone really needs that.” Honestly,
I also thought I’d never come across it, but it happened.
Imagine that the operator grants read access to the entire
/etc directory of the worker node instead of drilling down
to the folders that the container really requires. By that,
an attacker that compromised the container suddenly has
full read access to the /etc directory of the host.
Be very careful with hostPath mounts.
This constitutes a severe information leakage because

this directory often contains passwords, keys, or other
sensitive information. However, it gets even worse, com-
bined with a broken network isolation to the host net-
work. Why? Well, because the attacker could use the dis-
covered secrets from /etc to communicate with services
in the host’s network that require the leaked credentials.
Therefore, information leakage through read access to a
Figure 10: Mounting directories from the host file system should be avoided whenever possible. sensitive host directory plus a broken network isolation of
the Pod again means game over.
A little bit of checking the internet reveals that many

Unix sockets, like, for example, the sockets of Docker, re- You eliminate this issue by simply not using hostPath
side within this directory. Furthermore, it contains .pid mounts if possible. In case an application really requires
files of the host, also something valuable to an attacker. access to the host file system, you have two options:
That directory certainly contains many more valuable files
of the host. I believe you’ve got an idea of the sensitivity • Only mount what the container really needs. Do
of the folder now. not mount entire directories into a container when
the container only needs some sub-directories or
To highlight the problem, assume now that an attacker even specific files. Always narrow down the mount
managed to compromise a container with a mount of as far as possible, and never share what’s not re-
hostPath type for the /var/run directory of the host, quired by the container.
with permission to write to it. Further, as we’ve seen be- • Implement a proxy on the host. Sometimes it’s
fore, very often, containers run as the root user of the also possible to implement a service running on
underlying host. That’s an explosive combination! The at- the host node that proxies file system access. Of
tacker suddenly has write access to a very sensitive direc- course, that itself poses a security risk and should
tory of the worker node. What can the attacker use that be done very carefully. However, it potentially gives
for? Well, I mentioned things like Docker sockets earlier, you more control over what’s happening.
but also other vital files for applications running on the
worker node. The attacker now uses the hostPath mount ConfigMaps, a Dangerous Resource
to escape the container and write to files of the hosts / You’ve come across ConfigMaps multiple times through-
var/run directory, thereby potentially destroying applica- out this article. But I wasn’t explaining to you the danger
tions running on the underlying host. Depending on the that emerges from ConfigMaps when not used correctly.
configuration and the setting, the attacker can commu- Don’t get me wrong, ConfigMaps are awesome, and very
nicate with services having sockets in /var/run and that useful, when used correctly. However, when used to sim-
results in the worst case in code execution on the host ply parametrize everything that can be parametrized in
(for example, through spawning new containers using the a container, they turn into bullets against your cluster.
Docker socket). Yes, that means game over! Figure 10
summarizes the issues for the example cluster. To justify this hard statement, I want to explain the ef-
fect of inappropriately using ConfigMaps with several
Observe that in the example of Figure 10, the billing Pod examples. I’ll start with the most severe case, using a
and the invoice Pod both mount the /var/run directory of ConfigMap that contains a script. It happens very often,
the worker node. This implies that both Pods could write to unfortunately, especially in combination with containers
and read from files in this directory. Hence, a compromised that require some form of bootstrapping in terms of shell
billing or invoice Pod poses a real danger to the worker scripts.
node. Furthermore, if both Pods operate on the same files,
one malicious Pod could corrupt the other Pod. Suppose a Pod runs an init script before starting the appli-
cation in the container. Further, assume that Kubernetes
But that’s not the only danger emerging from hostPath mounts the init script from a ConfigMap into the Pod on
mounts. Suppose a container requires read access to some container creation. Then, all operators or entities having

write access to the ConfigMap can alter the init script.
That essentially corresponds to a code execution vulner-
ability. An attacker could simply alter the ConfigMap in
such a way that the intended application in the container
doesn’t start at all! Figure 11 shows an example.
In Figure 11, there’s the billing Pod. The billing Pod

requires the billing ConfigMap, which contains a shell
script named Init.sh. On startup, the billing app contain-
er launches the Init.sh script of the billing ConfigMap.
If an attacker overrides the billing ConfigMap, the billing
Pod gets compromised.
There are more reasons why ConfigMaps are dangerous. Very

often, developers store sensitive information in settings
files, like appsettings.json, for instance. Such sensitive Figure 11: Executing scripts from ConfigMaps provides the possibility to inject code.
information could be an API token, a database connection
string containing username and password or other creden-
tials. Hence, you have an information leakage through the The EncryptionConfiguration resource comes with sev-
ConfigMap. This kind of information leakage happens quite eral options, including the following:
often because it’s very convenient to retrieve the entire
configuration, including sensitive information, from one lo- • identity: Leave the secrets as is and don’t encrypt.
cation. However, from a security perspective, it poses a real This is not secure, so don’t use identity.
danger because everyone with access to the ConfigMap can • secretbox: Applies XSalsa20 and Poly1305. A strong
read out the sensitive information (remember the attacker solution.
with the ServiceAccountToken—this could also be such • aesgcm: Applies AES in Galois Counter Mode. Re-
an attacker!). Figure 12 shows an example of the billing quires a key rotation every 200k write operations.
ConfigMap that contains a user with name “admin” and • aescbc: Applies AES-CBC with PKCS#7 padding. Isn’t
password “test” (yes, very secure, I know). recommended by official Kubernetes website due to
known attacks against CBC.
The full ConfigMap is shown in the snippet below. • kms: This option allows to integrate the encryption
scheme with a third-party Key Management System
apiVersion: v1 (KMS). The preferred choice, if possible.
kind: ConfigMap
metadata:
name: billing-config
namespace: billing Configure encryption for data
data:
at rest in your cluster.
appsettings.json: |
{
"user": "admin",
"pw": "test" As you see, there are several options available, and you’d
} better use one of those. The identity, aesgcm, and aescbc
solutions I’d not consider, due to either no encryption, the
What are alternatives? How do you solve the issue? For necessity to manually rotate keys, or vulnerability to Pad-
scripts, I recommend that you don’t provide scripts in any ding Oracle attacks for CBC mode. Thus, you’re left with
form from the outside to a container. If possible, include either secretbox or kms. If possible, I’d go for the KMS
the scripts within the container image, thereby not pro- integration, but secretbox also offers a strong protection.
viding any chance to alter such scripts. Credentials should
always be stored within Kubernetes Secrets, another re- Why is it important to protect your secrets at rest in the
source type that I’ll discuss next. first place? Because everyone with access to etcd data-
base can readout your secret data at rest, resulting in a
Are Your Secrets Secure? security breach of your secrets.
Kubernetes provides you with resources of type Secrets.
They constitute the perfect resource for storing credentials, Running Out of Resources Is a Bad Thing
passwords, keys, and other sensitive information. So, we I’m slowly approaching the end of the introduction to
use Secrets, and that’s it? Our sensitive information is se- Kubernetes security. But before I’m done, I have two more
cure now? That would be nice, but no. Unfortunately, not. things I want to point out. Let’s have a look at the first
one. Recall that Kubernetes essentially distributes the
By default, Kubernetes doesn’t protect Secrets. It only workload across worker nodes. That’s the goal of Kuber-
applies a base64 encoding to them. Such an encoding netes. To be precise, the Kubernetes API server chooses
doesn’t offer any security. Hence, everyone with access to worker nodes to spawn new Pods on them. To that end,
the Secrets can decode them and read them. Luckily, Ku- the Kubernetes API server requires knowledge about the
bernetes provides the option to encrypt Secrets by con- available resources on each worker node. Observe that
figuring an EncryptionConfiguration. You’d better turn the worker nodes have limited resources, as with every
on your EncryptionConfiguration to protect your secrets. computational system.

So, what happens when you run out of resources? Sup-
pose that the entire cluster eats up too many resources
of your worker nodes to spawn new Pods. Well, in that
case, you have a Denial-of-Service situation, because the
cluster can’t maintain and run the cluster as the opera-
tor specified. Of course, the trivial solution to that is to
increase computational power. However, for the existing
resources that a cluster uses, it’s also vital to manage and
use them properly.
For that purpose, Kubernetes defines two different kinds

of resource types, namely ResourceQuotas and Limi-
tRanges. ResourceQuotas limit the resources that a sin-
gle Namespace uses. In contrast, LimitRanges limit the
resources that a single Pod uses. Both provide a powerful Figure 12: ConfigMaps potentially leak sensitive information.
tool to balance out the resources of the cluster. I illus-
trate the importance of them with two examples. Let’s
start with the first one.
Code Sample Consider the setting that Figure 13 shows. The figure shows
the example cluster with three Namespaces, namely the
You find an example project billing, shipping, and order Namespace. The shipping and
implementing some of the order Namespaces both have ResourceQuotas resources
security recommendations in place, essentially limiting the overall resources of all
from here at https://github. Pods within each of those Namespaces. By resources, I
com/apirker/k8s-sec. mean the sum of all CPU resources or the sum of the entire
Some of the code is also
memory that all Pods of a Namespace occupy together.
available connected to this
However, note that the billing Namespace doesn’t have a
article on www.CODEMag.com.
ResourceQuota resource in place. That essentially allows
the billing Namespace to bind all of the available resourc-
es of the cluster! So, in case an attacker manages to take
over a Pod (whose resources are unconstrained through
LimitRanges or similar) or even more within the billing
Namespace, the attacker could monopolize all the physi- Figure 13: If one namespace doesn’t have a
cal resources of the cluster, thereby bringing the entire ResourceQuota in place, that namespace could
cluster into a Denial-of-Service situation due to resource monopolize resources.
exhaustion. That’s bad for a cluster, because the goal of
Kubernetes is to have your microservices up and running
24/7. Figure 13 depicts the situation. sourceQuotas, here, only the shipping Namespace will
be affected. It’s still quite a severe situation for the en-
tire cluster if one Namespace isn’t working as it should.
Figure 14 shows the situation.
Use ResourceQuotas and LimitRanges
to limit the resources that a In summary, you’d better take care of the resources within
your cluster. Worker nodes and their resources are pre-
namespace or Pods uses, at most. cious because in Kubernetes, it’s all about how to dis-
tribute workload across available resources in a smart
way. The Code Snippet below shows the definition of a
That was for entire Namespaces. Now let’s have a ResourceQuota.
look at one individual Namespace. Consider the ship-
ping Namespace that has a ResourceQuota resource apiVersion: v1
in place to limit the overall resources within the entire kind: ResourceQuota
Namespace. Recall that there were two Services run- metadata:
ning in the shipping Namespace, the shipping Service, name: mem-cpu
and the logistic Service. Suppose each of those Services namespace: order
contains one Pod. Now, let’s assume that the shipping spec:
Pod has a Limits configuration (in the pod definition) hard:
in place, whereas the logistic Pod does not. Well, that’s requests.cpu: "1"
a quite asymmetrical situation. If an attacker now man- requests.memory: 1Gi
ages to take over the logistic Pod, the attacker could limits.cpu: "2"
monopolize all the resources of the shipping Namespace limits.memory: 2Gi
using the logistic Pod, thereby bringing the shipping Pod
into an interesting situation when it comes to acquir- Finally, There Are Also Users in Kubernetes
ing CPU or memory. It potentially results in a Denial-of- Kubernetes always requires operators in one or the other
Service situation for the Services running in the shipping form to set up the cluster or intervene if something isn’t
Namespace. In contrast to the previous example with Re- working as it should. Operators specify the Services run-

ning in the cluster, the ConfigMaps that Pods require,
and so on. How does Kubernetes now manage the permis-
sions an operator has?
For that, Kubernetes uses Role-based Access Control

(RBAC) through Roles and ClusterRoles resources. A
Role applies to a particular Namespace within the Ku-
bernetes cluster whereas a ClusterRole applies to the en-
tire cluster. Such resources, i.e., Roles and ClusterRoles,
are additive, and there are no “deny” rules. Both serve a
single purpose. The operator setting up RBAC defines the
permissions a certain Role or ClusterRole has regarding
resources within the cluster. For instance, suppose you
want to define that engineers with access to the shipping
Namespace can create new Pods. In that case, the opera-
tors define an Engineer Role for the shipping Namespace
that grants the verb create to the resource type Pod.
Roles and ClusterRoles build around resources and verbs. Figure 14: Limit and LimitRanges restrict the resource consumption of individual Pods.
The resource corresponds to the resources of a certain
type that the cluster runs, like, for example, Pods, Con-
figMaps, or other kinds of Kubernetes resources. The
verbs specify the action a certain Role or ClusterRole is
allowed to perform for that resource. Verbs include “get”
or “list,” but also “create,” for example. If a certain Role
or ClusterRole should be able to read Secrets, you grant
the “get” verb on Secrets. To associate a certain Role or
ClusterRole with an operator, you specify RoleBindings
or ClusterRoleBindings. Those two resources associate
subjects, like a user, with a Role or ClusterRole. As you
see, it’s straightforward to set up the access control for
your cluster.
However, there’s a catch. If you set it up with very per-

missive Roles and ClusterRoles, really bad things can
happen. Suppose you didn’t set up RBAC properly and an
attacker manages to take over an account of your clus-
ter. Depending on the Roles and ClusterRoles associated
with the account, the attacker could destroy your cluster. Figure 15: Risky RBAC permissions could result in a full
Figure 15 shows an example of what could happen. cluster/namespace takeover.
In Figure 15, the attacker managed to take over an ac-

count of the billing Namespace. The permissions for Let’s Summarize
this account appear very risky, because the attacker can We’ve come a long way. Now it’s time to wrap it up. What
exec into a Pod using the exec command in Kubernetes. did we learn from all this? First, I gave a brief intro-
This allows the attacker to open a remote shell into the duction to what Kubernetes is, how it works, and what
invoice Pod through a command to the Kubernetes API it does. I talked about different kinds of attackers and
server, thereby having shell access to the Pod. But the their capabilities. Next, I provided an overview of dif-
attacker could also create a new Pod because the Role ferent Pod security considerations, like auto mounting
of the account the attacker took over allows the verb service account tokens, for instance, or running a Pod as
“create” for the resource Pod. Finally, the attacker could root user. Then I gave an overview of network security in
even delete existing Pods due to the permissions granted Kubernetes, where NetworkPolicies play an important
by the Role. role. From there, I continued the security investigation
with dangerous host directory mounts. As you’ve seen,
a lot of issues can be mitigated by simply isolating the
worker nodes from your Pods. ConfigMaps also turned
Only allow verbs and resources out to be a potential problem when not used properly.
on Roles and ClusterRoles They could contain scripts or secret information. Then
I talked about Denial-of-Service situations in a cluster
that operators really require. due to missing resource limits and how to mitigate such
scenarios. Finally, I provided you with some insights
into RBAC in Kubernetes, and the danger when not used
properly.
As you see, having proper RBAC set up is extremely impor-
tant for a cluster, because it could lead to a full cluster or Alexander Pirker
Namespace compromise.

Building an Event-Driven .NET Core

App with Dapr in .NET 7 Core
Today, businesses are required to be capable of handling increasing amounts of data in a fast, efficient, and reliable manner.
Event-driven architecture (EDA) is an increasingly popular option to help companies process complex data streams in real-
time. Event-driven architecture has been gaining traction in the software development world lately, and with good reason.
In today’s digital age, it’s increasingly important for de- • Scalability: The extent to which the application can
velopers to be able to create event-driven applications serve requests by creating new instances of the ap-
quickly and efficiently. One technology that’s helping plication
make this possible is the Distributed Application Runtime • Synchronous model: The client sends an HTTP re-
(Dapr). This open-source framework enables developers to quest to the web server and then waits until it re-
build serverless and microservice applications using any ceives a response.
language and runtime, making it an ideal platform for
building modern cloud-native applications. The traditional communication pattern used in most web
applications is the request-response approach. This model
Joydip Kanjilal This article discusses event-driven architecture, its archi- involves the client making a synchronous HTTP request
joydipkanjilal@yahoo.com tectural components, the concepts related to Dapr, why to the web server and then waiting for a response. The
it’s useful, and how you can work with it in .NET 7 Core. request-response model is also used in a microservices
Joydip Kanjilal is an MVP architecture. Eventually, there’s a chain or series of HTTP
(2007-2012), software In order to work with the code examples discussed in this requests and responses working in a synchronous manner.
architect, author, and article, you need the following installed in your system:
speaker with more than
Now, let’s assume that one of the requests in the chain
20 years of experience.
• Visual Studio 2022 waiting for a response times out because it doesn’t re-
He has more than 16 years
• .NET 7.0 ceive a response from the web server. As a result, your
of experience in Microsoft
.NET and its related • ASP.NET 7.0 Runtime application becomes non-responsive or blocked. An in-
technologies. Joydip has • Dapr Runtime crease in the number of services increases synchronous
authored eight books, HTTP requests and responses, which explains why a failure
more than 500 articles, If you don’t already have Visual Studio 2022 installed, of one of the systems will also impact the other systems.
and has reviewed more you can download it here: https://visualstudio.microsoft.
than a dozen books. com/downloads/.
What Are Events and Notifications?
You’ll be building two event-driven applications using In the content of an event-driven applications, an event
Dapr. In this article, you’ll: is defined as a significant change of state that can trig-
ger an action. For example, a user enters login credentials
• Understand the importance of event-driven applica- in a login form and then presses the Login button. An
tions event occurs when something happens, like a user clicks
• Learn Dapr and its architectural building blocks on a button or presses a key. The application waits for
• Build a simple Minimal API application (named an event and when one occurs, it executes an appropri-
DaprDemo) in ASP.NET 7 Core ate handler. The handler can perform any task, including
• Configure the DaprDemo application to provide sup- changing the application’s state.
port for Dapr
• Connect to the DaprDemo Minimal API application An event has the following characteristics:
using Dapr HttpClient
• Connect to the DaprDemo Minimal API application • It serves as a record that something has occurred.
using .NET HttpClient • It’s lightweight, distributed, and encapsulates a
• Connect to the DaprDemo Minimal API application change of state.
using DaprClient • It can be distributed via channels that include
• Implement a Publish/Subscribe application using streaming and messaging.
Dapr • It’s immutable, which means that it can’t be altered
or removed.
• It can be persisted indefinitely.
From a Traditional Request- • It can be consumed an unlimited number of times
Response Model to an Event-Driven by an event consumer.
Approach
There are several challenges that modern-day web appli- A notification refers to a message used to inform the oc-
cations face: currence of an event within the application, such as a
new record added to the database, an email sent, etc.
• Availability: Whether one or more services are avail- Typically, this consists of a unique identifier, the details
able—up and running—to serve incoming requests of an event, and the context, such as the date, time, loca-
60 Building an Event-Driven .NET Core App with Dapr in .NET 7 Core codemag.com
tion, etc. An event consumer can determine whether the
event should be processed from this metadata.
Event Schema
An event schema denotes a specified format used to
define an event record. Although the producers publish
event records that comply with this schema, the consum-
ers should know this format to read one or more events.
Here’s how a typical event would look:
{
"message_type": "user_login", Figure 1: Demonstrating the components of an Event-Driven Architecture
"email": "someuser@example.com",
"content": "Thanks, you're now logged in!"
} consumer to process the messages asynchronously. Typi-
cally, an event-driven architecture is made up of the Pro-
ducer, the Message Broker, and the Consumer.
Event-Driven Architecture
An event-driven architecture is defined as a software A publisher publishes or creates a message to describe the
architectural pattern in which a conglomeration of de- event, converts it into a message, and presents it to the
coupled components is capable of publishing and sub- event router for further processing. The event producer
scribing to events asynchronously via an event broker. and the event consumer (also known as the event sub-
Event-driven architectures can create, identify, consume, scriber) are decoupled from each other.
and respond to events and can be asynchronous and mes-
sage-based. An event-driven architecture is a good choice A message broker acts as an intermediary to acquire,
for building applications that are loosely coupled, such as store, and deliver events to the event consumers. A mes-
microservices. Figure 1 shows the components of a typi- sage broker should be highly scalable and reliable and
cal event-driven architecture. ensure that events are not lost during a system failure.
It should be noted that the event broker is optional if
Benefits and Downsides you have a single event producer connected directly to a
The primary advantages of an event-driven architecture single event consumer, i.e., the event producer can send
include increased responsiveness, scalability, and agil- the messages to the event consumer directly.
ity. Responding to real-time information and integrating
additional services and analytics can instantly improve There are two types of message brokers: store-backed and
business processes and customer experiences. Most or- log-based. The former stores the events in a data store to
ganizations believe that the benefits of modernizing IT serve one or more consumers and then purges the events
infrastructure outweigh the costs associated with event- once they have been delivered, and the latter stores the
driven architecture. events in logs and persists the events even after they are
delivered.
In an event-driven architecture, because the event produc-
ers and consumers are decoupled from each other, the out- The consumer receives the message from the event broker
age of one service doesn’t affect the availability of other and performs the appropriate action, i.e., processes the
services. As a result, even when consumers are unavailable, events asynchronously. In other words, a consumer receives
producers can continue to produce event messages. Like- notifications of newly created events and processes those
wise, a consumer listens for the availability of new event events asynchronously. Figure 2 shows the producer, con-
messages but isn’t affected if the producer is down. sumer, and event broker in action where two of the three
consumers have subscribed to more than one producer.
Event-driven architecture provides several benefits:
An event-driven architecture works asynchronously and in
• Loose coupling a decoupled manner, which allows it to scale efficiently.
• Immutability The producer sends a notification when an event occurs
• Independent failure but it’s not bothered about the destination of the noti-
• Scalability fication, i.e., where the notification will eventually be
delivered.
However, there are certain downsides to using event-
driven architecture: Instead, the event broker (also known as an event router)
is responsible for distributing the events as appropriate.
• Increased complexity Event consumers (also known as sinks) start processing
• Difficulty in monitoring the events as they arrive. This allows all of the services
• Difficulty in debugging and troubleshooting to process the events asynchronously. Figure 3 illustrates
how an event broker works.
Components of Event-Driven
Architecture Event-Driven Architecture Patterns
In an event-driven architecture, the producers and con- Event-driven architectures have two different architec-
sumers are decoupled from one another, enabling the ture models: pub/sub and event streams.
codemag.com Building an Event-Driven .NET Core App with Dapr in .NET 7 Core 61
In a pub/sub model, each consumer gets messages in a ten to, process, and transmit events using a decoupled
topic in exactly the same order that they were received. architecture. In a typical event-driven architecture, the
By subscribing to event streams, this messaging pattern application is a conglomeration of different components
enables asynchronous communication between disparate that communicate with each other via events. Events are
system components. When an event is published, an messages sent between various components of the appli-
event notification is delivered to all subscribers who have cation. Events can be triggered using user interactions or
subscribed to the event. It should be noted that multiple triggers from external services via webhook.
event subscribers can listen to the event.
Key Concepts: Publishers, Subscribers, Sources, and Sinks
An event streaming model involves processing a sequence In contrast to conventional architectures that react to
of events in an asynchronous manner. Instead of deliver- user-initiated queries and deal with data in batches that
ing the events to the event subscribers, the published are added and altered at predefined intervals, event-
events are written to a stream store, usually a log. The driven architecture-based applications respond to events
event consumers check the stream store or the log to as they occur. Publishers, subscribers, sources, and event
determine whether new messages are available to be pro- consumers (sinks) are the essential principles of event-
cessed. Additionally, because the events are persisted, driven architecture.
the event consumers can join and start reading events
from an event stream at any point of time. A publisher is the component responsible for capturing
event data and storing it in a repository. The subscriber
is responsible for consuming the event data and respond-
Use Cases of Event-Driven Architecture ing to the event. A source denotes the component where
Some of the common use cases of event-driven architec- the event originates, and sinks represent the destinations
ture are: where the event subscribers send data.
• Real-time monitoring systems

• Microservices
How Do Event-Driven Applications
• Parallel processing Work?
• Connecting IoT devices for data ingestion/analytics Event-driven applications are governed by events that
• Payment processing can ascertain the flow of program execution. An event
• Fraud detection can happen inside or outside the program that requires
SPONSORED SIDEBAR: some action from the program. An event is any occurrence
that requires the application to respond. A user, another
Ready to Modernize What’s an Event-Driven Application? external system, or the application can trigger events.
a Legacy App? An event-driven application is one whose control flow is Events can be anything from user input to a new file be-
Need FREE advice on determined by events. It’s an application that can lis- ing uploaded to the server.
migrating yesterday’s
legacy applications to Events can also be generated by other systems outside
today’s modern platforms? the program, such as a database server that sends no-
Get answers by tifications when new data is available. When an event
taking advantage of occurs, the program looks for an event handler—a piece
CODE Consulting’s years of of code that knows how to respond to the event—and ex-
experience by contacting ecutes it. An event-driven application contains an event
us today to schedule loop that listens to events, and when it identifies one, it
your free hour of triggers a callback function.
CODE consulting call.
No strings. No commitment. Instead of delivering messages to a specific queue, event-
Nothing to buy. driven apps publish messages to a common topic. You can
For more information, then inform one or more event subscribers when an event
visit www.codemag.com/ occurs. Event-driven applications take advantage of an event
consulting or email us at broker responsible for transporting the events to their respec-
info@codemag.com.
Figure 2: The producer, consumer, and event broker in action. tive destinations. An event broker is defined as middleware
that’s responsible for delivering events to all subscribers.
Applications built on an event-driven model use a “push”

model of communication rather than a “pull” model. In this
model, the sender pushes messages to recipients as needed
instead of waiting for them to request information. The
advantage of this approach is that it allows many different
processes to share information without any one process
being overwhelmed by requests from its peers.
What’s Dapr?
Dapr is an open-source, language-agnostic, event-driven
runtime that enables developers to build resilient and
Figure 3: Demonstrating how event dispatch works in an event-driven architecture distributed applications that are portable, reliable, and
scalable. Dapr’s runtime handles the underlying infra- • Publish and subscribe: Provides support for scal-
structure, so developers can concentrate on developing able, secure messaging between the services
their applications rather than focusing on the infrastruc- • Bindings: As part of the bindings building block,
ture. Built on the actor model, it allows loose coupling enables external resources to be attached to it for
of components and helps to make your app more resilient the purpose of triggering a service or being called
when a failure occurs. from a service.
• Actors: Encapsulates the necessary logic and data
Dapr is a distributed application runtime that simplifies to manage state in reusable actor objects
the development of cloud-native applications, enabling • Configuration: Used for easy sharing of application
developers to focus on business logic rather than in- configuration changes, and for providing notifica-
frastructure components. Dapr has several architectural tions when changes to the configuration are made
components that make it an ideal choice for developing • Observability: Used to monitor and quantify mes-
applications in the cloud. sage calls across applications, services, and compo-
nents deployed using a Dapr sidecar.
Sidecar Architecture • Secrets: Allows access to external secret stores in a
Dapr can coexist with your service by using a sidecar, secure manner.
which allows it to operate in its memory process or con-
tainer. A sidecar refers to a secondary piece of software
that can be deployed alongside the main application and Using Dapr in Distributed Applications
run in tandem, typically in a separate process. Dapr is a great fit for event-driven architectures. Its
pub/sub model makes it easy to decouple services and
Because sidecars are external to the service they connect scale them independently. Dapr uses a pub/sub model for
to, they may offer isolation and encapsulation without communication between services. This means that each
impacting the service itself. This isolation enables you to service can subscribe to events from other services, and
create a sidecar using many different programming lan- publish its own events. Building blocks of Dapr provide
guages and platforms while allowing it to have its run- capabilities that are common to distributed applications
time environment. such as state management, service-to-service invocation,
pub/sub messaging, etc.
Why Use Dapr for Building Dapr reduces the inherent complexity associated with dis-
Event-Driven Applications? tributed microservice applications. Being event-driven,
Dapr provides a number of benefits for developers build- Dapr, plays an essential role in developing microservices-
ing event-driven applications: based applications because you can leverage Dapr to de-
sign an application that can react to events from external
• Portability: Dapr applications can be deployed on systems efficiently and produce events to notify other
any platform, including Kubernetes, Service Fabric, services of new facts.
and Azure Functions.
• Scalability: Dapr can scale horizontally and verti-
cally, meaning that applications can be scaled up Getting Started with Dapr
or down as needed without requiring code changes. You can get started with Dapr locally using the Dapr Com-
• Resilience: The actor model used by Dapr helps to mand-Line Interface (CLI). You can download a copy of it
make your app more resilient against failures. If one from here: https://docs.dapr.io/getting-started/install-
component fails, the others can continue working, dapr-cli/. You can configure Dapr to run in a self-hosted
ensuring that your app remains responsive even in mode, as a serverless solution such as Azure Container
the face of adversity. Apps, or deployed in Kubernetes. To verify whether the
• Flexibility: Developers can choose from a variety of Dapr CLI has been installed properly, use the following
programming languages when building Dapr apps, command at the console window or at the Windows Pow-
including C#, Java, Node.js, and Python. erShell window:
• Ease of use: Dapr apps can be deployed using a
simple YAML file, making it easy to get started with dapr
event-driven development.
• Loose coupling: A Dapr-based application’s compo- Assuming Dapr has been installed successfully on your
nents are loosely connected, meaning they may be computer, when you run the Dapr executable at the Dapr
built and deployed independently. As a result, your CLI command prompt, a list of all available commands of
application will be more modular, manageable, and the Dapr executable is displayed. Figure 4 captures the
simpler to maintain and extend over time. list of these commands at the Dapr CLI.
To initialize your Dapr environment, use the following

Dapr Architecture: command, as shown in Figure 5:
The Building Blocks of Dapr
Dapr encompasses a set of building blocks: dapr init
• Service-to-service invocation: Responsible for per- If you’d like to initialize Dapr without using Docker, you
forming secure, direct, service-to-service calls can use the following command instead:
• State management: Supports the creation of long-
running stateful and stateless services dapr init –slim
Figure 4: Verifying the Dapr CLI installation
Figure 5: Initializing the Dapr environment
Building Event-driven Applications 3. Specify the project name as DaprDemo and the path
Using Dapr in ASP.NET 7 where it should be created in the Configure your
new project window. If you want the solution file
In this section, you’ll examine how to use Dapr to build and project to be created in the same directory, you
event-driven applications in ASP.NET 7. can optionally check the “Place solution and proj-
ect in the same directory” checkbox. Click Next to
Create a New ASP.NET 7 Project in Visual Studio 2022 move on.
You can create a project in Visual Studio 2022 in several 4. In the next screen, specify the target framework as
ways. When you launch Visual Studio 2022, you’ll see the .NET 7 (Standard Term Support) and the authenti-
Start window. You can choose “Continue without code” cation type as well. Ensure that the “Configure for
to launch the main screen of the Visual Studio 2022 IDE. HTTPS,” “Enable Docker Support,” and the “Enable
OpenAPI support” checkboxes are unchecked be-
To create a new ASP.NET 7 Project in Visual Studio 2022: cause you won’t use any of these in this example.
5. Because you’ll be using minimal APIs in this example,
1. Start the Visual Studio 2022 IDE. uncheck the checkbox “Use controllers” (uncheck to
2. In the Create a new project window, select ASP.NET use minimal APIs).
Core Web API and click Next to move on. 6. Click Create to complete the process.
You’ll use this application in the subsequent sections in Alternatively, you can install these package(s) from the
this article. NuGet Package Manager Window. To install the required
packages into your project, right-click on the solution
Create the DaprDemo Minimal API and the select Manage NuGet Packages for Solution....
Open the Program.cs file or the Minimal API project you Now search for these package(s) one at a time in the
just created and replace the default-generated code with search box and install it/them.
the code shown in Listing 1.
Create a New Console Application Project in Visual
In Listing 1, I’ve created a record type named Product. Studio 2022
This record type is used in the HTTP GET endpoint /all- Let’s create a console application project that you’ll use
products that returns a list of records. You can update the for calling Dapr sidecar. You can create a project in Vi-
profiles section of the launchsettings.json file to launch sual Studio 2022 in several ways. When you launch Visual
the endpoint when your application starts, like this: Studio 2022, you’ll see the Start window. You can choose
Continue without code to launch the main screen of the
"profiles": { Visual Studio 2022 IDE.
"http": {
"commandName": "Project", To create a new Console Application Project in Visual Stu-
"dotnetRunMessages": true, dio 2022:
"launchBrowser": true,
"launchUrl": "allproducts", 1. Start the Visual Studio 2022 IDE.
"applicationUrl": "http://localhost:5239", 2. In the Create a new project window, select “Console
"environmentVariables": { App”, and click Next to move on.
"ASPNETCORE_ENVIRONMENT": "Development" 3. Specify the project name as DaprDemoClient and the
} path where it should be created in the Configure
}, your new project window.
"IIS Express": { 4. If you want the solution file and project to be cre-
"commandName": "IISExpress", ated in the same directory, you can optionally check
"launchBrowser": true, the “Place solution and project in the same direc-
"launchUrl": "allproducts", tory” checkbox. Click Next to move on.
"environmentVariables": { 5. In the next screen, specify the target framework you
"ASPNETCORE_ENVIRONMENT": "Development" would like to use for your console application.
} 6. Click Create to complete the process.
}
} You’ll use this application in the subsequent sections of
this article.
When you run the application, you’ll see the list of prod-
uct records displayed on your web browser. Install NuGet Package(s)
Because you’ll be using Dapr Client in this example, you
Install NuGet Package(s) should install the Dapr.Client package. You can do this
You can install the packages inside the Visual Studio 2022 from inside the Visual Studio 2022 IDE or by running the
IDE by running the following command(s) at the NuGet following command(s) at the NuGet Package Manager
Package Manager Console: Console:
Install-Package Dapr.AspNetCore Install-Package Dapr.Client
Listing 1: The Program.cs file of the Minimal API project

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build(); products.Add(new Product
{
app.MapGet("/allproducts", () => Id = Guid.NewGuid(),
{ Name = "LG TV",
List<Product> products = new List<Product>(); Quantity = 15,
Price = 55000
products.Add(new Product });
{ return products;
Id = Guid.NewGuid(), });
Name = "Sony TV",
Quantity = 25, app.Run();
Price = 90000
}); public record Product
{
products.Add(new Product public Guid Id { get; init; }
{ public string Name { get; init; }
Id = Guid.NewGuid(), public int Quantity { get; init; }
Name = "Samsung TV", public decimal Price { get; init; }
Quantity = 50, }
Price = 65000
});
Alternatively, you can install these package(s) from the --app-port 5239 -- dotnet run
NuGet Package Manager Window. To install the required
packages into your project, right-click on the solution Let’s now understand the different options of the Dapr
and the select Manage NuGet Packages for Solution.... command you used:
Now search for these package(s) one at a time in the
search box and install it/them. • app-id: Specifies the application or service ID for
service discovery
Using Dapr Sidecar with Dapr HttpClient • dapr-http-port: Specifies the HTTP port that Dapr
Lastly, open the Program.cs file of the console application should listen to; 3500 in this example
project you just created and replace the default generated • app-port: Specifies the port the application will lis-
code with the source code provided in Listing 2. ten to; 5239 in the example
• app-ssl: Optionally turns on HTTPS when Dapr in-
Execute the Application voked the application.
Navigate to the directory that contains your MinimalAPI proj- • dotnet run: Optionally executes your WebAPI.
ect file, and then execute the following command at the com-
mand window or the Windows PowerShell window to launch Next, navigate to the directory where the DaprDemoCli-
the productservice alongside a Dapr sidecar application: ent project file resides and run the productserviceclient
service alongside a Dapr sidecar:
dapr run --app-id productservice
--dapr-http-port 3500 dapr run --app-id productserviceclient
Listing 2: The Program.cs file of the Dapr HttpClient

using Dapr.Client; Console.WriteLine($"Id:{product.Id},
using System.Net.Http.Json; Name:{product.Name},
Quantity:{product.Quantity},
var client = DaprClient.CreateInvokeHttpClient(); Price: {product.Price}");
}
var products =
await client.GetFromJsonAsync Console.Read();
<List<Product>>(
"http://productservice/allproducts"); public record Product
{
Console.WriteLine public Guid Id { get; init; }
("Displaying product information"); public string Name { get; init; }
public int Quantity { get; init; }
foreach (var product in products) public decimal Price { get; init; }
{ }
Listing 3: The Program.cs file of the .NET HttpCLient

using System.Net.Http.Json; Name:{product.Name}, Quantity:{product.Quantity},
Price: {product.Price}");
var client = new HttpClient(); }
var products =
await client.GetFromJsonAsync<List<Product>>( Console.Read();
"http://localhost:3500/v1.0/invoke/
productservice/method/allproducts"); public record Product
{
Console.WriteLine("Displaying product information"); public Guid Id { get; init; }
public string Name { get; init; }
foreach (var product in products) public int Quantity { get; init; }
{ public decimal Price { get; init; }
Console.WriteLine($"Id:{product.Id}, }
Listing 4: The Program.cs file of the DaprClient

using Dapr.Client; Name:{product.Name}, Quantity:{product.Quantity},
using System.Net.Http.Json; Price: {product.Price}");
}
using var client = new DaprClientBuilder().Build();
Console.Read();
var products =
await client.InvokeMethodAsync<List<Product>>( public record Product
HttpMethod.Get, "productservice", "allproducts"); {
public Guid Id { get; init; }
Console.WriteLine("Displaying product information"); public string Name { get; init; }
public int Quantity { get; init; }
foreach (var product in products) public decimal Price { get; init; }
{ }
Console.WriteLine($"Id:{product.Id},
Figure 6: Connecting to Dapr Sidecar using Dapr HttpClient
--app-protocol http Listing 5: The Subscriber Minimal API application

--dapr-http-port 3501 -- dotnet run const string PUBSUB = "orderpubsub";
const string TOPIC = "orders";
Figure 6 shows the output when both these applications
var builder = WebApplication.CreateBuilder(args);
are executed: var app = builder.Build();
Using Dapr Sidecar with .NET HttpClient app.MapGet("/orderprocessing/subscribe", () => {

Console.WriteLine
To call Dapr sidecar with a HttpClient, replace the source ($"The Dapr pub/sub is now subscribed to pubsub:
code of the Program.cs file of the DaprDemoClient project {PUBSUB}, topic: {TOPIC}");
with the code shown in Listing 3. });
app.MapPost("/orders", (Order order) => {

Using Dapr Sidecar with DaprClient Console.WriteLine
To call Dapr sidecar with a DaprClient, replace the source ($"The subscriber received Order Id:
code of the Program.cs file of the DaprDemoClient project {order.Id}");
return Results.Ok(order);
with the code given in Listing 4. });
await app.RunAsync();
Implementing a Publish and
public record Order
Subscribe (Pub/Sub) Application {
In this section, you’ll build a simple pub/sub applica- public int Id { get; set; }
public string? Product_Code { get; set; }
tion. The application consists of two projects: a Minimal }
API project and a Console Application project. Follow the
steps mentioned earlier to create the Minimal API and
Console Application projects.
Execute the Publish and
Create the Subscriber Subscribe Application
Replace the default-generated source code of the Pro- Navigate to the folder where the project file of the Mini-
gram.cs file pertaining to the Minimal API project mal API project resides and execute the following com-
(the Publisher in this example) with the code given in mand at the command window or the Windows PowerShell
Listing 5. window to launch the orderprocessing service alongside a
Dapr sidecar application:
Create the Publisher
Replace the default-generated code of the Program.cs file dapr run --app-id orderprocessing
pertaining to the Console Application with the piece of --app-port 7001 --dapr-http-port 3501
code shown in Listing 6. -- dotnet run
Listing 6: The Publisher Console application
using System.Text; for (int i = 1; i <= 5; i++)
using System.Text.Json; {
var order = new Order() { Id = i,
var baseURL = "http://localhost:3500"; Product_Code = "P000" + i.ToString()};
const string PUBSUB = "orderpubsub"; var content = new StringContent
const string TOPIC = "orders"; (JsonSerializer.Serialize
const string APP_ID = "orderprocessing"; <Order>(order), Encoding.UTF8,
"application/json");
Console.WriteLine($"Publishing to baseURL: var response = await httpClient.
{baseURL}, Pubsub Name: {PUBSUB}, Topic: {TOPIC} "); PostAsync($"{baseURL}/orders", content);
Console.WriteLine($"Published Order Id:
var httpClient = new HttpClient(); {order.Id}");
httpClient.DefaultRequestHeaders. }
Accept.Add(new System.Net.Http.Headers.
MediaTypeWithQualityHeaderValue public record Order
("application/json")); {
httpClient.DefaultRequestHeaders.Add public int Id { get; set; }
("dapr-app-id", APP_ID); public string Product_Code { get; set; }
}
Figure 7: The Publish/Subscribe application in action!
Next, navigate to the folder where the Console Applica- Conclusion

tion project file resides and run the orderpublisher service Dapr offers easy-to-use tools that make developing
alongside a Dapr sidecar: and deploying event-driven applications easy. With its
strong focus on scalability, resiliency, and portability,
dapr run --app-id orderpublisher Dapr is an ideal choice for developers looking to cre-
--app-protocol http --dapr-http-port 3500 ate robust, scalable, and distributed applications with
-- dotnet run .NET Core.
You can hit this endpoint using Postman or Fiddler or any Note that, for the sake of simplicity, I’ve used Order and
other HTTP Client. Alternatively, open your web browser Product record types twice in the code examples—you
and browse this link to register the Dapr pub/sub subscrip- can create a class library that contains these two types
tion: http://localhost:7001/orderprocessing/subscribe and then import them into your projects to avoid code
redundancy.
When you run both the applications, you’ll be able to see
orders published in one command window and subscribed Joydip Kanjilal
in another, as shown in Figure 7:
Architects:
The Case for Software Leaders
The software industry has a poor success record. In fact, all of IT does. Throughout phases of various types of formal leaders,
including CIOs, project managers, and scrummasters, the project success rate struggles to get to 20%. In this article, you’ll
explore the multi-decade history of the software industry, some of the attempts at progress, and the results. You’ll also explore
what it takes to be a leader in software. At the end of this of the natural forces at play in software projects.
article, you will be able to:
This twenty-year period from 1960 to 1989 was the early
• Identify the main eras of the software industry part of Mr. Brooks’ career, and throughout the book, he
• Understand the average level of success and failure shares the nature of early software systems and comput-
over time ers. These computers didn’t come with a general-purpose
• List the responsibilities of a software leader operating system. The programmers loaded every bit of
• Decide whether the calling of software leadership code the computer was to run. Software applications
is for you weren’t a thing. We just called them “programs.” Operat-
ing Systems were still in development. Moreover, every Jeffrey Palermo
programmer did as he saw fit. There were no norms. No
Custom Software: A History of Failure methodologies. No standard processes. No large technol-
jeffrey@clear-measure.com
JeffreyPalermo.com
For as many technical advances as the software (and IT) ogy vendors suggesting a “normal” way to proceed. The @JeffreyPalermo
industries have seen over the decades, the rate of fail- name of the game was to figure out how to make the
Jeffrey Palermo is the
ure, outages, business disruptions, and lost investment computer do something. Chief Architect, and
is staggering. To illustrate this, consider the Standish Chairman of Clear Measure,
Group. This organization has been publishing studies 1980-2000: The Waterfall Period Inc., a software archi-
for decades, beginning with the original CHAOS report The Standish Group has labeled the period from 1980 to tecture company that
of 1994: https://www.standishgroup.com/sample_re- around 2000 as the Waterfall Period. This period saw the empowers its client’s
search_files/chaos_report_1994.pdf. In 1994, the CHAOS rise of software processes. Universities began computer development teams to be
report contends that only 16.2% of software projects are science curriculums. CASE (computer-aided software en- self-sufficient: moving
deemed a success. Success means that the initial promise gineering) tools were developed and taught as being the fast, delivering quality,
of the system was delivered on time, on budget, and with wave of the future where software programs would be and running their systems
the needed level of stability while running the system. programmed automatically by visual specifications of the with confidence.
The Standish Group has continued to catalog projects and required behavior. These two decades saw the right of the
publish reports, roughly every five years, and the 2020 IT project manager and the establishment of the PMBOK, Jeffrey has been recog-
report shows an alarming result. The success rate overall PMI’s Project Management Body of Knowledge with the nized as a Microsoft MVP
is still only 16.2%! Although the report studies a wide first report in 1987 and then the official first version in since 2006 and has spoken
at national conferences
range of IT projects, this is alarming for our industry. For 1996.
such as Microsoft Ignite,
readers interested in the source reports, I encourage the
TechEd, VS Live, and
reading of the Standish Group material. During this period, the mainframe computer introduced
DevTeach. He has founded
segregation into businesses and gave rise to the IT de- and run several software
partment. After all, if a business was going to invest in user groups and is the
one large expensive computer that multiple people could author of several print
Even after 25 years since share, then someone had to be responsible for keeping books, video books, and
it available and in good repair. At this point, program- many articles. A Christian,
the original CHAOS report, mers could concentrate on writing their program and then graduate of Texas A&M
software project success, on running it through a remote terminal. But because any
program had the capability to crash the shared mainframe University (BA), and the
average, struggles to achieve 20%. Jack Welch Management
server, IT administrators began developing restrictions to
protect the many from the few. In larger companies, this Institute (MBA), an Eagle
resulted in IT departments that separated the program- Scout, and an Iraq war
Let’s now take a look at some major trends in the software mer from the server administrator. This segregation would veteran, Jeffrey likes to
industry according to the Standish Group. take the next few decades to undo. spend time with his family
of five camping and riding
dirt bikes.
1960-1980: The Wild West As these departments grew, IT project managers rose as a
Our industry has now topped 60 years since inception. common role in the industry and PMI served as an educat-
Arguably, the late 1950s saw the first business software ing and certifying body to give credibility to the role. The
systems, but the cases to study are few. Fred Brooks is processes espoused by the PMI’s PMBOK are now known as
one of the early computer programmers (before the more Waterfall. Essentially the concept of “gathering require-
common term, software developer or software engineer ments” because the root cause for long project phases
came about). In his book, The Mythical Man-Month, he and rigidity is downstream. In this period, the Standish
writes a series of essays in which he reasons about some Group published its first CHAOS reports showing the world
codemag.com Architects: The Case for Software Leaders 69

that the cumulative efforts by all involved were yielding field, we’re seeing that the industry, as a whole, is getting
a 16.2% success rate. bigger, but it isn’t getting better. The industry needs a
reboot. With the latest edition of the CHAOS report show-
2000-2020: The Agile Period ing that the role of a project manager or the absence
Even though “The Agile Manifesto” wasn’t authored until thereof makes no difference in the likelihood of success,
2001, the late nineties saw the establishment of what you have to speculate about what can be done to operate
would become the new way of thinking in software. Kent a software team that’s better than the industry’s 16.2%.
Beck’s 1999 book, “Extreme Programming Explained,”
suggested software team practices that, in my view, raised
the work of computer programming into an engineering
discipline. Many of the practices of Extreme Programming
Software project managers
are now mainstays of agile and DevOps teams. don’t make a difference in
the success rate of projects.
The Agile Manifesto and the accompanying rise of scrum
training and scrummaster certifications broke long soft- Projects fail, even those that
ware phases into smaller phases. Initially suggested at include a project manager.
month-long phases, called “sprints,” teams that under-
stood the forces at play started to drop the notion of any
kind of phase or iteration. When programmable virtual
servers became widely available, the last barrier to itera- Mixed Results of Modern Software
Leadership Is Always
in High Demand
tion-less software development stepped aside. Initiatives
This period is also the period that shifted from the server The last two decades of advances in the software industry
John Maxell, the author of 70+
leadership books, asserts that being a unit of hardware to being a slice of hardware. have led to some fantastic successes. No one can argue
“everything rises and falls with In concept, the multi-user mainframe server was reborn that new companies are not being formed on the backs
leadership.” This has proven to at the data center level when large hypervisor clusters of successful software projects. Individuals benefit from
be a universal truth regardless running VMWare or Hyper-V were offered by hosting com- highly capable software applications in the palm of their
of industry. In software, some panies as easy VM Servers. AWS perfected the virtualized hands. Many people shop at grocery stores where the
job titles allude to leadership. data center and Microsoft’s Azure bet the farm on this checkout lines are software programs rather than store
The growth of the number model and took it further with specific runtimes so that clerks. Software has replaced the paper map as a common
of software developers in our software teams wouldn’t even have to deal with the con- item in a car glove box. More and more homes have tele-
industry has put a strain on cept of a server operating system. Azure has copied from visions running software that streams television rather
education avenues. With the AWS and AWS has copied from Azure in a data center than receiving it over the airwaves or coaxial cable. With
focus being on producing battle that has propelled the available technical infra- all the advances, you’ve got to wonder why some compa-
people who can write code, structure that can be used to run server-based software. nies see a return on their investment but a disproportion-
the number of programmers ate number of companies do not.
has doubled twice in the DevOps came along in 2010 as a long-needed protest to
last twenty years. But the the segregated IT departments created in the 90s. DevO- Digital Transformations Mostly Fail
education avenues are not ps, powered by virtualization and cloud technology, sug- The Boston Consulting Group (BCP) publishes research on
producing people who are gested that no divide should exist between those writing the IT industry. They find that 80% of businesses have
set up for software leadership.
software and those running it. Many organizations have plans to move forward with some form of digital transfor-
Software leaders still tend to
adopted that approach, including large enterprises such mation. To translate that away from executive-speak, this
be homegrown. And we don’t
as Microsoft. In this period, the role of the software tester means that more and more companies are going to up-
have enough. Throughout my
lifetime, leaders have always saw its demise as well. At the beginning of this period, grade software systems, modernize data centers, develop
been in high demand, and it some organizations maintained even ratios of testers to net-new systems, and generally revamp their technology.
likely will always be that way. developers, but now, only specialized organizations main- With email, file servers, and ERP systems becoming very
tain the role of dedicated software testers. mature and generally available, the technology initiatives
of most companies involve custom programming. So, al-
This period has also been the source of explosive growth. though these projects will likely include server and net-
Stack Overflow publishes a developer survey that shows working infrastructure, the world is becoming more and
that every six to seven years in this period, the number more of a software world.
of software developers has doubled, and the growth rate
isn’t slowing down. BCG’s 2020 study is interesting because they find that 70%
of digital transformation projects fail. Although this is a
2020+: The Infinite Flow Period little better than the Standish Group found, we’re still on
The Standish Group’s CHAOS report epilogue predicts that the wrong end of the 80/20 rule. (Readers who’d like to
we are in the next period, and they classify it as the In- read more can refer to the BCG’s writeup of the report:
finite Flow Period that will see the abandonment of the Flipping the Odds of Digital Transformation Success | BCG.)
concept of a software project as well as the elimination
of the formal role of technical project manager. We can Bad Software Costs Billions
already see that this has happened in organizations that In 2020, CISQ, the Consortium for Information & Software
earn revenue from operating complex software systems. Quality, performed a study that yielded depressing results.
The 2020 CHAOS report sheds a pretty damning light on The group studies the costs of poor-quality software. Their
the way the industry has chosen to manage itself. Still study can be found here (https://www.it-cisq.org/the-
with only a 16.2% success rate, the industry is in great cost-of-poor-software-quality-in-the-us-a-2020-report/)
peril. With the growth in the number of developers in the and can be summarized into three cost types
70 Architects: The Case for Software Leaders codemag.com

• $260B: the cost of unsuccessful software projects In times of trial, the leader takes the responsibility, the
• $520B: the cost of poor quality in legacy systems blame, and the stumbles. In times of triumph, the leader
• $1.56T: the cost of operational software failures steps aside and shines the light and rewards on the team.
These figures are for 2020 alone and are for just the Unit-
ed States. If the technology industry as a whole is around
$1.5T, these figures suggest a very upside-down return on John Maxell, author of 70+ books
investment equation. on leadership says: “Everything
Anecdotally, I’ve been involved in many conversations
rises or falls on leadership.”
over 25 years in the profession. These conversations in-
volve software developers lamenting that they spend too
much time fixing bugs or investigating production issues. Over the 60+ years of our software industry, much has been
Others lament that the IT group is seen just as a cost cen- established. Many processes and paradigms have been
ter, only to have hiring freezes and layoffs in a downturn. tried. Many have succeeded. Most of them have failed at
When you look at these statistics, and if they correlate least once. The knowledge of how to succeed is there. Unfor-
to similar things, you might conclude that overall, in the tunately, that knowledge isn’t universally applied. I have a
United States, the IT industry as a whole is, indeed, a hypothesis. It’s because of uneven leadership within orga-
cost center. Now, I haven’t cited statistics on revenues nizations producing and running custom software.
produced by software and IT investments. Those are hard
to come by. Because companies are still aggressively in- What Software Leaders Actually Do
vesting in technology, you can conclude that even with Some people reading this article are currently software
all the mess, it’s still worth it. But from within the indus- leaders. Some aspire to be. Some work under the guidance
try, we must do better. of one. Some wish their organization had one.
It might be convenient to consider common job titles

that suggest leadership:
$1.5T is lost in the US every year
to operational software failures. • Development manager
• Lead engineer
• Chief architect
• Software manager
Teams That Beat the Averages • Project manager
This article so far has focused on the poor results, on
average, that our industry produces. But there are teams I’ve seen people occupying each of these titles yet not
that beat the averages. There are teams that flip the filling the boots of leadership. I’ve seen those in roles
scales, that rarely fail. Several organizations are perform- with lowly titles demonstrating strong leadership for a
ing ongoing research into the practices that lead to good software team. Let’s shy away from job titles for a mo-
software delivery results. DevOps Research and Assess- ment, as there isn’t a clear correlation between job titles
ment (DORA) is one such organization. DORA’s Accelerate and leadership. In a perfect world, titles that confer au-
State of DevOps Report analyzes the results of software thority or responsibility would reliably be reserved for
teams and the practices they use. The report can show those with the leadership skills to fill them. But there are
a correlation between team behavior and team results. too many exceptions to ignore.
DORA was acquired by Google in 2019 and operates from
this website: https://cloud.google.com/devops/. Responsibilities of the Software Leader
Any analogy of leadership outside of technology will both
A key finding of DORA’s State of DevOps report was that apply and fall short. You can imagine an army captain
elite performers take full advantage of automation. From organizing a company of soldiers for a mission. Done well,
builds to testing to deployments and even security con- the soldiers have every resource they need, are trained
figuration changes, elite performers have a seven times in every skill required on the mission, and they operate
lower change failure rate and over 2,000 times faster together flawlessly to get the job done. Once successful,
time to recover from incidents. DORA also provides some the captain passes on the rewards and accolades to the
metrics that elite teams can track to ensure they remain soldiers. If not successful, the captain doesn’t pass the
among elite performers. blame to any soldier in the company. Instead, he accepts
the blame and accountability.
Everything Rises and Falls Another analogy that both applies as well as falls short is
on Leadership that of a shepherd. The shepherd has a flock of sheep. The
John Maxwell is the author of over 70 leadership books. shepherd provides what the sheep need, including protec-
People in every industry have used his books to identify tion, food, water, and safety. When successful, the sheep
and grow leadership. His books and seminars frequently yield an abundance of wool and many healthy new lambs.
include the now-famous quote: “Everything rises and falls If any danger threatens the flock, the shepherd faces it
on leadership.” head-on and directly to save the sheep.
It’s not hard to spot a leader. Others around the leader are Software developers are neither soldiers going to war nor
better because of the presence or influence of the leader. sheep producing wool, but in both analogies, the leader
codemag.com Architects: The Case for Software Leaders 71

provides for, cares for, protects, and is with leader should read. It’s called “Accelerate: The When these are achieved, the team gains con-
those under his leadership. If you apply this di- Science of Lean Software and DevOps: Build- fidence in themselves and pushes for even
rectly to a software team, here are some of the ing and Scaling High Performing Technology higher levels of performance. When these are
high-level responsibilities of a software leader: Organizations” by Nicole Forsgren, PhD, et al. achieved, business executives realize what an
This book applies available research to some internal asset the software organization has
• Form your team. Whether you inherited a practices and measurements that are correlated become. When these are achieved, your team
team or are forming it from scratch, you with high-performing software teams. This is will be an outlier in the Standish Group re-
must make sure it has the capabilities re- one example of knowledge that strong software search, beating the averages by an enormous
quired for the job. leaders bring into their organizations. At this margin.
• Equip your team. From computers to tools point in our industry, a software leader must do
to processes, you must make sure your more assembly of knowledge than independent
team has everything necessary to succeed. research of it. The DORA metrics of high-per-
• Design the working environment. The forming software teams are as follows: Strong software leaders
team needs an environment in which to
work. This includes where raw materials • Deployment frequency. Elite teams are able
empower their teams
are located, where work activities take to perform multiple deployments per day. to move fast,
place, and where finished products flow. • Mean lead time for changes. Elite teams deliver quality,
Virtual spaces are still part of the envi- are able to deploy code into production
ronment. within one hour of making a commit. and run their systems
• Set vision and targets. Your people need • Change failure rate. Elite teams have less with confidence.
a clear shared vision. And they need tar- than 15% of deployment resulting in the
gets that are close enough to see. need for an immediate fix.
• Prioritize the work. The team needs the • Time to restore service or Mean Time to
work to be in a clear sequence. Research Recovery (MTTR). Elite teams can fix a Conclusion
has shown that 2/3 of prioritized work nor- customer impact within one hour.
mally is of little value to software users. I hope you’re enlightened, a bit worried, but
Picking the right 1/3 makes a difference. There’s a lot behind each of these metrics. Engi- also inspired by this article. Over the decades,
• Measure the team. “A players” love be- neering practices like test-driven development, our industry has seen macro trends come and
ing measured. They intrinsically work for continuous integration, and automated deploy- go. It has seen great technological advances,
the grade. “C players” don’t want to be ments, and others are required in order to make but what it hasn’t seen, on average, is much
measured. “A players” will course-correct these metric thresholds possible. A strong soft- improvement in success. Rather than new tech-
and press on to achieve the targets set ware leader is educated and practiced in all of nology or innovation, our industry needs lead-
for them. the needed processes to achieve these results. ers. It needs you. The knowledge is out there.
• Monitor and adjust. Over time, the met- A strong leader has developed the people and The innovations are out there. The industry
rics collected will yield areas that need to communication skills to cast vision and align needs leaders like you to take the lead, accept
be adjusted. everyone in the same direction. A strong leader responsibility, take authority for future success,
• Strengthen your team. Every team mem- is able to identify and remove distractions so and make a change. Software leadership is not
ber needs a professional development that focus is simplified down to just the essen- a job title. You can do it. If you feel the call-
path. Don’t expect each team member to tials. Ultimately, what a business wants out of ing in your heart, it doesn’t matter your age or
automatically do that for themselves. a software team is for it to: level of experience. Take the lead. Do the work.
Make the change.
Results of a Strong Software Leader • Move fast
The DORA organization’s leaders have pub- • Deliver quality Jeffrey Palermo
lished a book that every aspiring software • Run stable software in production
(Continued from 74) Ask yourself if you would accept shoddy work as opposed to differently situated employees,
from a contractor in your home. Of course, that doesn’t absolve executive management
a two-way street because it isn’t in our na- you wouldn’t. Nobody would or should. But from its core oversight duties and responsibili-
ture to dredge up past transgressions to be used we collectively tolerate it in our projects. Re- ties. What often happens is a slow depletion of
against us! Therefore, to be responsible about sponsibility requires oversight and leadership. what I refer to as “organizational knowledge.”
putting people on the path so that they may Consider that Southwest Airlines and the FAA Such knowledge, generally, is at the very least, an
be responsible requires a culture of transpar- have different perspectives but with much in understanding of how things operate. When we
ency. This is the essence of what continuous common. staff things out, over time, organizations often
improvement is all about. Such efforts require defer the requirement for such direct knowledge
egos being checked at the door. Because the Let’s take the FAA scenario first. That situation, in favor of the external entities. Once organiza-
serious fact is this: The work we do matters. It based on news reports, appears to be an over- tions start to lose grip on how their technical
matters to other people because they depend sight/contractor scenario. A public agency, the infrastructure operates, they become captive to
on our technical work. To support such a cul- FAA will need to be transparent about whatever these external entities. It’s the height of irre-
ture, it requires the buy-in and trust of senior after-action report is published. The point here sponsibility on the part of management when it
leadership. That same senior leadership needs is that although the C-suite may find it benefi- causes an organization to lose grip of its internal
to finally listen to the technical staff that has cial to financially and legally organize in a way understanding of IT matters. Despite the differ-
been trying to tell them the facts for years. that uses non-geographically co-located entities ence in organizations, the FAA and Southwest
72 Architects: The Case for Software Leaders codemag.com

CODE COMPILERS
situations are similar as to technical debt and Clean%20as%20approved%20by%20Board%20

the damage that scourge has on an organization. 8-1-18.pdf. Pay special attention to sections
The organizational differences illustrate the dif- like conflict of interest and corporate oppor-
ferent paths that can be encountered with tech- tunities. When we collectively and individu- Mar/Apr 2023
nical debt accrual. ally act in a way that restricts the organiza- Volume 24 Issue 2
tion from the opportunities that it should be
Southwest Airlines, based on news accounts, able to avail itself of through innovation, we’re Group Publisher
is a good example of a set of processes that not acting in the organization’s best interest Markus Egger
can work, but not at scale. Being resilient in and therefore, we’re not acting responsibly. Associate Publisher
the face of many variables is a scaling charac- We likely didn’t affirmatively decide to violate Rick Strahl
teristic. In general, the sensitivity that large the rules, although actions that do just that Editor-in-Chief
systems with manual components have is that make intent irrelevant because the damage will Rod Paddock
they can’t be easily or feasibly simulated. The be done!
Managing Editor
system and its components have evolved over Ellen Whitney
time. It’s simply “The System” with thousands There may in fact be real legal jeopardy as
and thousands of components, software, pro- more of these issues arise. And more of these Contributing Editor
John V. Petersen
cesses, people, and all the interdependencies issues will arise. An example of legal jeopardy
therein. No one or group of people is likely to is the recently filed Southwest shareholder Content Editor
Melanie Spiller
know how it all works. As for the right mix of class action lawsuit: https://simpleflying.com/
events that will cause the system to melt down, southwest-shareholder-federal-securities-law- Editorial Contributors
there may be theories as to what they are, but violation-class-action/. I foresee a day when, Otto Dobretsberger
Jim Duffy
until you’re in it or very close to it, you won’t depending on the harm, we may very well see Jeff Etter
know for sure. Therefore, you’re forced to react, more criminal and civil sanctions levied on per- Mike Yeager
not respond. sons and organizations. The law provides for it.
Writers In This Issue
It just needs to be enforced. It’s the consum- Joydip Kanjilal Julie Lerman
It doesn’t matter much why past choices were ers, the public, the very folks that either use Sahil Malik Rod Paddock
made. The harms are known and those harms the technology we build or are affected by it Jeffrey Palermo John Petersen
Alexander Pirker Paul D. Sheriff
can be proximately traced to things that give that will eventually demand legislation. People Shawn Wildermuth
rise to technical debt. News accounts appear demand accountability and the bill eventu-
to confirm that, for years, there have been nu- ally comes due. Responsible management keeps Technical Reviewers
Markus Egger
merous warnings from staff. Good leadership that in mind. Rod Paddock
involves listening to your people instead of
Production
kicking the can down the road. At some point, Where does all that begin? As Ted eloquently Friedl Raffeiner Grafik Studio
folks just need the wherewithal to be able to wrote seven years ago in these pages, it begins www.frigraf.it
set about fixing issues that have been around with ourselves, the most important tool in our Graphic Layout
for years without fear or favor. Because that’s toolbox. We need to take care that we remem- Friedl Raffeiner Grafik Studio in collaboration
with onsight (www.onsightdesign.info)
what taking care is all about: not causing ber to invest in ourselves so that we’re best
harm. situated to take on responsibility, not only Printing
for ourselves, but on behalf of others. And Fry Communications, Inc.
800 West Church Rd.
At the outset, those kinds of value decisions to make it all work, others need to do for us Mechanicsburg, PA 17055
are made within ourselves first. How we make as well.
those decisions, specifically what we choose Advertising Sales
Tammy Ferguson
to prioritize, ignore, etc., that’s our internal Nothing substantial in our history has occurred 832-717-4445 ext 26
process. That process is driven by an inter- without cooperation. A lot of that history isn’t tammy@codemag.com
nal code. Such codes are often informed by, or good, of course but that same force of coopera-
Circulation & Distribution
at least should be informed by, externalities tion can achieve a different, good result. It’s General Circulation: EPS Software Corp.
including applicable rules, regulations, and within all of us to decide what path we each Newsstand: American News Company (ANC)
codes of ethics, a concept that Ted addresses will take. This is something you can’t ask Chat-
Subscriptions
well in his column. If yours is a public com- GPT to provide an answer for. Whatever group Subscription Manager
pany, then every year, as part of SEC require- you’re part of, whatever objective that group Colleen Cade
ments, public companies are required under collectively works toward, you all need to be ccade@codemag.com
Sarbanes-Oxley, to disclose whether they have aligned, even if that means rejecting the objec- US subscriptions are US $29.99 for one year. Subscriptions
a code of ethics and whether any waivers to tive! The opposite of cooperation is opposition. outside the US are US $50.99. Payments should be made
that code have been approved by the board of If you have opposing forces within your group, in US dollars drawn on a US bank. American Express,
MasterCard, Visa, and Discover credit cards accepted.
directors. they’ll cancel each other out. Your organization Back issues are available. For subscription information,
will incur the expense, with not only no ben- e-mail subscriptions@codemag.com.
These governance tasks matter. Well-run orga- efit, but likely an impairment to existing sys-
Subscribe online at
nizations are managed by serious and respon- tems. In other words, your organization spent www.codemag.com
sible people who take care to “honor the deal.” money to be in a worse position. Put another
To borrow Rod Paddock’s phrase as to what way (and I give the credit to my superstar edi- CODE Developer Magazine
6605 Cypresswood Drive, Ste 425, Spring, Texas 77379
companies profess regarding what they do ver- tor Melanie for these words that sum it up per- Phone: 832-717-4445
sus what they do, is “a deal a deal?” Consider fectly): “We all play nicely together or we all
Southwest Airlines’ published code of ethics hang together.”
as of 8/1/2018: https://www.southwestair-
linesinvestorrelations.com/~/media/Files/S/ John V. Petersen
Southwest-IR/Code%20of%20Ethics%20-%20
codemag.com CODA: On Responsibility: Part II 73

CODA
CODA:
On Responsibility: Part II
As I sit down to write this next CODA installment, the January/February 2023 issue is available for
consumption, as are the events incident to Southwest Airlines and the FAA. In the printed physical
magazine world, content must be assembled in a layout long in advance of the publication date.
That’s how we ensure a quality CODE Magazine sponsible decision is how technical debt may umn, Ted called out to me over the issue of
product. Our magazine analog is not so differ- be quantified. Technical debt is an insidious ethics, the law, and liability. Although I’ve
ent from a software analog where we must also thing because it’s invisible. It isn’t recorded generally written about those things in the
take care as we proceed along the software de- in the financials in the sense that debt is usu- past, with the new Southwest Airlines and FAA
velopment journey for a given project. ally referenced. Nevertheless, you know that context, in conjunction with this once, pres-
technical debt exists. It exists in the corpus ent, and perhaps future topic of responsibility,
The goal for software is the same as with the of our software in how it was designed, built, regardless of what was written in the past, it
magazine, to deliver a quality product. Quality tested, and deployed. To the extent that our bears repeating in a new way with the basic
isn’t some abstract thing in this case, because existing platform isn’t receptive to changes question of how we can improve our condi-
to make some determination regarding quality, that reflect current innovation that competi- tion. How do we become more responsible?
it must be measurable. Quality is real, not an tive, top players in an industry implement, Perhaps it’s a matter of first learning how to
abstract thing. The quality of the software we technical debt exists. One way to quantify be responsible. And before that, we must each
build is entirely dependent on how we build it. technical debt is to equate it to the annual be willing to accept responsibility and be ac-
EBITA from such foregone opportunities. The countable; first to ourselves, and then to the
The recent events of Southwest Airlines and the implication is that every year, an organization team and the organization. It is important to
FAA, and the not-so-distant events with the accrues more technical debt, even if it hasn’t remember that “we” is just a group of indi-
Boeing 737 Max-8, require me to look at this spent another cent on that project. Technical vidual “I”s.
topic anew because although we understand debt is about opportunity costs. Organizations
what quality is, collectively, when confronted that squander corporate opportunities are ir- The difficulty with large systems that have ex-
with scenarios that require attention to es- responsible. isted for a long time (like Southwest’s) is that
tablish, maintain, and enhance quality, we’re many, many people; past, present, and future,
not interested in investing in such support have or will have an impact on that system.
infrastructure. Nevertheless, quality is desired Why and how things are as they are isn’t nearly
because of the benefits that are incident to For an industry that prides as important as the fact that they are this way
quality. It doesn’t seem very responsible to, on itself on its analytical and our collective response to that fact. That
one hand, desire the benefits of quality with- collective response is just an aggregation of
out doing the things necessary to earn a posi- ability and abstract mental individual responses, from a variety of profes-
tive quality designation. Whether something is processing, we often don’t sionals and disciplines that have been brought
of good quality is in the eye of the beholder. to bear on a solution. At a fundamental level,
Quality is a report card. It’s a judgment. And we do a great job applying those individual responses are best served when
can’t be the judges of our own work. That’s what that mental skill to the most they’re in sync with some stated overarching
makes a good measure, because it can’t be tar- principles. Think of these principles as a com-
geted. At its most abstract, quality is simply
important element of pass or North Star to serve as a guide.
the result of something else. the programmer’s tool
chest – ourselves. What if such stated principles exist or exist but
Goals are important. They help a group focus aren’t known? That’s why the most important
on some common thing. But that isn’t enough. tool in the programmer’s toolbox is ourselves.
We can’t just focus on the end. How we get We’re the most important tool because we each
there matters because it has a knock-on effect That’s the reason I decided to scour the CODE possess the ability to act appropriately at the
to what we end up with. It may very well be Magazine archives to get a sense of what’s “last responsible moment.” Whether or not we
that instead of focusing on quality, perhaps we been written before in these pages. I encoun- act appropriately at the last responsible mo-
should focus on those things we have control tered some of my previous work and that of ment is another question. If it seems in your
over that, when practiced, tend to make qual- others. One article that stuck out was from projects that you’re always reacting, and put-
ity enhancement more likely, not less probable. my back-page predecessor Ted Neward. In his ting out fires, this applies to you. To be re-
That’s what taking care is about. Managed Coder Column from May/June 2016 sponsible, we must be responsible about being
(https://www.codemag.com/Article/1605121/ responsible.
With all the bad news recently about South- Managed-Coder-On-Responsibility), he wrote
west Airlines and the FAA and their antiquated On Responsibility wherein he raised the fol- Another twist on that is that serious people
manual process, the first thing I thought of lowing assertion: treat serious matters seriously. This is where
was technical debt. I hold to the notion that rigorous honesty must be embraced. But that’s
every irresponsible decision an IT organization As insightful as that quote was seven years
makes, the hard financial costs with that irre- ago, it’s even more so now. Later in his col- (Continued on page 73)
74 CODA: On Responsibility: Part II codemag.com

DREADING
SHARING
THAT YOUR
APPLICATION
CAUSED A
DATA BREACH?
CODE SECURITY’S APPLICATION TESTING SERVICES CAN HELP YOU PREVENT THAT BREACH.
• My development team practices secure coding techniques Yes No
• My applications have been tested for security vulnerabilities Yes No
• Code Audits and Penetration Testing have been performed by a third party Yes No
CODE Security experts can help you identify and correct security vulnerabilities in your products,
services and your application’s architecture. Additionally, CODE Training’s hands-on Secure Coding for
Developers training courses educates developers on how to write secure and robust applications.
Contact us today for a free consultation and details about our services.
codemag.com/security
OLD
TECH HOLDING
YOU BACK?
Are you being held back by a legacy application that needs to be modernized? We can help.
We specialize in converting legacy applications to modern technologies. Whether your application
is currently written in Visual Basic, FoxPro, Access, ASP Classic, .NET 1.0, PHP, Delphi…
or something else, we can help.
codemag.com/legacy

CODE 2-2023 Web

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CODE 2-2023 Web

Uploaded by

Copyright:

Available Formats

Cloud Computing, Razor, Kubernetes, Dapr, Chrome Debugging

Hidden Secrets of Event Systems Exploring (More)

ADDRESS POTENTIAL SECURITY PROBLEMS

32  lean Shave: Razor Pages

49 Kubernetes Security for Starters 6 Editorial

4 Table of Contents codemag.com

The Way of the Coder

reasons for this. Building the software uncovers

Or how many times have you heard this one. “I

MORE THAN JUST

Chrome Debugging Tips

8 Chrome Debugging Tips codemag.com

The problem this causes is finding a particular file on a

Prettify JavaScript Files

The one big disadvantage is that such files become im-

Your Personal CodePen

To access the snippet editor, go to the sources tab, click

Although this is great, you may be wondering about a se-

codemag.com Chrome Debugging Tips 9

There’s another way. The JavaScript console lets you liter-

Let’s say you have a complex JavaScript method, as be-

Running this generates a random set of characters every

Figure 6: The snippet editor built inside of dev tools

Figure 7: Warning when adding a filesystem folder to dev tools

10 Chrome Debugging Tips codemag.com

Now feel free to play around with the other options as

Below this UX and not visible in the screenshot are a

In the actual network log, you can see a number of other

codemag.com Chrome Debugging Tips 11

Finally, you can also import/export the entire network trace

Record User Journeys

All of this is possible via Chrome dev tools. To do so, visit

Bonus Tip: Dark Mode

The other thing that has changed is age. Although my

12 Chrome Debugging Tips codemag.com

In the bookmarks bar, I create a bookmark. You can name

In the URL, I put the code shown in Listing 1 (the line

I know there are extensions that can do this for me auto-

Figure 15: Dark mode works quite well.

I could easily come up with another 10 tips. Although the

codemag.com Chrome Debugging Tips 13

Create Your Own SQL Compare

14 Create Your Own SQL Compare Utility Using GetSchema() codemag.com

Each collection has its own restrictions array, but most

Get All Tables and Views

Add two using statements to bring in the namespaces for

codemag.com Create Your Own SQL Compare Utility Using GetSchema() 15

16 Create Your Own SQL Compare Utility Using GetSchema() codemag.com

string conn = "Data Source=Localhost; // Get All Tables in the Database

using SqlConnection cn = new(conn); // Display Column Names

Try It Out // Get All Views in the Database

Try It Out cn.Open();

// Display Column Names

codemag.com Create Your Own SQL Compare Utility Using GetSchema() 17

using SqlConnection cn = new(conn); // Display Data

18 Create Your Own SQL Compare Utility Using GetSchema() codemag.com

Get Metadata about Restrictions

codemag.com Create Your Own SQL Compare Utility Using GetSchema() 19

20 Create Your Own SQL Compare Utility Using GetSchema() codemag.com

public class SchemaBase public override string ToString()

codemag.com Create Your Own SQL Compare Utility Using GetSchema() 21

// Use ExceptBy to locate differences

32 lean Shave: Razor Pages

49 Kubernetes Security for Starters 6 Editorial

CREATE DATABASE ANOTHER_DEMO Rod Paddock