Azure Test Plans vs Katalon

Pros
Azure Test Plans Katalon
The easy-to-use, browser-based test It is well suited for quick automation and
management solution provides all the avoiding dependency on coding skilled
capabilities required for planned manual resources.
testing, user acceptance testing,
exploratory testing, and gathering
feedback from stakeholders.

Configure lightweight charts to track Features to automate web, mobile app

your manual test results using the chart and Web Services.
types of your choice, and pin the charts to
your dashboard to easily analyse these
results.
It Control how long to keep test results It provides record and play back
capability

Controls automated test cases with It supports both manual and script mode
a Team Build pipeline that generates
builds containing the test binaries.

It can be integrated with azure pipelines It can be integrated with tools like Git,
to run the automation scripts. Jira etc.

Better Test Maintenance for manual User-friendly GUI,

testing.
Productive IDE for API automation

Built to work with modern frameworks

Better Test Maintenance

It provides in-built templates for

maintaining the object repository, test
cases and custom keywords

Sustainable and scalable platform

Cons
Azure Test Plans
Integration is needed to run the
Automation
With other framework and Scripting
language like selenium
No IDE for Automation, Use Visual
Studio or Eclipse to create test scripts.
Manual tester Won’t be able to switch it
to automation very easily as it needs
learning to create automation scripts.
Integration with Microsoft products is
easy, but with non-Microsoft products it
is more difficult, and you have to make a
lot of configuration changes to integrate
Integration is needed with azure pipeline
to run the automated test scripts
Required more efforts for test
maintenance and efforts to manage
pipeline
Katalon
It is not well suited for complex projects
where high customization is required
It’s not an open source, Katalon
studio(basic) is free to use and for other
services they have pricing.
As of now there is no support for
distributed testing.
Lack of scripting languages
Unlike Selenium and TestComplete, the
only scripting language supported by
Katalon is Groovy. This scripting
language is in the Java family so anyone
who knows Java can use it. But, users
would like to see more languages
supported.
It cannot automate desktop applications
unlike some licensed tools like UFT and
TestComplete.
Katalon Browser Support

Desktop Version on Version on Note

Browsers Windows macOS
Internet Explorer 9, 10, 11 NA Required IE
configurations: Interne
t Explorer
Configurations.

Microsoft Edge Current NA Refer to this page for

the current status of
Edge WebDriver.

Firefox 56+ To use Firefox 57

with Katalon
Studio, please
use Katalon
Studio v5.1+

Google Chrome 58+

Opera Not Supported
Safari 5.1+ 12

References:
https://docs.katalon.com/katalon-studio/docs/supported-environments.html#system-
requirements
https://docs.microsoft.com/en-us/azure/devops/test/run-automated-tests-from-test-hub?
view=azure-devops
https://azure.microsoft.com/en-us/blog/deep-dive-into-azure-test-plans/
https://docs.katalon.com/katalon-studio/docs/overview.html#documentations