Download as pdf or txt
Download as pdf or txt
You are on page 1of 18

Welcome To Day 5

ZPA Support/TAM Bootcamp Training 201


By: Davis Altamirano
Class Starts at 8:00 am PDT / 9:00 CDT / 11:00 am EDT

1 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting 101

2 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Module 5 Objectives

By the end of the module, you will be able to identify the most common
issues related to the ZPA connectivity

Objectives
● Being able to isolate the main issues related to each specific situation
● Apply best practices to avoid misconfiguration issues

3 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Using ZPA Diagnostics
What can you do on your side first before asking customer for data

• Setup filters to identify the Application Domain and Username affected


• Check Policy Name, Action, Connector Name
• Identify which “Internal Status Code” and use the Status Code reference link below to
identify if Error Codes reported:
• These codes represent normal information codes and error codes with potential corrective
actions.
https://help.zscaler.com/zpa/about-zpa-session-status-codes
• Find and view the “Raw JSON” – If Zscaler ticket is open, please download the “Raw JSON” and
add to support case

4 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Identifying the ZPA Domain and App Segment
What can you do on your side first before asking customer for data
• Identify if Domain is ZPA:
• Using ZCC ZSATunnel.log file, search file for domain name of application and look for
these type messages:
• “DBG DNS: Domain=your.domain.com is ZPN!” – This message shows that a
domain is seen by ZCC as a ZPA domain
• “ZPN: Control Message Response Data: {"zpn_client_app:” – This message will also
show domain , type of transport protocol (TCP/UDP) and the configured ports that
are part of application segment

• Find application segment in Administration >> Application Segment


• Find Access Policy in Administration >> Access Policy
• Identify if Application segment domain and ports are configured for type of access from client.
For example, it is TCP or UDP and what ports are needed?

5 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Auth / SAML Issue
What can you do on your side first before asking customer for data

SAML Data missing

• Confirm their saml assertion has the required groups


• https://samlsp.private.zscaler.com/auth/v2/login?domain=companydomainhere.com&ssotype=test
note that you need to modify the company domain
• If saml has all the attributes, check if the user is still failing or if they are now succeeding
• If user is still failing ensure they logged out fully and logged back in to ensure that the saml
assertion presented is fully up to date.
• If all of this fails confirm if the diagnostic log shows the saml attribute
• if attribute is present in diagnostic log but policy match is failing this needs to be raised as a bug in
the policy engine
• If attribute is not present in the log but in the assertion attribute, then this needs to be raised as a
bug against saml parsing

6 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
SCIM Issues
What can you do on your side first before asking customer for data

SCIM failures

• Ensure the user has fully synced.


• Check that the user has the required scim attributes in the scim view
• Important required attributes, these being missing or incorrect block all zapp access:
• active = true

7 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Common Issues

8 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Case 1 : “Connection Error”
Problem: Zscaler Client Connector shows “Connection Error”
Cause:
Connection error appears in Zscaler Client Connector when there is a connectivity issue from the user's machine to Zscaler
ZPA Cloud.

Solution:
Step 1: Identify the Nearest Broker
1.) In order to get the nearest data center node or broker ZPA does a DNS resolution to any.broker.prod.zpath.net and the
DNS response contains the nearest broker IP address. Below steps can be taken to identify broker IP address.

Clear Zscaler App logs and gather a fresh set of logs with captures along with enabling packet captures on the App make sure
that we capture the connection error event.
Looks for ZSATunnel log
Search for any.broker.prod.zpath.net.

2.) Ensure access to Zscaler brokers IP is not restricted by any corporate firewalls, as our infrastructure evolves and expands.
Zscaler strongly recommends that the Zscaler Client Connector have unrestricted outbound access to the Internet on port 443,
to ensure access to all Zscaler brokers as our infrastructure evolves and expands.

3.) In some cases, the DNS resolution to any.broker.prod.zpath might fail in which case we will get the Connection Error on
ZPA, Please ensure that DNS resolver is functioning properly

9 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Case 2 : “SE: Policy Not Configured For Access”
Problem: Unable to access application ZPA diagnostic logs show error “SE: Policy not configured
for access”
Cause:
There could be multiple reason why ZPA couldn’t find access-policy for the application. Some of the reasons are below
a.) No access policy created for the user.
b.) Access-policy is defined based on SAML and SCIM attributes.
c.) Access-policy is defined on the basis of posture profile.
d.) Access policy is defined based on the trusted network criteria
e.) Access-policy is defined based on the client type.

Solution:
1.) Verify the above policies by navigating to ZPA Portal> Administration > Access-policy

a.) Verify that access-policy is created


b.) Verify that user is a part of right SAML or SCIM group is policy is based on that.
c.) Verify that user is passing the posture profile. ZPA Portal > Diagnostic Select the username and click on user
metadata.
d.) Verify that the user is in the right trusted network.
e.) Verify that Client type is correct.

10 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Case 3 : “CA: Application Not Reachable”
Problem: Unable to access application ZPA Diagnostic logs show error below “CA: Application not
reachable”
Cause:
This means none of the configured app connectors for the application either cannot resolve the server or cannot reach the
server.

Solution:
1.) Login to ZPA connector check below command in order to identify a DNS or a network problem

a.) Ping the destination server and see if we have connectivity and DNS resolution. For example, ping <destination
server ip or domain>
b.) telnet on specified port number and see if the TCP socket is open. For example, telnet <destination server>:<port>
c.) Run a tracepath to identify the network path in order to troubleshoot the transit network. For example, tracepath
<destination server>

11 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Case 4 : “SE: Timeout Policy Blocked Access”
Problem: Unable to access application ZPA Diagnostic logs show below error “SE: Timeout policy
blocked access”
Cause:
This means the timeout policy has triggered the Z-App to re-authenticate into the application.

Solution:
1.) Please ask the user to re-authenticate to the ZPA on ZCC. If the ZCC shouldn’t be prompted for re-authentication, please
review the timeout policy under ZPA Portal > Administration > Timeout Policy

12 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Case 5 : “SE: Application Policy Blocked Access”
Problem: Unable to access application ZPA Diagnostic logs show below error “SE: Application
policy blocked access”
Cause:
This means that the access has been denied by an access policy and could be result of multiple overlapping policies

Solution:
1.) Review the Access-policy configuration, Name of the access-policy can be found in the same Diagnostic log under
Access Policy Name.

2.) Overlapping Policies Case:


a.) If there are 2 policies in the access-policy segment and 1st allows access to a subnet (For Ex: 10.0.0.0/8 and
second rule denied access to server 10.84.71.10. Even if 1st policy is allowed still selects the second policy which
blocks the traffic
b.) This is by design, if there are 2 access policies for the same destination ZPA will pick the access policy that matches
the most specific application segment. For more information refer to the URL below https://help.zscaler.com/zpa/about-
policies

13 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting Tools Overview
Key tools for you to troubleshoot issues with customers

• CloudView
• Cloudy/Grafana (ZPA-Triage Team has access)
• Looking Glass for MTRs (from )
• Wireshark Tips using filters
• Fiddler Tips (Application Issues)

Few Tips when troubleshooting:

● Capture on the ZCC Client(more->start capture)


● If possible captures on the app connector as well (simultaneous)
● ZCC Logs
● As much details as possible with time frames and time zones

14 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Troubleshooting Tools Overview
Key tools for you to troubleshoot issues with customers

• Internal reference Docs:


https://confluence.corp.zscaler.com/display/ET/Connector+aka+Assistant
https://confluence.corp.zscaler.com/display/ET/Application+access
https://confluence.corp.zscaler.com/display/ET/Broker
https://confluence.corp.zscaler.com/display/ET/Library
https://confluence.corp.zscaler.com/display/CC/Escalating+ZPA+Issues+to+Engineering

15 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Lab Exercises

16 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Lab Exercise #1
Create a SetUp

• Login to your Azure/AWS account shared for ZIA


∙ Create a Setup with the below resources,
Connector
Windows Machine (with ZCC Installed)
Web Server (If possible that can be used for BA)
PSE

• Check the below:


Access to the Application from the windows machine is successful.
If the WebServer is accessible as BA test the same.
Connect to the Application via the PSE.

17 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION
Thank you

21 ©2021 Zscaler, Inc. All rights reserved. ZSCALER CONFIDENTIAL INFORMATION SECURING YOUR DIGITAL TRANSFORMATION

You might also like