812197 _ 2023 - Data Not Visible in Forensic Lab of ETD System

9/22/23, 11:01 AM 812197 / 2023 - Data not visible in Forensic lab of ETD system
Incident: 812197 / 2023 - Data not visible in Forensic lab of ETD system
ID 002075129500008121972023
Customer 681953 - EEP - Ethiopia Electric Power
Installation 0021197165 - T_S/4HOP
System TDS - TDSDB
Component SAP Enterprise Threat Detection (BC-SEC-ETD)
Status In Processing by SAP
Priority High
Communication
15.09.2023 10:05:09 CET - Problem Description: Anurag Kumar (S0025438044)
--- Product/Function ---

No predicted product
User selected product: SAP Enterprise Threat Detection
Predicted product function: SAP Enterprise Threat Detection > Enterprise Threat Detection (Recommended)
Predicted component: BC-SEC-ETD-CLD (Product Function)
--- Steps to Reproduce ---
*Note To SAP:
I give approval for SAP Support to use these Steps to Reproduce, while connected to my impacted non-production environments, even if the steps result in a
change being made and my approval remains valid until the issue is resolved, unless I inform SAP Support otherwise.
--- Description ---

Hi there,
We have installed SAP ETD system and performed the configuration part on both ABAP and ETD side.
So from ABAP , we can check from SETD, data is flowing successfully to ETD system but not visible in the Forensic lab.
I am attaching the KAFKA and LOGCOLLECTIOR logs and assuming that the problem is here. Requesting you to please check the logs and help us to
resolve the issue ASAP.
Thank you so much.
Regards,
Rachit
15.09.2023 11:15:28 CET - Info for SAP: Anurag Kumar (S0025438044)
Attachment uploaded
Attachment uploaded
Attachment uploaded
18.09.2023 10:43:19 CET - Info for Customer: SAP
Dear Customer,
have you checked our troubleshooting guide for missing data from ABAP systems? Please check "SAP Enterprise Threat Detection does not receive log data
from an ABAP system" in
https://help.sap.com/docs/SAP_ENTERPRISE_THREAT_DETECTION/e8c03e3e8bc84d61a882844233cc1499/e49c9db881624b6eb546135347d48544.html?
locale=en-US
If not, please go through all the steps and let us know after that.
https://userapps.support.sap.com/sap(bD1lbiZjPTAwMQ==)/support/incident/print/default.htm?pointer=002075129500008121972023 1/10
Best regards,
Pavel
Hi Pavel,
We already checked the Troubleshooting guide. We are only facing issues with KAFKA2HANA service. Log collector and Normalizer issues doesn't report
any error.
Below is the error.
"2023-09-15T18:06:36.854+0000 ERROR org.apache.kafka.clients.consumer.KafkaConsumer [KafkaConsumer-JSONMasterData]: [Consumer

clientId=consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-24, groupId=SAP Enterprise Threat Detection Kafka to HANA Writer
PROD] Failed to close coordinator java.lang.InterruptedException java.lang.InterruptedException"
Also attaching the Kafka2Hana and Kafka log file.
Please check once and help to resolve.
Thank you
Rachit
Attachment uploaded
Attachment uploaded
Attachment uploaded
Hi team,
This is an very urgent for us to resolve the issue ASAP as the consultant are waiting for it to complete. So , we request you to please help to resolve the
issue as early as possible.
Thank you ,
Rachit
Dear Customer,
we are working on this issue and we will return to you as soon as possible.
Best regards,
Pavel
20.09.2023 12:08:19 CET - Business Impact: SAP
Business Impact:
Go live date for production is already passed.

Going live SAP ETD.
Implementation.
The system has to deliver 20 days back.
This issue is blocking the delivery of the system.
How many users/ members affected : multiple users are affected.
This is a showstopper issue.
Financial loss : Not sure
Deadline : Have to resolve by today 20th Sept, as the issue is stopping the team to hand it over to the customer.
Your interaction with SAP Support
Channel: Call from customer
Interaction summary:
Requester: Mr. Rachit

Request: To raise the priority to very high and speed up.
Business Impact:

Going live SAP ETD.
Implementation.
Additional Comment: Raised to high

Advised the P1 criteria
Contact :
Mr. Rachit
+91 8813886827
Hi team,
Adding below kafka logs here. May be this can help to understand the issue.
" journalctl -u kafka|grep failed

Sep 16 09:09:15 eepetd01 kafka[1947]: connection.failed.authentication.delay.ms = 100
Sep 16 09:09:22 eepetd01 kafka[1947]: [2023-09-16 09:09:22,019] INFO [GroupCoordinator 0]: Preparing to rebalance group SAP Enterprise Threat
Detection Normalizer PROD in state PreparingRebalance with old generation 6 (__consumer_offsets-24) (reason: Adding new member consumer-SAP
Enterprise Threat Detection Normalizer PROD-1-b7071668-40cc-4633-92f2-0df2a5800a47 with group instance id None; client reason: rebalance failed due
to MemberIdRequiredException) (kafka.coordinator.group.GroupCoordinator)
Sep 16 09:10:21 eepetd01 kafka[1947]: [2023-09-16 09:10:21,868] INFO [GroupCoordinator 0]: Member consumer-SAP Enterprise Threat Detection Kafka
to HANA Writer PROD-22-73c1c669-dd71-43c8-8e29-2cb7fb525797 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
removing it from the group (kafka.coordinator.group.GroupCoordinator)
to HANA Writer PROD-12-3422e044-32fb-4fea-ba18-9f6274fd0e34 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-16-1aab11b7-2dd1-4d93-adf2-7b1af11d4205 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-20-496ebdf7-6b4b-45e2-b102-bae6517109ab in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-6-5bae8294-f66a-4bc9-a629-6897a940153b in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-18-cb50ec0e-13ee-4b62-86cd-2f72940d146b in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-19-d6db1c93-97d0-4e9b-a5f1-40352a7f875e in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-11-886529e2-4a3b-44ed-8567-1e22f90e0c6b in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-3-f0586316-1249-4bd1-bae5-187929c20d9b in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-9-5c9af744-52f0-4de8-a16d-e5cb747600fd in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-23-523add8b-bbd1-4a97-85d3-0321b9a34b1e in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-2-6bcdca06-343e-4472-9a85-2f0e8e6668bc in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-8-5f881985-d2d2-4d4f-a85a-3475eef51e15 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-1-647c9cdd-5f62-4394-affa-8cc4eccc3855 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-13-3d1fe760-f4de-4244-9331-100f880306f7 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-15-e08da7d3-6957-4692-85f5-861847d8517b in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-7-369e44a6-b3a2-4bfe-960e-544a35ef74b4 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-24-1660a5a7-1cb1-494f-992b-89a6d38678d2 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-14-7a350c91-adc1-4502-8be9-8bec6b032fe2 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-21-69c58005-67ce-4f38-86f8-7de245045e19 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-4-001f1191-3f10-4e86-8e2a-b1a004b16e77 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-10-643f2ea0-3e27-4cb1-b013-f3fde80d670a in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-5-1c107b87-36d8-420f-b4e9-534c95b83e97 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
to HANA Writer PROD-17-a5116aba-76c4-4fc1-b4d5-96191f2423c2 in group SAP Enterprise Threat Detection Kafka to HANA Writer PROD has failed,
Sep 16 09:10:21 eepetd01 kafka[1947]: [2023-09-16 09:10:21,988] INFO [GroupCoordinator 0]: Member consumer-SAP Enterprise Threat Detection
Normalizer PROD-3-5fce0ec3-1b97-42eb-bb8d-456516a92894 in group SAP Enterprise Threat Detection Normalizer PROD has failed, removing it from the
group (kafka.coordinator.group.GroupCoordinator)
Normalizer PROD-1-480f1aa8-2dc8-40b2-8fee-c16ed9c56207 in group SAP Enterprise Threat Detection Normalizer PROD has failed, removing it from the
Normalizer PROD-2-a9c4953f-020c-46f1-94ee-e5070ef07d07 in group SAP Enterprise Threat Detection Normalizer PROD has failed, removing it from the
Detection Kafka to HANA Writer PROD in state PreparingRebalance with old generation 22 (__consumer_offsets-31) (reason: Adding new member
consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-1-4e0a47f8-5870-4a96-a89d-5b1aeda937fd with group instance id None; client
reason: rebalance failed due to MemberIdRequiredException) (kafka.coordinator.group.GroupCoordinator)
consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-4-e6d06249-4dcc-4f69-9866-84eb361c847c with group instance id None; client
consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-8-5926f3d7-4d8e-4881-b3f1-73ec10979339 with group instance id None; client
consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-6-bcf1e697-68c9-4095-b3f3-d58c0a7d012f with group instance id None; client
consumer-SAP Enterprise Threat Detection Kafka to HANA Writer PROD-16-5eadd4e1-3263-4d57-8a04-37fc002bd1a9 with group instance id None; client
eepetd01:/opt/etd/kafka_2_hana/logs #"
Regards,
Rachit
Hello team,
Sorry to ping you again but this is a very urgent issue which need to be resolved ASAP. So. could you please check once and update.
Also I am working in IST hours so it would be good if you provide an update within next two hours , so that I would be available incase of any input would be
required from our side.
Thanks,
Rachit
Contact name: Anurag Kumar

Contact phone or email: +251-
Rachit requested an update.
I informed him that the developer for this issue is located in Germany.

Going live SAP ETD.
Implementation.
Additional Comment: Raised to high

Advised the P1 criteria
Contact :
Mr. Rachit
+91 8813886827
Contact phone or email: +251-

Requester: Mr. Rachit

Request: To speed up and requesting for an update.
Business Impact:

Going live SAP ETD.
Implementation.
Additional Comment: Team kindly check and help the customer with an update.
Contact :
Mr. Rachit
+91 8813886827
Hi Rachit,
could you please attach the log collector configuration xml file and the kafka2hana configuraiton xml file please?
I would also like to check first on the ABAP system that sends logs that this is working. Could you maintain the ABAP system in remote connection login
details please?
Thank you,
Jutta
Hello Jutta,
We have configured our GRC system to send data to ETD and Remote connection is already open.
Also I am attaching all streaming XML files along with SETD transaction screenshot.
Regards,
Rachit
Attachment uploaded
Attachment uploaded
Attachment uploaded
Attachment uploaded
Attachment uploaded
Attachment uploaded
Attachment uploaded
Hi Anurag,
in GRC 001 I found an error for the sender:
{msg:"Malformed JSON. Details: com.google.gson.JsonSyntaxException: Missing field in JSON: EXTRACTIO...
400 Bad Request
The version of the new ABAP extractor does not fit to your log collector version, the message that is sent is different from what is expected.
We are preparing an ABAP note that makes these versions compatible.
Best regards,
Jutta
Sorry, I pressed on the wrong button. I did not want to sent the ticket back to you, but just inform you.
Could you please
Hi Anurag,
how have you implemented the new ABAP Extractor for ETD? Have you implemented the latest version of this note
https://me.sap.com/notes/3313550
Best regards
Jutta
Hello Jutta,
Thank you so much for helping us. Given SAP note has been implemented.
After that In Open Protocol logs, below Auth error was coming.
"User ETD_CLNT_COM is not authorized to execute the Reader for type USERCON (Authorization Object S_ETD_MD)
ETD Error Code: CX_SETD_NO_AUTH_LOG_READER; Refer to SAP Note 2957945 for further details "
I have created a new role and assigned all the missing authorization in that and assigned to user ETD_CLNT_COM.
There is an another below error in Open Protocols.
"Work item generator was called with an invalid configuration

ETD Error Code: CX_SETD_INV_CFG_WI_GENERATOR; Refer to SAP Note 2957945 for further details"
I tried to open the given SAP note but this is not available for us.
In the logs i can see that GRC system started to send data to ETD through given RFC.
" Sender finished

Successfully sent 1 kilobytes of master data to destination SAPGRC_TO_ETD
Sender found 1 prepared log packages
Sender started for time range 21.09.2023 19:02:05 to 21.09.2023 19:03:00 (UTC) "
But I forensic labs , still no data available.
Could you please check once and suggest what else we are missing.
Thank you so much.
Regards,
Rachit
Contacts
Role Name Time Zone Primary Phone Secondary Phone E-Mail
Anurag Kumar
Reporter Centra +91 9346882902 anurag@orane.in
(S0025438044)
System Opener UTC+00
Attachments
File Name Description File Type File Size Created By Created On
Logcollector_logs_15_SE S0025438044 - Anurag Kumar

TXT 125.0 KB 15.09.2023 10:57:39 CET
PT.txt 9/15/2023, 2:27:38 PM (S0025438044)
KAFKA_LOGS_15_SEPT S0025438044 - Anurag Kumar

TXT 91.4 KB 15.09.2023 10:57:39 CET
.txt 9/15/2023, 2:27:38 PM (S0025438044)
Normalizer_Logs_15_SE S0025438044 - Anurag Kumar

TEXT/PLAIN 42.7 KB 15.09.2023 11:15:26 CET
PT.txt 9/15/2023, 2:45:26 PM (S0025438044)
KAFKA_LOGS_15_SEPT S0025438044 - Anurag Kumar

TEXT/PLAIN 91.4 KB 15.09.2023 11:18:55 CET
.txt 9/15/2023, 2:48:55 PM (S0025438044)
KAFKA2HANA- S0025438044 - Anurag Kumar

TEXT/PLAIN 99.7 KB 18.09.2023 16:02:04 CET
LOGS_18092023.txt 9/18/2023, 7:32:04 PM (S0025438044)
S0025438044 - Anurag Kumar

Kafka_logs_18_SEPT.txt TEXT/PLAIN 213.2 KB 18.09.2023 16:26:31 CET
9/18/2023, 7:56:31 PM (S0025438044)
etd- S0025438044 - Anurag Kumar

APPLICATION/XML 3.4 KB 21.09.2023 13:22:22 CET
coldstorage_config.xml 9/21/2023, 4:52:22 PM (S0025438044)

etd-loglearner_config.xml APPLICATION/XML 1.0 KB 21.09.2023 13:23:16 CET
9/21/2023, 4:53:16 PM (S0025438044)

kafka_2_hana_config.xml 9/21/2023, 4:53:59 PM (S0025438044)

logcollector_config.xml 9/21/2023, 4:54:23 PM (S0025438044)

etd-normalizer_config.xml APPLICATION/XML 2.9 KB 21.09.2023 13:25:54 CET
9/21/2023, 4:55:54 PM (S0025438044)

transporter_config.xml 9/21/2023, 4:57:31 PM (S0025438044)
APPLICATION/VND.OPE
GRC_LOG_CONFIGURA S0025438044 - NXMLFORMATS- Anurag Kumar
261.0 KB 21.09.2023 13:28:42 CET
TION.docx 9/21/2023, 4:58:42 PM OFFICEDOCUMENT.WO (S0025438044)
RDPROCESSINGML.D
Action Log
Changed On Changed At Changed By Action Old Value New Value
Anurag Kumar 0020751295 0000812197

Friday 15.09.2023 10:05:09 CET Incident created
(S0025438044) 2023
Anurag Kumar
10:05:09 CET Component BC-SEC-ETD-CLD
(S0025438044)
Anurag Kumar
10:07:37 CET Memo/Text changed Problem Description Problem Description
(S0025438044)
Anurag Kumar
10:57:39 CET Memo/Text changed Problem Description Problem Description
(S0025438044)
Anurag Kumar
10:57:39 CET Status Not Sent to SAP Sent to SAP
(S0025438044)
Anurag Kumar
11:15:28 CET Memo/Text changed Info for SAP
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
11:46:37 CET SAP Status Sent to SAP In Processing by SAP
11:46:54 CET SAP Component BC-SEC-ETD-CLD BC-SEC-ETD
Monday 18.09.2023 10:43:23 CET SAP Status In Processing by SAP Customer Action
10:43:23 CET SAP Memo/Text changed Info for Customer
Anurag Kumar
15:36:24 CET Status Customer Action Sent to SAP
(S0025438044)
15:36:29 CET SAP Memo/Text changed Info for SAP
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
Tuesday 19.09.2023 21:08:17 CET Memo/Text changed Info for SAP
(S0025438044)
Wednesday 20.09.2023 09:11:44 CET SAP Memo/Text changed Info for Customer
12:08:19 CET SAP Memo/Text changed Business Impact
12:08:20 CET SAP Priority changed Medium High
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Thursday 21.09.2023 07:11:53 CET SAP Memo/Text changed Info for Customer
11:54:56 CET SAP Status In Processing by SAP Customer Action
Anurag Kumar
(S0025438044)
13:19:50 CET SAP Status Customer Action In Processing by SAP
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
Anurag Kumar
(S0025438044)
15:56:51 CET SAP Status Customer Action In Processing by SAP
Anurag Kumar
(S0025438044)
22.09.2023 07:26:52 CET

812197 _ 2023 - Data Not Visible in Forensic Lab of ETD System

Uploaded by

Copyright:

Available Formats

You might also like

812197 _ 2023 - Data Not Visible in Forensic Lab of ETD System

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

812197 _ 2023 - Data Not Visible in Forensic Lab of ETD System

Uploaded by

Copyright:

Available Formats

9/22/23, 11:01 AM 812197 / 2023 - Data not visible in Forensic lab of ETD system

--- Product/Function ---

--- Steps to Reproduce ---

--- Description ---

Thank you so much.

15.09.2023 11:15:28 CET - Info for SAP: Anurag Kumar (S0025438044)

15.09.2023 11:18:56 CET - Info for SAP: Anurag Kumar (S0025438044)

15.09.2023 11:18:57 CET - Info for SAP: Anurag Kumar (S0025438044)

18.09.2023 10:43:19 CET - Info for Customer: SAP

18.09.2023 15:36:23 CET - Info for SAP: Anurag Kumar (S0025438044)

Below is the error.

"2023-09-15T18:06:36.854+0000 ERROR org.apache.kafka.clients.consumer.KafkaConsumer [KafkaConsumer-JSONMasterData]: [Consumer

Also attaching the Kafka2Hana and Kafka log file.

Please check once and help to resolve.

18.09.2023 15:37:20 CET - Info for SAP: Anurag Kumar (S0025438044)

18.09.2023 16:02:04 CET - Info for SAP: Anurag Kumar (S0025438044)

18.09.2023 16:26:32 CET - Info for SAP: Anurag Kumar (S0025438044)

19.09.2023 21:08:17 CET - Info for SAP: Anurag Kumar (S0025438044)

20.09.2023 09:11:44 CET - Info for Customer: SAP

20.09.2023 12:08:19 CET - Business Impact: SAP

Go live date for production is already passed.

20.09.2023 12:09:10 CET - Info for Customer: SAP

Your interaction with SAP Support

Channel: Call from customer

Requester: Mr. Rachit

Go live date for production is already passed.

Additional Comment: Raised to high

20.09.2023 13:14:20 CET - Info for SAP: Anurag Kumar (S0025438044)

" journalctl -u kafka|grep failed

20.09.2023 18:20:37 CET - Info for SAP: Anurag Kumar (S0025438044)

20.09.2023 19:31:00 CET - Info for Customer: SAP

Your interaction with SAP Support

Contact name: Anurag Kumar

Rachit requested an update.

Go live date for production is already passed.

Additional Comment: Raised to high

21.09.2023 07:11:53 CET - Info for Customer: SAP

Your interaction with SAP Support

Contact phone or email: +251-

Requester: Mr. Rachit

Go live date for production is already passed.

21.09.2023 13:19:48 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:22:23 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:23:16 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:24:00 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:24:24 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:25:54 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:27:31 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 13:28:43 CET - Info for SAP: Anurag Kumar (S0025438044)

21.09.2023 15:55:47 CET - Info for Customer: SAP

in GRC 001 I found an error for the sender:

{msg:"Malformed JSON. Details: com.google.gson.JsonSyntaxException: Missing field in JSON: EXTRACTIO...

400 Bad Request

We are preparing an ABAP note that makes these versions compatible.

21.09.2023 15:56:48 CET - Info for Customer: SAP

Could you please

21.09.2023 19:03:16 CET - Info for Customer: SAP

21.09.2023 21:44:44 CET - Info for SAP: Anurag Kumar (S0025438044)

There is an another below error in Open Protocols.