Service Request Number Customer Ticket Number Report Date Incident Occurred Date

9165206 2018-01-04 04:56:47 2018-01-04 04:56:47

9499771 2018-03-13 23:27:12 2018-03-13 23:27:12

9502339 2018-03-14 12:51:46 2018-03-14 12:51:46

9507009 2018-03-15 07:42:50 2018-03-15 07:42:50

9580446 2018-03-29 04:22:44 2018-03-29 04:22:44

9602355 2018-04-02 23:56:05 2018-04-02 23:56:05

9617164 2018-04-05 11:17:05 2018-04-05 11:17:05

9617229 2018-04-05 11:33:34 2018-04-05 11:33:34

 Expected Response Date Actual Response Date Response SLA Performance Expected Restore Date
2018-01-04 05:13:24 No SLA

2018-03-14 00:27:12 2018-03-13 23:27:48 Win 2018-03-23 23:27:12

2018-03-14 13:21:46 2018-03-14 12:51:46 Win 2018-03-17 12:51:46

2018-03-15 08:42:50 2018-03-15 07:43:59 Win 2018-04-14 07:42:50

2018-03-29 04:52:44 2018-03-29 04:22:44 Win 2018-04-01 04:22:44

2018-04-03 00:26:05 2018-04-02 23:56:50 Win 2018-04-05 23:56:05

2018-04-05 11:47:05 2018-04-05 11:18:04 Win 2018-04-05 23:17:05

2018-04-05 12:33:00 2018-04-05 11:34:27 Win 2018-04-15 11:33:34

Expected Restore Date with Travel Actual Restore Date Restore SLA performance
No SLA

2018-03-23 23:27:12 2018-03-15 08:17:46 Win

2018-03-17 12:51:46 Lost

2018-04-14 07:42:50 Undue

2018-04-01 04:22:44 Lost

2018-04-05 23:56:05 2018-04-03 05:37:20 Win

2018-04-05 23:17:05 Lost

2018-04-15 11:33:34 Undue

Expected Workaround Date with Suspend Actual Workaround Date Workaround SLA Performance
No SLA

No SLA

No SLA

No SLA

No SLA

No SLA
No SLA

No SLA
 Expected Close Date Expected Close Date with Suspend Resolved Date Resolve SLA Performance
No SLA

2018-05-12 23:27:12 2018-05-12 23:27:12 Undue

2018-07-12 12:51:46 2018-07-12 12:51:46 Undue

No SLA

2018-07-27 04:22:44 2018-07-27 04:22:44 Undue

2018-07-31 23:56:05 2018-07-31 23:56:05 Undue

2018-05-20 11:17:05 2018-05-20 11:17:05 Undue

2018-06-04 11:33:00 2018-06-04 11:33:00 Undue

Total Suspend Duration(Day) Closed Date Last Updated Date Type Severity
43.51 2018-04-05 00:31:04 CS - Technical Request Minor

0.00 2018-03-29 02:26:12 CS - Technical Request Minor

0.00 2018-03-27 10:56:01 CS - Technical Request Major

0.00 2018-04-05 10:16:30 CS - Technical Request Minor

0.00 2018-04-05 07:29:52 CS - Technical Request Major

0.00 2018-04-03 05:47:20 CS - Technical Request Major

0.00 2018-04-05 12:22:36 CS - Technical Request Major

0.00 2018-04-05 11:35:42 CS - Technical Request Minor

Customer Severity Urgency On-Site Flag Status Stage Channel Logged By
Minor Schedule action Customer Agreed Suspend L1 Stage Email H84088578

Minor Schedule action L1-Work in Progress L1 Stage Email R80014838

Major Schedule action L1-Solution Review L1 Stage Phone CWX510806

Minor Schedule action Solution Provided L1 Stage Email H84088578

Major Schedule action L2-Work in Progress L2 Stage Phone CWX510806

Major Schedule action L1-Work in Progress L1 Stage Phone J84064266

Major Schedule action L1-Work in Progress L1 Stage Email J84079399

Minor Schedule action L1-Work in Progress L1 Stage Email J84079399

Incident Level Resolution Code Problem Code Contract Number
None Client-Huawei Product-local Maintenance Unknown No SLA

None Unknown Unknown 00060416007527

None Client-Huawei Product-local Maintenance Unknown NSF201801309406

None Client-Huawei Product-local Maintenance Unknown NSF201801309406

None Unknown Unknown 00060418002324

None Unknown Unknown 00060418002324

None Unknown Unknown 00060416007527

None Unknown Unknown 00060416007527

 Service Item Defect No.in DTS DTS No.in release notes Escalate to RDE Flag
N

Non phone-Technical Assistance Service N

Non phone-Technical Assistance Service N

Phone-Technical Assistance Service N

Non phone-Technical Assistance Service N

Phone-Technical Assistance Service N

Non phone-Technical Assistance Service N

Non phone-Technical Assistance Service N

Result Affect Type Escalate to PSE Flag Escalate to PSE Date Escalate to PSE Target Date
Y 2018-02-08 22:55:54

N 2018-03-19 23:27:12

Y 2018-03-14 14:44:19 2018-03-26 12:51:46

Y 2018-03-21 06:40:08

Y 2018-04-05 05:34:01 2018-04-10 04:22:44

N 2018-04-14 23:56:05
N 2018-04-09 23:17:05

N 2018-04-11 11:33:30
Escalate to RDE Date Escalate to RDE Target Date Customer Region Customer Office Country
Latin America Region Peru Rep Office Peru

Latin America Region Peru Rep Office Peru

2018-04-07 14:44:19 Latin America Region Peru Rep Office Peru

Latin America Region Peru Rep Office Peru

2018-04-29 05:34:01 Latin America Region Peru Rep Office Peru

Latin America Region Peru Rep Office Peru

Latin America Region Peru Rep Office Peru

Latin America Region Peru Rep Office Peru

 Customer Type Customer Group Customer Name Contact Type Contact Full Name
Carrier Telefonica Telefonica del Peru S.A.A Customer Elias Guzman

Carrier Telefonica Telefonica del Peru S.A.A Customer Miguel Benito Orellana

Carrier Telefonica Telefonica del Peru S.A.A Employee WX510806,Cristhian Eduardo Reyes Alcala

Carrier Telefonica Telefonica del Peru S.A.A Customer Denys Espillco

Carrier Telefonica Telefonica del Peru S.A.A Employee WX510806,Cristhian Eduardo Reyes Alcala

Carrier Telefonica Telefonica del Peru S.A.A Customer Luis Felipe Quispe
Carrier Telefonica Telefonica del Peru S.A.A Customer Miguel Benito Orellana

Carrier Telefonica Telefonica del Peru S.A.A Customer Miguel Benito Orellana
 Contact Work Phone Email Address
51-999040957 Elias.guzman@telefonica.com

005112102254 // 0051995833671 miguel.benito@telefonica.com

cristhian.reyes.wx@huawei.com

996086647 denys.espillco@telefonica.com

cristhian.reyes.wx@huawei.com

51-938947020 luisfe.quispe@telefonica.com
005112102254 // 0051995833671 miguel.benito@telefonica.com

005112102254 // 0051995833671 miguel.benito@telefonica.com

 Employee Organization TAC Employee Organization Name
Spanish TAC(Mexico) Spanish TAC(Mexico)-OSS-Peru-CSE

Spanish TAC(Mexico) Spanish TAC(Mexico)-ON-Peru-CSE

Unknown Unknown

Spanish TAC(Mexico) Spanish TAC(Mexico)-OSS-Peru-CSE

Unknown Unknown

Spanish TAC(Mexico) Spanish TAC(Mexico)-Optical Network-CSE

Spanish TAC(Mexico) Spanish TAC(Mexico)-ON-Peru-CSE

Spanish TAC(Mexico) Spanish TAC(Mexico)-ON-Peru-CSE

 Employee Name Department Code Department Name Product Line
,00754937,PABLO ROBERTO NAVA ACOSTA, 031054 TAC Mexico Network

,00749923,David Leal Rodriguez, 031054 TAC Mexico Network

WX510806,WX510806,Cristhian Eduardo Reyes Alcala, Unknown Unknown Network

,00754937,PABLO ROBERTO NAVA ACOSTA, 031054 TAC Mexico Network

WX510806,WX510806,Cristhian Eduardo Reyes Alcala, Unknown Unknown Network

,00749923,David Leal Rodriguez, 031054 TAC Mexico Network

,80015039,Ernesto Reyes Cuenca, 031054 TAC Mexico Network

,80015039,Ernesto Reyes Cuenca, 031054 TAC Mexico Network

 Product Class SPDT PDT Product
NCE NCE-T iManager U2000-T iManager U2000-T

Transmission Network WDM Metro WDM OptiX OSN 9800 U16

Transmission Network WDM Backbone WDM PDT OptiX OSN 8800

NCE NCE-T iManager U2000-T iManager U2000-T

NCE NCE-T iManager U2000-T iManager U2000-T

Transmission Network WDM Backbone WDM PDT OptiX OSN 8800

Transmission Network WDM Metro WDM OptiX OSN 9800 U16

Transmission Network WDM Metro WDM OptiX OSN 9800 U16

Product Number Category Impact Unit Product Revision
6692804 Optical Network 202

OFFE00004425 Optical Network

6046100 Optical Network 164

6692804 Optical Network 202

6692804 Optical Network 306

6046100 Optical Network

OFFE00004425 Optical Network

OFFE00004425 Optical Network

 Product Revision Desc
iManager U2000-T V200R016(EOFS:, EOS:)

OptiX OSN 8800 T32 V100R009C10SPC001T

iManager U2000-T V200R016(EOFS:, EOS:)

iManager U2000-T V200R016C60CP2112

 Problem Summary
Peru|Telefonica|U2000 T| Problems with performances report| Site: Lima

Peru | Telefonica | OSN 9800 | Alarm OSC_LOS on 0-09-12ST2. Site: PUN_9800UPS_202E-1

Unidireccional trails in OSN ORO_8800T32_210E-1_LA OROYA(M)

Peru|Telefonica|U2000 T| Can't see the board of the ATN in the U2000 of TX

Unidirectional SDH trails Old Reference SR 9037495

SR | Peru | Telefonica | OSN 8800 | Spare TN12LSCT01 | Site: San Isidro

Peru | Telefonica | OSN 9800 | OLP_PS alarm | Site: ICA_9800UPS_905F-1

Peru | Telefonica | OSN 9800 | COMMUN_FAIL alarm | Site: MAN_9800U16_102E-2

 Resolution Summary
Please set the delay according to HQ suggestions

SDH trail is a function of U2000, we have discussed with U2000 expert and they already provide a method to check this kind of unidirection trail problem. please
follow the guide provided by U2000 expert Ansiwei.

the device can’t access NMS server by sftp/ftp channel

Outage Type Impact Business Number of Network Elements Impacted Partial Outage Rate
 Suspend Planned End Date SLA Due Date Reference Entitlement Check Result Cyber Security Flag
2018-04-06 01:17:52 No contract, provide service without SLA N

2018-05-12 23:27:12 Provide service by contract N

2018-07-12 12:51:46 Provide service by contract N

Provide service by contract N

2018-07-27 04:22:44 Provide service by contract N

2018-07-31 23:56:05 Provide service by contract N

2018-05-20 11:17:05 Provide service by contract N

2018-06-04 11:33:00 Provide service by contract N

Escalation Level Current Assignee Unresolved Duration(day)
Never Escalation 00754937,PABLO ROBERTO NAVA ACOSTA, 92

Never Escalation 00749923,David Leal Rodriguez, 24

Never Escalation WX510806,Cristhian Eduardo Reyes Alcala, 23

Never Escalation 00754937,PABLO ROBERTO NAVA ACOSTA, 22

Never Escalation 00314465,Yang Jianguo, 8

Never Escalation 00749923,David Leal Rodriguez, 4

Never Escalation 80015039,Ernesto Reyes Cuenca, 1

Never Escalation 80015039,Ernesto Reyes Cuenca, 1

Overdue duration(day)

-36

-97

-112

-116
-44

-59
 Publish Notes Latest Update Content
Current Progress: local office is tracking this issue, customer's request needs development. Next Plan: give anaswe to the customer.Next Update Date:2018-03-24
11:57:05
Current Progress: we are pending LRO to confirm the results of the replacement of the transmission board Next Plan: once we have those results we will be able
to confirm the root cause:Next Update Date:2018-04-04 17:10:56

Solution Analysis: [Root Cause] the device can’t access NMS server by sftp/ftp channel [01:01:02.395],Info ,
[T12171],CNeDecoupleMibOper::queryFileTransferState OperStatus is error!, FlhOperStatus = 7, strSourceFile = cfcard:/15.225531.zip, strDestFile =
ftproot/pnp/router_v5/atn_910950/card/15.225531.zip, iTransStatus = 7 [FileTransfer2Net|CNeDecoupleEntity|-1] [01:01:02.456],Warn ,
[T12171],FileTransfer2Net failed!, iRet = 603980178 [CollectPnpData|CNeDecoupleEntity|-1] [01:01:02.532],Warn ,[T12171],get file from device failed!, try get
pnp from NMS., iRet = 603980178 [HandleCollectPnpData|CNeDecoupleTask|-1] Resolution Detail: 1 check SNMP and Test snmp . Administration-NE
Communicate Parameter—NE Access Protocol Parameters. The Test result must be succeess 1 switch the channel to ftp and sync NE, then reopen the Ne
Explorer, check the Bord type is recover 2 if after step 1 ,problem still exist ,please try ftp NMS server on device If ftp failed ,that we may check the device
configuration on device 3 if step 2 is success ,then may collect some information (the early info don’t capture the packets) , 3-1 snoop -d bge0 -o
/export/home/snoop_20180321.cap host device_IP You can [#ifconfig -a] comand to get network card name, e.g. one machine ip is 10.71.164.66, run ifconfig -a
comand in this machine: # ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.71.164.66 netmask fffffe00 broadcast 10.71.165.255 ether
0:3:ba:a2:9b:c9 the network card name of 10.71.164.66 is bge0 then Reopen the NE explorer , then sync / sync Description on the Down right , then you can stop
capture //Risk of Solution:Non-Risk Operation //

Next Update Date:2018-04-06 16:36:35

 80% SLA Date Publish Notes Latest Update Date Publish Notes Latest Update Type
2018-03-16 02:22:06 Work Notes

2018-04-30 23:27:12 2018-03-27 07:11:44 Work Notes

2018-06-18 12:51:46

2018-04-05 10:11:26 Solution Description

2018-07-03 04:22:44

2018-07-07 23:56:05 2018-04-03 05:37:25 Work Notes

2018-05-11 11:17:05

2018-05-23 11:32:34
 Notes Latest Update Date Notes Latest Update Type Notes Next Update Notes Latest Visibility
2018-03-16 02:22:06 Work Notes 2018-03-25 01:57:05 Publish

2018-03-27 07:11:44 Work Notes 2018-04-05 06:10:56 Publish

2018-03-27 10:56:01 Solution Description Public

2018-04-05 10:11:26 Solution Description Publish

2018-04-05 07:29:52 Work Notes 2018-04-09 07:26:58 Public

2018-04-03 05:37:25 Work Notes 2018-04-07 05:36:35 Publish

 Notes Latest Update Content
Current Progress: local office is tracking this issue, customer's request needs development. Next Plan: give anaswe to the customer.Next Update Date:2018-03-24
11:57:05
Current Progress: we are pending LRO to confirm the results of the replacement of the transmission board Next Plan: once we have those results we will be able
to confirm the root cause:Next Update Date:2018-04-04 17:10:56
Solution Analysis: There are many unidirection trails and each trail contains so many NEs, it is impossible to check cross connections and fiber connections for each
NE from device side. I talked this with Yangjianguo today, Ansiwei has already provided the steps which indicate how to analysis for this situation. Please try
following the steps for all other problem trails. If not clear about the steps, please create ticket for U2000 and escalate it to L2, Yangjianguo will help to explain in
detail with U2000 r&d together. Otherwise, you can consider the suggestion from Ansiwei, look for help from professional teams. Resolution Detail: SDH trail
is a function of U2000, we have discussed with U2000 expert and they already provide a method to check this kind of unidirection trail problem. please follow the
guide provided by U2000 expert Ansiwei. // Risk of Solution:Non-Risk Operation

Solution Analysis: [Root Cause] the device can’t access NMS server by sftp/ftp channel [01:01:02.395],Info ,
[T12171],CNeDecoupleMibOper::queryFileTransferState OperStatus is error!, FlhOperStatus = 7, strSourceFile = cfcard:/15.225531.zip, strDestFile =
ftproot/pnp/router_v5/atn_910950/card/15.225531.zip, iTransStatus = 7 [FileTransfer2Net|CNeDecoupleEntity|-1] [01:01:02.456],Warn ,
[T12171],FileTransfer2Net failed!, iRet = 603980178 [CollectPnpData|CNeDecoupleEntity|-1] [01:01:02.532],Warn ,[T12171],get file from device failed!, try get
pnp from NMS., iRet = 603980178 [HandleCollectPnpData|CNeDecoupleTask|-1] Resolution Detail: 1 check SNMP and Test snmp . Administration-NE
Communicate Parameter—NE Access Protocol Parameters. The Test result must be succeess 1 switch the channel to ftp and sync NE, then reopen the Ne
Explorer, check the Bord type is recover 2 if after step 1 ,problem still exist ,please try ftp NMS server on device If ftp failed ,that we may check the device
configuration on device 3 if step 2 is success ,then may collect some information (the early info don’t capture the packets) , 3-1 snoop -d bge0 -o
/export/home/snoop_20180321.cap host device_IP You can [#ifconfig -a] comand to get network card name, e.g. one machine ip is 10.71.164.66, run ifconfig -a
comand in this machine: # ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.71.164.66 netmask fffffe00 broadcast 10.71.165.255 ether
0:3:ba:a2:9b:c9 the network card name of 10.71.164.66 is bge0 then Reopen the NE explorer , then sync / sync Description on the Down right , then you can stop
capture //Risk of Solution:Non-Risk Operation //
Current Progress: According to Ansiwei’s analysis show us this Bidirectional trail in old Database (when VC4 server trail was fine): Now this trail is Unidirectional (In
Yellow): NE: ORO_8800T32_210E-1_LA OROYA(M) Level Source Slot Source TimeSlot/Path Sink Slot Sink TimeSlot/Path Service Origin VC4
Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-25-N4SLD64-1(SDH-1) VC4:4 Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-25-N4SLD64-2(SDH-2) VC4:14 Create
Manually UNIDIRECTIONAL VC4 Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-24-N4SLD64-1(SDH-1) VC4:9 Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-25-
N4SLD64-1(SDH-2) VC4:4 Create Manually BIDIRECTIONAL VC4 Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-25-N4SLD64-1(SDH-2) VC4:4
Shelf0(ORO_8800T32_210E-1_LA OROYA(S))-24-N4SLD64-1(SDH-1) VC4:9 Create Manually Please Ansiwei and Yang, confirm to notify to customer that need to
delete 24-1 VC4:9 --------> 25-1 VC4:4 and then create 25-2 VC4:14----------> 25-1 VC4:4 ? Customer request us that the suggestion we give to them must
solve the problem. Next Plan: Please feedback database backup.Next Update Date:2018-04-08 18:26:58

Next Update Date:2018-04-06 16:36:35

Drilling Test Flag Work Notes Link State Reject to L2 times
N Work Notes

N Work Notes

N Work Notes

N Work Notes

N Work Notes

N Work Notes
N Work Notes

N Work Notes