Incidents
The shared ticket inbox of the NOC shift.
P4 INC-1006 · Medium · reported by Samuel Rey (Monitoring)
Bern: hardware alarm, no user impact
SLA breached
-
Samuel Rey (Monitoring) Report 14:01:00
One of the Bern devices raised an environment alarm at 14:01. No impact visible so far. Can the NOC take a look?
-
NOC-POC System 14:49:01
SLA breached: nobody solved this incident in time. The fault was repaired.
Post-mortem
- Root cause
- A power supply of core-brn-2 failed; the second one carried the load.
- Fault
- PSU failure
- Right answer
- D: Replace the failed power supply of core-brn-2
- Solved by
- Nobody: SLA breached
PS2 of core-brn-2 failed at 14:01:00; the remaining supply carries the load.
Causal chain
- A power supply of core-brn-2 failed; the second one carried the load.
- What was noticed: a hardware alarm while the device kept running.
Why the other options were wrong
- Replace the failed power supply of core-brn-1 Wrong: it assumed a power supply of core-brn-1 had failed. core-brn-1 was healthy: no alarm, its sensors and power supplies were normal. The cause: A power supply of core-brn-2 failed; the second one carried the load.
- Roll back the last configuration change on core-brn-1 Wrong: it assumed a configuration change on core-brn-1 broke it. core-brn-1 was healthy: no alarm, its sensors and power supplies were normal. The cause: A power supply of core-brn-2 failed; the second one carried the load.
- Roll back the last configuration change on core-brn-2 Wrong: it assumed a configuration change on core-brn-2 broke it. Right place, wrong fault: core-brn-2 had no such problem. The cause: A power supply of core-brn-2 failed; the second one carried the load.
Timeline
- 14:01:00 INC-1006 opened: Bern: hardware alarm, no user impact (Samuel Rey (Monitoring))
- 14:49:01 SLA breached; fault repaired
Inbox
0- No incidents match the filters.