Scenarios
About 1 minutesLet's look at a scenario firstFictional example; numbers are illustrative
Each shift faces hundreds of alerts, with repeated alerts from the same source.
Shift handovers rely on handwritten notes, taking about 70 minutes per shift.
Qiu Ziang, Backend Developer at Qilan Intelligence, dealt with an online incident last night. The post-mortem requires documenting the sequence, impact, root cause hypotheses, and improvement items, taking about three hours.
Methods
About 2 minutesMethod: Five Steps, Read-Only, No Action
Check each item before posting
Prompts and Examples
About 4 minutesPrompt 1: Alert Merging, Read-Only, No Action
Suggested timeYou are an operations shift supervisor assistant with ten years of experience. You only read and do not take action. Below are the alerts for a shift (already de-identified: devices are written as Device 01, etc.). Please: 1. Merge them into groups based on likely common root causes, and list the time and quantity of items in each group; 2. List the top three items worth reviewing first, and explain why; 3. Count the number of occurrences for each type of alert; 4. End with 'Information I did not see.' Rules: Only merge; do not determine severity levels or provide any action commands. For alerts not provided to you, do not write 'No anomalies.' Use only the content I provide; if something is missing, write 'To Be Supplemented,' and do not make things up. Alerts: [Paste]
Prompt 2: Post-Mortem Draft, Separate Facts and Speculation
Suggested timeBelow is the time-and-phenomenon record of an online incident (already de-identified). Please: 1. Sort by time and mark time gaps: which periods have no record of what anyone did; 2. Divide into two columns: the 'Facts' column contains only the time and phenomena explicitly stated in the records; the 'Speculation' column contains reasons guessed by people in the records, with each item marked 'To Be Verified'; 3. Draft the post-mortem structure: Process, Impact (numbers to be verified by me), Cause (write only 'To Be Confirmed'), Improvement Items (write 'To Be Confirmed' for responsible person and deadline); 4. End with 'Information I did not see.' Rules: Do not write speculation in the Cause section; do not fabricate times. Record: [Paste]
Example: Organizing an incident record into a timelineFictional example; numbers are illustrative
Original Record
21:40 Version 2 deployment completed; 21:52 Response time increased; 22:05 Someone in the group chat said 'It might be that the database connections are full'; 22:30 Decision to roll back; 22:41 Response time returned to normal.
Organized
- Facts Column: Four time points: 21:40, 21:52, 22:30, 22:41
- Speculation Column: The comment at 22:05, marked 'To Be Verified'
- Time Gap: Between 21:52 and 22:05, there is no record of what anyone did
- Reason: To be confirmed
Common Biases and Remedies
Mistaking hypotheses for causes
Remedy: The Cause section should only state 'To be confirmed.' Move all sentences containing words like 'possibly' or 'probably' to the Hypotheses section.
Overlooking low-priority but recurring alerts
Remedy: Instruct the system to count the occurrences of each alert type. Even if an alert is low-priority, it should be listed if it occurs frequently.
It states 'All other alerts are normal'
Remedy: Require it to report the number of alerts reviewed versus the total number. Randomly check a few samples. If the numbers do not match, it has not reviewed all alerts.
It provides handling suggestions or commands
Remedy: Remind it that the role is read-only for handling; delete any commands. Decisions on handling are made by the Shift Lead.
Mini-Exercises, Quizzes, and Takeaways
About 3 minutesThree quick questions to check yourself
Question 1: Who writes the 'Cause' in the post-mortem?
Question 2: It states 'All other alerts are normal.' What do you do?
In this lesson, take away four key phrases.
- Only include the timestamp and the incident description in de-sensitized data; do not post source code, keys, or customer logs.
- Sort the timeline chronologically and mark any time gaps.
- Separate the facts column from the speculation column; label all speculation as 'pending verification'.
- Treat alerts as read-only; do not write 'no anomaly' for items you haven't read.
Next step: Create your own initial draft of a post-mortem.
Run both prompts using de-sensitized records approved by your company.
- Select a de-sensitized duty log, sort it chronologically, and separate it into facts and speculation columns.
- Take three items from the merged results and cross-check them against the original records.
Show the full transcript (video narration)
Incident Post-Mortem and Alert Consolidation
This lesson covers incident post-mortems and alert consolidation. We will do two things: organize messy duty logs into a timeline and draft an initial post-mortem; consolidate hundreds of alerts into groups and select a few key items for human review. First, let's look at a scenario.
Let's look at a scenario first
Let's look at a scenario, which is fictional, with illustrative numbers. Hong Zhiyuan, a Network Operations Engineer at Duchuan Communications, faces hundreds of alerts per shift, with repeated alerts from the same source. Shift handovers rely on handwritten notes, taking about 70 minutes per shift. Qiu Ziang, a Backend Developer at Qilan Intelligence, dealt with an online incident last night. The post-mortem requires documenting the sequence, impact, root cause hypotheses, and improvement items, taking about three hours. Both often make similar mistakes: incorrect timeline order, treating hypotheses as confirmed causes, and improvement items lacking assigned owners.
Method: Five Steps, Read-Only, No Action
The process has five steps. Step 1: Provide only the de-identified time and phenomenon. Write 'Device 01' for devices and 'Customer A' for customers. Do not paste the entire log, source code, or network addresses. Step 2: Sort by time and mark time gaps. Step 3: Divide into 'Facts' and 'Speculation' columns; all speculation must be marked 'To Be Verified.' Step 4: Merge alerts into groups, list the top three items worth reviewing first, and count the occurrences of each type. Step 5: A human confirms: the cause is written by a person, numbers are verified against monitoring, and improvement items include the responsible person and deadline. It only reads and does not take action; merging is just a suggestion.
Check each item before posting
Before posting, please check each item. First, the content contains only time and phenomena, with no source code, keys, or customer logs. Second, devices, customers, and addresses have all been replaced with codes. Third, I know the reasons it provides are only hypotheses to be verified. Fourth, I will cross-reference the original records and spot-check two merging results. Additionally, a reminder: review the company's AI usage policy first. Source code, keys, and customer network data must not be posted in public AI applications. The materials for this exercise are fictional and can be posted directly.
Prompt 1: Alert Merging, Read-Only, No Action
The first prompt is for alert merging, read-only, and no action. It merges alerts based on likely common root causes, lists the top three items worth reviewing, counts the occurrences of each type of alert, and ends with 'Information I did not see.' The rules are strict: do not determine severity levels, do not provide action commands, and for alerts not provided to it, do not write 'No anomalies.' If something is missing, write 'To Be Supplemented,' and do not make things up.
Prompt 2: Post-Mortem Draft, Separate Facts and Speculation
The second prompt is for the post-mortem draft. First, sort by time and mark time gaps; then divide into 'Facts' and 'Speculation' columns, with each speculation item marked 'To Be Verified'; then draft the post-mortem structure, writing 'To Be Confirmed' in the Cause column and for the responsible person and deadline of improvement items. There are only two rules: do not write speculation in the Cause section, and do not fabricate times.
Example: Organizing an incident record into a timeline
Consider a fictional incident record with illustrative numbers. The original record contains five entries: 21:40 – Version 2 release completed; 21:52 – Response time increased; 22:05 – A team member suggested the database connections might be exhausted; 22:30 – Decision to rollback; 22:41 – Response time returned to normal. After organizing the data, the Facts section lists four timestamps, while the statement about database connections is moved to the Hypotheses section, marked as pending verification. There is no record between 21:52 and 22:05, creating a time gap. The reason for the rollback at 22:30 is not documented, requiring human input to fill the gap.
Common Biases and Remedies
There are four common biases. First, mistaking hypotheses for causes; the remedy is to mark the Cause section only as 'To be confirmed.' Second, overlooking low-priority but recurring alerts; the remedy is to have it count occurrences and list those that appear frequently. Third, it states 'All other alerts are normal'; the remedy is to require it to report the number of alerts reviewed and randomly check a few samples—if the numbers do not match, it has not finished reviewing. Fourth, it provides handling suggestions or commands; the remedy is to remind it that the role is read-only for handling and delete any commands. If improvement items lack an owner and deadline, mark them as 'To be confirmed' and do not allow the system to assign them itself.
Quiz Question 1
Let's take the quiz. Question 1: Who writes the 'Cause' in the post-mortem? A. Dragon Warrior writes it directly, as it looks the most professional. B. A person confirms it; Dragon Warrior can only provide hypotheses, which must be marked 'Pending Verification'. C. The Shift Lead writes it based on impression, then has Dragon Warrior polish it. The answer is B. Dragon Warrior can only provide hypotheses; causes must be confirmed by a person, and all hypotheses must be marked 'Pending Verification'.
Quiz Question 2
Question 2: It states 'All other alerts are normal.' What do you do? A. Accept it directly and include it in the handover document. B. Require it to list the number of alerts reviewed and randomly check a few samples. C. Write 'No anomalies' in the improvement items. The answer is B. Alerts that have not been reviewed cannot be marked as 'No anomalies'; if the numbers do not match, it has not finished reviewing.
In this lesson, take away four key phrases.
Let's review: take away these four phrases from this lesson. First, only include the timestamp and incident description in de-sensitized data; do not post source code, keys, or customer logs. Second, sort the timeline chronologically and mark any time gaps. Third, separate the facts column from the speculation column; label all speculation as 'pending verification'. Fourth, treat alerts as read-only; do not write 'no anomaly' for items you haven't read.
Next step: Create your own initial draft of a post-mortem.
Next, use de-sensitized records approved by your company to run both prompts. Select a log, sort it chronologically, and separate it into facts and speculation columns; then take three items from the merged results and cross-check them against the original records. This video's narration is AI-generated. Thanks for watching.