侠Liangshen AIailiangshen.com
AI Implementation Workshop · Short topic lessons · Lesson 3

Let it critique itself

The answer looks right, but where might it be wrong?

Freeabout 9 minNo experience neededWeb lesson + slides + video

This is a skills module. You will practice asking the Dragon Warrior to review its own answers. Fluent speech does not mean the content is correct.

By the end of this lesson, you can

  • Ask it to list the three most likely errors in the answer, along with the reasons
  • Use the prompt 'List all content not provided by me' to identify its additions
  • For external facts, click the source links to find the original sentences
  • Complete a verification record form

Takeaway: A verification record form and a list of common pitfalls

01

Opening

About 1 minutes

First, consider a scenarioFictional example; numbers are illustrative

Finance Specialist Chen Wei

She asked the Dragon Warrior to summarize a reimbursement policy into a guide for new colleagues.

It reads smoothly, but she feels uneasy: Are the spending limits and approval levels just fabricated for the sake of flow?

Fluent speech and accuracy are two different things.

Fortunately, there is a solution: ask it to review its own answer.
02

Procedure

About 4 minutes

Three-step critique

Identify three issuesAsk it to list the three most likely errors, ranked by risk, and explain the reasoning for each.
→
List the fabricated parts.Go through sentence by sentence and list all numbers, titles, field names, and IDs that were not provided by you.
→
Verify it yourself.Check against the original source material item by item. If the original text cannot be found, it does not count.
This technique helps you highlight key points, but it does not replace human verification.
Tip

Two prompt templates for 'critique'.

Suggested time
Critique.
Please review your previous response.
1. List the three most likely errors, ranked from highest to lowest risk.
2. For each error, specify: the exact sentence, why it might be wrong, the potential consequences of the error, and which source material you should check against.
3. List all instances involving dates, amounts, names, quantities, field names, and IDs, noting whether the basis was provided by you or added by the AI.
4. If you are unsure, simply state 'I am not sure.' Do not provide statements that sound confident.
List all content not provided by me.
Please go through your previous response sentence by sentence and list all content not found in the materials I provided: fabricated numbers, titles, experiences, field names, function names, clause IDs, and promises. Indicate the sentence where each appears. I will decide whether to delete them; do not delete them yourself.
Each point must specify a single sentence and indicate which source material to check against.

External facts: Click the link, find the original sentence.Key verification points.

I only see the link.

It provided a link, stating 'Source here.'

True verification.

Click both sources, find the specific number or original sentence within the pages, and check the date and applicable region.

A clickable link does not mean the answer is substantiated. If you cannot find the original sentence, treat it as unsubstantiated.

When it goes in circles, stop the loss using this sequence.

You must make the final judgment yourself.
03

Practice and Recording

About 2 minutes

Verify the five columns of the record sheet

Date

Which day this message is from, or which day it refers to

Which section

The specific sentence or number

Its statement

What is written in the answer

The original text's statement

What your reference material says, with the source cited

Consistency

Choose one: consistent, inconsistent, or not in the original text

Tip

5-minute exercise: Find one of its errors

Suggested time
Tech version material (fictional, ready to paste)
Field table (use this as the standard):
task_id: The ID of the export task
range_days: The range of days for the export
format: The export format, with values of 'table' or 'text'

API description (needs verification):
Pass task_id to specify the task; range_days to specify the export days; format to specify the format; retry_limit to specify the number of retries after failure, defaulting to three. After the export is complete, the system returns a download link.
Question
Which content in the API description is not in the field table? List them one by one and explain where you should verify them.
The answer should include retry_limit and the sentence 'returns a download link'.
04

Quiz and Wrap-up

About 2 minutes

Three quick questions to check yourself

Question 1: If the link it provides opens, does that mean the answer is supported?

Answer B.An open link only means the page exists; you must find the specific original sentence and check the date and applicable region. If you cannot find it, treat it as unsupported.
Show the full transcript (video narration)
Let it critique itself

This is a skills module. You will practice asking the Dragon Warrior to review its own answers. Fluent speech does not mean the content is correct. In nine minutes, you will practice a verification method.

After completing this module, you will be able to

After completing this module, you will be able to do four things. First, ask it to list the three most likely errors in the answer, along with the reasons. Second, use the prompt 'List all content not provided by me' to identify its additions. Third, for external facts, click the source links to find the original sentences. Fourth, complete a verification record form. Your takeaway is this form and a list of common pitfalls.

First, consider a scenario

First, consider a fictional scenario; numbers are illustrative. Finance Specialist Chen Wei asked the Dragon Warrior to summarize a reimbursement policy into a guide for new colleagues. It reads smoothly, but she feels uneasy: Are the spending limits and approval levels just fabricated for the sake of flow? The Dragon Warrior's fluent speech does not guarantee accuracy. Fortunately, there is a simple solution: ask it to review its own answer.

Three-step critique

The 'critique' process has three steps. Step 1: Ask it to list the three most likely errors, ranked from highest to lowest risk, and explain the reasoning for each. Step 2: Go through sentence by sentence and list everything that was not provided by you: numbers, titles, field names, and IDs. You decide whether to delete them. Step 3: You check against the original source material, marking each item off one by one. This technique helps you highlight key points, but it does not replace human verification.

Two prompt templates for 'critique'.

Here are two prompt templates. The first asks it to list the three most likely errors, specifying for each: the exact sentence, why it might be wrong, and which source material I should check. The second asks it to list all content not provided by me, sentence by sentence. I decide whether to delete them; it should not delete them itself. For uncertain points, it should simply state 'I am not sure.'

External facts: Click the link, find the original sentence.

For verifying external facts, follow these three steps. First, click at least two sources. Second, find the specific number or original sentence within the pages; if you cannot find it, it does not count. Third, check the date and applicable region to ensure they are current and relevant to your location. A clickable link does not mean the answer is substantiated.

When it goes in circles, stop the loss using this sequence.

When it goes in circles, stop the loss in this order. First, halt and ask it to write in two columns: confirmed facts and things that are still assumptions. Then paste the original text back and ask it to answer based only on the original. If it is still messy, start a new conversation and bring only the summary. If it keeps apologizing but the answer remains wrong, stop urging or scolding. Instead, set rules, ask the opposite question, and verify yourself. Make the final judgment yourself.

Verify the five columns of the record sheet

The verification record sheet has five columns: Date, Section, Its statement, Original text's statement, and Consistency. If the original text does not mention something, record that it is not there. Keeping this record over time will help you identify whether it commonly makes mistakes with dates, amounts, responsible parties, or field names in your type of work, so you can prioritize checking these areas in the future.

5-minute exercise: Find one of its errors

The exercise has an office version and a tech version; here we discuss the tech version. The material is fictional: the field table has only three fields, but the API description adds retry_limit and mentions returning a download link. Paste both sections to it and ask it to list the content not in the field table one by one. You should see retry_limit and the sentence about returning a download link. Then verify each item one by one and fill in the verification record sheet.

Quiz

Here is a quiz. If the link it provides opens, does that mean the answer is supported? A. Yes, an open link means there is support. B. No, you must find the specific number or original sentence in the page. C. Yes, as long as it claims to be an official source. The answer is B. An open link only means the page exists; you must also find the specific original sentence and check the date and applicable region.

End of this lesson

End of lesson. Take a passage it wrote and the specific point you identified for verification. Check whether it uses phrases like 'sent' or 'verified'; those phrases also need verification. Its self-critique may be incomplete. The final safeguard remains your responsibility. Thank you for watching. Narration synthesized by AI.