Translating SRT Subtitles with the Three-Step Reflection Method

Andrew Ng's "Three-Step Reflection Translation Method" is very effective. It improves translation quality by having the model self-examine the translation results and propose improvements. However, directly applying this method to SRT format subtitle translation presents some challenges.

Special Requirements of SRT Subtitle Format

SRT format subtitles have strict formatting requirements:

• First line: Line number (numeric)
• Second line: Two timestamps, connected by -->, in the format hours:minutes:seconds,milliseconds
• Third line and subsequent lines: Subtitle text content

Subtitles are separated by two blank lines.

Example:

1
00:00:01,950 --> 00:00:04,430
Several molecules have been discovered in the five-star system,

2
00:00:04,720 --> 00:00:06,780
We are still multiple universes away from third-type contact.

3
00:00:07,260 --> 00:00:09,880
Weibo has been carrying out filming missions for years now,

4
00:00:10,140 --> 00:00:12,920
Many previously difficult-to-capture photos have been transmitted recently.

Common Problems in SRT Translation

When using AI to translate SRT subtitles, the following problems may occur:

• Format errors:
  • Missing line numbers or duplicate timestamps
  • Translating English symbols in timestamps into Chinese symbols
  • Merging adjacent subtitle texts into one line, especially when the previous and next sentences form a complete grammatical sentence.
• Translation quality issues:
  • Even with strict prompt restrictions, translation errors often occur.

Examples of Common Errors:

• Subtitle text merging leading to blank lines

• Format disorder

• Line number being translated

• Inconsistent number of original and resulting subtitles

As mentioned above, when the previous and next subtitles are grammatically part of the same sentence, they are likely to be translated into a single subtitle, resulting in a missing subtitle entry in the result.

Format errors directly prevent subsequent processes that rely on SRT from working. Different models exhibit different errors and error probabilities. Generally, more intelligent models are more likely to return valid and compliant content, while locally deployed small-scale models are almost unusable.

However, given the improvement in translation quality offered by the three-step reflection method, I still tried it. I eventually chose to use gemini-1.5-flash for a small test, mainly because it's intelligent enough and free. Apart from frequency limitations, it has virtually no restrictions.

Writing Prompt Ideas

Write prompts according to Andrew Ng's three-step reflection workflow:

• The first step requires the AI to translate literally.
• The second step requires evaluating the literal translation and providing optimization suggestions.
• The third step re-translates based on the optimization suggestions.

The difference is that the prompts are strengthened to ensure that the returned content is in a valid SRT format, although it may not always comply perfectly.

Building a Simple API

One problem with the three-step reflection method is the significantly higher token consumption. The prompts become longer, the output results become longer, and due to Gemini's frequency limitations, exceeding the limit will return a 429 error. It's necessary to pause for a period of time after each request.

Use Flask to build a backend API, and use Bootstrap5 to create a simple single page frontend. The overall interface is as follows:

Obviously, using Gemini in China requires a VPN.

• Simultaneous translation lines: refers to the number of subtitle lines in a single translation request. If it's too large, it may exceed the token limit and cause errors. If it's too small, it's meaningless. A value within 30-100 is recommended, with a default of 50.
• Pause seconds after translation: Prevents requests from being too frequent and causing a 429 error. It will pause for 10 seconds after each request before making the next request.

Example of a returned result:

<step1_initial_translation>
1
00:00:01,950 --> 00:00:04,430
Several molecules have been discovered in the five-star system,

2
00:00:04,720 --> 00:00:06,780
We are still multiple universes away from third-type contact.

3
00:00:07,260 --> 00:00:09,880
Weibo has been carrying out filming missions for years now,

4
00:00:10,140 --> 00:00:12,920
Many previously difficult-to-capture photos have been transmitted recently.

5
00:00:13,440 --> 00:00:17,500
In early June, astronomers published this photo in Nature,

6
00:00:18,040 --> 00:00:19,180
Outside the blue core,

7
00:00:19,360 --> 00:00:21,380
There\'s also this circle of orange light,

8
00:00:21,900 --> 00:00:23,740
This is a new drama-scale sweet donut,

9
00:00:24,380 --> 00:00:25,640
This is a portal.

10
00:00:26,280 --> 00:00:28,100
This is the generation ring of an alien civilization,

</step1_initial_translation>

<step2_reflection>
* **Line 1:** "Five-star system" is likely a mistranslation.  It probably refers to a five-member committee or group, not a star system. Clarify the context.
* **Line 2:** "Multiple universes" seems like an over-exaggeration.  Rephrase for clarity and accuracy.
* **Line 3:** "Weibo" should be explained as a Chinese social media platform.  "Filming missions" is unclear.  Does it mean "posting videos/images"?
* **Line 8:** "Drama-scale sweet donut" is a nonsensical literal translation. Figure out the intended meaning.
* **Line 9:** "Portal" seems out of context.  Verify the intended meaning.
* **Line 10:** "Generation ring" is likely a mistranslation.  Clarify the context.
* **Line 11:** "Organic polycyclic aromatic hydrocarbons" is overly technical for a general audience. Simplify if possible.
* **Line 12 and 14:** Use the correct formatting for the galaxy\'s name: SPT0418-47.
* **Line 15:** "It hasn\'t been shortened" is awkward. Remove or rephrase.
* **Line 28:**  The name of the organization and the resource should be translated more naturally and accurately.  Consider breaking this long line into two for better readability.
* **Line 29:**  "Cute plush dolls" may sound childish. Consider rephrasing as "animated characters" or similar.
* **Line 35:** "James Webb Space Telescope" should be used consistently throughout. Shortening to "Webb Telescope" after the first mention is acceptable.
* **Line 44:** "SPD048" is likely a typo.  It should be SPT0418-47 to be consistent.
* **Line 45-46:**  "Standard beautiful photo" is redundant.  Simplify to "beautiful photo".
* **Line 48:** "Grovitational Lenshin" is a typo. Correct to "Gravitational Lensing".
* **Line 50:** The sentence is incomplete. Finish the thought.

</step2_reflection>

<step3_refined_translation>
1
00:00:01,950 --> 00:00:04,430
Several molecules have been discovered in the five-member group\'s area of focus.

2
00:00:04,720 --> 00:00:06,780
We are still far from making contact with extraterrestrial life.

3
00:00:07,260 --> 00:00:09,880
The James Webb Space Telescope has been capturing images for a year now,

4
00:00:10,140 --> 00:00:12,920
and has recently transmitted many previously unseen photos.

5
00:00:13,440 --> 00:00:17,500
In early June, astronomers published this image in Nature.

6
00:00:18,040 --> 00:00:19,180
Outside the blue core,

7
00:00:19,360 --> 00:00:21,380
there\'s a ring of orange light.

8
00:00:21,900 --> 00:00:23,740
This is a large, ring-shaped structure.

9
00:00:24,380 --> 00:00:25,640
This is being investigated.

10
00:00:26,280 --> 00:00:28,100
This is thought to be a sign of an early galaxy.

</step3_refined_translation>

Extract the text within the <step3_refined_translation></step3_refined_translation> tags from the result, which is the translation result.

A Simple Package is Built, Interested Parties Can Download and Try it Locally

Download directly, unzip, and double-click app.exe to automatically open the above UI in your browser. Enter the key applied for in Gemini, fill in the proxy address, select the SRT subtitle file to be translated, select the target language to translate to, and try the result.

Q1: How does the reflection workflow differ from traditional machine translation?

A1: The reflection workflow introduces a self-assessment and optimization mechanism, simulating the thinking process of a human translator, and can produce more accurate and natural translation results.

Q2: How long does it take to use the reflection workflow?

A2: Although the reflection workflow requires multiple AI processing steps, it usually only takes 10-20 seconds longer than the traditional method. Considering the improvement in translation quality, this time investment is worthwhile.

Q3: Can the reflection workflow guarantee that the subtitle translation result is always a valid SRT?

A3: No, it may still have problems such as blank lines and inconsistencies in the number of subtitles compared to the original subtitles. For example, if the following subtitle only has 3-5 characters and is grammatically continuous with the previous sentence, the translation result is likely to be merged into one.

---

---

Added a function to the tool to support uploading video or audio files at the same time. With the help of Gemini, the audio and video are converted into subtitles, and the subtitles can be translated at the same time, and the translation results are returned.

Gemini's large model itself supports both text form and audio and video form, so a request can realize the transcription of audio and video into subtitles and translation.

For example, if an English-speaking video is sent to Gemini and specified to be translated into Chinese, the returned result is a Chinese subtitle.

1. Translate Subtitles Only

You can paste the SRT format subtitle content in the left text box, or directly click the "Upload SRT Subtitle" button to select a subtitle file from your local computer.

Then set the target language you want to translate to, and you can use the "Three-Step Reflection Translation Method" to command Gemini to perform the translation task. The returned result is output to the right text box. Click the "Download button" in the lower right corner to save it as an SRT file to your local machine.

2. Transcribe Audio and Video into Subtitles

Click the "Upload Audio and Video to Transcribe to Subtitles" button on the left, select any audio or video file to upload. After the upload is complete, submit it. After processing, Gemini will return the subtitle content recognized from the speech in the audio and video, and the effect is good.

If the target language is also specified, then after Gemini recognizes the speech, it will continue to translate the result into the language you specified before returning it. That is, it completes the two tasks of generating subtitles and translating subtitles at the same time.

Download Address:

https://github.com/jianchang512/ai2srt/releases/download/v0.2/windows-ai2srt-0.2.7z