LocalLLM Local Large Model Translation

If you have some hands-on skills, you can also deploy a large model locally and use it for translation. This section introduces the deployment and usage methods using Tongyi Qianwen as an example.

1. Download the exe and run it successfully

Open the website https://ollama.com/download

Click to download. After the download is complete, double-click to open the installation interface, and click Install to complete.

After completion, a black or blue window will automatically pop up, enter 3 words ollama run qwen and press Enter, it will automatically download the Tongyi Qianwen model

Wait for the model to finish downloading, no proxy is required, the speed is quite fast

After the model is automatically downloaded, it will run directly. When the progress reaches 100% and the "Success" character is displayed, it means that the model has been successfully run. At this point, the installation and deployment of the Tongyi Qianwen large model is completely completed, and you can use it happily. Isn't it super simple?

The default interface address is http://localhost:11434

If the window is closed, how do I open it again? It's also very simple, open the computer's start menu, find "Command Prompt" or "Windows PowerShell" (or directly press Win+q and enter cmd to search), click to open it, and enter ollama run qwen to complete.

2. Use it directly in the console command window

As shown in the figure, when this interface is displayed, you can actually directly enter text in the window to start using it.

3. Of course, this interface may not be very friendly, so let's get a friendly UI

Open the website https://chatboxai.app/zh and click Download

After downloading, double-click, wait for the interface window to open automatically

Click "Start Setting", in the pop-up floating layer, click the top model, select "Ollama" in the AI model provider, fill in the API domain name http://localhost:11434, select Qwen:latest in the model drop-down menu, and then save it.

The usage interface displayed after saving, use your imagination and use it freely.

4. Fill in the API into the video translation dubbing software

1. Open the menu--Settings--Local compatible OpenAI large model, add a model in the middle text box ,qwen, after adding it is as follows, and then select the model

2. Fill in http://localhost:11434 in the API URL, and fill in SK at will, such as 1234

3. Test whether it is successful, save it if it is successful, and use it

5. Call in the code

Ollama provides an api interface compatible with openai, you can directly use the openai library to call, you only need to change the model name to qwen

from openai import OpenAI

client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)

response = client.chat.completions.create(
  model="qwen",
  messages=[
    {"role": "system", "content": "You are a professional multilingual translation expert."},
    {"role": "user", "content": "Translate the content I sent you into English, only return the translation, do not answer questions, do not confirm, do not reply to this content, start translating from the next line\nThe weather is nice today!\nIt's very sunny, we have no class in the afternoon.\nThis is really cool"}
  ]
)
print(response.choices[0].message.content)

The effect is not bad

7. What other models can be used

In addition to Tongyi Qianwen, there are many other models that can be used, and the usage method is equally simple, just 3 words ollama run model name

Open this address https://ollama.com/library You can see all the model names, copy the name you want to use, and then execute ollama run model name.

Do you remember how to open the command window? Click the start menu, find Command Prompt or Windows PowerShell

For example, I want to install the openchat model

Open Command Prompt, enter ollama run openchat, press Enter and wait until it shows Success.

Precautions:

Most AI translation channels may limit the number of requests per minute. If an error message prompts that the request frequency is exceeded during use, you can set a number of seconds through the software Menu--Tools/Advanced Settings--Advanced Settings/set.ini--Pause after translation, such as 30, that is, wait 30 seconds after each translation before performing the next translation to avoid exceeding the limit frequency error.