pyVideoTrans

English

简体中文

English

简体中文

Appearance

PyInstaller Packaging Pitfalls: From Silent Crashes to a Clear Path ​

In software development, packaging a smoothly running Python project into a standalone executable (EXE) is a critical step for delivering it to users. PyInstaller is undoubtedly the king in this field. However, sometimes this king can bring us quite a few troubles.

I recently encountered a typical problem. My project had always been packaged smoothly with PyInstaller. But after upgrading the torch2.7 library, the packaging command pyinstaller sp.spec suddenly failed. It would run for a moment and then exit silently without any error messages.

This article documents how I found the root cause of this "silent crash" and ultimately resolved it. At the same time, I will take this opportunity to systematically share my experience with PyInstaller, especially on the Windows platform.

1. The Mysterious "Silent Crash" ​

When I ran the packaging command in the virtual environment, the log output stopped abruptly halfway:

... (previous logs omitted)
127592 INFO: Loading module hook 'hook-numba.py' ...
127608 INFO: Loading module hook 'hook-llvmlite.py' ...

(venv) F:\python\pyvideo>_

The cursor blinked quietly—no Error, no Traceback, nothing.

This situation is very tricky. An exit without an error usually indicates a problem at a lower level, likely a crash of the Python interpreter itself. Libraries like PyTorch, NumPy, and Numba contain a lot of C/C++ compiled code at their core. When PyInstaller analyzes these libraries, if there are version incompatibilities or conflicts, it can cause memory errors, leading the entire process to crash directly without generating a Python-level error report.

Since the problem occurred after upgrading torch, the root cause could almost certainly be pinned on version compatibility.

2. Finding Clues: Making the Error Visible ​

Faced with a "silent crash," our first priority is to make it "speak."

My first thought was: could it be that the PyInstaller version is too old and doesn't recognize the new version of torch? So I executed:

pip install --upgrade pyinstaller

After upgrading to the latest version, I ran the packaging command again. A miracle happened—the previous "silent crash" disappeared! But in its place, a new, clear error message appeared:

... (logs)
182529 INFO: Processing standard module hook 'hook-h5py.py' ...

=============================================================
A RecursionError (maximum recursion depth exceeded) occurred.
...

RecursionError! The problem finally became clear.

This error tells us that PyInstaller entered too deep into recursive calls while analyzing the project dependencies.

Imagine PyInstaller as a detective. To find all the files needed for the program to run, it starts from your main script sp.py, sees what it imports (e.g., torch), then checks what torch imports (e.g., scipy), and then what scipy imports, and so on, tracing layer by layer.

For massive libraries like torch, the internal module interdependencies form a huge, intricate web. After an upgrade, this web might have become even more complex, causing the detective to loop around too many times, eventually exceeding Python's default "recursion depth" safety limit, and the program simply gives up.

3. The One-Strike Solution: Increase the Recursion Limit ​

Fortunately, PyInstaller's error message helpfully provided the solution. We just need to relax this limit in the .spec file.

The .spec file is PyInstaller's "blueprint" for packaging, giving us fine-grained control over the packaging process.

I opened my sp.spec file and added two lines of code at the very beginning:

python

# Add these two lines to solve RecursionError
import sys
sys.setrecursionlimit(5000)

This line of code tells Python: "Increase the recursion depth limit from the default 1000 to 5000."

After saving, I ran pyinstaller sp.spec again. The packaging process passed the previously stuck point smoothly and finally generated the executable successfully. Problem solved.

This experience teaches us that when packaging projects containing large scientific computing libraries (like PyTorch, TensorFlow, SciPy, etc.), RecursionError is a common issue, and increasing the recursion limit is the most direct and effective solution.

4. Diving into the .spec File: The Art of Packaging ​

Since the .spec file was key to solving the problem, let's take a deeper look at it. While typing pyinstaller script.py directly in the command line is simple, for complex projects, carefully editing a .spec file is the more professional and reliable approach.

We can use the pyi-makespec script.py command to generate a basic .spec file and then modify it.

Below, let's explore the details using my sp.spec file as an example.

python
# sp.spec

import sys
sys.setrecursionlimit(5000)

import os, shutil
from PyInstaller.utils.hooks import collect_data_files

# --- 1. Define resources to include and hidden imports ---
hidden_imports = [
    'funasr', 'modelscope', 'transformers', # etc.
    'scipy.signal',
]
datas = []
datas += collect_data_files('funasr')
datas += collect_data_files('modelscope')

# --- 2. Analysis Phase ---
a = Analysis(
    ['sp.py'],  # Your main program entry
    pathex=[],
    binaries=[],
    datas=datas, # Include all non-code files
    hiddenimports=hidden_imports, # Tell PyInstaller about libraries it might miss
    excludes=[], # Optionally exclude certain libraries
    # ... other parameters
)

# --- 3. Package Python Modules (PYZ) ---
pyz = PYZ(a.pure, a.zipped_data)

# --- 4. Create Executable (EXE) ---
exe = EXE(
    pyz,
    a.scripts,
    name='sp',          # Define your program name here
    console=False,      # False means a GUI program without a console window
    icon='videotrans\\styles\\icon.ico', # Specify your icon here
    # ... other parameters
)

# --- 5. Collect All Files into the Final Directory (COLLECT) ---
coll = COLLECT(
    exe,
    a.binaries,
    a.datas,
    name='sp',
)

# --- 6. Custom Post-Build Operations ---
# This is a very powerful feature of the .spec file, allowing execution of arbitrary Python code after packaging
os.makedirs("./dist/sp/videotrans", exist_ok=True)
shutil.copytree("./videotrans/prompts", "./dist/sp/videotrans/prompts", dirs_exist_ok=True)
shutil.copy2("./voice_list.json", "./dist/sp/voice_list.json")
# ... more file copy operations

Key Concepts Explained:

  • hiddenimports: Some libraries use dynamic imports (e.g., importlib.import_module()), which PyInstaller's static analyzer might not "see." Adding these library names as strings to the hiddenimports list directly tells PyInstaller: "Don't forget these guys."
  • datas: Your program might need some non-.py files, such as .json configuration files, images, model files, etc. datas is used to package these data files.
    • collect_data_files('some_library') is a convenient helper function that automatically collects all data files accompanying a specific library.
    • You can also add them manually in the format [('source_file_path', 'relative_path_in_packaged_directory')]. For example, [('config.json', '.')] places config.json in the same directory as the executable.
  • EXE: This is key for customizing the executable.
    • name: Defines the name of sp.exe.
    • icon: Specifies an .ico file as the program's icon. This is a crucial step for making the program look more professional on Windows.
    • console: True creates a program with a black console window (suitable for command-line tools), False creates one without (suitable for GUI programs).
  • Post-Build Script: After COLLECT, you can write arbitrary Python code. In my example, I used shutil.copytree and shutil.copy2 to copy directories and files that don't need to be packed into the EXE but must be placed alongside it, such as configuration files, documentation, model weights, etc. This offers great flexibility.

5. Other Common Issues on the Windows Platform ​

Besides RecursionError, you might encounter other issues when using PyInstaller on Windows:

  1. Missing DLLs: Sometimes PyInstaller misses certain dynamic link libraries (.dll). The program crashes immediately upon running, prompting that a DLL is missing. The solution is to find that DLL file and manually add it in the binaries parameter of Analysis, in a format similar to datas.

  2. File Path Issues: After packaging, the program can't find resource files. This happens because the behavior of relative paths changes in packaged mode. The correct approach is to use the following code to get a reliable base path:

    python
    import sys
import os

if getattr(sys, 'frozen', False):
    # If in packaged state (.exe)
    base_path = os.path.dirname(sys.executable)
else:
    # If in normal running state (.py)
    base_path = os.path.dirname(os.path.abspath(__file__))

# Then use base_path to construct your resource paths
config_path = os.path.join(base_path, 'config.ini')

    Note: This code is very reliable for one-folder mode. For one-file mode, when the program runs, it extracts to a temporary directory, and sys._MEIPASS points to that temporary directory.

  3. False Positives by Antivirus Software: This is a persistent problem, especially when using the one-file (--onefile) mode. Because PyInstaller's bootloader needs to extract files into memory before execution, this behavior resembles certain malware.

    • Recommendation: Prefer the one-folder mode (the default in .spec). It's more stable, starts faster, and is less likely to trigger false positives. Then send the entire folder to the user.
    • If you must use one-file mode, try updating PyInstaller to the latest version or compiling the bootloader from source yourself, though this is more complex.

Conclusion ​

PyInstaller is a powerful tool, but it inevitably faces challenges when dealing with the increasingly complex Python ecosystem. This troubleshooting experience, from "silent crash" to RecursionError, reaffirms a few simple truths:

  1. Clear error messages are half the battle won. When encountering problems, first find a way to make them "speak." Updating related toolchains (like PyInstaller itself) can sometimes provide unexpected clues.
  2. The .spec file is important. Spend some time learning it, and you'll be able to handle various complex packaging needs with ease.
  3. Be cautious when upgrading large libraries. After upgrading core dependencies (like torch, tensorflow), packaging scripts often need corresponding adjustments.

References ​