pyVideoTrans

English

简体中文

English

简体中文

Appearance

A PyInstaller Packaging Story: From Silent Crashes to a Breakthrough ​

In software development, packaging a smoothly running Python project into a standalone executable (EXE) is a crucial step for user delivery. PyInstaller is undoubtedly the king in this domain. But sometimes, even this king can cause us a lot of trouble.

I recently encountered a classic problem. My project had always been packaged successfully with PyInstaller. That is, until I upgraded the torch2.7 library. Suddenly, the packaging command pyinstaller sp.spec failed. It would run for a moment and then exit silently, leaving no error messages behind.

This article documents how I navigated this "silent crash" dilemma to find the root cause and ultimately solve it. I'll also take this opportunity to systematically discuss my experiences with PyInstaller, especially the quirks on the Windows platform.

I. The Mysterious "Silent Crash" ​

When I ran the packaging command in my virtual environment, the log output would stop halfway and vanish:

... (previous logs omitted)
127592 INFO: Loading module hook 'hook-numba.py' ...
127608 INFO: Loading module hook 'hook-llvmlite.py' ...

(venv) F:\python\pyvideo>_

The cursor just blinked quietly. No Error, no Traceback, nothing.

This situation is extremely tricky. An exit without an error usually means the problem lies at a much lower level, likely a crash within the Python interpreter itself. Libraries like PyTorch, NumPy, and Numba contain a large amount of C/C++ compiled code. If PyInstaller encounters version incompatibilities or conflicts while analyzing these libraries, it can trigger a memory error, causing the entire process to crash instantly, without enough time to generate a Python-level error report.

Since the problem appeared right after upgrading torch, the root cause was almost certainly related to version compatibility.

II. Finding Clues and Making the Error Surface ​

Faced with this "silent lamb," our first priority is to make it "speak up."

My first thought was, maybe my PyInstaller version is too old and doesn't recognize the new version of torch? So I executed:

pip install --upgrade pyinstaller

After upgrading to the latest version, I ran the packaging command again. A miracle happened! The previous "silent crash" was gone. But in its place was a new, explicit error message:

... (logs)
182529 INFO: Processing standard module hook 'hook-h5py.py' ...

=============================================================
A RecursionError (maximum recursion depth exceeded) occurred.
...

RecursionError! The problem was finally clear.

This error tells us that PyInstaller got stuck in excessively deep recursive calls while analyzing the project's dependencies.

Imagine PyInstaller as a detective. To find all the files your program needs to run, it starts with your main script, sp.py, sees what it imports (like torch), then checks what torch imports (like scipy), then what scipy imports... and so on, tracing the chain layer by layer.

Giant libraries like torch have a massive, intricate web of internal module references. After the upgrade, this web likely became even more complex, causing the detective to run in too many circles, eventually exceeding the "recursion depth" safety limit set by Python, forcing the program to give up.

III. The Direct Fix: Increasing the Recursion Limit ​

Fortunately, PyInstaller's error message kindly provided the solution. We just need to relax this limit in the .spec file.

The .spec file is PyInstaller's "design blueprint," giving us fine-grained control over the packaging process.

I opened my sp.spec file and added two lines at the very top:

python

# Add these two lines to solve the RecursionError
import sys
sys.setrecursionlimit(5000)

This code tells Python: "Please increase the recursion depth limit from the default of 1000 to 5000."

After saving, I ran pyinstaller sp.spec again. The packaging process smoothly passed the point where it had been stuck and successfully generated the executable file. Problem solved.

This experience teaches us that when packaging projects that include large scientific computing libraries (like PyTorch, TensorFlow, SciPy, etc.), RecursionError is a common issue, and increasing the recursion limit is the most direct and effective solution.

IV. A Deep Dive into the .spec File: The Art of Packaging ​

Since the .spec file was key to solving the problem, let's take a closer look at it. While simply typing pyinstaller script.py in the command line is easy, for complex projects, carefully editing a .spec file is the more professional and reliable approach.

We can generate a basic .spec file with the pyi-makespec script.py command and then modify it.

Below, let's examine my sp.spec file to see the ins and outs.

python
# sp.spec

import sys
sys.setrecursionlimit(5000)

import os, shutil
from PyInstaller.utils.hooks import collect_data_files

# --- 1. Define resources and hidden imports ---
hidden_imports = [
    'funasr', 'modelscope', 'transformers', # etc.
    'scipy.signal',
]
datas = []
datas += collect_data_files('funasr')
datas += collect_data_files('modelscope')

# --- 2. Analysis Phase ---
a = Analysis(
    ['sp.py'],  # Your main script entry point
    pathex=[],
    binaries=[],
    datas=datas, # Include all non-code files
    hiddenimports=hidden_imports, # Tell PyInstaller about libraries it might miss
    excludes=[], # You can explicitly exclude certain libraries if needed
    # ... other parameters
)

# --- 3. Package Python Modules (PYZ) ---
pyz = PYZ(a.pure, a.zipped_data)

# --- 4. Create the Executable (EXE) ---
exe = EXE(
    pyz,
    a.scripts,
    name='sp',          # Define your program name here
    console=False,      # False for a GUI application without a console window
    icon='videotrans\\styles\\icon.ico', # Specify your icon here
    # ... other parameters
)

# --- 5. Collect All Files into the Final Directory (COLLECT) ---
coll = COLLECT(
    exe,
    a.binaries,
    a.datas,
    name='sp',
)

# --- 6. Custom Post-Build Actions ---
# This is a very powerful feature of .spec files, allowing you to run arbitrary Python code after packaging
os.makedirs("./dist/sp/videotrans", exist_ok=True)
shutil.copytree("./videotrans/prompts", "./dist/sp/videotrans/prompts", dirs_exist_ok=True)
shutil.copy2("./voice_list.json", "./dist/sp/voice_list.json")
# ... more file copying operations

Core Concepts Explained:

  • hiddenimports: Some libraries use dynamic imports (e.g., importlib.import_module()), which PyInstaller's static analyzer might not "see." Adding the names of these libraries as strings to the hiddenimports list is like telling PyInstaller directly: "Don't forget these guys."
  • datas: Your program might need non-.py files, such as .json configuration files, images, model files, etc. datas is used to package these data files.
    • collect_data_files('some_library') is a convenient helper function that automatically collects all data files bundled with a specific library.
    • You can also add them manually in the format [('source_file_path', 'relative_path_in_dist_folder')]. For example, [('config.json', '.')] would place config.json in the same directory as the executable.
  • EXE: This is where you customize the executable.
    • name: Defines the name of sp.exe.
    • icon: Specifies an .ico file to use as the program's icon. This is a key step to make your program look professional on Windows.
    • console: True creates a program with a black console window (suitable for command-line tools), while False creates one without (suitable for GUI applications).
  • Post-build script: After the COLLECT step, you can write any Python code. In my example, I use shutil.copytree and shutil.copy2 to copy directories and files that shouldn't be bundled inside the EXE but need to be placed alongside it, such as configuration files, documentation, or model weights. This provides tremendous flexibility.

V. Other Common Issues on the Windows Platform ​

Besides RecursionError, you might encounter other problems when using PyInstaller on Windows:

  1. Missing DLLs: Sometimes PyInstaller might miss certain dynamic-link libraries (.dll). The program crashes on launch, reporting a missing DLL. The solution is to find that DLL file and manually add it to the binaries parameter in the Analysis section, using a format similar to datas.

  2. File Path Issues: After packaging, the program can't find its resource files. This is because the behavior of relative paths changes in packaged mode. The correct approach is to use the following code to get a reliable base path:

    python
    import sys
import os

if getattr(sys, 'frozen', False):
    # If running in a bundled app (.exe)
    base_path = os.path.dirname(sys.executable)
else:
    # If running in a normal environment (.py)
    base_path = os.path.dirname(os.path.abspath(__file__))

# Then use base_path to construct your resource paths
config_path = os.path.join(base_path, 'config.ini')

    Note: This code is very reliable for one-folder mode. For one-file mode, the program unpacks to a temporary directory at runtime, and sys._MEIPASS will point to that temporary directory.

  3. False Positives from Antivirus Software: This is a notorious, long-standing issue, especially when using one-file (--onefile) mode. This is because PyInstaller's bootloader needs to unpack files into memory before execution, a behavior similar to some malware.

    • Recommendation: Prioritize using one-folder mode (the default for .spec files). It's more stable, starts faster, and is less likely to trigger false positives. You can then distribute the entire folder to users.
    • If you must use one-file mode, try updating PyInstaller to the latest version or compiling the bootloader from source yourself, though this is more complex.

Conclusion ​

PyInstaller is a powerful tool, but it inevitably faces challenges when dealing with the increasingly complex Python ecosystem. This journey from a "silent crash" to a RecursionError reaffirmed a few simple truths:

  1. A clear error message is half the battle won. When you encounter a problem, first find a way to make it "speak up." Updating the relevant toolchain (like PyInstaller itself) can sometimes provide unexpected clues.
  2. The .spec file is crucial. Spend some time learning it, and you'll be able to handle all sorts of complex packaging requirements with confidence.
  3. Be cautious when upgrading large libraries. After upgrading core dependencies (like torch or tensorflow), your packaging script will likely need adjustments as well.

References ​