Update dependencies to allow for Windows ARM build#1018
Conversation
Bump versions for Qt (6.9.1), PyQt6 (6.9.1), cryptography (46.0.1), scipy (1.16.2), cffi (2.0.0), and add numpy (1.26.4) to pip_requirements_core. This ensures compatibility with recent upstream changes and improves security and stability.
Test Results2 404 tests 2 389 ✅ 23s ⏱️ Results for commit 854f128. ♻️ This comment has been updated with latest results. |
wawanbreton
left a comment
There was a problem hiding this comment.
Some minor remarks, however with this change we completely lose the optimized version of numpy on Windows. I see 2 options:
- Test if that actually makes a noticeable change when using Cura on Windows (e.g. performance while painting). Hopefully this is not really required.
- Add a windows sub-version for ARM that does not use the optimized numpy
Bumped Qt, PyQt6, and PyQt6-Qt6 versions from 6.9.1 to 6.10.0 in extra_dependencies and pip_requirements_core. Updated associated hashes for PyQt6 and PyQt6-Qt6. Also moved numpy hashes to the correct section for consistency.
|
The package for the numpy + mkl is only effective for Intel cpu on windows devices, we can make a comparison and if deemed to be not necessary it helps to clean things up. We can use this version as measure for the comparison against the baseline with that package. |
Update conandata.yml requirements to use pyarcus/5.12.0-alpha.0 and uvula/1.1.0 to match the new package releases.
| if _spec and _spec.submodule_search_locations: | ||
| _qt6_bin = os.path.join(list(_spec.submodule_search_locations)[0], "Qt6", "bin") | ||
| for _dll in ["Qt6Core.dll", "Qt6Network.dll", "Qt6DBus.dll", "Qt6Gui.dll", "Qt6Widgets.dll"]: | ||
| ctypes.windll.kernel32.LoadLibraryW(os.path.join(_qt6_bin, _dll)) |
There was a problem hiding this comment.
This piece of code feels extreeeeeemely hacky 😆 I know it is only in the package testing, not in production code, but isn't there a way to make it cleaner ? Or is it somehow a recommended necessary hack ?
Also the comment mentions Windows ARM, but the if will probably apply it also to Windows x86_64
| import sys | ||
|
|
||
| if sys.platform == "win32": | ||
| # On Windows ARM64, Python's import mechanism uses LoadLibraryExW with | ||
| # LOAD_LIBRARY_SEARCH_DEFAULT_DIRS which does NOT search PATH. | ||
| # Two components needed by PyQt6 are only accessible via PATH: | ||
| # 1. python3.dll - the stable ABI DLL (conan cpython loads python312.dll, not python3.dll) | ||
| # 2. Qt6 DLLs in PyQt6\Qt6\bin\ | ||
| # Pre-loading them via ctypes (which uses LoadLibraryW legacy path search) | ||
| # puts them in the process module table where LoadLibraryExW finds them as already-loaded. | ||
| import ctypes, os, importlib.util | ||
| ctypes.windll.kernel32.LoadLibraryW("python3.dll") | ||
| _spec = importlib.util.find_spec("PyQt6") | ||
| if _spec and _spec.submodule_search_locations: | ||
| _qt6_bin = os.path.join(list(_spec.submodule_search_locations)[0], "Qt6", "bin") | ||
| for _dll in ["Qt6Core.dll", "Qt6Network.dll", "Qt6DBus.dll", "Qt6Gui.dll", "Qt6Widgets.dll"]: | ||
| ctypes.windll.kernel32.LoadLibraryW(os.path.join(_qt6_bin, _dll)) |
There was a problem hiding this comment.
| import sys | |
| if sys.platform == "win32": | |
| # On Windows ARM64, Python's import mechanism uses LoadLibraryExW with | |
| # LOAD_LIBRARY_SEARCH_DEFAULT_DIRS which does NOT search PATH. | |
| # Two components needed by PyQt6 are only accessible via PATH: | |
| # 1. python3.dll - the stable ABI DLL (conan cpython loads python312.dll, not python3.dll) | |
| # 2. Qt6 DLLs in PyQt6\Qt6\bin\ | |
| # Pre-loading them via ctypes (which uses LoadLibraryW legacy path search) | |
| # puts them in the process module table where LoadLibraryExW finds them as already-loaded. | |
| import ctypes, os, importlib.util | |
| ctypes.windll.kernel32.LoadLibraryW("python3.dll") | |
| _spec = importlib.util.find_spec("PyQt6") | |
| if _spec and _spec.submodule_search_locations: | |
| _qt6_bin = os.path.join(list(_spec.submodule_search_locations)[0], "Qt6", "bin") | |
| for _dll in ["Qt6Core.dll", "Qt6Network.dll", "Qt6DBus.dll", "Qt6Gui.dll", "Qt6Widgets.dll"]: | |
| ctypes.windll.kernel32.LoadLibraryW(os.path.join(_qt6_bin, _dll)) | |
| import sys | |
| import platform | |
| if sys.platform == "win32" and platform.machine() == "ARM64": | |
| # On Windows ARM64, Python's import mechanism uses LoadLibraryExW with | |
| # LOAD_LIBRARY_SEARCH_DEFAULT_DIRS which does NOT search PATH. | |
| import os, importlib.util | |
| os.add_dll_directory(os.path.dirname(sys.executable)) # python3.dll (stable ABI) | |
| _spec = importlib.util.find_spec("PyQt6") | |
| if _spec and _spec.submodule_search_locations: | |
| _qt6_bin = os.path.join(list(_spec.submodule_search_locations)[0], "Qt6", "bin") | |
| if os.path.isdir(_qt6_bin): | |
| os.add_dll_directory(_qt6_bin) # all Qt6 DLLs |
@wawanbreton was mostly in: "make package, stop giving error mode" 🤣
Had the hack locally but didn't commit initially, don't remember why I included it at the end, BUT:
Can probably do something like above using os.add_dll_directory() to register directories directly with that search, so both python3.dll and Qt6 DLLs are found without ctypes tricks. At least as far as I could find (need to test 😝)
There was a problem hiding this comment.
That would already be a bit cleaner indeed, and also safer in the sense that you don't hard-code the DLLs names. But it does feel super weird that we have to do that at all...
There was a problem hiding this comment.
Probably should be done in the runner itself but I honestly didn't look into it. Also given then number of workarounds we have in place for the windows installer, at the time it didn't pop up as an unexpected requirement 🤷
https://github.com/Ultimaker/cura-workflows/blob/524f0058ab5cf4ddd40cbaa03376f70ab65c0a28/.github/workflows/cura-installer-windows.yml#L116-L150
twisted (dev dependency) imports TypeVar that is added in 4.4.0+
This pull request updates the version of the
cpythondependency in theconanfile.pyto ensure the project uses the latest stable release.Dependency updates:
cpythonrequirement from version3.12.2to3.12.7in therequirementsmethod ofconanfile.py.CURA-12814