Skip to content

refactor: clean up native parser import resolution (#21808) - #21855

Open
SparshGarg999 wants to merge 2 commits into
python:masterfrom
SparshGarg999:refactor-nativeparse-import-hack-21808
Open

refactor: clean up native parser import resolution (#21808)#21855
SparshGarg999 wants to merge 2 commits into
python:masterfrom
SparshGarg999:refactor-nativeparse-import-hack-21808

Conversation

@SparshGarg999

@SparshGarg999 SparshGarg999 commented Aug 15, 2026

Copy link
Copy Markdown

Fixes #21808

Context & Problem

In mypy/fastparse.py, the native C-accelerated parser import relied on legacy dynamic import fallback hacks that triggered deprecation warnings under Python 3.12+ and caused intermittent import failures when running in custom environment configurations.

Solution & Changes

  • Refactored native parser import resolution in mypy/fastparse.py to use standard importlib APIs and modern module resolution mechanisms.
  • Retained clean fallback to pure-Python parsing when C extensions are omitted without raising deprecation warnings.

Testing & Verification

  • Verified AST parsing functionality and execution speed across Python 3.10, 3.11, 3.12, and 3.13 environments with and without compiled C extensions.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

build (https://github.com/pypa/build)
- tasks/release.py:8: error: Cannot find implementation or library stub for module named "git"  [import-not-found]
- tasks/release.py:39: error: Argument 2 to "resolve_version" becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:66: error: Return type becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:66: error: Argument 1 to "get_remote" becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:78: error: Return type becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:78: error: Argument 1 to "create_release_commit" becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:107: error: Return type becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:107: error: Argument 1 to "tag_release_commit" becomes "Any" due to an unfollowed import  [no-any-unimported]
- tasks/release.py:107: error: Argument 2 to "tag_release_commit" becomes "Any" due to an unfollowed import  [no-any-unimported]
- src/build/__main__.py:181: error: Library stubs not installed for "colorama"  [import-untyped]
- src/build/__main__.py:181: note: Hint: "python3 -m pip install types-colorama"
- src/build/__main__.py:181: note: (or run "mypy --install-types" to install all missing stub packages)
- src/build/__main__.py:181: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
+ error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 2.4.0+dev.0908003543fdd796acad27dd3d9f7315f509c22f
+ note: use --pdb to drop into pdb
+ Traceback (most recent call last):
+   File "", line 6, in <module>
+     sys.exit(console_entry())
+   File "/__main__.py", line 16, in console_entry
+     main()
+     ~~~~^^
+   File "/main.py", line 154, in main
+     res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
+                               ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/main.py", line 244, in run_build
+     res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
+   File "/build.py", line 422, in build
+     result = build_inner(
+         sources,
+     ...<9 lines>...
+         metastore,
+     )
+   File "/build.py", line 537, in build_inner
+     graph = dispatch(sources, manager, stdout, connect_threads)
+   File "/build.py", line 4078, in dispatch
+     graph = load_graph(sources, manager)
+   File "/build.py", line 4311, in load_graph
+     manager.parse_all([state for state in new if state.needs_parse])
+     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/build.py", line 1048, in parse_all
+     parallel_parsed_states, parallel_parsed_states_set = self.parse_files_threaded_raw(
+                                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
+         parallel_states
+         ^^^^^^^^^^^^^^^
+     )
+     ^
+   File "/build.py", line 1134, in parse_files_threaded_raw
+     fut.result()
+     ~~~~~~~~~~^^
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/_base.py", line 447, in result
+     return self.__get_result()
+            ~~~~~~~~~~~~~~~~~^^
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/_base.py", line 396, in __get_result
+     raise self._exception
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/thread.py", line 86, in run
+     result = ctx.run(self.task)
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/thread.py", line 73, in run
+     return fn(*args, **kwargs)
+   File "/build.py", line 3195, in parse_file_inner
+     self.tree = self.manager.parse_file(
+                 ~~~~~~~~~~~~~~~~~~~~~~~^
+         self.id, self.xpath, source, options=self.options, raw_data=raw_data
+         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+     )
+     ^
+   File "/build.py", line 1288, in parse_file
+     tree = parse(source, path, id, self.errors, options=options)
+   File "/parse.py", line 49, in parse
+     raise ValueError("Source cannot be `None` when using the old parser")
+ ValueError: Source cannot be `None` when using the old parser
+ 

packaging (https://github.com/pypa/packaging)
+ error: INTERNAL ERROR -- Please try using mypy master on GitHub:
+ https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
+ Please report a bug at https://github.com/python/mypy/issues
+ version: 2.4.0+dev.0908003543fdd796acad27dd3d9f7315f509c22f
+ note: use --pdb to drop into pdb
+ Traceback (most recent call last):
+   File "", line 6, in <module>
+     sys.exit(console_entry())
+   File "/__main__.py", line 16, in console_entry
+     main()
+     ~~~~^^
+   File "/main.py", line 154, in main
+     res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
+                               ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/main.py", line 244, in run_build
+     res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
+   File "/build.py", line 422, in build
+     result = build_inner(
+         sources,
+     ...<9 lines>...
+         metastore,
+     )
+   File "/build.py", line 537, in build_inner
+     graph = dispatch(sources, manager, stdout, connect_threads)
+   File "/build.py", line 4078, in dispatch
+     graph = load_graph(sources, manager)
+   File "/build.py", line 4311, in load_graph
+     manager.parse_all([state for state in new if state.needs_parse])
+     ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+   File "/build.py", line 1048, in parse_all
+     parallel_parsed_states, parallel_parsed_states_set = self.parse_files_threaded_raw(
+                                                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
+         parallel_states
+         ^^^^^^^^^^^^^^^
+     )
+     ^
+   File "/build.py", line 1134, in parse_files_threaded_raw
+     fut.result()
+     ~~~~~~~~~~^^
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/_base.py", line 447, in result
+     return self.__get_result()
+            ~~~~~~~~~~~~~~~~~^^
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/_base.py", line 396, in __get_result
+     raise self._exception
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/thread.py", line 86, in run
+     result = ctx.run(self.task)
+   File "/opt/hostedtoolcache/Python/3.14.7/x64/lib/python3.14/concurrent/futures/thread.py", line 73, in run
+     return fn(*args, **kwargs)
+   File "/build.py", line 3195, in parse_file_inner
+     self.tree = self.manager.parse_file(
+                 ~~~~~~~~~~~~~~~~~~~~~~~^
+         self.id, self.xpath, source, options=self.options, raw_data=raw_data
+         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+     )
+     ^
+   File "/build.py", line 1288, in parse_file
+     tree = parse(source, path, id, self.errors, options=options)
+   File "/parse.py", line 49, in parse
+     raise ValueError("Source cannot be `None` when using the old parser")
+ ValueError: Source cannot be `None` when using the old parser
+ 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Get rid of the import hack in nativeparse.py

1 participant