gh-101758: Clean Up Uses of Import State

[ericsnowcurrently]

This change is almost entirely moving code around and hiding import state behind internal API. We introduce no changes to behavior, nor to non-internal API. (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.) The motivation is to simplify a number of upcoming changes.

Specific changes:

• move existing import-related code to import.c, wherever possible
• add internal API for interacting with import state (both global and per-interpreter)
• use only API outside of import.c (to limit churn there when changing the location, etc.)
• consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
• add macros for import state in import.c (to simplify changing the location)
• group code in import.c into sections
• remove _PyState_AddModule()¹

I avoided some related changes that would slightly change behavior, like the order of interpreter/runtime init. I may pursue those separately later.

• Issue: Isolate PyModuleDef to Each Interpreter for Extension/Builtin Modules #101758

Footnotes

1. I've also removed _PyState_AddModule() from the stable ABI manifest. It was part of "public" API since PEP 3121 was implemented 15 years ago (before a stable ABI existed) but was removed from limited API 6 years ago and then moved to internal API 4 years ago. Presumably it inadvertently slipped into the stable ABI manifest (with PEP 652) due to PC/python3dll.c. ↩

[ericsnowcurrently]

@brettcannon, this is what I was talking about. 😄

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO 3.x has failed when building commit b2fc549.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/567/builds/3626) and take a look at the build logs.
4. Check if the failure is related to this commit (b2fc549) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/567/builds/3626

Failed tests:

• test_asyncio

Failed subtests:

• test_subprocess_consistent_callbacks - test.test_asyncio.test_subprocess.SubprocessThreadedWatcherTests.test_subprocess_consistent_callbacks

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

415 tests OK.

10 slowest tests:

• test_tools: 3 min 16 sec
• test_gdb: 2 min 48 sec
• test_concurrent_futures: 2 min 31 sec
• test_multiprocessing_spawn: 1 min 38 sec
• test_multiprocessing_fork: 1 min 27 sec
• test_signal: 1 min 14 sec
• test_multiprocessing_forkserver: 1 min 13 sec
• test_asyncio: 1 min 4 sec
• test_logging: 44.6 sec
• test_math: 41.1 sec

1 test failed:
test_asyncio

18 tests skipped:
test_check_c_globals test_devpoll test_ioctl test_kqueue
test_launcher test_msilib test_nis test_peg_generator
test_perf_profiler test_startfile test_tix test_tkinter test_ttk
test_winconsoleio test_winreg test_winsound test_wmi
test_zipfile64

1 re-run test:
test_asyncio

Total duration: 5 min 7 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 771, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 763, in main
    self.assertEqual(events, [
AssertionError: Lists differ: [('pi[69 chars]), 'process_exited', 'pipe_connection_lost', '[17 chars]ost'] != [('pi[69 chars]), 'pipe_connection_lost', 'pipe_connection_lo[17 chars]ted']


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 771, in test_subprocess_consistent_callbacks
    self.loop.run_until_complete(main())
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.lto/build/Lib/test/test_asyncio/test_subprocess.py", line 763, in main
    self.assertEqual(events, [
AssertionError: Lists differ: ['process_exited', ('pipe_data_received', 1, b'stdout')] != [('pipe_data_received', 1, b'stdout'), ('p[95 chars]ted']

[ericsnowcurrently]

I'm fairly sure that buildbot failure is unrelated to this PR. If the buildbot fails the next commit then I'll look into this failure further.

[markshannon]

OOI, did you benchmark this? Startup times are sensitive to the changes in the import system.