hass-core/homeassistant/util
J. Nick Koston 03eea7bd3f
Avoid subprocess memory copy when c library supports posix_spawn (#87958)
* use posix spawn on alpine

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy all the memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
1b736838e6/Lib/subprocess.py (L745)

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* Avoid subprocess memory copy when c library supports posix_spawn

By default python 3.10 will use the fork() which has to
copy memory of the parent process (in our case
this can be huge since Home Assistant core can use
hundreds of megabytes of RAM). By using posix_spawn
this is avoided and subprocess creation does not
get discernibly slow the larger the Home Assistant
python process grows.

In python 3.11 vfork will also be available
https://github.com/python/cpython/issues/80004#issuecomment-1093810689
https://github.com/python/cpython/pull/11671 but we won't
always be able to use it and posix_spawn is considered safer
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14

The subprocess library doesn't know about musl though
even though it supports posix_spawn https://git.musl-libc.org/cgit/musl/log/src/process/posix_spawn.c
so we have to teach it since it only has checks for glibc
1b736838e6/Lib/subprocess.py (L745)

The constant is documented as being able to be flipped here:
https://docs.python.org/3/library/subprocess.html#disabling-use-of-vfork-or-posix-spawn

* missed some

* adjust more tests

* coverage
2023-02-13 09:02:51 -05:00
..
yaml Update Union typing (4) [Py310] (#86427) 2023-01-23 09:04:40 +01:00
__init__.py Stricter pylint message control (#86154) 2023-01-20 13:47:55 +01:00
aiohttp.py Enable Ruff TRY004 (#86811) 2023-01-30 14:06:52 +01:00
async_.py Import ParamSpec from typing [Py310] (#86413) 2023-01-23 07:28:43 +01:00
color.py Code styling tweaks to core utils & YAML loader (#85433) 2023-01-08 22:01:55 -08:00
decorator.py Improve typing [util.decorator] (#67087) 2022-02-23 20:58:42 +01:00
distance.py String formatting and max line length - Part 7 (#84532) 2022-12-27 11:18:56 +01:00
dt.py Optimize fetching weekly/monthly/yearly statistics (#87747) 2023-02-09 21:22:32 -05:00
enum.py Cache try_parse_enum (#87911) 2023-02-12 03:57:36 -06:00
executor.py Remove ThreadPoolExecutor shutdown backport (#66735) 2022-02-17 10:39:33 -06:00
file.py Use os.path.dirname() for getting the directory name of a path (#81504) 2022-11-14 15:21:35 +01:00
json.py Code styling tweaks to core utils & YAML loader (#85433) 2023-01-08 22:01:55 -08:00
location.py Enable Ruff D212 (#87347) 2023-02-03 23:08:48 +01:00
logging.py Enable Ruff D212 (#87347) 2023-02-03 23:08:48 +01:00
network.py Add is_host_valid util (#76589) 2022-09-11 11:12:04 -05:00
package.py Avoid subprocess memory copy when c library supports posix_spawn (#87958) 2023-02-13 09:02:51 -05:00
percentage.py Fix an extra quote typo in the percentage util (#87161) 2023-02-02 19:59:08 -06:00
pil.py Code styling tweaks to core utils & YAML loader (#85433) 2023-01-08 22:01:55 -08:00
pressure.py Add unitless unit converter (#85694) 2023-01-12 09:20:00 +01:00
process.py Move disallow-any-generics to mypy.ini (#63917) 2022-01-11 13:33:25 -08:00
read_only_dict.py Make TypeVars private (1) (#68205) 2022-03-17 18:52:38 +01:00
speed.py Add unitless unit converter (#85694) 2023-01-12 09:20:00 +01:00
ssl.py Code styling tweaks to core utils & YAML loader (#85433) 2023-01-08 22:01:55 -08:00
temperature.py String formatting and max line length - Part 7 (#84532) 2022-12-27 11:18:56 +01:00
thread.py Ensure shutdown does not deadlock (#49282) 2021-04-18 08:55:51 -10:00
timeout.py Add Self typing (4) [mypy 1.0] (#87601) 2023-02-06 22:30:22 -06:00
ulid.py Sync event timed_fired and the context ulid time (#71854) 2022-05-14 12:12:08 -07:00
unit_conversion.py Add Mega Joule as valid unit of energy (#86055) 2023-01-25 14:13:42 +01:00
unit_system.py Update unit system unit conversions (#87632) 2023-02-08 10:52:54 +01:00
uuid.py Improve performance of generating non-cryptographically secure uuids (#41314) 2020-10-07 16:37:01 +02:00
variance.py Import ParamSpec from typing [Py310] (#86413) 2023-01-23 07:28:43 +01:00
volume.py String formatting and max line length - Part 7 (#84532) 2022-12-27 11:18:56 +01:00