Detect lingering threads after tests (#37270)

* Detect lingering threads after tests

* Make sure cast is setup before checking state

* Make sure we ask executors of old hass to shutdown

We are not waiting here, just hoping for the best

* Make sure all instances of hass and executors is stopped.

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

* Also apply hass stopping to scripts

* Adjust to changes how we set up executor

* Add new CoreState.stopped

Co-authored-by: Paulus Schoutsen <balloob@gmail.com>

tests/scripts/test_check_config.py

@@ -1,6 +1,8 @@
 """Test check_config script."""
 import logging
 
+import pytest
+
 from homeassistant.config import YAML_CONFIG_FILE
 import homeassistant.scripts.check_config as check_config
 
@@ -23,6 +25,11 @@ BASE_CONFIG = (
 BAD_CORE_CONFIG = "homeassistant:\n  unit_system: bad\n\n\n"
 
 
+@pytest.fixture(autouse=True)
+async def apply_stop_hass(stop_hass):
+    """Make sure all hass are stopped."""
+
+
 def normalize_yaml_files(check_dict):
     """Remove configuration path from ['yaml_files']."""
     root = get_test_config_dir()