Commit graph

137 commits

Author SHA1 Message Date
J. Nick Koston
6f66f37fc7
Shutdown database engine before waiting for executor shutdown (#117339)
* Close database connection before stopping the executor

related issue #117004

* Close database connection before waiting for database executor to finish

related issue #117004

* fix test

* fix

* drop unused arg

* Revert "drop unused arg"

This reverts commit 8a9fe6a24a.

* docstring

* comment
2024-08-22 16:47:22 +02:00
Erik Montnemery
5d3c57ecfe
Freeze integration setup timeout for recorder during non-live migration (#122431) 2024-07-22 18:48:55 -05:00
Erik Montnemery
3dc36cf068
Improve error handling when creating new SQLite database (#122406)
Co-authored-by: J. Nick Koston <nick@koston.org>
2024-07-22 14:16:11 -05:00
Erik Montnemery
20fc5233a1
Add recorder data migrator class to clean up states table (#122069) 2024-07-22 13:04:01 -05:00
Erik Montnemery
76cd53a864
Improve error handling when recorder schema migration fails (#122397) 2024-07-22 18:55:12 +02:00
Erik Montnemery
b14e8d1609
Remove SchemaValidationStatus.valid (#122394) 2024-07-22 17:33:13 +02:00
Erik Montnemery
e8b88557ee
Refactor recorder schema migration (#122372)
* Refactor recorder schema migration

* Simplify

* Remove unused imports

* Refactor _migrate_schema according to review comments

* Add comment
2024-07-22 16:53:54 +02:00
Erik Montnemery
cbe94c4706
Fix typo in recorder persistent notification (#122374) 2024-07-22 12:02:17 +02:00
Erik Montnemery
2f47312eeb
Fix recorder setup hanging if non live schema migration fails (#122242) 2024-07-20 13:10:23 +02:00
Erik Montnemery
436a38c1d2
Revert "Fix recorder setup hanging if non live schema migration fails" (#122232) 2024-07-20 12:29:08 +02:00
Erik Montnemery
153b69c971
Fix recorder setup hanging if non live schema migration fails (#122207) 2024-07-20 11:17:40 +02:00
Erik Montnemery
9970b7eece
Refactor recorder data migration (#121009)
* Refactor recorder data migration

* Fix stale docstrings

* Don't store a session object in BaseRunTimeMigration instances

* Simplify logic in EntityIDMigration.migration_done

* Fix tests
2024-07-16 21:50:19 +02:00
J. Nick Koston
73f6e3c07b
Narrow sqlite database corruption check to ensure disk image is malformed (#121947)
* Narrow sqlite database corruption check to ensure disk image is malformed

The database corruption check would also replace the database when it
locked externally instead of only when its malformed.

This was discovered in https://github.com/home-assistant/core/issues/121909#issuecomment-2227409124
when a user did a manual index creation while HA was online

* tweak

* tweak

* fix

* fix
2024-07-14 23:23:07 +02:00
J. Nick Koston
19d2d023ab
Ensure states table rebuild still happens if the event_id index was removed (#121938)
* Ensure states table rebuild still happens if the event_id index was removed

If ix_states_event_id was removed by the foreign key still
exists, the states table would not get rebuilt. This should
not happen under normal circumstances and seems to only be
possible if the index was removed manually or Home
Assistant was restarted forcefully in the middle
of a previous migration from years ago.

* cover

* fix tests

* mysql wont allow at that point but thats ok as long as its gone at the end
2024-07-14 23:20:52 +02:00
J. Nick Koston
43596f22a4
Fix recorder max backlog calculation being too small (#121739) 2024-07-11 09:09:35 +02:00
J. Nick Koston
c81d5a1ac2
Handle empty entity_id in the recorder filter (#121681)
fixes #111745
2024-07-10 09:53:38 -05:00
Erik Montnemery
6eeb70151d
Use BIGINT SQL type for ID columns (#121025) 2024-07-05 10:42:39 -05:00
Erik Montnemery
61f1c8d963
Fix leak of SQLAlchemy engine objects in recorder (#121085) 2024-07-03 12:17:52 -05:00
Erik Montnemery
e26b4554e6
Improve logic when retrying establishing database connection (#121047) 2024-07-03 16:22:21 +02:00
Marc Mueller
b798d76706
Update mypy-dev to 1.11.0a9 (#120289) 2024-06-24 09:57:15 +02:00
J. Nick Koston
9082dc2a79
Reduce recorder overhead when entity filter is empty (#119631) 2024-06-14 08:43:51 +02:00
J. Nick Koston
0888233f06
Make Recorder dialect_name a cached_property (#117922) 2024-05-29 09:23:40 +02:00
Sid
c2b3bf3fb9
Enable Ruff RET502 (#115139) 2024-05-21 22:19:33 +02:00
Marc Mueller
26a599ad11
Use PEP 695 for function annotations (1) (#117658) 2024-05-18 11:45:54 +02:00
Sid
2cc916db6d
Replace pylint broad-except with Ruff BLE001 (#116250) 2024-05-07 14:00:27 +02:00
J. Nick Koston
6339c63176
Improve recorder and worker thread matching in RecorderPool (#116886)
* Improve recorder and worker thread matching in RecorderPool

Previously we would look at the name of the threads. This
was a brittle if because other integrations may name their
thread Recorder or DbWorker. Instead we now use explict thread
ids which ensures there will never be a conflict

* fix

* fixes

* fixes
2024-05-05 16:25:10 -04:00
Marc Mueller
d4500cf945
Improve recorder event typing (#115253) 2024-04-08 14:56:18 -10:00
J. Nick Koston
ca5ed274cb
Deprecate calling async_listen and async_listen_once with run_immediately (#115169) 2024-04-08 10:07:54 -10:00
Marc Mueller
a0e6fd6ec5
Add improved typing for event fire and listen methods (#114906)
* Add EventType implementation

* Update integrations for EventType

* Change state_changed to EventType

* Fix tests

* Remove runtime impact

* Add tests

* Move to stub file

* Apply pre-commit to stub files

* Fix ruff PYI checks

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-04-07 13:28:24 -10:00
Sid
6587ee20db
Enable Ruff TRY300 (#114437)
* Enable Ruff TRY300

* Update validation.py

* Address review comments
2024-03-30 10:37:59 +01:00
Sid
a5b609f081
Enable ruff TRY401 (#114395)
* Enable ruff TRY401

* fix tests
2024-03-29 07:20:36 +01:00
Erik Montnemery
4d7a434254
Don't access States.last_reported_ts before it's added (#114333) 2024-03-28 00:44:01 +01:00
Erik Montnemery
a3059fe504
Record state.last_reported (#114201)
* Record state.last_reported

* Include last_reported in parts of the history API

* Use a bulk update

* fix refactoring error

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-03-27 12:48:06 +01:00
J. Nick Koston
d8acd90370
Run recorder shutdown tasks eagerly (#113018) 2024-03-26 22:25:45 -10:00
J. Nick Koston
06f356a038
Avoid expensive db queries at startup to check if data is migrated (#113773) 2024-03-19 15:27:40 -10:00
Sid
6ee273a548
Clean up unneeded ruff noqa directives (#113616) 2024-03-16 09:48:37 -10:00
J. Nick Koston
af6f2a516e
Guard against db corruption when renaming entities (#112718) 2024-03-08 11:34:07 -10:00
Marc Mueller
a033574ee2
Add empty line after module docstring [r] (#112703) 2024-03-08 15:05:07 +01:00
J. Nick Koston
f9dc92a9a0
Fix recorder ws_info blocking the event loop (#110657)
* Fix recorder ws_info blocking the event loop

Fixes
```
2024-02-15 06:37:55.423 WARNING (MainThread) [asyncio] Executing <Task pending name=websocket_api.async:ws_info coro=<_handle_async_response() running at /usr/src/homeassistant/homeassistant/components/websocket_api/decorators.py:26> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()] created at /usr/local/lib/python3.12/asyncio/tasks.py:712> cb=[set.remove()] created at /usr/src/homeassistant/homeassistant/core.py:653> took 0.332 seconds
```

* no instance did not actually work
2024-02-16 15:11:09 +01:00
Marc Mueller
438d3b01b9
Use builtin TimeoutError [o-s] (#109682) 2024-02-05 12:14:37 +01:00
J. Nick Koston
a22244707b
Create an issue when database backups fail because the system runs out of resources (#109020) 2024-01-30 10:23:58 -10:00
J. Nick Koston
d8c6534aff
Refactor recorder for common event case (#106753)
Almost 99% of items that are put into the recorder queue are
Events. Avoid wrapping them in tasks since we have to unwrap
them right away and its must faster to check for both RecorderTask
and Events since events are the common case.
2024-01-07 22:31:40 -05:00
J. Nick Koston
da1d5fc862
Increase max bind vars based on database version (#101464) 2023-10-06 07:59:38 +02:00
Erik Montnemery
0dc21504f5
Remove support for excluding attributes in recorder platforms (#100679) 2023-09-24 14:45:06 +02:00
J. Nick Koston
50382a609c
Create recorder futures with loop.create_future() (#100049) 2023-09-10 11:24:57 -05:00
Erik Montnemery
7168e71860
Log unexpected exceptions causing recorder shutdown (#99445) 2023-09-02 09:51:06 -05:00
Marc Mueller
a9ade1f84d
Use asyncio.timeout [core] (#98447) 2023-08-15 08:36:05 -05:00
J. Nick Koston
b2e708834f
Add slots to the StateMachine class (#95849) 2023-07-05 14:00:37 +02:00
J. Nick Koston
d9b43fc43f
Significantly speed up recorder event listener (#93532)
* Significantly speed up recorder event listener

This code is called every time an event happens since it
subscribes to all events. Its our most frequently called
listener out of the box.

It used to have a seperate filter function but it was
later combined after core had some previous refactoring.

It was never optimized after that happened.

This change reduces the run time by ~70%

* decruft
2023-05-25 21:44:12 -04:00
J. Nick Koston
f09abb0f2e
Fix race in tracking pending writes in recorder (#93414) 2023-05-23 21:47:31 +02:00