Commit graph

27 commits

Author SHA1 Message Date
Marc Mueller
2decf6c023
Use dict instead of MutableMapping [recorder] (#115318) 2024-04-09 15:08:31 -10:00
Sid
0d66d298ec
Enable Ruff RET504 (#114528)
* Enable Ruff RET504

* fix test

* Use noqa instead of cast

* fix sonos RET504

---------

Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2024-04-06 11:07:37 +02:00
Erik Montnemery
4d7a434254
Don't access States.last_reported_ts before it's added (#114333) 2024-03-28 00:44:01 +01:00
Erik Montnemery
a3059fe504
Record state.last_reported (#114201)
* Record state.last_reported

* Include last_reported in parts of the history API

* Use a bulk update

* fix refactoring error

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2024-03-27 12:48:06 +01:00
Sid
6ee273a548
Clean up unneeded ruff noqa directives (#113616) 2024-03-16 09:48:37 -10:00
Erik Montnemery
397bde4652
Add missing continue in legacy recorder history function (#113535)
Co-authored-by: Jan Bouwhuis <jbouwh@users.noreply.github.com>
2024-03-15 21:20:39 +01:00
Marc Mueller
aa16a9d707
Add empty line after module docstring (3) (#112750) 2024-03-08 20:38:34 +01:00
Marc Mueller
a033574ee2
Add empty line after module docstring [r] (#112703) 2024-03-08 15:05:07 +01:00
Alex Hermann
bdef0ba6e5
Significantly improve performance for some cases of the history start time state query (#99450)
* recorder: Apply filter in the outer query too

Function _get_start_time_state_for_entities_stmt() produced a query which
is dead-slow in my installation. On analysis, the outer query produced
millions of rows which had to be joined to the subquery. The subquery has
a filter which would eliminate almost all of the outer rows.

To speed up the query, apply the same filter to the outer query, so way
less rows have to be joined.

This reduced the query time on my system from more than half an hour to
mere milliseconds.

* lint

* merge filter

---------

Co-authored-by: J. Nick Koston <nick@koston.org>
2023-11-28 16:23:49 -06:00
J. Nick Koston
6f8734167f
Bump SQLAlchemy to 2.0.21 (#99745) 2023-09-20 16:19:53 +02:00
Ville Skyttä
34b47a2597
Remove unnnecessary pylint configs from components [m-r]* (#98924) 2023-08-24 00:56:50 +02:00
Franck Nijhof
3e85a29b86
Move overlapping pylint rules to ruff, disable mypy overlap (#94359) 2023-06-27 17:42:46 +02:00
J. Nick Koston
f251c464e2
Fix history start time state returning None with some postgresql versions (#93555)
* Add additional coverage to history websocket api

related issue #93258

* Add additional coverage to history websocket api

related issue #93258

* Fix results when union query ends up at the end instead of front

* Apply suggestions from code review

* resort

* zero instead

* fix exception

* fix tests
2023-05-25 22:18:19 -04:00
J. Nick Koston
34b824a27b
Reduce number of columns when selecting attributes for history (#91717) 2023-04-22 14:21:08 +02:00
J. Nick Koston
95fcdc5684
Speed up sql queries where ORM rows are not needed (#91839)
* Speed up logbook and history queries where ORM rows are not needed

This avoids having sqlalchemy wrap Result in ChunkedIteratorResult
which has additional overhead we do not need for these cases

* more places

* anything that uses _sorted_statistics_to_dict does not need orm rows either
2023-04-21 23:28:07 -04:00
J. Nick Koston
4e6937d20f
Avoid multiple round trips to the database for history API calls (#91193)
* delete more code

* tweak

* tweak

* wrappers

* restore lost performance

* restore lost performance

* restore lost performance

* compact

* reduce

* fix refactor

* DRY

* tweak

* delete the start time state injector

* move away the legacy code

* tweak

* adjust

* adjust

* tweak

* ignore impossible

* fix a bug where the first start was changed to the start time when there was no previous history recorded before

* avoid the empty scan most cases

* postgresql

* fixes

* workaround for mariadb < 10.4

* remove unused

* remove unused

* adjust

* bail early

* tweak

* tweak

* fix more tests

* fix recorderrun being init in the future in the test

* run history tests on schema 30 as well

* Revert "run history tests on schema 30 as well"

This reverts commit d798b100ac.

* reduce

* cleanup

* tweak

* reduce

* prune

* adjust

* adjust

* adjust

* reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code

* Revert "reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code"

This reverts commit bf974e103e.

* fix test

* Revert "Revert "reverse later is faster because the index is in forward order and the data size we are reversing is much smaller even if we are in python code""

This reverts commit 119354499e.

* more coverage

* adjust

* fix for table order

* impossible for it to be missing

* remove some more legacy from the all states
2023-04-11 22:38:23 -04:00
J. Nick Koston
d0d4ab6056
Require a list of entity ids when fetching history (#90992) 2023-04-08 16:14:44 -10:00
J. Nick Koston
d21433b6af
Ensure filters are generated inside the lambda locks (#90418) 2023-03-28 20:50:10 +02:00
J. Nick Koston
4c45c3c63b
Add a faster query for get_last_state_changes when the number of states is 1 (#90211)
* Add a faster query for get_last_state_changes when the number of states is 1

related issue #90113

* Add a faster query for get_last_state_changes when the number of states is 1

related issue #90113

* coverage

* Apply suggestions from code review
2023-03-24 09:39:55 -04:00
J. Nick Koston
96225bb287
Rename recorder run_history to table_managers.recorder_runs_manager (#90070) 2023-03-21 18:38:33 -10:00
J. Nick Koston
affb48d271
Avoid joining states_meta for statistics queries (#89941) 2023-03-19 16:44:35 -10:00
J. Nick Koston
7f3e4cb3af
Guard against selecting all invalid entity_ids in history (#89929)
If all the entity_ids that were provided do not exist we would
end up passing an empty list of ids to the SQL query which
would do an unbounded select
2023-03-19 22:03:12 -04:00
J. Nick Koston
ed27dae173
Small cleanups to recorder history (#89774)
* Small cleanups to recorder history

* Small cleanups to recorder history

* fixes

* flake8 cannot figure it out
2023-03-15 23:44:33 -04:00
J. Nick Koston
a244749712
Make StatesMetaManager thread-safe when an entity_id is fully deleted from the database and than re-added (#89732)
* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* refactor to make StatesMetaManager threadsafe

* reduce

* comments
2023-03-15 08:54:02 -04:00
J. Nick Koston
85ca94e9d4
Mark database sessions that do not write data as read_only (#89600)
* Mark sessions that do not write data as read_only

* Mark sessions that do not write data as read_only
2023-03-12 21:33:28 -04:00
J. Nick Koston
c41f91be89
Deduplicate entity_id in the states table (#89557) 2023-03-12 10:01:58 -10:00
J. Nick Koston
50c31a5355
Move legacy database queries and models to prepare for schema v38 (#89532) 2023-03-11 11:26:30 -10:00