Commit graph

81 commits

Author SHA1 Message Date
J. Nick Koston
db503c0daa
Always use UTC time for the MariaDB/MySQL session (#87583) 2023-02-09 11:39:45 -06:00
Erik Montnemery
94519de8dd
Upgrade SQLAlchemy to 2.0.2 (#86436)
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00
Marc Mueller
b478b4fa16
Add ParamSpec TypeAliases [mypy 1.0] (#87597) 2023-02-07 15:56:26 +01:00
J. Nick Koston
3aa744ed19
Fix indent on slow_range_in_select for MySQL/MariaDB (#87581)
We would calculate this value and throw it away
because it was only used on first_connection
2023-02-06 23:59:54 -05:00
J. Nick Koston
c11f3ffa17
Speed up purge time with newer MariaDB versions (#87409)
* Speed up purge time with newer MariaDB versions

* fix

* document

* document

* document

* rename

* self review

* Update homeassistant/components/recorder/util.py

* fixes
2023-02-05 17:58:34 -05:00
J. Nick Koston
5284837c8f
Add a repair issue when using MariaDB is affected by MDEV-25020 (#87040)
closes https://github.com/home-assistant/core/issues/83787
2023-01-31 20:42:07 +01:00
Franck Nijhof
d4955a3d87
Replace assert False by raising an error (#86686) 2023-01-26 16:51:43 +01:00
Marc Mueller
45b4b0e990
Import ParamSpec from typing [Py310] (#86413)
* Import ParamSpec from typing [Py310]

* Update additional imports
2023-01-23 07:28:43 +01:00
Franck Nijhof
79b52a2b41
Stricter pylint message control (#86154) 2023-01-20 13:47:55 +01:00
Franck Nijhof
c5dedb7a79
Code styling tweaks to the recorder integration (#86030) 2023-01-16 19:51:11 +01:00
Ville Skyttä
ae302bbec0
Make use of str.removeprefix and .removesuffix (#85584) 2023-01-13 12:19:38 +01:00
Franck Nijhof
fed8f905c8
Correct return type of two raise-only methods in recorder (#85048) 2023-01-03 12:15:45 +01:00
Franck Nijhof
94755a5773
String formatting and max line length - Part 4 (#84445)
Co-authored-by: jjlawren <jjlawren@users.noreply.github.com>
2022-12-23 13:27:27 +01:00
Erik Montnemery
2fe8e95309
Add helper to calculate statistic period start and end (#82493)
* Add helper to calculate statistic period start and end

* Don't parse values in resolve_period

* Add specific test for resolve_period

* Improve typing

* Move to recorder/util.py

* Extract period schema
2022-11-26 13:00:40 -05:00
Erik Montnemery
d1facab71c
Add guard in recorder retry function (#80585) 2022-10-19 10:05:55 +02:00
Marc Mueller
3a9ecab98a
Improve iterable typing (1) (#79295) 2022-10-01 00:12:39 +02:00
Joakim Sørensen
0caf998547
Bump awesomeversion from 22.6.0 to 22.8.0 (#77436) 2022-08-28 14:52:23 -04:00
Erik Montnemery
fd6ffef52f
Support non-live database migration (#72433)
* Support non-live database migration

* Tweak startup order, add test

* Address review comments

* Fix typo

* Clarify comment about promoting dependencies

* Tweak

* Fix merge mistake

* Fix some tests

* Fix additional test

* Fix additional test

* Adjust tests

* Improve test coverage
2022-07-22 15:11:34 +02:00
Erik Montnemery
606d544157
Use recorder get_instance function to improve typing (#75567) 2022-07-22 11:58:26 +02:00
J. Nick Koston
00a79635c1
Revert "Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191)" (#73917) 2022-06-23 13:59:55 -05:00
J. Nick Koston
7ae8bd5137
Remove sqlalchemy lambda_stmt usage from history, logbook, and statistics (#73191) 2022-06-07 16:15:50 -07:00
Erik Montnemery
5f2b4001f3
Separate recorder database schema from other classes (#72977)
* Separate recorder database schema from other classes

* fix logbook imports

* migrate new tests

* few more

* last one

* fix merge

Co-authored-by: J. Nick Koston <nick@koston.org>
2022-06-07 14:41:43 +02:00
J. Nick Koston
4dbc1ed7a7
Add dialect (database engine) and version to recorder system health data (#72339) 2022-05-22 21:29:30 -07:00
Erik Montnemery
edd7a3427c
Remove support for databases without ROW_NUMBER (#72092) 2022-05-18 21:52:38 -05:00
Erik Montnemery
037f6947d8
Fail recorder setup with unsupported dialect or version (#70888) 2022-05-18 16:52:46 +02:00
J. Nick Koston
f3c582815c
Convert statistics to use lambda_stmt (#71903)
* Convert stats to use lambda_stmt

- Since baked queries are now [deprecated in 1.4](https://docs.sqlalchemy.org/en/14/orm/extensions/baked.html#module-sqlalchemy.ext.baked) the
  next step is to convert these to `lambda_stmt`

https://docs.sqlalchemy.org/en/14/core/connections.html#quick-guidelines-for-lambdas

* Update homeassistant/components/recorder/statistics.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>
2022-05-18 16:22:21 +02:00
J. Nick Koston
a4c1bcefb9
Tune sqlite based on configured settings (#72016) 2022-05-17 18:12:15 -05:00
J. Nick Koston
98809675ff
Convert history queries to use lambda_stmt (#71870)
Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2022-05-15 10:47:29 -05:00
J. Nick Koston
a8f1dda004
Use ciso8601 for parsing MySQLdb datetimes (#71818)
* Use ciso8601 for parsing MySQLDB datetimes

The default parser is this:

5340191feb/MySQLdb/times.py (L66)

* tweak

* tweak

* add coverage for building the MySQLdb connect conv param
2022-05-13 19:26:09 -05:00
J. Nick Koston
6922209ddb
Ensure all mysql tables get optimized (#71538) 2022-05-08 12:15:06 -07:00
J. Nick Koston
a8aa0e1cca
Add Estimated Database Size to the recorder system health (#71463) 2022-05-07 21:02:54 -07:00
Marc Mueller
cf90e34776
Type recorder retry decorator (#70993) 2022-04-28 21:04:41 +02:00
J. Nick Koston
72fffde77a
Fix spelling of periodic in recorder (#69658)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-07 23:37:02 -10:00
J. Nick Koston
ec131d685e
Auto repack the database on the second sunday of the month (#69314) 2022-04-04 21:39:12 -10:00
Marc Mueller
53110f8cb7
Update pylint to 2.13.2 (#68704) 2022-03-27 16:08:24 +02:00
Marc Mueller
911b159281
Cleanup after pylint update (#68657) 2022-03-26 00:34:12 +01:00
J. Nick Koston
225f7a989b
Add strict typing for recorder util (#68681) 2022-03-25 15:03:46 -07:00
J. Nick Koston
c5c34bc0d7
Typing and code quality for recorder history (#68647) 2022-03-24 17:58:38 -07:00
J. Nick Koston
7deeb92485
Switch sqlalchemy execute to use .all() instead of list() on the iterator (#68540) 2022-03-22 19:24:21 -10:00
Stefan Agner
0a9927d18e
Avoid locking the database for non-SQLite backends (#63847)
* Avoid locking the database for non-SQLite backends

Currently we only have a lock implementation for SQLite. Just return
success for all other databases as they are not expected to store data
in the config directory and the caller can assume that a backup can
be safely taken.

This fixes `RuntimeError: generator didn't yield` errors when creating
a backup with the current Supervisor dev builds.
2022-01-11 16:17:56 +01:00
Erik Montnemery
bceeaec2f8
Remove duplicated statistics rows (#61146)
* Remove duplicated statistics

* Fix misleading docstring

* Pylint knows best

* Correct test

* Oops

* Prevent insertion of duplicated statistics

* Tweak

* pylint

* Add models_schema_23.py

* Tweak
2021-12-13 14:15:36 +01:00
Stefan Agner
f0006b92be
Allow to lock SQLite database during backup (#60874)
* Allow to set CONF_DB_URL

This is useful for test which need a custom DB path.

* Introduce write_lock_db helper to lock SQLite database

* Introduce Websocket API which allows to lock database during backup

* Fix isort

* Avoid mutable default arguments

* Address pylint issues

* Avoid holding executor thread

* Set unlock event in case timeout occures

This makes sure the database is left unlocked even in case of a race
condition.

* Add more unit tests

* Address new pylint errors

* Lower timeout to speedup tests

* Introduce queue overflow test

* Unlock database if necessary

This makes sure that the test runs through in case locking actually
succeeds (and the test fails).

* Make DB_LOCK_TIMEOUT a global

There is no good reason for this to be an argument. The recorder needs
to pick a sensible value.

* Add Websocket Timeout test

* Test lock_database() return

* Update homeassistant/components/recorder/__init__.py

Co-authored-by: Erik Montnemery <erik@montnemery.com>

* Fix format

Co-authored-by: J. Nick Koston <nick@koston.org>
Co-authored-by: Erik Montnemery <erik@montnemery.com>
2021-12-07 13:16:24 +01:00
Erik Montnemery
470b01e4ce
Move async_migration_in_progress (#59087) 2021-11-05 10:40:56 +01:00
Erik Montnemery
be4e9f91b6
Change minimum supported SQLite version to 3.31.0 (#59073) 2021-11-04 16:34:35 +01:00
Erik Montnemery
dfa50a842a
Simplify recorder PgSQL version checks (#58533) 2021-10-27 16:05:40 +02:00
Erik Montnemery
e9ba5f3b4b
Warn when recorder connects to an unsupported database (#58161) 2021-10-26 13:41:59 +02:00
Erik Montnemery
8ef8838801
Correct detection of row_number support for MariaDB (#57663) 2021-10-14 11:19:39 -07:00
Erik Montnemery
0139bfa749
Detect if mysql and sqlite support row_number (#57475) 2021-10-11 21:17:18 -07:00
Marc Mueller
565a9fea6b
Import Callable from collections.abc (2) (#56776) 2021-09-29 14:06:51 +02:00
Erik Montnemery
5976f898da
Add WS API for removing statistics for a list of statistic_ids (#55078)
* Add WS API for removing statistics for a list of statistic_ids

* Refactor according to code review, enable foreign keys support for sqlite

* Adjust tests

* Move clear_statistics WS API to recorder

* Adjust tests after rebase

* Update docstring

* Update homeassistant/components/recorder/websocket_api.py

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>

* Adjust tests after rebase

Co-authored-by: Paulus Schoutsen <paulus@home-assistant.io>
2021-09-27 23:30:13 +02:00