Commit graph

419 commits

Author SHA1 Message Date
J. Nick Koston
c35661947a
Optimize fetching statistics by avoiding timestamp conversions (#87845)
* Optimize fetching statistics by avoiding timestamp conversions

* remove accidential unrelated change

* fix test

* recreate so we handle timezone changes
2023-02-10 11:26:46 -05:00
J. Nick Koston
cbaf4764e7
Ensure MySQL tests cleanup connections and raise an exception if they do not (#87767)
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
Co-authored-by: Erik <erik@montnemery.com>
2023-02-09 16:27:53 -06:00
J. Nick Koston
509de02044
Ensure recorder still shuts down if the final commit fails (#87799) 2023-02-09 15:12:40 -06:00
J. Nick Koston
abf0c87e40
Migrate statistics to use timestamp columns (#87321) 2023-02-09 12:24:19 -06:00
Erik Montnemery
2fae82a9ef
Really ensure recorder test fixture is setup before hass fixture (#87796) 2023-02-09 11:43:14 -06:00
J. Nick Koston
db503c0daa
Always use UTC time for the MariaDB/MySQL session (#87583) 2023-02-09 11:39:45 -06:00
J. Nick Koston
c05a7b29e6
Terminate stale MySQL connections at the end of test runs (#87794) 2023-02-09 11:13:13 -06:00
J. Nick Koston
03b47f1931
Disable test_entity_ids on MariaDB (#87766)
Dropping the database after this test will fail on
MySQL and hang forever because it causes an InnoDB deadlock

```
| 2042 | root   | localhost:52698 | NULL               | Query   |   41 | Waiting for table metadata lock | DROP DATABASE `homeassistant-test` |    0.000 |
```
2023-02-09 15:30:35 +01:00
J. Nick Koston
f4a81a8b5b
Disable test_migration_in_progress on MySQL (#87765)
This test was recently enabled on MySQL in
https://github.com/home-assistant/core/pull/87753

Since it the migration is still in progress in the background
when the test ends, it causes deadlock with InnoDB when its dropped
out from under it
2023-02-09 15:27:27 +01:00
J. Nick Koston
483b0cd017
Skip db locking test with postgresql/MySQL (#87760)
* Skip db locking test with postgresql/MySQL

https://github.com/home-assistant/core/pull/87756#issuecomment-1423828389

* switch to the guards instead
2023-02-09 13:58:32 +01:00
J. Nick Koston
77cdde94f3
Turn back off test_shutdown_before_startup_finishes test on MySQL (#87761) 2023-02-09 03:12:10 -06:00
Erik Montnemery
58618b2978
Improve recorder tests to use MariaDB / PostgreSQL (#87756) 2023-02-09 02:18:20 -06:00
Erik Montnemery
b0f5bdd504
Improve migration tests to use MariaDB / PostgreSQL database (#87753) 2023-02-09 08:42:32 +01:00
epenet
ba85fdcd61
Add type hints to integration tests (q-s) (#87706) 2023-02-08 08:12:54 -08:00
Erik Montnemery
94519de8dd
Upgrade SQLAlchemy to 2.0.2 (#86436)
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-08 08:17:32 -06:00
Erik Montnemery
720f51657d
Add CI job which runs recorder tests on PostgreSQL (#80614)
Co-authored-by: Franck Nijhof <git@frenck.dev>
Co-authored-by: J. Nick Koston <nick@koston.org>
2023-02-07 09:46:14 -06:00
epenet
59ca7780fa
Add typing to tests with single hass argument (#87631) 2023-02-07 15:01:16 +01:00
J. Nick Koston
0a1d5c85ad
Optimize history.get_last_state_changes query (#87554)
fixes undefined
2023-02-07 07:50:44 -06:00
epenet
4142f0d15d
Add return type to tests without arguments (#87613)
* Add return type to tests without arguments

* Black

* Cancel fixture amends
2023-02-07 14:20:06 +01:00
epenet
aa00114c2f
Add return type to async tests without arguments (#87612) 2023-02-07 10:26:56 +01:00
Marc Mueller
342b406dc0
Add Self typing (1) [mypy 1.0] (#87598) 2023-02-06 22:29:47 -06:00
J. Nick Koston
c7980bda82
Fix recorder run history during schema migration and startup (#87492)
Fix recorder run history during schema migration

RunHistory.get and RunHistory.current can be called before
RunHistory.start. We need to return a RecorderRuns object
with the recording_start time that will be used when start
it called to ensure history queries still work as expected.

fixes #87112
2023-02-05 22:12:41 -05:00
J. Nick Koston
c11f3ffa17
Speed up purge time with newer MariaDB versions (#87409)
* Speed up purge time with newer MariaDB versions

* fix

* document

* document

* document

* rename

* self review

* Update homeassistant/components/recorder/util.py

* fixes
2023-02-05 17:58:34 -05:00
epenet
19e27f4f52
Ensure recorder api aligns with sensor converters (#87257) 2023-02-04 14:28:47 -06:00
J. Nick Koston
389fc515a1
Prevent overly large attributes from being stored in the database (#87105) 2023-02-02 11:49:33 -06:00
Franck Nijhof
5e81d28116
Update black to 23.1.0 (#87188) 2023-02-02 18:35:24 +01:00
epenet
58596d205c
Ensure sensor and recorder converters align (#87138) 2023-02-02 10:36:03 +01:00
J. Nick Koston
1fd58b6cb7
Fix a few cartesian products in recorder tests (#87106)
Fixes
```
recorder/test_init.py:251: SAWarning: SELECT statement has a cartesian product between FROM element(s) "states" and FROM element "state_attributes".  Apply join condition(s) between each element to resolve.
```
2023-02-02 07:47:09 +01:00
J. Nick Koston
fb55933b0a
Reduce chance of queue overflow during schema migration (#87090)
Co-authored-by: Franck Nijhof <git@frenck.dev>
2023-02-01 18:34:40 +01:00
J. Nick Koston
5284837c8f
Add a repair issue when using MariaDB is affected by MDEV-25020 (#87040)
closes https://github.com/home-assistant/core/issues/83787
2023-01-31 20:42:07 +01:00
Erik Montnemery
53c5f02ca2
Remove some dead code from recorder (#86697) 2023-01-30 18:37:48 +01:00
J. Nick Koston
c612a92cfb
Use python defaults for comparing State, LazyState, and Event objects (#86856)
* Speed up comparing State and Event objects

Use default python implementation for State and Event __hash__ and __eq__

The default implementation compared based on the id() of the object
which is effectively what we want here anyways. These overrides are
left over from the days when these used to be attrs objects

By avoiding implementing these ourselves all of the equality checks
can happen in native code

* tweak

* adjust tests

* write out some more

* fix test to not compare objects

* more test fixes

* more test fixes

* correct stats tests

* fix more tests

* fix more tests

* update sensor recorder tests
2023-01-29 13:31:43 -05:00
Franck Nijhof
a79885ceaf
Enable Ruff SIM117 (#86783) 2023-01-27 11:52:49 +01:00
Franck Nijhof
e4a78420b8
Enable Ruff PT013 (#86757)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2023-01-27 09:09:46 +01:00
Erik Montnemery
fea30c1ce9
Terminate strings at NUL when recording states and events (#86687) 2023-01-26 11:11:03 +01:00
J. Nick Koston
52ea64d1d0
Fix repr for States and Events without a timestamp (#86391) 2023-01-22 08:11:42 -10:00
epenet
30bf0634fe
Add per-file-ignore to pylint (#86289) 2023-01-22 17:26:24 +01:00
J. Nick Koston
233332c3a0
Fix fetching history include_start_time_state when timezone is not UTC (#85983) 2023-01-16 09:01:32 +01:00
Michael
72d3fa6d89
Replace the usage of unit constants by enumerations in Tests [o-r] (#85936) 2023-01-15 14:50:24 +01:00
Erik Montnemery
b0d4b73874
Add unitless unit converter (#85694)
* Add unitless unit converter

* Adjust type hints

* Adjust tests

* Rename to UnitlessRatioConverter
2023-01-12 09:20:00 +01:00
J. Nick Koston
d81febd3f4
Small speed up to frequently called datetime functions (#85399) 2023-01-08 09:42:29 -10:00
J. Nick Koston
b8a1537b58
Improve performance of fetching and storing history and events with the database (#84870) 2023-01-02 13:26:08 -10:00
Erik Montnemery
587381440f
Fix repairing datetime precision for PostgreSQL (#83351) 2022-12-05 22:33:06 -05:00
Joakim Plate
c576a68d33
Upgrade pytest-aiohttp (#82475)
* Upgrade pytest-aiohttp

* Make sure executors, tasks and timers are closed

Some test will trigger warnings on garbage collect, these warnings
spills over into next test.

Some test trigger tasks that raise errors on shutdown, these spill
over into next test.

This is to mimic older pytest-aiohttp and it's behaviour on test
cleanup.

Discussions on similar changes for pytest-aiohttp are here:
https://github.com/pytest-dev/pytest-asyncio/pull/309

* Replace loop with event_loop

* Make sure time is frozen for tests

* Make sure the ConditionType is not async

  /home-assistant/homeassistant/helpers/template.py:2082: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
    def wrapper(*args, **kwargs):
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

* Increase litejet press tests with a factor 10

The times are simulated anyway, and we can't stop the normal
event from occuring.

* Use async handlers for aiohttp

tests/components/motioneye/test_camera.py::test_get_still_image_from_camera
tests/components/motioneye/test_camera.py::test_get_still_image_from_camera
tests/components/motioneye/test_camera.py::test_get_stream_from_camera
tests/components/motioneye/test_camera.py::test_get_stream_from_camera
tests/components/motioneye/test_camera.py::test_camera_option_stream_url_template
tests/components/motioneye/test_camera.py::test_camera_option_stream_url_template
  /Users/joakim/src/hass/home-assistant/venv/lib/python3.9/site-packages/aiohttp/web_urldispatcher.py:189: DeprecationWarning: Bare functions are deprecated, use async ones
    warnings.warn(

* Switch to freezegun in modbus tests

The tests allowed clock to tick in between steps

* Make sure skybell object are fully mocked

Old tests would trigger attempts to post to could services:

```
DEBUG:aioskybell:HTTP post https://cloud.myskybell.com/api/v3/login/ Request with headers: {'content-type': 'application/json', 'accept': '*/*', 'x-skybell-app-id': 'd2b542c7-a7e4-4e1e-b77d-2b76911c7c46', 'x-skybell-client-id': '1f36a3c0-6dee-4997-a6db-4e1c67338e57'}
```

* Fix sorting that broke after rebase
2022-11-29 22:36:36 +01:00
Erik Montnemery
f869ce9d06
Validate common statistics DB schema errors on start (#79707)
* Validate common statistics db schema errors on start

* Fix test

* Add tests

* Adjust tests

* Disable statistics schema validation in tests

* Update after rebase
2022-11-29 10:16:08 +01:00
Erik Montnemery
a038314d8b
Add display unit to WS recorder/get_statistics_metadata (#82870) 2022-11-28 13:46:57 -05:00
Erik Montnemery
2fe8e95309
Add helper to calculate statistic period start and end (#82493)
* Add helper to calculate statistic period start and end

* Don't parse values in resolve_period

* Add specific test for resolve_period

* Improve typing

* Move to recorder/util.py

* Extract period schema
2022-11-26 13:00:40 -05:00
Erik Montnemery
b94e1e9ef8
Fire events when long term statistics is updated (#82492)
* Fire events when long term statistics is updated

* Allow the new events to be subscribed to by anyone

* Address review comments

* Finish renaming events

* Finish renaming events

* Fix do_adhoc_statistics

* Adjust tests

* Adjust tests
2022-11-24 22:01:36 +01:00
Erik Montnemery
607a0e7697
Reduce size of get_statistics_during_period WS API response (#82131) 2022-11-16 17:36:30 +01:00
Erik Montnemery
9b8f94363c
Fix statistic_during_period for data with holes (#81847) 2022-11-16 12:46:29 +01:00