If a user manually migrated their database to MySQL or PostgresSQL
and incorrectly created the timestamp columns as float we would
fail to correct them to double because when we migrated to use
timestamps for the columns I missed that we needed to change the
columns and types for µs precision
- If the user had previously duplicated data we could end up
picking the next metadata_id and there could be stale rows
in the database that have that metadata_id. This can only happen
from bad manual migrations (which is what this is function
is validating in the first place). To solve this we now insert
data with a future date and look at the latest inserted row
instead of the first.
Example
```
['stored_statistics',
defaultdict(<class 'list'>,
{'recorder.db_test_schema': [{'end': 948589200.0,
'last_reset': None,
'max': None,
'mean': 2021.0,
'min': None,
'start': 948585600.0,
'state': None,
'sum': 394.5068},
{'end': 1601946000.000001,
'last_reset': 1601942400.000001,
'max': 1.000000000000001,
'mean': 1.000000000000001,
'min': 1.000000000000001,
'start': 1601942400.000001,
'state': 1.000000000000001,
'sum': 1.000000000000001}]})]
```
Update the calander event trigger logic to have more exhaustive coverage. The
trigger will now use a timespan to create an explicit window for considering
upcoming events. The start/end of the time span is now more explicit, rather
than getting it from the alarm time.
The trigger is now broken into composable pieces:
- A timespan object for more explicitly managing the time window
- A function to get events during a time span
- A function to process upcoming events and determine the trigger times
The existing listener is now just responsible for scheduling alarms and glue.
This fixes bug with DST handling where the conversion back and forth between
UTC and timezone ends up dropping events during the jump forward. In practice,
an event was returned from the scanning, but it was never fired by the trigger
because (1) it was filtered out of the interval and (2) the event list was
previously cleared every iteration so it would get dropped.
Future improvements can bake more invariant checking into this structure.
* Fix cpu thrashing during purge after all legacy events were removed
We now remove the the index of of event ids on the states table when its
all NULLs to save space. The purge path needs to avoid checking for legacy
rows to purge if the index has been removed since it will result in a full
table scan each purge cycle that will always find no legacy rows to purge
* one more place
* drop the key constraint as well
* fixes
* more sqlite
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* remove exception catch since the threading.excepthook will actually catch this in production
* fix a few missed ones
* threadsafe
* Update homeassistant/components/recorder/table_managers/statistics_meta.py
* coverage and optimistic caching
* Add local calendar diagnostics platform
* Use redaction from ical
* Update diagnostics for new ical version
* Apply suggestions from code review
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
* Use snapshot tests for local calendar diagnostics
* Setup diagnostics directly in tests rather than via dependencies
---------
Co-authored-by: epenet <6771947+epenet@users.noreply.github.com>
* Initial commit
* Add websocket test tool
* Small tweak
* Tiny cleanup
* Make pipeline work with frontend branch
* Add some more info to start event
* Fixes
* First voice assistant tests
* Remove run_task
* Clean up for PR
* Add config_flow.py
* Remove CLI tool
* Simplify by removing stt/tts for now
* Clean up and fix tests
* More clean up and API changes
* Add quality_scale
* Remove data from run-finish
* Use StrEnum backport
---------
Co-authored-by: Paulus Schoutsen <balloob@gmail.com>
* Add filters to climate and light service descriptions
* Allow specifying enums directly
* Update service descriptions
* Adjust test
* Cache entity features
* Lint
* Improve error handling, add list of known base components
* Don't allow specifying an entity feature as int
