* Add volume flow rate conversions
* Add missing translations
* Adjust liter unit and add gallons per minute
* Adjust to min instead of m for minutes
* Add matching class for number
* Add some tests for number and sensor platform
* Add deprecated constants
* Add explicit list of flow rate for check
This reverts commit 105171af31.
* Support calculating changes between consecutive sum statistics
* Add support for unit conversion when calculating change
* Don't include sum in WS response unless requested
* Improve tests
* Break out calculating change to its own function
* Improve test coverage
* Speed up logbook and history queries where ORM rows are not needed
This avoids having sqlalchemy wrap Result in ChunkedIteratorResult
which has additional overhead we do not need for these cases
* more places
* anything that uses _sorted_statistics_to_dict does not need orm rows either
* Reduce cache key size for queries that only need single columns
These queries only cared about a single row but would select
the whole set of columns from the orm object
* wrap it
* Break out statistics schema repairs into a repairs module
A future PR will add repairs for events, states, etc
* reorg
* reorg
* reorg
* reorg
* fixes
* fix patch targets
* name space rename
If a user manually migrated their database to MySQL or PostgresSQL
and incorrectly created the timestamp columns as float we would
fail to correct them to double because when we migrated to use
timestamps for the columns I missed that we needed to change the
columns and types for µs precision
- If the user had previously duplicated data we could end up
picking the next metadata_id and there could be stale rows
in the database that have that metadata_id. This can only happen
from bad manual migrations (which is what this is function
is validating in the first place). To solve this we now insert
data with a future date and look at the latest inserted row
instead of the first.
Example
```
['stored_statistics',
defaultdict(<class 'list'>,
{'recorder.db_test_schema': [{'end': 948589200.0,
'last_reset': None,
'max': None,
'mean': 2021.0,
'min': None,
'start': 948585600.0,
'state': None,
'sum': 394.5068},
{'end': 1601946000.000001,
'last_reset': 1601942400.000001,
'max': 1.000000000000001,
'mean': 1.000000000000001,
'min': 1.000000000000001,
'start': 1601942400.000001,
'state': 1.000000000000001,
'sum': 1.000000000000001}]})]
```
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* Avoid database executor job to fetch statistic metadata on cache hit
Since we will almost always have a cache hit fetching
statistic meta data we can avoid an executor job
* remove exception catch since the threading.excepthook will actually catch this in production
* fix a few missed ones
* threadsafe
* Update homeassistant/components/recorder/table_managers/statistics_meta.py
* coverage and optimistic caching
* Make sql subqueries threadsafe
fixes#89224
* fix join outside of lambda
* move statement generation into a seperate function to make it easier to test
* add cache key tests
* no need to mock hass
* Avoid asking recorder platforms for list_statistic_ids when already complete
If we already had all the data needed for list_statistic_ids, we would
still query recorder platforms and throw away the results
* Update homeassistant/components/recorder/statistics.py
* Load pending state attributes and event data ids at startup
Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries
* lru
* fix hang
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* cleanup
* Fix statistics_at_time query not using index
fixes#82411
* fix refactoring error
* fix query so sqlalc does not get confused
* split it
* write as subquery
* reduce
* cleanup
* reduce
* Revert "reduce"
This reverts commit 43b4b55778.
* Optimize fetching statistics
* speed up
* avoid double groupby
* avoid another loop
* tweak flow
* fixes
* tweak
* avoid a new dt object in the cache for week/month
* avoid a new dt object in the cache for week/month
