* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* refactor to make StatesMetaManager threadsafe
* reduce
* comments
Drop duplicated indices from schema
https://docs.percona.com/percona-toolkit/pt-duplicate-key-checker.html
```
% pt-duplicate-key-checker --databases fresh
ALTER TABLE `fresh`.`events` DROP INDEX `ix_events_event_type_id`;
ALTER TABLE `fresh`.`states` DROP INDEX `ix_states_metadata_id`;
ALTER TABLE `fresh`.`statistics` DROP INDEX `ix_statistics_metadata_id`;
ALTER TABLE `fresh`.`statistics_short_term` DROP INDEX `ix_statistics_short_term_metadata_id`;
```
* Split context id migration into states and events tasks
Since events can finish much earlier than states we
would keep looking at the table because states as not
done. Make them seperate tasks
* add retry dec
* fix migration happening twice
* another case
* Deduplicate event_types in the events table
* Deduplicate event_types in the events table
* more fixes
* adjust
* adjust
* fix product
* fix tests
* adjust
* migrate
* migrate
* migrate
* more test fixes
* more test fixes
* fix
* migration test
* adjust
* speed up
* fix index
* fix more tests
* handle db failure
* preload
* tweak
* adjust
* fix stale docs strings, remove dead code
* refactor
* fix slow tests
* coverage
* self join to resolve query performance
* fix typo
* no need for quiet
* no need to drop index already dropped
* remove index that will never be used
* drop index sooner as we no longer use it
* Revert "remove index that will never be used"
This reverts commit 461aad2c52.
* typo
* Fix data migration never finishing when database has invalid datetimes
If there were impossible datetime values in the database (likely
from a manual sqlite to MySQL conversion) the conversion would
never complete
* Update homeassistant/components/recorder/migration.py
* Make sql subqueries threadsafe
fixes#89224
* fix join outside of lambda
* move statement generation into a seperate function to make it easier to test
* add cache key tests
* no need to mock hass
* Avoid asking recorder platforms for list_statistic_ids when already complete
If we already had all the data needed for list_statistic_ids, we would
still query recorder platforms and throw away the results
* Update homeassistant/components/recorder/statistics.py
Ensure new tables are created using InnoDB
InnoDB is the only supported engine to use with MariaDB
or MySQL as we currently have large keys in the states
table that will not work with MyIASM. Other storage
engines including Aria will likely work fine, but they
are not officially supported.
* Load pending state attributes and event data ids at startup
Since we queue all events to be processed after startup
we can have a thundering herd of queries to prime the
LRUs of event data and state attributes ids. Since we
know we are about to process a chunk of events we can
fetch all the ids in two queries
* lru
* fix hang
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* Fix recorder LRU being destroyed if event session is reopened
We would clear the LRU in _close_event_session but
it would never get replaced with an LRU again so
it would leak memory if the event session is reopened
* cleanup
* Mark PostgreSQL range select as fast
Currently we were using the slow range select workaround for
PostgreSQL that was original developed for MariaDB but
its actually slower on PostgreSQ
fixes#83253
* Mark PostgreSQL range select as fast
Currently we were using the slow range select workaround for
PostgreSQL that was original developed for MariaDB but
its actually slower on PostgreSQ
fixes#83253
* Fix statistics_at_time query not using index
fixes#82411
* fix refactoring error
* fix query so sqlalc does not get confused
* split it
* write as subquery
* reduce
* cleanup
* reduce
* Revert "reduce"
This reverts commit 43b4b55778.
* Remove default from created statistics schema
We were still inserting created times because even though
None was passed when creating the object explictly, the
default would still be used
* adjust column
* preserve original pre sql alc 2.0 behavior
* Adjust size of recorder LRU based on number of entities
If there are a large number of entities the cache would
get thrashed as there were more state attributes being
recorded than the size of the cache. This meant we had
to go back to the database to do lookups frequently when
an instance has more than 2048 entities that change
frequently
* add a test
* do not actually record 4096 states
* patch target
* patch target
