Commit graph

37 commits

Author SHA1 Message Date
J. Nick Koston
8b067e83f7
Initial orjson support take 3 ()
* Initial orjson support take 2

Still need to work out problem building wheels

--

Redux of  /  Now possible since the following is solved:
 (comment)

This implements orjson where we use our default encoder.  This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently.  If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).

Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)

Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups).  I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes

If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`

Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">

* use for views as well

* handle UnicodeEncodeError

* tweak

* DRY

* DRY

* not needed

* fix tests

* Update tests/components/http/test_view.py

* Update tests/components/http/test_view.py

* black

* templates
2022-06-22 21:59:51 +02:00
J. Nick Koston
c365454afb
Revert "Initial orjson support ()" ()
This was causing the wheels to fail to build. We need
to workout why when we don't have release pressure

This reverts commit d9d22a9556.
2022-05-31 13:51:55 -07:00
J. Nick Koston
d9d22a9556
Initial orjson support () 2022-05-31 12:18:11 -07:00
Franck Nijhof
8f4caf4141
Clean up accessing event helpers via hass () 2022-05-17 19:36:29 +02:00
J. Nick Koston
24d7a464e1
Refactor logbook to reduce overhead and complexity () 2022-05-08 21:47:26 -07:00
J. Nick Koston
b8442d9340
Add json decode caching to logbook () 2022-05-01 19:33:31 -05:00
J. Nick Koston
fe6a4bfb1d
Remove EVENT_TIME_CHANGED and EVENT_TIMER_OUT_OF_SYNC ()
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-09 09:05:54 -10:00
Marc Mueller
be7ef6115c
Make TypeVars private (1) () 2022-03-17 18:52:38 +01:00
Franck Nijhof
fa09cf663e
Update black to 22.1.0 () 2022-02-05 14:19:37 +01:00
Marc Mueller
77ee72cbb9
Import Callable from collections.abc (1) () 2021-09-29 16:32:11 +02:00
Marc Mueller
fabd73f08b
Update typing 03 () 2021-03-17 21:46:07 +01:00
J. Nick Koston
c9df42b69a
Add support for pre-filtering events to the event bus () 2021-02-14 09:42:55 -10:00
J. Nick Koston
6b7a7939d2
Include the first seen context data in the logbook api ()
* Include the context_entity_id in the logbook api

context_entity_id is the first entity seen during
a time period that includes the context

* update test

* more of them

* include friendly name

* pylint wants a ternary

* Refactor

* performance

* fix homekit context

* Fix self describing events

* Fix external_events
2020-08-24 12:44:40 -05:00
Paulus Schoutsen
f49ce5d1b4
Protect loop set default executor ()
Co-authored-by: J. Nick Koston <nick@koston.org>
2020-07-06 15:58:53 -07:00
J. Nick Koston
34ccb6588c
Cleanup async_track_state_change and augment docstring ()
* Cleanup async_track_state_change and augment docstrings.

Skip from_state and to_state matching in
async_track_state_change when they are None

Optimize the state change listener for the most
common use case: no to_state and from_state
matching.

* Update benchmark to be more realistic (previously we assumed only one entity was present in the whole instance)

* Add more tests to ensure behavior is preserved

* Ensure new behavior matches test

* remove MATCH_ALL from zone automation since its the default anyways

* Might as well use async_track_state_change_event instead since MATCH_ALL is removed
2020-07-05 17:31:33 -05:00
J. Nick Koston
4b2ebf5487
Ensure removed entities are not displayed in logbook () 2020-07-03 23:08:46 -07:00
mdegat01
6c7355785a
Add support for glob matching to entity filters ()
* Added GLOB capability to entityfilter and every place that uses it. All existing tests are passing

* added tests for components affected by glob change

* fixed flake8 error

* mocking the correct listener

* mocking correct bus method in azure test

* tests passing in 3.7 and 3.8

* fixed formatting issue from rebase/conflict

* Checking against glob patterns in more performant way

* perf improvments and reverted unnecessarily adjusted tests

* added new benchmark test around filters

* no longer using get with default in entityfilter

* changed filter name and removed logbook from filter benchmark

* simplified benchmark tests from feedback

* fixed apache tests and returned include exclude schemas to normal

* fixed azure event hub tests to properly go through component logic

* fixed azure test and clean up for other tests

* renaming test files to match standard

* merged mqtt statestream test changes with base

* removed dependency on recorder filter schema from history

* fixed recorder tests after merge and a bunch of lint errors
2020-06-23 20:02:29 -05:00
J. Nick Koston
59e43ab6e4
Prefilter more logbook events in sql ()
* Prefilter more logbook events in sql

Prefilter sensor events in _keep_event before humanify

Cache static attribute lookup

Reduces logbook execution time by ~35%

* fix mocking in benchmark

* Update tests for logbook users
2020-06-21 10:50:58 -07:00
Franck Nijhof
b9b1cee403
Enable pylint import-outside-toplevel () 2020-04-04 17:07:36 +02:00
Ville Skyttä
221d5205e4
Upgrade mypy to 0.770, tighten config a bit ()
* Upgrade mypy to 0.770, related cleanups

https://mypy-lang.blogspot.com/2020/03/mypy-0770-released.html

* Clean up config and make it a notch stricter, address findings
2020-03-12 11:52:20 +01:00
Paulus Schoutsen
d28d1ff657
Add JSON benchmark ()
* Add JSON benchmark

* Fix logbook benchmarks

* Move logbook import back
2020-03-11 11:50:16 -07:00
Paulus Schoutsen
8aea538662
Allow teaching logbook about events ()
* Allow teaching logbook about events

* Use async_add_executor_job

* Fix tests
2020-03-05 11:55:50 -08:00
Paulus Schoutsen
07fa844c43
Speed up validate_entity_id ()
* Speed up validate_entity_id

* Add some more invalid entity IDs

* Adjust regular expression

* Extend and sort test cases

* Update regular expression, more cases, faster

* Adjust tests, allow start with number, disallow double underscore

Co-authored-by: Franck Nijhof <frenck@frenck.nl>
2020-02-24 08:35:02 -08:00
Bas Nijholt
67c56c860d Sort imports according to PEP8 for 'homeassistant' folder ()
Components are already done
2019-12-09 16:42:10 +01:00
Franck Nijhof
2f0eb07624 Migrate legacy typehints in core to PEP-526 ()
* Migrate legacy typehints in core to PEP-526

* Fix one type
2019-09-03 20:36:04 -07:00
Franck Nijhof
decf13b948 Use literal string interpolation in core (f-strings) () 2019-08-23 09:53:33 -07:00
Paulus Schoutsen
4de97abc3a Black 2019-07-31 12:25:30 -07:00
Ville Skyttä
e8e84fb764 Type check homeassistant.scripts ()
* Run mypy on homeassistant.scripts, disabling bunch of checks for now

* Declare async_initialize in AuthProvider

* Add some type hints

* Remove unreachable code

* Help mypy out

* Script docstring fixes
2019-07-24 13:18:40 -07:00
Paulus Schoutsen
f995ab9d54
Don't pass in loop ()
* Don't pass in loop

* Revert some changes

* Lint + Axis revert

* reinstate loop

* Fix a test

* Set loop

* Update camera.py

* Lint
2019-05-22 21:09:59 -07:00
Anders Melchiorsen
0c8a31b8ec Memory optimization for logbook () 2019-03-01 23:23:45 -08:00
Paulus Schoutsen
fbc1c41673 Logbook context ()
* Convert logbook to use attr

* Add context to events

* Enhance logbook

* Lint

* Fix logbook entry

* Don't use intermediary classes for logbook entries
2018-10-01 16:12:25 +02:00
Ville Skyttä
3246b49a45 Upgrade pylint to 2.1.0 ()
* Upgrade pylint to 2.1.0

* Remove no longer needed pylint disables
2018-08-04 15:22:22 +02:00
Julius Mittenzwei
16cb7388ee Removing asyncio.coroutine syntax from HASS core ()
* changed asyncio.coroutine syntax to new async def/await

* removed py34 from tox environment

* reverted some changes within entity.py

* -

* reverted changes within bootstrap.py

* reverted changes within discovery.py

* switched decorators

* Reverted change within aiohttp_client.py

* reverted change within logging.py

* switched decorators

* Await lock properly

* removed asyncio.coroutine from test
2018-02-25 03:38:46 -08:00
Anders Melchiorsen
f9ee29a5cd Logbook speedup ()
* Optimize logbook filtering

* Avoid State construction during Events filtering

* Move tuple creation out of loop

* Add benchmark
2018-02-21 10:35:55 -08:00
Fabian Affolter
47e31dc9ee Fixes for PEP257 ()
* Fixes for PEP257

* More updates
2018-01-20 22:35:38 -08:00
Paulus Schoutsen
8f06b35dfc Optimize event matcher ()
* Optimize event matcher

* Tweak order of checks

* Add a benchmark for time_changed helper

* Add state change benchmark

* fix lint
2017-10-10 22:26:03 +02:00
Paulus Schoutsen
9f5bfe28d1 Add initial benchmark framework ()
* Add initial benchmark framework

* Use timer from timeit
2017-05-30 21:34:40 -07:00