Initial orjson support take 3 (#73849)
* Initial orjson support take 2 Still need to work out problem building wheels -- Redux of #72754 / #32153 Now possible since the following is solved: ijl/orjson#220 (comment) This implements orjson where we use our default encoder. This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently. If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently). Areas where this makes a perceptible difference: - Anything that subscribes to entities (Initial subscribe_entities payload) - Initial download of registries on first connection / restore - History queries - Saving states to the database - Large logbook queries - Anything that subscribes to events (appdaemon) Cavets: orjson supports serializing dataclasses natively (and much faster) which eliminates the need to implement `as_dict` in many places when the data is already in a dataclass. This works well as long as all the data in the dataclass can also be serialized. I audited all places where we have an `as_dict` for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups). I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes If it turns out to be a problem we can disable this with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it will fallback to `as_dict` Its quite impressive for history queries <img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png"> * use for views as well * handle UnicodeEncodeError * tweak * DRY * DRY * not needed * fix tests * Update tests/components/http/test_view.py * Update tests/components/http/test_view.py * black * templates
This commit is contained in:
parent
9ac28d2076
commit
8b067e83f7
22 changed files with 149 additions and 80 deletions
|
@ -7,6 +7,8 @@ import json
|
|||
import logging
|
||||
from typing import Any
|
||||
|
||||
import orjson
|
||||
|
||||
from homeassistant.core import Event, State
|
||||
from homeassistant.exceptions import HomeAssistantError
|
||||
|
||||
|
@ -30,7 +32,7 @@ def load_json(filename: str, default: list | dict | None = None) -> list | dict:
|
|||
"""
|
||||
try:
|
||||
with open(filename, encoding="utf-8") as fdesc:
|
||||
return json.loads(fdesc.read()) # type: ignore[no-any-return]
|
||||
return orjson.loads(fdesc.read()) # type: ignore[no-any-return]
|
||||
except FileNotFoundError:
|
||||
# This is not a fatal error
|
||||
_LOGGER.debug("JSON file not found: %s", filename)
|
||||
|
@ -56,7 +58,10 @@ def save_json(
|
|||
Returns True on success.
|
||||
"""
|
||||
try:
|
||||
json_data = json.dumps(data, indent=4, cls=encoder)
|
||||
if encoder:
|
||||
json_data = json.dumps(data, indent=2, cls=encoder)
|
||||
else:
|
||||
json_data = orjson.dumps(data, option=orjson.OPT_INDENT_2).decode("utf-8")
|
||||
except TypeError as error:
|
||||
msg = f"Failed to serialize to JSON: {filename}. Bad data at {format_unserializable_data(find_paths_unserializable_data(data))}"
|
||||
_LOGGER.error(msg)
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue