8000 export: apply delta encoding for better compression rate · Issue #690 · cosmos/iavl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
export: apply delta encoding for better compression rate #690
Closed
@yihuang

Description

@yihuang

Snapshot is always created and processed in order, taking advantage of that, we can apply delta encodings to some fields:

  • version
    IAVL tree has the nice property that parent node's version is always bigger than or equal to children's version, and in snapshot, nodes are added in post-order, where parent node is put after children nodes, so we can only record the delta value, to be specific parent version - max(left version, right version), smaller value results in smaller encoded size with varint encoding.
  • key
    After the export: don't export branch node's key in snapshot #688 improvement, we only have leaf keys in snapshot, they are ordered, so delta encoding can also potentially have some good compression rate there.
  • height
    since IAVL tree has the invariant that parent height = max(left height, right height) + 1, I think we don't need to encode the height field at all in the snapshot.

The version and height compression rely on the fact that nodes are exported in post-order. the key compression only need the leaf nodes are exported in order.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0