feature: ADR 101 pull companion API #1292
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* proto: Add VersionService Signed-off-by: Thane Thomson <connect@thanethomson.com> * proto: make proto-gen Signed-off-by: Thane Thomson <connect@thanethomson.com> * config: Add gRPC server section Signed-off-by: Thane Thomson <connect@thanethomson.com> * rpc/grpc: Add server with support for version service Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Add gRPC server Signed-off-by: Thane Thomson <connect@thanethomson.com> * rpc/grpc: Add client with support for version service Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Add gRPC server test of version service Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entries Signed-off-by: Thane Thomson <connect@thanethomson.com> * rpc: Fix test broken by introduction of gRPC service Some of the RPC tests run Comet nodes in goroutines, causing port conflicts. This randomizes the port assigned to the gRPC server, reducing the likelihood of port conflicts. Signed-off-by: Thane Thomson <connect@thanethomson.com> * rpc/grpc: Update log message format to conform to RPC log style Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* initial logic for the gRPC block service (#1094) * initial logic for gRPC block client and test (#1094) * block response and request (#1094) * add configuration for block service (#1094) * use pointer for request parameter (#1094) * change block service GetBlock response (#1094) * return block information (#1094) * convert core type to proto type (#1094) * hookup server (#1094) * hooking directly to the BlockStore instead of Environment (#1094) * changing client return type, use core type (#1094) * convert from proto to core type (#1094) * implemented proper logic to test block service (#1094) * return the latest height if height is 0 (#1094) * adding proper grpc error handling and return (#1094) * rename rpc and message in proto to match ADR-101 spec (#1094) * renaming service and client methods structs to match ADR (#1094) * additional error handling (#1094) * rename get block request and response (#1094) * update method name in the client (#1094) * proto entries for GetLatestHeight logic (#1094) * added logic for streaming new blocks as part of GetLatestHeight (#1094) * ensure subcribers have their own unique id (#1094) * client logic to use a channel parameter (#1094) * added test for GetLatestHeight (#1094) * better streaming test to prevent error (#1094) * remove empty line Co-authored-by: Thane Thomson <connect@thanethomson.com> * removing UNARY_RPC to prevent lint on server streaming (#1094) * remove server reflection, doesn't work well with gogoproto (#1094) * handle negative height parameter (#1094) * remove `Block` from `GetBlockLatestHeight` Co-authored-by: Thane Thomson <connect@thanethomson.com> * changed method name and fixes to test (#1094) * fixes from PR feedback (#1094) * remove vars Co-authored-by: Thane Thomson <connect@thanethomson.com> * fix declaration Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplify error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * remove line Co-authored-by: Thane Thomson <connect@thanethomson.com> * remove var declaration, not needed Co-authored-by: Thane Thomson <connect@thanethomson.com> * use simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplest error return Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplify error handling Co-authored-by: Thane Thomson <connect@thanethomson.com> * simplified the logic for subscriber id (#1094) * more fixs based on PR feedback (#1094) * generated new protos (#1094) * remove else logic to allow compilation (#1094) * added logging capabilities to the block service and added log messages (#1094) * ensure node is a valid testing node (#1094) * added more conditions to handle subscription cancel (#1094) * use break in the for loop (#1094) * renamed ResultBlock to Block (#1094) * refactored the client logic use a channel for errors (#1094) * simplify subscriber name Co-authored-by: Thane Thomson <connect@thanethomson.com> * adding error to log Co-authored-by: Thane Thomson <connect@thanethomson.com> * update logging Co-authored-by: Thane Thomson <connect@thanethomson.com> * Simplified and improved logging (#1094) * added logic to drop the height publish if channel full (#194) * improved test to ensure is not light or seed node (#1094) * refactored block service to use one channel with result type (#1094) * blockservice: Apply service name to all logs Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Capitalize start of all log messages Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Remove unnecessary logs Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Add trivial RPC error response tracing mechanism for easier debugging Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Simplify constructor Signed-off-by: Thane Thomson <connect@thanethomson.com> * blockservice: Extract type assertion Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entries Signed-off-by: Thane Thomson <connect@thanethomson.com> * docs: Update configuration-related content Signed-off-by: Thane Thomson <connect@thanethomson.com> * grpc/client: Format Signed-off-by: Thane Thomson <connect@thanethomson.com> * grpc: Extract function to validate or update block height from request Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* Add block results grpc endpoint * Make requested changes * Fix linting * Lint proto * Fix test race condition * Update test order * Update rpc/grpc/server/services/blockresultservice/service.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update rpc/grpc/server/services/blockresultservice/service.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update config/toml.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update rpc/grpc/server/services/blockresultservice/service.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update rpc/grpc/server/services/blockresultservice/service.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Format protos * Format upstream changes * Test height for failing grpc call * Test w additional logging * Remove accidental formal * Revert "Remove accidental formal" This reverts commit 81cb702. * Make additional req changes * Requested changes for formatting/client signature * Add changelog entry * Add separate endpoint for latest BlockResults * Fix linting * Clean * Update changelog * Update rpc/grpc/server/services/blockresultservice/service.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update test --------- Co-authored-by: Thane Thomson <connect@thanethomson.com>
* Added pruning mechanism for blocks and abci block results * Added support for datacompanion and application retain heights --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Thane Thomson <connect@thanethomson.com>
* proto: Add proto files for the PruningService * proto: generate *.pb.go files for PruningService * config: add grpc.privileged incl. pruning_service As described in ADR-101, add the node configuration section named grpc.privileged to control the privileged server socket, containing a pruning_service section for the pruning service. * config: add pruning service config to the template * grpc: package for privileged server The privileged server optionally instantiated with the pruning service. * node: add setup for the privileged gRPC server * rpc: enable PruningService in test helper config Also stop the makeAddrs function from growing grotesquely repetitive and replace it with makeAddr returning a single, supposedly random, local address string. * grpc: privileged client with PruningService Add client-side support for the privileged connection that features an optionally enabled pruning service. * e2e tests for PruningService * config: refer to [storage.pruning] section in config.toml Replace potentially confusing text in the comments on the pruning service configuration. Co-authored-by: Thane Thomson <connect@thanethomson.com> * grpc: tracing with error logs for PruningService --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Thane Thomson <connect@thanethomson.com>
* Metrics to report on the data companion retain height * Added metric to report the application retain height as well * Added metrics to report the blockstore base height and the abci results base height --------- Co-authored-by: Thane Thomson <connect@thanethomson.com>
* Applied @serigo-mena's comments * Update state/pruner.go Co-authored-by: Thane Thomson <connect@thanethomson.com> * Applied further comments from PR * Fixed batching interval --------- Co-authored-by: Thane Thomson <connect@thanethomson.com>
* grpc: Add close method to client Signed-off-by: Thane Thomson <connect@thanethomson.com> * grpc: Add close method to privileged client Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Close gRPC clients after testing Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>
Signed-off-by: Thane Thomson <connect@thanethomson.com>
* state: Invert logic of whether retain heights are set to be more readable Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor findMinRetainHeight logic for clarity and to respect config Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename findMinRetainHeight to findMinBlockRetainHeight for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename Pruner.SetApplicationRetainHeight to SetApplicationBlockRetainHeight for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename pruner SetCompanionRetainHeight to SetCompanionBlockRetainHeight for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - shorten local variable name Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - rename local method name for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - remove redundant condition Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - use helper instead of redefining logic Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - shorten local variable names Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - shorten local variable names Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor Pruner.SetApplicationBlockRetainHeight logic The application retain height should be set regardless of what the companion retain height is. Pruning should take place based on whichever of the two values is lower. There is no need to consider the companion retain height in this logic. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor Pruner.SetCompanionBlockRetainHeight logic Similar refactor to that done for Pruner.SetApplicationBlockRetainHeight. There is no need to consider the application retain height during this function call. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor Pruner.SetABCIResRetainHeight logic Aligns the logic of this method similar to that in the other retain height setter methods. Also fixes a logic bug where setting the retain height would skip updating the metrics. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - shorten local variable names Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Simplify pruner logic assuming retain heights are always set We can simplify the pruner logic if we assume that the initial retain heights are always set on node startup. Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Tell pruner whether companion is enabled Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Expand error message detail Signed-off-by: Thane Thomson <connect@thanethomson.com> * state+store: Align tests with new logic Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - shorten local variable names Signed-off-by: Thane Thomson <connect@thanethomson.com> * rpc/grpc: Return trace IDs in error responses from pruning service Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Refactor/expand pruning service initialization logic Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Minor cosmetic tweaks to gRPC tests Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Enable data companion pruning Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Refactor - extract pruner creation as function Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Only enable companion-based pruning on full nodes and validators Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Only start ABCI results pruning routine when data companion is enabled Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Fix TestMinRetainHeight logic Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Fix companion retain height initialization logic Signed-off-by: Thane Thomson <connect@thanethomson.com> * test/e2e: Expand testing framework Allows explicit control over whether or not pruning should take a data companion into account for full nodes or validators in our E2E framework. This allows greater flexibility in the E2E framework to test nodes both with and without data companion-related functionality. Expands our standard CI manifest to expect data companions attached to two of the nodes. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename background routines Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Apply change from code review Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>
This reverts commit 64c36ae.
Signed-off-by: Thane Thomson <connect@thanethomson.com>
3 tasks
|
Keeping as a draft while folks make more changes to the feature branch. As those PRs are merged they'll automatically show up here. |
* done stateDB writes batching * remove forgotten debug print * benchmarking for state db save * state db save benchmarking * batching in store * benchmarking for store batching * Added pruning mechanism -initial version * removed comments * Rewrore pruning mechanism as service. Working code but tests need to change. * Minor edits * Removed comments * Fixed linter * Fixed linter * Fixed state and block store tests Proper ticker used instead of sleep for pruning service job * Fixed failing http test * fixed linter errors and reduced sleep in http test * Fixed e2e test and failing linter * Applied some of @thanethomson's PR comments * add changelog * Separated retain height functions * fixed linter and test * Fixed code in select min height` * Fixed bugs causing failing tests * Added config flag for companion initial height Fixed failing test due to wrong check on blockstore base height * Tested setting the companion initial height via config. Moved pruner initialization into blockExecutor again. * Added config for pruning frequency * minor * Added additional test for companion retain height setting * script for adjasting test loadtime parameeters * Added simple testing tool for goleveldb * measuring tool + pruning setting * draft of tx_index pruning * using bus to prune tx_index * draft of pruning * working prototipe on tx_index txIndexer pruning * consensus: Fix test broken due to missing state store mocks Signed-off-by: Thane Thomson <connect@thanethomson.com> * config: Refactor to match updated ADR ADR-101 was updated recently to reflect the desired configuration file changes - this updates the code to match the ADR. It also changes the "frequency" variable to more accurately be called an "interval", since it represents a sleep period between pruning operations as opposed to the number of times it should be run per second. The interval is also defined as a time.Duration now to facilitate more ergonomic usage from the configuration file. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Format Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename pruner sleep time to "interval" Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor pruner config Make the pruner config private such that its configuration is fixed at runtime. With the previous approach, the PrunerInterval method could be abused to change the interval at runtime since it returns a function that can operate on the public type. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Fix minor nits in function docstrings Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Remove unused method Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Remove unnecessary OnStart/OnStop logic The base service OnStart/OnStop logic is purposefully empty, so there's no need to explicitly call it. And by not defining the OnStop method on Pruner the base service's OnStop (a noop) will automatically be called because of Go's "inheritance" mechanism. Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Make pruner belong to Node primarily Move the pruner "ownership" the Node. This doesn't actually have a meaningful effect on the lifetime of the pruner, but it is more of a signal to readers of the code that it should be considered a service in its own right, and the block executor simply has access to it to trigger one method call. Signed-off-by: Thane Thomson <connect@thanethomson.com> * store: Format comment for readability Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Add docstring for PruneABCIResponses Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor pruningRoutine to condense logging Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor pruningRoutine - extract function for block pruning Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor pruningRoutine - extract function for ABCI result pruning Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Only save companion retain height on startup if not yet present Expand the check on node creation to only set the companion retain height if 3 conditions are met: 1. The companion retain height has not yet been set previously. 2. The companion is enabled. 3. The configured companion retain height is > 0. Signed-off-by: Thane Thomson <connect@thanethomson.com> * config: Impose restriction on pruning interval to be > 0 Signed-off-by: Thane Thomson <connect@thanethomson.com> * test for txindex pruning * node: Fail construction if setting data companion retain height fails Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Start pruner on node startup after state sync Signed-off-by: Thane Thomson <connect@thanethomson.com> * light: Remove sleep from test Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Handle error cases where retain height keys are not set Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Make Pruner.FindMinRetainHeight private Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Improve logging in block executor pruning Signed-off-by: Thane Thomson <connect@thanethomson.com> * block indexer pruning + test * Format Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Export FindMinRetainHeight exclusively for testing Signed-off-by: Thane Thomson <connect@thanethomson.com> * node: Extract function for initializing companion retain height Signed-off-by: Thane Thomson <connect@thanethomson.com> * Add changelog entries Signed-off-by: Thane Thomson <connect@thanethomson.com> * docs: Update configuration-related content Signed-off-by: Thane Thomson <connect@thanethomson.com> * finilizing * undoing testing setup * Revert "done stateDB writes batching" This reverts commit 78d8d38. * Revert "benchmarking for state db save" This reverts commit f3b757a. * Revert "batching in store" This reverts commit 7ca253f. * Revert "benchmarking for store batching" This reverts commit 2fa2e38. * Revert "add changelog" This reverts commit ec60b4a. * Revert "fixed linter and test" This reverts commit fac927e. * Revert "remove forgotten comments" This reverts commit fae7d59. * fixed go.sum * shut up linter * add changelog * node: Stop pruner when the node stops Signed-off-by: Thane Thomson <connect@thanethomson.com> * Remove redundunt variable Co-authored-by: Thane Thomson <connect@thanethomson.com> * remove redundunt search for block heights * rebased to PR 1150 * state: Refactor ABCI response pruning Signed-off-by: Thane Thomson <connect@thanethomso 6302 n.com> * state: Add parameter names to Store interface for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename helper for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Optimize ABCI responses pruning Avoid iterating through all possible heights when pruning ABCI responses. Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fixed couting of the last height pruned for abci respnses and blocks * Set application retain height on startup to avoid pruning by the data companion before the application has indicated so * Apply suggestions from code review Co-authored-by: Thane Thomson <connect@thanethomson.com> * Set app retain height on startup only if it was not set before * Fixed linter * Simplified abci res pruning * fixed tests to support indexerService in pruner * forgotten file * updated test * ADR 101: Recommended updates to pruner service (#1201) * state: Reference retain heights instead of pruned heights for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Explicitly assign variable values for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Refactor - early returns Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Add mutex to Pruner to serialize retain height setting requests Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Add convenience getters to Pruner Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Export Pruner errors Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Rename PrunerInterval option to WithPrunerInterval for clarity Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Fix error message references Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * ADR 101: Add observer for pruner (#1183) * state: Add observer for Pruner Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Wire in pruner observer Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Make test more deterministic Signed-off-by: Thane Thomson <connect@thanethomson.com> * Fix renames Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Simplify NoopPrunerObserver usage Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Fix order of expected/actual params in assertion Signed-off-by: Thane Thomson <connect@thanethomson.com> * state: Clarify field meanings Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> * persisting last retained height for the block indexer * separate index pruning * adjust go.sum * Separated ABCI result pruning into separate routine Added observer into the pruner tests. * optimizing event search * linter fix * pruning benchmark * fix linter * persisting indexer retain height * indexer last retain height * fix linter * Update state/pruner.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * remove reduntant dependencies * added separate persisting of block and tx indexer retain heights * ADR-101: implement gRPC `PruningService` (#1154) * proto: Add proto files for the PruningService * proto: generate *.pb.go files for PruningService * config: add grpc.privileged incl. pruning_service As described in ADR-101, add the node configuration section named grpc.privileged to control the privileged server socket, containing a pruning_service section for the pruning service. * config: add pruning service config to the template * grpc: package for privileged server The privileged server optionally instantiated with the pruning service. * node: add setup for the privileged gRPC server * rpc: enable PruningService in test helper config Also stop the makeAddrs function from growing grotesquely repetitive and replace it with makeAddr returning a single, supposedly random, local address string. * grpc: privileged client with PruningService Add client-side support for the privileged connection that features an optionally enabled pruning service. * e2e tests for PruningService * config: refer to [storage.pruning] section in config.toml Replace potentially confusing text in the comments on the pruning service configuration. Co-authored-by: Thane Thomson <connect@thanethomson.com> * grpc: tracing with error logs for PruningService --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev> Co-authored-by: Thane Thomson <connect@thanethomson.com> * Update state/txindex/indexer_service.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update state/indexer/block/kv/kv.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * ADR-101: Metrics to monitor the pruning (#1234) * Metrics to report on the data companion retain height * Added metric to report the application retain height as well * Added metrics to report the blockstore base height and the abci results base height --------- Co-authored-by: Thane Thomson <connect@thanethomson.com> * using built in bytes function * fix missing import * reducing self-written functions * exporting getKeys for testing * removing indexer service as middle layer between indexers and pruner * separate tx and block indexers * Update state/pruner.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update state/pruner.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * fix naming * revert go.mod go.sum changes * Update state/pruner.go Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * fix ill merge resolution * batch pruning * block indexer: non-nil keyArray * block indexer: remove event keys record optimization * block indexer: remove getEventKeys optimization * periodically flushing pruning batch * revert package renaming, delete unused functions * minor changes * .changelog: add missing method to pruner update * txindexer: change panic to returning an error * remove exposal of GetKeys outside testing --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> Co-authored-by: Adi Seredinschi <a@seredinschi.net> Co-authored-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Mikhail Zabaluev <mikhail@informal.systems> Co-authored-by: Andy Nogueira <me@andynogueira.dev>
* Added changelog for pruning metrics * Apply PR review Co-authored-by: Thane Thomson <connect@thanethomson.com> --------- Co-authored-by: Thane Thomson <connect@thanethomson.com>
* added tx and block indexer pruning metrics * add changelog * added base height metric for tx and block indexer
Signed-off-by: Thane Thomson <connect@thanethomson.com>
3 tasks
* initial doc structure for data companion (#1296) * add some boilerplate content (#1296) * add boilerplate for gRPC docs (#1296) * adding more content for DC docs (#1296) * added more information for pruning retain height (#1296) * added more info to fetch data via gRPC (#1296) * more content, some refactoring and rewrite (#1296) * added info about latest height stream (#1296) * removed gRPC doc for now (#1296) * Update docs/data-companion/pruning.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update docs/data-companion/intro.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * changes on intro based on feedback (#1296) * better structure and content (#1296) * added info about block and tx index pruning (#1296) * minor addition (#1296) * Update docs/data-companion/pruning.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update docs/data-companion/grpc.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update docs/data-companion/grpc.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update docs/data-companion/pruning.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> * Update docs/data-companion/pruning.md Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com> --------- Co-authored-by: Jasmina Malicevic <jasmina.dustinac@gmail.com>
* done gRPC endpoint for TxIndexer and BlockIndexer pruning * re-ordering function signatures (#1260) * added .changelog --------- Co-authored-by: Andy Nogueira <me@andynogueira.dev>
3 tasks
* Fixed changelog * Deleted changelog entries from 0.38
3 tasks
andynog
reviewed
Sep 20, 2023
| @@ -0,0 +1,2 @@ | |||
| - `[config]` Add `[grpc.block_results_service]` gRPC configuration `BlockResultsService` | |||
There was a problem hiding this comment.
Suggested change
| - `[config]` Add `[grpc.block_results_service]` gRPC configuration `BlockResultsService` | |
| - `[config]` Add `[grpc.block_results_service]` section to configure gRPC `BlockResultsService` |
to match the BlockService changelog entry
Comment on lines
+1
to
+2
| - `[grpc]` Add `BlockService` with client to facilitate fetching of blocks and | ||
| streaming of the latest committed block height |
There was a problem hiding this comment.
Suggested change
| - `[grpc]` Add `BlockService` with client to facilitate fetching of blocks and | |
| streaming of the latest committed block height | |
| - `[grpc]` Add `BlockService` with client to fetch blocks for a given height or latest and | |
| streaming of the latest committed block height |
| @@ -0,0 +1,2 @@ | |||
| - `[rpc/grpc]` Add gRPC client with support for version service | |||
There was a problem hiding this comment.
Suggested change
| - `[rpc/grpc]` Add gRPC client with support for version service | |
| - `[grpc]` Add gRPC client with support for version service |
should this be just [grpc] to match the entries for other services e.g. BlockService. I see PruningService and VersionService use [rpc/grpc] but BlockService and BlockResultsService use [grpc]. I will not add comments on each section. Just a comment here, please feel free to change in other places if this makes sense.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #816.
PR checklist
.changelog(we use unclog to manage our changelog)docs/orspec/) and code comments