Tags: M-nick808/insight
Tags
Merge pull request thirdweb-dev#196 from thirdweb-dev/05-15-fix_mappi… …ng_keys_when_replacing_data fix mapping keys when replacing data
Merge pull request thirdweb-dev#194 from thirdweb-dev/04-23-fix_sign_… …on_insert fix sign on insert
Merge pull request thirdweb-dev#192 from thirdweb-dev/04-17-make_inse… …rts_and_reorgs_atomic Make inserts and reorg handling atomic
add blob fields to transactions (thirdweb-dev#190) ### TL;DR Added support for EIP-4844 blob transaction fields in the transaction data model. ### What changed? - Added two new fields to the `Transaction` struct: - `MaxFeePerBlobGas` - to store the maximum fee per blob gas - `BlobVersionedHashes` - to store the versioned hashes of the blobs - Updated the `TransactionModel` struct to include these new fields - Modified the transaction serialization logic to handle the new blob-related fields - Updated the ClickHouse database schema and insertion logic to store the new fields ### How to test? 1. Process transactions from a network that supports EIP-4844 (like Ethereum mainnet post-Dencun upgrade) 2. Verify that blob transactions are correctly stored with their `MaxFeePerBlobGas` and `BlobVersionedHashes` values 3. Query the transactions table to confirm the new fields are populated for blob transactions ### Why make this change? This change adds support for EIP-4844 (blob transactions) introduced in the Ethereum Dencun upgrade. Blob transactions include additional fields that need to be captured and stored in our system to maintain complete transaction data. Without these fields, we would be missing critical information about blob transactions, which are important for layer 2 rollups and other scaling solutions.
publishing blockchain data (thirdweb-dev#176) ### TL;DR Added Kafka publisher functionality to stream blockchain data to external systems. ### What changed? - Added a new Publisher module that can stream blocks, transactions, traces, and events to Kafka topics - Implemented configuration options for the publisher in the root command flags - Added serialization methods for blockchain data models - Integrated the publisher with the Committer and ReorgHandler to publish data during normal processing and reorgs - Added filtering capabilities for transactions and events based on addresses and topics ### How to test? 1. Enable the publisher with the `--publisher-enabled` flag 2. Configure Kafka broker(s) with `--publisher-brokers` 3. Enable specific data types with their respective flags: - `--publisher-blocks-enabled` - `--publisher-transactions-enabled` - `--publisher-traces-enabled` - `--publisher-events-enabled` 4. Set topic names and filters as needed 5. Verify data is being published to the configured Kafka topics ### Why make this change? This change enables real-time streaming of blockchain data to external systems via Kafka, allowing for more flexible data processing pipelines and integration with other applications. The publisher can be used to build real-time analytics, monitoring systems, or to feed data into other databases or processing frameworks.
Merge pull request thirdweb-dev#178 from thirdweb-dev/vt-transfers-api feat: add endpoint to query transfers table
use consistent read when deleting data (thirdweb-dev#171) ### TL;DR Added `ForceConsistentData: true` to query filters in block deletion operations. ### What changed? Modified the query filters in `deleteBlocks`, `deleteLogs`, `deleteTransactions`, and `deleteTraces` functions to include `ForceConsistentData: true` parameter when retrieving data before deletion. ### How to test? 1. Attempt to delete blocks and their associated data 2. Verify that the deletion operations complete successfully 3. Confirm that all related data (logs, transactions, traces) is consistently deleted ### Why make this change? Ensures data consistency during block deletion operations by forcing the query to return complete and consistent datasets before performing deletions. This prevents potential partial or inconsistent data deletions that could corrupt the database state.
Merge pull request thirdweb-dev#156 from thirdweb-dev/02-19-keep_trac… …k_of_last_committed_block_in_memory Track last committed block in memory
Merge pull request thirdweb-dev#154 from thirdweb-dev/02-18-fix_dupli… …cate_inserts fix duplicate inserts
create an endpoint to query token balances (thirdweb-dev#136) ### TL;DR Added a new API endpoint to fetch token balances by type (ERC20, ERC721, ERC1155) for a given address. ### What changed? - Added `/balances/:owner/:type` endpoint to retrieve token balances - Created materialized views in ClickHouse to track token balances from transfer events - Implemented balance querying with support for pagination and filtering - Added support for hiding zero balances - Introduced token balance data structures and query filters ### How to test? 1. Start the API server 2. Query the endpoint: `/balances/{address}/{token_type}` - `token_type`: "erc20", "erc721", or "erc1155" - Optional query params: - `hide_zero_balances`: true/false - `token_address`: specific token contract - `page` and `limit`: for pagination - `sort_by` and `sort_order`: for sorting ### Why make this change? To provide a convenient way to fetch token balances for addresses across different token standards, enabling better wallet integration and portfolio tracking capabilities.
PreviousNext