8000 feat: uncomment other tests fixing pending issues by outerlook · Pull Request #868 · trufnetwork/node · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: uncomment other tests fixing pending issues #868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Mar 25, 2025

Conversation

outerlook
Copy link
Contributor
@outerlook outerlook commented Mar 24, 2025

Description

⚠️ Current version is not performing so well -- a lot worse than expected, making it unusable on real nested queries. There sure is a lot of room for SQL optimizations, reducing correlated scalar queries and etc. I thought we could reach to a state that is just slightly better on first interaction, but the query times are exploding with additional layers of stream nesting. I should revisit this after the Argentina goal is complete https://github.com/trufnetwork/truf-data-provider/issues/427

  • bump kwil to include array feat engine: add native array func implementations kwil-db#1489
  • uncommented and fixed additional tests, including benchmark that permitted us to understand current state
  • implemented a series of batch version of basic functions, to be able to batch insert streams in sufficient time
  • created some missing actions to make tests pass, such as disable taxonomy
  • fixed get index behavior with a new implementation
  • deleted extraneous files such as system contract, etc

Related Problem

How Has This Been Tested?

tests passing, but definitely will need to be revisited

This commit deletes the `config.go`, `stream_manager.go`, and `stream_storage_test.go` files from the integration storage tests. These files were previously disabled for the kwil-db v0.10.0 upgrade and are no longer needed, streamlining the test suite and removing obsolete code. This cleanup aims to enhance maintainability and clarity in the testing framework.
This commit introduces a new SQL action `get_nonexistent_streams` that checks for the existence of streams based on provided data providers and stream IDs, returning those that do not exist. Additionally, it enhances the existing `is_stream_owner` action with a check to ensure the stream exists before proceeding.

In the test suite, the `testStreamDeletion` function has been updated to validate the existence of streams before and after deletion, ensuring that the new functionality is properly tested. These changes aim to improve error handling and robustness in stream management.
This commit renames the SQL action `get_nonexistent_streams` to `filter_streams_by_existence`, updating its functionality to allow filtering based on the existence of streams, controlled by the `ExistingOnly` flag. The action now returns either existing or non-existing streams as specified. Corresponding changes have been made in the test suite to reflect this new action name and functionality, ensuring that the tests validate the correct behavior of stream existence checks. These modifications aim to improve clarity and maintainability in the stream management system.
This commit introduces several new SQL actions to enhance metadata retrieval for streams. The actions include `get_latest_metadata`, `get_latest_metadata_int`, `get_latest_metadata_ref`, `get_latest_metadata_bool`, and `get_latest_metadata_string`, each designed to fetch the latest metadata values based on specified parameters. These additions aim to improve the functionality and flexibility of metadata queries within the stream management system.
This commit refines the authorization logic within the `002-authorization.sql` file by simplifying checks for stream visibility and wallet permissions. Key changes include replacing multiple loops with direct metadata retrieval functions, improving readability and efficiency. Additionally, the logic now includes checks for stream ownership and default visibility settings, ensuring a more robust authorization process. These modifications aim to enhance clarity and maintainability in the authorization mechanisms for stream access.
This commit introduces a new SQL action `disable_taxonomy` that allows for the disabling of taxonomies based on the provided data provider, stream ID, and group sequence. The action includes a check to ensure that the wallet is authorized to write before proceeding with the update. This enhancement aims to improve the management of taxonomy states within the system, providing a mechanism to effectively disable taxonomies as needed.
…xed value calculations

This commit introduces two new SQL actions: `get_index_primitive` and `get_index_composed`. The `get_index_primitive` action calculates indexed values relative to a base value for primitive streams, while `get_index_composed` performs similar calculations for composed streams. Both actions include enhanced permission checks and improved error handling to ensure robust functionality. These additions aim to enhance the indexing capabilities within the stream management system, providing more flexibility in data retrieval and analysis.
…tence function

This commit introduces the `DisableTaxonomy` action, allowing for the disabling of taxonomies based on the provided data provider, stream ID, and group sequence. It includes error handling to ensure robust functionality. Additionally, the `FilterStreamsByExistence` function has been updated to call a renamed SQL action `filter_streams`, enhancing clarity in the stream filtering process. These changes aim to improve taxonomy management and streamline stream existence checks within the system.
This commit introduces a new test for visibility settings in the `common_test.go` file, validating read permissions for both owners and non-owners of streams. The test checks default visibility, updates the read visibility to private, and verifies the expected access outcomes. Additionally, the implementation refactors the stream locator setup to utilize predefined stream IDs, improving clarity and maintainability in the test suite.
… functionality

This commit refactors the `complex_composed_test.go` file by reintroducing the test functions with enhanced structure and clarity. Key changes include the addition of a `SeedScripts` parameter in the `TestComplexComposed` function to utilize seed scripts for setup, and the implementation of `StreamLocator` for composed streams in the test functions. The tests now validate the expected outcomes for various scenarios, including record retrieval, index calculations, and handling of edge cases. These modifications aim to improve the maintainability and robustness of the test suite for complex composed streams.
This commit refactors the `index_change_test.go` file by reintroducing test functions with enhanced structure and clarity. Key changes include the addition of a `SeedScripts` parameter in the `TestIndexChange` function to utilize seed scripts for setup, and the implementation of `StreamLocator` for the streams in the test functions. The tests now validate the expected outcomes for various scenarios, including index calculations and handling of edge cases. These modifications aim to improve the maintainability and robustness of the test suite for index change functionality.
…ionality

This commit refactors the `composed_test.go` file by reintroducing test functions with enhanced structure and clarity. Key changes include the addition of a `SeedScripts` parameter in the `TestComposed` function to utilize seed scripts for setup, and the implementation of `StreamLocator` for composed streams in the test functions. The tests now validate the expected outcomes for various scenarios, including reco
8000
rd retrieval, taxonomy management, and error handling for non-owner actions. These modifications aim to improve the maintainability and robustness of the test suite for composed streams.
This commit deletes the `system_contract_test.go` file, which contained tests for the system contract that are no longer in use. The removal aims to streamline the test suite by eliminating outdated and unnecessary code, thereby improving maintainability and clarity in the testing framework.
/**
* get_latest_metadata_int: Retrieves the latest metadata value for a stream.
*/
CREATE OR REPLACE ACTION get_latest_metadata_int(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these were implemented only to make easier to avoid errors on the schema development. I still recommend using only get metadata externally, as there it's easier to deduplicate code

…actions

This commit introduces several new SQL actions to improve the management of streams and their metadata. Key additions include `cre
8000
ate_streams`, `is_stream_owner_batch`, `is_primitive_stream_batch`, `stream_exists_batch`, and batch permission checks for writing to streams. These actions streamline the process of creating multiple streams, checking ownership and primitive status in batches, and validating permissions efficiently. Additionally, existing actions have been refactored for better clarity and performance, ensuring a more robust and maintainable stream management system.
…lity

This commit refactors the benchmark package by reorganizing the code structure, enhancing type safety, and improving the clarity of function definitions. Key changes include the introduction of a new setup mechanism for schemas, the implementation of a more robust method for executing stream procedures, and the addition of improved error handling. The refactor aims to streamline the benchmarking process, making it more maintainable and efficient while ensuring better integration with the existing system components.
This commit deletes the `init.go` and `init_test.go` files, which contained the implementation and tests for the system contract initialization that are no longer in use. The removal aims to streamline the codebase by eliminating outdated and unnecessary components, thereby improving maintainability and clarity in the project structure.
This commit refactors multiple test files, enhancing the clarity and maintainability of the test suite. Key changes include the removal of unused imports, the introduction of new test functions for creating multiple streams in a single transaction, and the refactoring of existing tests to utilize updated structures and procedures. Additionally, the metadata handling in tests has been improved, ensuring better validation of metadata insertion, retrieval, and disabling. These modifications aim to streamline the testing process and improve the robustness of the test suite.
@outerlook outerlook self-assigned this Mar 25, 2025
@outerlook outerlook requested a review from MicBun March 25, 2025 18:31
@outerlook outerlook marked this pull request as ready for review March 25, 2025 18:31
Copy link
Member
@MicBun MicBun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

Copy link
pr-time-tracker bot commented Mar 25, 2025

@outerlook
⚠️⚠️⚠️
You must submit the time spent on this PR.
⚠️⚠️⚠️

@MicBun MicBun merged commit b6d697c into refactor/node-2.0 Mar 25, 2025
8 of 10 checks passed
@MicBun MicBun deleted the feat/uncomment-tests branch March 25, 2025 21:12
@MicBun MicBun linked an issue Mar 25, 2025 that may be closed by this pull request
Sign up for free < 6622 strong>to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Problem: Commented tests still exist on the codebase
2 participants
0