8000 [Feature] Support write and read decimal256 by stephen-shelby · Pull Request #59778 · StarRocks/starrocks · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[Feature] Support write and read decimal256 #59778

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 16, 2025
Merged

[Feature] Support write and read decimal256 #59778

merged 1 commit into from
Jun 16, 2025

Conversation

stephen-shelby
Copy link
Contributor
@stephen-shelby stephen-shelby commented Jun 10, 2025 •
edited
Loading

Why I'm doing:

What I'm doing:

the changes of this PR are all to ensure that decimal256 can be correctly compiled and read-written. some traits aren't implemented yet because we need other features to test them. the main change of this patch is that we define int256 as the storage for decimal256. This PR is only the simplest version for int256 arithmetic operations. We will optimize it in the subsequent PR.

Currently, StarRocks uses multiple widths to represent decimals

  1. Decimal (P <= 18, S), LogicalType is Decimal64, stored using int64, delegate LogicalType is BIGINT
  2. Decimal (P > 18 & P <= 38, S), LogicalType is Decimal128, stored using int128, delegate LogicalType is LARGEINT
    For the newly added Decimal(P > 38 & P <= 76, S), the LogicalType is Decimal256, which is represented by the custom int256, and its delegate LogicalType is INT_256.

Storage Layer

use int256 as storage #59808

Computing layer

Currently, StarRocks has abstracted a set of good computing layer expressions for Decimal, DecimalV3Column, which is well adapted to the precision and scale unique to decimal. For Decimal256, it can be reused directly.
DecimalV3Column is a fixed length column. We can reuse the serialized and deserialized implement. And we need to adapt the corresponding hash function for int256.
Some core definitions as follow up:

template class DecimalV3Column<int32_t>;
template class DecimalV3Column<int64_t>;
template class DecimalV3Column<int128_t>;
template class DecimalV3Column<int256_t>;

template <>
struct RunTimeTypeTraits<TYPE_INT256> {
    using CppType = starrocks::int256_t;
    using ColumnType = FixedLengthColumn<starrocks::int256_t>;
};

template <>
struct RunTimeTypeTraits<TYPE_DECIMAL256> {
    using CppType = int256_t;
    using ColumnType = Decimal256Column;
};

using Decimal256Column = DecimalV3Column<int256_t>;

Read-Written behavior

The read and write behavior is consistent with the previous decimal.
use decimal(50, 48) for example

  • insert into test_decimal(d1) select 1.222222; work normal
  • insert into test_decimal(d1) select 11111111111111111111111111111111111111111111.222222; Successfully written. Since the scale is 48, the integer part plus 48 has exceeded the maximum integer that can be represented by 256 bits, So fe directly casts the value to null.
  • insert into test_decimal(d1) select 333; throw an exception on the be sideERROR 1064 (HY000): Insert has filtered data. Because the integer part plus the scale exceeds the maximum value represented by a 50-digit integer.

Others behavior changes

  • When inserting, you can write a integer that exceeds 38 and has a maximum of 76 digits. However, you cannot write the maximum 77-digit decimal integer that can be represented by 256 bits. This is the same as decimal128.
  • TYPE_INT is only used for TYPE_DECIMAL256 storage and won't be exposed to users.

Fixes #issue
#59645

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.5
    • 3.4
    • 3.3

@stephen-shelby stephen-shelby requested review from a team as code owners June 10, 2025 07:04
@stephen-shelby stephen-shelby requested review from a team as code owners June 10, 2025 07:04
@stephen-shelby stephen-shelby linked an issue Jun 10, 2025 that may be closed by this pull request
Support decimal256 #59645
Closed
@wanpengfei-git wanpengfei-git requested a review from a team June 10, 2025 07:05
@stephen-shelby stephen-shelby force-pushed the read_write_decimal branch 3 times, most recently from 6b14595 to 00e5b92 Compare June 10, 2025 13:30
alvin-celerdata
alvin-celerdata reviewed Jun 10, 2025
View reviewed changes
@stephen-shelby stephen-shelby force-pushed the read_write_decimal branch 2 times, most recently from 13bfac7 to f993d98 Compare June 11, 2025 04:27
@stephen-shelby stephen-shelby mentioned this pull request Jun 11, 2025
Merged
22 tasks
satanson
satanson previously approved these changes Jun 13, 2025
View reviewed changes
@stephen-shelby stephen-shelby force-pushed the read_write_decimal branch 3 times, most recently from 24fbea7 to 1a1ce00 Compare June 13, 2025 14:37
@stephen-shelby
[Feature] Support write and read decimal256
435b407 
Signed-off-by: stephen <stephen5217@163.com>
@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed

Issues
11 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@github-actions GitHub Actions
Copy link

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 7 / 8 (87.50%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/analysis/DecimalLiteral.java 4 5 80.00% [292]
🔵 com/starrocks/catalog/ScalarType.java 3 3 100.00% []

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

❌ fail : 48 / 77 (62.34%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/column/datum.h 0 1 00.00% [175]
🔵 be/src/column/decimalv3_column.cpp 0 4 00.00% [111, 112, 113, 114]
🔵 be/src/types/int256.h 0 2 00.00% [646, 647]
🔵 be/src/column/column_visitor_mutable.cpp 0 2 00.00% [46, 65]
🔵 be/src/column/column_visitor.cpp 0 2 00.00% [46, 64]
🔵 be/src/column/fixed_length_column_base.cpp 0 4 00.00% [322, 323, 324, 325]
🔵 be/src/storage/key_coder.h 1 6 16.67% [307, 309, 311, 315, 316]
🔵 be/src/connector/mysql_connector.cpp 1 5 20.00% [465, 466, 467, 468]
🔵 be/src/runtime/types.h 2 4 50.00% [210, 211]
🔵 be/src/types/logical_type.cpp 4 6 66.67% [114, 115]
🔵 be/src/storage/rowset/zone_map_index.cpp 2 3 66.67% [93]
🔵 be/src/exprs/expr.cpp 1 1 100.00% []
🔵 be/src/storage/key_coder.cpp 1 1 100.00% []
🔵 be/src/types/logical_type.h 3 3 100.00% []
🔵 be/src/storage/rowset/encoding_info.cpp 2 2 100.00% []
🔵 be/src/util/decimal_types.h 4 4 100.00% []
🔵 be/src/storage/chunk_helper.cpp 4 4 100.00% []
🔵 be/src/exprs/literal.cpp 3 3 100.00% []
🔵 be/src/exec/tablet_sink.cpp 2 2 100.00% []
🔵 be/src/column/column_visitor_adapter.h 2 2 100.00% []
🔵 be/src/storage/decimal_type_info.cpp 5 5 100.00% []
🔵 be/src/runtime/types.cpp 5 5 100.00% []
🔵 be/src/storage/tablet_schema.cpp 3 3 100.00% []
🔵 be/src/column/column_helper.cpp 1 1 100.00% []
🔵 be/src/storage/types.cpp 2 2 100.00% []

@kangkaisen kangkaisen merged commit 5d3a511 into StarRocks:main Jun 16, 2025
53 of 56 checks passed
AntiTopQuark pushed a commit to AntiTopQuark/starrocks that referenced this pull request Jun 19, 2025
@stephen-shelby @AntiTopQuark
[Feature] Support write and read decimal256 (StarRocks#59778)
ba6094d 
Signed-off-by: stephen <stephen5217@163.com>
Signed-off-by: AntiTopQuark <AntiTopQuark1350@outlook.com>
dengliu pushed a commit to dengliu/starrocks that referenced this pull request Jun 20, 2025
@stephen-shelby @dengliu
[Feature] Support write and read decimal256 (StarRocks#59778)
ef7a135 
Signed-off-by: stephen <stephen5217@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alvin-celerdata alvin-celerdata alvin-celerdata left review comments

@satanson satanson satanson approved these changes

@Seaven Seaven Seaven approved these changes

@kangkaisen kangkaisen kangkaisen approved these changes

Assignees

@stephen-shelby stephen-shelby

Labels
PROTO-REVIEW version:3.6
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support decimal256
6 participants
@stephen-shelby @satanson @Seaven @kangkaisen @alvin-celerdata @wanpengfei-git
0