8000 feat(storylog): 实现日志导出 v1.5版本并优化相关功能 by PaienNate · Pull Request #1287 · sealdice/sealdice-core · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(storylog): 实现日志导出 v1.5版本并优化相关功能 #1287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

feat(storylog): 实现日志导出 v1.5版本并优化相关功能 #1287

wants to merge 2 commits into from

Conversation

PaienNate
Copy link
Contributor
  • 新增 Parquet 文件格式支持- 实现分页查询以提高日志导出性能
  • 添加版本控制以支持不同 API 版本
  • 优化骰点简报生成逻辑， 引入 gorm-cursor-paginator 和 parquet-go 依赖

关联染色器PR： sealdice/story-painter#36

同时做了如下处理：

  1. 采用游标分页方式进行文件写入，并增加十秒写入限制，从而应当能解决#1284的问题（若问题确实为内存内驻留太多数据的话）。
  2. 将log stat内的骰点函数采用gjson重构，从而减少无用的数据在内存驻留。

性能结论：
在随机3W条数据下，压缩后的文件约为900KB。相比于纯文本下3M的数据有极大的性能提升。

目前设置为V1.5版本，Client代号Parquet。

另外：由于木落染色器并未正确实现Client的标注，这导致目前虽然设置为V1.5版本且染色器使用PR内提供的版本，染色器仍无法进行染色。

PaienNate added 2 commits June 8, 2025 11:45
@PaienNate
feat(storylog): 实现日志导出 v1.5版本并优化相关功能
a4e3a1a 
- 新增 Parquet 文件格式支持- 实现分页查询以提高日志导出性能
- 添加版本控制以支持不同 API 版本
- 优化骰点简报生成逻辑，
 引入 gorm-cursor-paginator 和 parquet-go 依赖
@PaienNate
lint happy
35a2b8e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@PaienNate
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
0