8000 fix: change limit to 100k by sokumon · Pull Request #32419 · frappe/frappe · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix: change limit to 100k #32419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 6, 2025
Merged

fix: change limit to 100k #32419

merged 1 commit into from
May 6, 2025

Conversation

sokumon
Copy link
Contributor
@sokumon sokumon commented May 6, 2025

No description provided.

@sokumon sokumon added backport version-14-hotfix backport to version 14 backport version-15-hotfix Backport the PR to v15 labels May 6, 2025
@ankush ankush merged commit b34b6d7 into frappe:develop May 6, 2025
25 checks passed
mergify bot pushed a commit that referenced this pull request May 6, 2025
(cherry picked from commit b34b6d7)
mergify bot pushed a commit that referenced this pull request May 6, 2025
(cherry picked from commit b34b6d7)
ankush pushed a commit that referenced this pull request May 6, 2025
(cherry picked from commit b34b6d7)

Co-authored-by: Soham Kulkarni <77533095+sokumon@users.noreply.github.com>
ankush pushed a commit that referenced this pull request May 6, 2025
(cherry picked from commit b34b6d7)

Co-authored-by: Soham Kulkarni <77533095+sokumon@users.noreply.github.com>
@iamejaaz
Copy link
Member

Related to support ticket: 37774 ??

frappe-pr-bot pushed a commit that referenced this pull request May 13, 2025
## [14.96.3](v14.96.2...v14.96.3) (2025-05-13)

### Bug Fixes

* 'not set' equivalence in query conditions (backport [#28015](#28015)) ([#32450](#32450)) ([8cf37ef](8cf37ef))
* change limit to 100k ([#32419](#32419)) ([#32433](#32433)) ([91d2148](91d2148))
* **db_query:** double-escaped value (backport [#32376](#32376)) ([#32446](#32446)) ([abb7098](abb7098))
frappe-pr-bot pushed a commit that referenced this pull request May 13, 2025
## [15.68.1](v15.68.0...v15.68.1) (2025-05-13)

### Bug Fixes

* 'not set' equivalence in query conditions (backport [#28015](#28015)) ([#32451](#32451)) ([273bb79](273bb79))
* add deprecation warning for social module energy points ([6764a9e](6764a9e))
* **apps:** don't skip app if no permission hook (backport [#32492](#32492)) ([#32505](#32505)) ([af3f337](af3f337))
* change limit to 100k ([#32419](#32419)) ([#32434](#32434)) ([1faeb91](1faeb91))
* conflicts in yarn.lock ([7f724ad](7f724ad))
* correct yarn.lock ([f1ff8ed](f1ff8ed))
* **db_query:** double-escaped value ([#32376](#32376)) ([#32447](#32447)) ([4e3a1f4](4e3a1f4))
* don't allow attaching an file invalid url ([4008fcf](4008fcf))
* layout for file uploader ([#32476](#32476)) ([7ae3b3e](7ae3b3e))
* remove ui warnings for now ([6721178](6721178))
* resolve conflicts in yarn.lock ([d985c3a](d985c3a))
* revert python changes and do via js ([7082a04](7082a04))
* rq worker count (backport [#31800](#31800)) ([#32491](#32491)) ([1884b75](1884b75))
* **TextEditor:** table actions as handle to quills toolbar ([432626e](432626e))
* translation backport command (CLI) ([#32485](#32485)) ([0237077](0237077))
* update email template hex code according to espresso (backport [#32401](#32401)) ([#32490](#32490)) ([00f91fd](00f91fd))

### Performance Improvements

* split maintenance tasks into separate queue (backport [#32413](#32413)) ([#32461](#32461)) ([844314d](844314d))
* transform ifnull into two conditions (backport [#32377](#32377)) ([#32457](#32457)) ([1d49173](1d49173))
pedropais00 pushed a commit to pedropais00/frappe that referenced this pull request May 14, 2025
@ashish-greycube
Copy link
Contributor

@ankush this contradicts
https://support.frappe.io/helpdesk/my-tickets/13605

Kindly help resolve or change design asap, as it will affect all enterprise users or any instance with good data volume & completely unacceptable solution to download to excel and view it.

@sokumon

@ankush
Copy link
Member
ankush commented May 19, 2025

@ashish-greycube this is hard to fix without fixing the underlying problem: frappe/datatable#216

Currently, those with resource-constrained machines can't do anything if the report fails to render. Falling back to export is at least better than not being able to do anything.

Few possible fixes @sokumon @iamejaaz:

@jigneshpshah
Copy link
  • till the time it is resolved , can we keep limit to 500k ?
  • and i hope export to excel will be forced upon the user, only if the number of rows getting rendered is greater than hardcoded limit ?
  • for in general viewing of GL / SL (e.g. for a specific voucher or specific party accounts / ledger accounts / stock movement ) having less than 500k rows in the result set , it would work as usual without forcing users to download excel ?

@ankush
Copy link
Member
ankush commented May 19, 2025

Is >100k rows even usable? I have a fairly recent machine with top-end Ryzen chip and 16GB RAM, I can render 1M rows, but it's hardly usable after rendering. It lags everywhere and filters etc don't work.

I think for data this large, there won't be any option to render it in the browser beyond a point. The best we can do is pagination, which will again restrict a lot of other features.

I made a small fix here that will allow increasing the limit to maybe 200K: frappe/datatable#217


for in general viewing of GL / SL (e.g. for a specific voucher or specific party accounts / ledger accounts / stock movement ) having less than 500k rows in the result set , it would work as usual without forcing users to download excel ?

Yeah, it should work fine. Individual vouchers rarely have even >100 lines.

till the time it is resolved , can we keep limit to 500k ?

We can't, at least not by default... because those with weaker machines won't even be able to view the report or export it. The best we can do right now is make it configurable.

@sokumon can you make this configurable? System settings > reporting, add a new config for max report rows.

@jigneshpshah
Copy link

thanks for the clarifications, the insights and the quick fix to allow it upto 200k.

Is >100k rows even usable?

use case : when users have to search a specific record or bunch of records ( specially accounts users, in a Financial year) , they tend to load everything without much of filtering and via scrolling they try to search..

continuously changing the filters to narrow the results to keep it less than 100k leads to a major UX issue and feels like being handcuffed !

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport version-14-hotfix backport to version 14 backport version-15-hotfix Backport the PR to v15
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0