8000 Chunked-Uploading function totally NOT working since version 0.55.2 · Issue #979 · rejetto/hfs · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Chunked-Uploading function totally NOT working since version 0.55.2 #979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
physicsdolphin opened this issue Apr 27, 2025 · 8 comments
Open

Chunked-Uploading function totally NOT working since version 0.55.2 #979

physicsdolphin opened this issue Apr 27, 2025 · 8 comments
Labels
bug Something isn't working fixed-next-release

Comments

@physicsdolphin
Copy link
physicsdolphin commented Apr 27, 2025 •
edited
Loading

Sorry for this late issue since I've been really busy working on my degree dissertation recently. I first discovered this on Jan.2025. It's really a critical problem that forced me to use 0.55.0 since Dec.2024 ┭┮﹏┭┮

BEFORE

  • Are you using a reverse-proxy, like nginx?
    • Check if your problem is caused by your proxy, try without it.

NO, the issue occurs when using direct ip:port to access HFS web interface, even on localhost. And of course occurs on every other situation

To Reproduce
Steps to reproduce the behavior:
1.set up HFS web server on windows server 2022
2.set "split uploading in chunks" size to 100MB (because the server also functions behind a cloudflare argo tunnel with 100MB free chunked size as maximum.)
3.use windows 11 or even the server itself as test client, and upload a file directly via chrome browser to HFS web interface. The file size should be larger than 95.4% of your chunked-encoding size set in step-2.

Describe the bug

The first chunk upload is fine, until 95.4% size of your chunked-encoding size set in step-2, then:
before 0.55.0 (included), the upload will run smoothly when uploading each chunk, and there's nothing more than a minor time gap before switching to next chunk's uploading. The uploading function works 100% fine;
in 0.55.3 0.56.0 and so on, you will stuck on TCP-zero window (I've used wireshark to identify this) right after uploading the first chunk to server, which prevents any further uploading (zero-speed), the uploading progress and the uploading panel indicator on your browser will stuck forever. If you refresh the browser, you'll see a temp file($HFSuploading...) in target directory with 95.4% of your chunked-encoding size(actually the first chunk...) ;
in 0.57.0(tested on 25/04/28), you will be prompted that "uploading failed"(at least not stuck forever), leaving a temp file($HFSuploading...) in target directory with 95.4% of your chunked-encoding size(actually the first chunk...);

Expected behavior
The uploading function works 100% fine.

Screenshots

uploading result:
Image

Image

admin settings:

Image

test on 0.55.0 with no settings changed:

Image Image

【I‘ve redacted user and ip in the following log】
Server log for failed upload: (0.57.0) HTTP 400 Bad request
[28/Apr/2025:04:00:56 +0800] "PUT /interchange%20station/test.rar?notificationChannel=upload-2n4wead9p5&giveBack=1741874115909&partial=481835828&existing=skip HTTP/1.1" 400 11 "{"size":481835828,"speed":1404769178}"

Server log for SUCCESSFUL upload (before 0.55.0(included)):
[28/Apr/2025:04:07:32 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&existing=skip HTTP/1.1" 200 10 "{"size":50000000}"
[28/Apr/2025:04:07:32 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=50000000&existing=skip HTTP/1.1" 200 10 "{"size":100000000}"
[28/Apr/2025:04:07:33 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=100000000&existing=skip HTTP/1.1" 200 10 "{"size":150000000}"
[28/Apr/2025:04:07:33 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=150000000&existing=skip HTTP/1.1" 200 10 "{"size":200000000}"
[28/Apr/2025:04:07:33 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=200000000&existing=skip HTTP/1.1" 200 10 "{"size":250000000}"
[28/Apr/2025:04:07:34 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=250000000&existing=skip HTTP/1.1" 200 10 "{"size":300000000}"
[28/Apr/2025:04:07:34 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=300000000&existing=skip HTTP/1.1" 200 10 "{"size":350000000}"
[28/Apr/2025:04:07:34 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=350000000&existing=skip HTTP/1.1" 200 10 "{"size":400000000}"
[28/Apr/2025:04:07:34 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&partial=y&resume=400000000&existing=skip HTTP/1.1" 200 10 "{"size":450000000}"
[28/Apr/2025:04:07:35 +0800] "PUT /interchange%20station/test1.rar?notificationChannel=upload-nx37g2ofaL&resume=450000000&existing=skip HTTP/1.1" 200 42 "{"size":481835828}"

Environment (please complete the following information):

  • Server OS: windows server 2022 Datacenter 10.0.20348
  • Tested client OS: windows server 2022 Datacenter 10.0.20348, windows 11 pro workstation 24H2 10.0.26100
  • HFS Version 0.57.0 (EVERY VERSION SINCE 0.55.2(included) HAS THIS CRITICAL ISSUE!)
  • Tested Browser: Microsoft Edge Stable 135.0.3179.98, Google Chrome Stable 135.0.7049.115

Additional context

I‘m sure that 95.4% comes from 100%/1.024/1.024 (you can double-check this), maybe this will help you locate the problem...

Image

Image

I tried to set other value for chunk size. The stucking position is always 95.4% of the chunk size you set.
@physicsdolphin physicsdolphin added the bug Something isn't working label Apr 27, 2025
@physicsdolphin physicsdolphin changed the title Uploading function totally NOT working since version 0.55.2 Chunked-Uploading function totally NOT working since version 0.55.2 Apr 28, 2025
@rejetto
Copy link
Owner
rejetto commented May 1, 2025

thanks for your detailed report!
i'm studying it

@rejetto
Copy link
Owner
rejetto commented May 1, 2025

i started testing and i've immediately found a problem with split-uploads, that was introduced in 0.57.0-beta5. Next release will fix such bug, but I assume it is not the one your are talking about, so I'll keep testing.
I don't have Windows 2022. I'll try using Win11 as server.

@rejetto
Copy link
Owner
rejetto commented May 1, 2025

i made a test uploading to a win11 from a chrome135, and it worked fine.
the split was set to 50mb, and the file was 97mb.
the result you posted, with the error code 400 was caused by the bug i just fixed, so it doesn't count.
can you please make a test using this version, that has the bug fixed?
https://github.com/user-attachments/files/19998859/hfs-windows-x64-0.57.2-beta1.zip

to easily install it go to Admin/Home and right-click on check-for-updates
8000

@physicsdolphin
Copy link
Author

Tested on 0.57.2-beta1 via directly replacing the executable on windows server 2022 Datacenter 10.0.20348 with server-side config file untouched. Clients use the same browsers mentioned above.

Chunked uploading now works fine with 50MB and 100MB chunk-uploading size. Direct-ip link and domain behind cloudflare argo tunnel both works.

The uploading behavior has gone back to "there's nothing more than a minor time gap before switching to next chunk's uploading".

Image

Image

Image

Maybe the size calculation affected on something like per-chunk content verification when uploading.

Anyway, it seems that your fix on error code 400 also fix this chunk-uploading issue. Thanks!

@rejetto
Copy link
Owner
rejetto commented May 7, 2025

i've found other problems with the upload.
for example 2 big upload (bigger than the "split" size), got stuck at the second,
and also it got stuck in case of disconnections.

i published 0.57.2-beta2 that includes these fixes. Let me know if you can give it a look, please.

@physicsdolphin
8000 Copy link
Author

I've updated 0.57.2-beta2 via the same method above (replace executable with config untouched).

Result:

2 big upload (bigger than the "split" size), got stuck at the second

Image
Currently I have not met with this problem but will keep watching.

and also it got stuck in case of disconnections

Tested this by changing my server's outgoing proxy node while uploading to simulate disconnection scenario, and current behavior I've seen is that when proxy node is switched (broken connection), the uploading process will automatically restart from the beginning rather than reporting 'FAILED'(in 0.57.0 and 0.57.2-beta1). I think it's great to implement automatic re-uploading when connection is unstable, while a configurable failure-limit may work better (e.g. Uploading-Max-Retries: 5 (1~10) ).

@rejetto
Copy link
Owner
rejetto commented May 10, 2025

if a limit in the retry is useful for some use case, i will introduce it, but i'd like to undestand why.

also, i worked on a mechanism that will resume on disconnection from the point you left.

@physicsdolphin
Copy link
Author

if a limit in the retry is useful for some use case, i will introduce it, but i'd like to undestand why.

also, i worked on a mechanism that will resume on disconnection from the point you left.

The max-retries option maybe useful for users with a not-so-stable connection, especially if it's a metered-connection. Continuous fail-retries during large-file uploading when network isn't stable enough may quickly use up client/server's data quota.

Break-point resuming is definitely great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working fixed-next-release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rejetto @physicsdolphin
0