Description
Description
When using aria2c, the User-Agent header behaves inconsistently depending on how it is specified:
- If
User-Agentis set via aria2c's configuration file (e.g.,aria2.conf), it is placed as the first header in the HTTP request. - If
User-Agentis set via the--headerflag, it is placed as the last header, which is understandable and aligns with user expectations.
This discrepancy causes issues with certain servers, such as those protected by Cloudflare, which enforce strict validation based on header order.
Expected Behavior
The User-Agent header should be inserted in a well-defined position, consistent with standard browser behavior (see Additional Context: Header Order in Modern Browsers below).
Observed Behavior
The User-Agent header is inserted:
- First when specified via
aria2.conf. - Last when specified via
--header.
This inconsistency can lead to problems on servers that validate requests based on header order.
Steps to Reproduce
- Create a minimal
aria2.conffile:user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 - Download a file and log the HTTP headers:
aria2c --console-log-level=debug "https://ordoiuris.pl/sites/default/files/inline-files/Edukacja_seksualna_kontra_prawa_rodzicow_0.pdf" - Compare this with a download using
--header:aria2c --header="User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36" --console-log-level=debug "https://ordoiuris.pl/sites/default/files/inline-files/Edukacja_seksualna_kontra_prawa_rodzicow_0.pdf"
- Observe that the
User-Agentheader appears in different positions.
Environment
- aria2 version: v1.37.0
- Operating System: Microsoft Windows 10 version 22h2
Suggested Fix
Standardize the header order so that User-Agent is treated consistently, even if specified via aria2.conf. It should not be placed as the first header field by default.
Additional Context: Header Order in Modern Browsers
Modern browsers, such as Chrome, Firefox, and even legacy tools like FlashGet, send HTTP headers in a consistent order, typically positioning User-Agent after headers like Accept and Host. This predictable behavior aligns with the expectations of many servers, including those protected by middleware like Cloudflare. Any deviation from this convention, such as placing User-Agent as the first header, can lead to unexpected rejections despite adherence to HTTP specifications.