Description
I did this
I'm observing that when a URL contains already-encoded characters (e.g. %3A), the canonical request generated by cURL's AWS SigV4 signing (in lib/http_aws_sigv4.c, particularly in the canon_string() function) does not match the expected canonical request, leading to signature mismatches.
For example, when a URL containing %3A is used, the canonical string produced seems to miss the inclusion of the x-amz-security-token header in the signed headers, resulting in a signature error.
I have compared this behavior with awscurl, which handles the encoding correctly. I suspect that the logic in canon_string() (lines XYZ–ABC in lib/vauth/aws_sigv4.c) is not correctly normalizing already-encoded characters.
Could you please review this section and consider a fix so that pre-encoded URL components are handled consistently?
I expected the following
curl "https://s3tables.us-east-1.amazonaws.com:443/tables/arn%3Aaws%3As3tables%3Aus-east-1%3A022954301426%3Abucket%2Fjasoehartablebucket/jasoeharnamespace/jasoehartable/encryption" \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
-H "x-amz-security-token: $AWS_SESSION_TOKEN" \
--aws-sigv4 "aws:amz:us-east-1:s3tables" \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'User-Agent: agent'
Running the above command fails to handle the %3A encoded characters already present in the URL. If I un-encode the URL, the request fails to the service. Encoding it however fails signing.
The error coming back:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'GET
/tables/arn%253Aaws%253As3tables%253Aus-east-1%253A022954301426%253Abucket%252Fjasoehartablebucket/jasoeharnamespace/jasoehartable/encryption
So it looks like it is encoding the '%' again and failing to recognize the '%3A' as a valid hex digit
Example with single quote and unencoded characters
curl 'https://s3tables.us-east-1.amazonaws.com:443/tables/arn:aws:s3tables:us-east-1:022954301426:bucket/jasoehartablebucket/table/0e65f8e0-0d5c-4791-8010-120e0ebf8bfc/jasoeharnamespace/jasoehartable/encryption' \
--user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
-H "x-amz-security-token: $AWS_SESSION_TOKEN" \
--aws-sigv4 "aws:amz:us-east-1:s3tables" \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'User-Agent: agent'
{"message":"Unable to determine service/operation name to be authorized"}
curl/libcurl version
curl 8.13
operating system
Ubuntu 22.04