8000 HADOOP-18516: [Backport to 3.4] [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication by anujmodi2021 · Pull Request #6855 · apache/hadoop · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

HADOOP-18516: [Backport to 3.4] [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication #6855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 7, 2024
Merged

HADOOP-18516: [Backport to 3.4] [ABFS][Authentication] Support Fixed SAS Token for ABFS Authentication #6855

merged 1 commit into from
Jun 7, 2024

Conversation

anujmodi2021
Copy link
Contributor
@anujmodi2021 anujmodi2021 commented May 31, 2024 •
edited
Loading

Description of PR

Jira: https://issues.apache.org/jira/browse/HADOOP-18516
PR on trunk: #6552
Commit cherry-picked: d8b485a

Original PR Description:

This PR introduces a new configuration for Fixed SAS Tokens: "fs.azure.sas.fixed.token"

Using this new configuration, users can configure a fixed SAS Token in the account settings files itself. Ideally, this should be used with SAS Tokens that are scoped at a container or account level (Service or Account SAS), which can be considered to be a constant for one account or container, over multiple operations.

The other method of using a SAS Token remains valid as well, where a user provides a custom implementation of the SASTokenProvider interface, using which a SAS Token are obtained.

When an Account SAS Token is configured as the fixed SAS Token, and it is used, it is ensured that operations are within the scope of the SAS Token.

The code checks for whether the fixed token and the token provider class implementation are configured. In the case of both being set, preference is given to the custom SASTokenProvider implementation. It must be noted that if such an implementation provides a SAS Token which has a lower scope than Account SAS, some filesystem and service level operations might be out of scope and may not succeed.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 18m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 7 new or modified test files.
_ branch-3.4 Compile Tests _
+1 💚 mvninstall 49m 25s branch-3.4 passed
+1 💚 compile 0m 38s branch-3.4 passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 compile 0m 34s branch-3.4 passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚 checkstyle 0m 30s branch-3.4 passed
+1 💚 mvnsite 0m 39s branch-3.4 passed
+1 💚 javadoc 0m 37s branch-3.4 passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 0m 32s branch-3.4 passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚 spotbugs 1m 5s branch-3.4 passed
+1 💚 shadedclient 38m 29s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 28s the patch passed
+1 💚 compile 0m 30s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javac 0m 30s the patch passed
+1 💚 compile 0m 26s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚 javac 0m 26s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 checkstyle 0m 19s the patch passed
+1 💚 mvnsite 0m 29s the patch passed
+1 💚 javadoc 0m 25s the patch passed with JDK Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1
+1 💚 javadoc 0m 24s the patch passed with JDK Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
+1 💚 spotbugs 1m 3s the patch passed
+1 💚 shadedclient 38m 28s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 13s hadoop-azure in the patch passed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
161m 20s
Subsystem Report/Notes
Docker ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6855/1/artifact/out/Dockerfile
GITHUB PR #6855
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
uname Linux 161c5677ec56 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.4 / b5b8992
Default Java Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.22+7-post-Ubuntu-0ubuntu220.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_402-8u402-ga-2ubuntu1~20.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6855/1/testReport/
Max. process+thread count 529 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6855/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@anujmodi2021
Copy link
Contributor Author

:::: AGGREGATED TEST RESULT ::::

============================================================
HNS-OAuth

[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 76
[WARNING] Tests run: 411, Failures: 0, Errors: 0, Skipped: 57

============================================================
HNS-SharedKey

[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 28
[WARNING] Tests run: 411, Failures: 0, Errors: 0, Skipped: 44

============================================================
NonHNS-SharedKey

[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 604, Failures: 0, Errors: 0, Skipped: 268
[WARNING] Tests run: 411, Failures: 0, Errors: 0, Skipped: 47

============================================================
AppendBlob-HNS-OAuth

[WARNING] Tests run: 137, Failures: 0, Errors: 0, Skipped: 2
[WARNING] Tests run: 620, Failures: 0, Errors: 0, Skipped: 78
[WARNING] Tests run: 411, Failures: 0, Errors: 0, Skipped: 81

Time taken: 55 mins 25 secs.

@anujmodi2021 anujmodi2021 marked this pull request as ready for review May 31, 2024 08:01
@anujmodi2021 anujmodi2021 mentioned this pull request May 31, 2024
Merged
4 tasks
@anujmodi2021
Copy link
Contributor Author

@steveloughran @mukund-thakur
This should be good to merge.

Thanks

@steveloughran steveloughran merged commit 93c787b into apache:branch-3.4 Jun 7, 2024
steveloughran
steveloughran reviewed Jun 7, 2024
View reviewed changes
Copy link
Contributor
@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
merged. I like low-conflict backports...its why I want to keep 3.4 and 3.5 in sync

@anujmodi2021 anujmodi2021 deleted the branch_3.4_fixedSAS branch July 17, 2024 04:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@steveloughran steveloughran steveloughran left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@anujmodi2021 @hadoop-yetus @steveloughran
0