8000 feat(og): Foward host header by JasonLovesDoggo · Pull Request #370 · TecharoHQ/anubis · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(og): Foward host header #370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Apr 29, 2025
Merged

feat(og): Foward host header #370

merged 13 commits into from
Apr 29, 2025

Conversation

JasonLovesDoggo
Copy link
Member
@JasonLovesDoggo JasonLovesDoggo commented Apr 26, 2025 •
edited by Xe
Loading

also contains the changes of #367 as I did not want to deal with merge conflicts later. Feel free to close #367 if you decide to merge this before that.

Closes #242
Supersedes / Closes #251
Closes #367

Checklist:

  • Added a description of the changes to the [Unreleased] section of docs/docs/CHANGELOG.md
  • Added test cases to the relevant parts of the codebase
  • Ran integration tests npm run test:integration (unsupported on Windows, please use WSL)

@JasonLovesDoggo
feat(ogtags): enhance target URL handling for OGTagCache, support Uni…
3e50b77 
…x sockets

Closes: TecharoHQ#323 TecharoHQ#319
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
docs: update CHANGELOG.md to include Opengraph passthrough support fo…
e1c1d23 
…r Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
docs: update CHANGELOG.md to include Opengraph passthrough support fo…
4249d2f 
…r Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
feat(ogtags): add option to consider host in Open Graph tag cache key
7c0ec7f 
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
feat(ogtags): add option to consider host in OG tag cache key
8d43e39 
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
test(ogtags): enhance tests for OGTagCache with host consideration sc…
0f62035 
…enarios

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo JasonLovesDoggo mentioned this pull request Apr 27, 2025
Closed
@JasonLovesDoggo
Merge branch 'main' into json/oghostheader
b4bf973 
Signed-off-by: Jason Cameron <git@jasoncameron.dev>

# Conflicts:
#	lib/anubis.go
@Xe Xe modified the milestones: v1.17.0: Asahi sas Brutus, v1.18.0: Varis zos Galvus Apr 27, 2025
@SlyEcho
Copy link
Contributor
SlyEcho commented Apr 27, 2025

For some reason, it tries to make a request with HTTPS to the target even if it has only http:// specified there.

{
  "time": "2025-04-27T19:22:50.592896361Z",
  "level": "ERROR",
  "source": {
    "function": "github.com/TecharoHQ/anubis/lib.(*Server).RenderIndex",
    "file": "/src/lib/http.go",
    "line": 59
  },
  "msg": "failed to get OG tags",
  "user_agent": "Mozilla",
  "accept_language": "",
  "priority": "",
  "x-forwarded-for": "80.235.34.138",
  "x-real-ip": "80.235.34.138",
  "err": "http get failed: Get \"https://mdon.ee/@slyecho/114297494827082438\": tls: failed to verify certificate: x509: certificate signed by unknown authority"
}

My Anubis setup:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mdon-ee-anubis
  labels: &labels
    app.kubernetes.io/name: mastodon
    app.kubernetes.io/instance: mdon.ee
    app.kubernetes.io/component: anubis
spec:
  replicas: 1
  selector:
    matchLabels: *labels
  template:
    metadata:
      labels: *labels
    spec:
      enableServiceLinks: false
      containers:
        - name: anubis
          image: cumtr.ee/henri/anubis:pr-370 # this is just built from JasonLovesDoggo:json/oghostheader
          imagePullPolicy: Always
          envFrom:
            - secretRef: {name: mdon-ee-anubis}
          env:
            - name: TARGET
              value: 'http://mdon-ee-web'
            - name: SLOG_LEVEL
              value: DEBUG
            - name: COOKIE-DOMAIN
              value: mdon.ee
            - name: OG_PASSTHROUGH
              value: 'true'
            - name: OG_CACHE_CONSIDER_HOST
              value: 'true'
          ports:
            - name: http
              containerPort: 8923

This is working well for Anubis's own reverse proxy.

@JasonLovesDoggo
refactor(ogtags): extract constants for HTTP timeout and max content …
a07ee0f 
…length

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
refactor(ogtags): restore fetchHTMLDocument method for cache key gene…
3ac5bae 
…ration

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
Copy link
Member Author

For some reason, it tries to make a request with HTTPS to the target even if it has only http:// specified there.

{
  "time": "2025-04-27T19:22:50.592896361Z",
  "level": "ERROR",
  "source": {
    "function": "github.com/TecharoHQ/anubis/lib.(*Server).RenderIndex",
    "file": "/src/lib/http.go",
    "line": 59
  },
  "msg": "failed to get OG tags",
  "user_agent": "Mozilla",
  "accept_language": "",
  "priority": "",
  "x-forwarded-for": "80.235.34.138",
  "x-real-ip": "80.235.34.138",
  "err": "http get failed: Get \"https://mdon.ee/@slyecho/114297494827082438\": tls: failed to verify certificate: x509: certificate signed by unknown authority"
}

My Anubis setup:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mdon-ee-anubis
  labels: &labels
    app.kubernetes.io/name: mastodon
    app.kubernetes.io/instance: mdon.ee
    app.kubernetes.io/component: anubis
spec:
  replicas: 1
  selector:
    matchLabels: *labels
  template:
    metadata:
      labels: *labels
    spec:
      enableServiceLinks: false
      containers:
        - name: anubis
          image: cumtr.ee/henri/anubis:pr-370 # this is just built from JasonLovesDoggo:json/oghostheader
          imagePullPolicy: Always
          envFrom:
            - secretRef: {name: mdon-ee-anubis}
          env:
            - name: TARGET
              value: 'http://mdon-ee-web'
            - name: SLOG_LEVEL
              value: DEBUG
            - name: COOKIE-DOMAIN
              value: mdon.ee
            - name: OG_PASSTHROUGH
              value: 'true'
            - name: OG_CACHE_CONSIDER_HOST
              value: 'true'
          ports:
            - name: http
              containerPort: 8923

This is working well for Anubis's own reverse proxy.

Hmm I wasn't able to replicate using your setup but I added an extra guard which should force the proto to be http. Would you mind building and trying again?

@SlyEcho
Copy link
Contributor
SlyEcho commented Apr 27, 2025

I think I found the issue, Mastodon is redirecting to the HTTPS protocol. This does not happen during normal reverse proxy operation because it also sends the X-Forwarded-Proto: https header.

And actually, I'm not sure hardcoding http:// is the right thing, if some user wants to proxy a TLS-enabled service, Anubis should still try to fetch it.

@JasonLovesDoggo
refactor(ogtags): replace maxContentLength field with constant and en…
680be0f 
…sure HTTP scheme is set correctly

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
fix(fetch): add proxy headers
d3baee4 
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
@JasonLovesDoggo
Merge branch 'main' into json/oghostheader
ba104eb
@SlyEcho
Copy link
Contributor
SlyEcho commented Apr 27, 2025

Yeah, now it seems to work. 👍

@Xe Xe merged commit 4184b42 into TecharoHQ:main Apr 29, 2025
4 checks passed
@JasonLovesDoggo JasonLovesDoggo deleted the json/oghostheader branch April 29, 2025 12:38
@jaredallard
Copy link
Contributor
jaredallard commented May 1, 2025 8000

Thanks for carrying this forward! I totally got caught up with work/life 😄 🙌

@JasonLovesDoggo
Copy link
Member Author

Thanks for carrying this forward! I totally got caught up with work/life 😄 🙌

Yeah no worries! I'm on a warpath to making "Making OpenGraph Great Again" 😂. Your PR was a great starting place

@BrewTestBot BrewTestBot mentioned this pull request May 1, 2025
Merged
JasonLovesDoggo added a commit to JasonLovesDoggo/anubis that referenced this pull request Jun 17, 2025
@JasonLovesDoggo
feat(og): Foward host header (TecharoHQ#370)
1c4c509 
* feat(ogtags): enhance target URL handling for OGTagCache, support Unix sockets

Closes: TecharoHQ#323 TecharoHQ#319
Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* docs: update CHANGELOG.md to include Opengraph passthrough support for Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* docs: update CHANGELOG.md to include Opengraph passthrough support for Unix sockets

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* feat(ogtags): add option to consider host in Open Graph tag cache key

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* feat(ogtags): add option to consider host in OG tag cache key

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* test(ogtags): enhance tests for OGTagCache with host consideration scenarios

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): extract constants for HTTP timeout and max content length

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): restore fetchHTMLDocument method for cache key generation

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(ogtags): replace maxContentLength field with constant and ensure HTTP scheme is set correctly

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* fix(fetch): add proxy headers

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

---------

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Xe Xe Xe approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.17.1: Asahi sas Brutus: Echo 1
Development

Successfully merging this pull request may close these issues.

OG passthrough: does not send correct Host header
4 participants
@JasonLovesDoggo @SlyEcho @jaredallard @Xe
0