Permalink
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
...
- 19 commits
- 84 files changed
- 6 contributors
Commits on Jun 17, 2025
Commits on Jun 19, 2025
-
fix: improve S3 credential handling and chunk processing in ClickHous…
…e sync (#3071) - Retrieve fresh AWS credentials for each part and chunk to prevent expiration during long-running operations - Refactor chunk handling to process individual files instead of wildcards - Changed pushDataToS3 to return array of AvroFile pointers - Updated pushS3DataToClickHouse to process each chunk separately - Add detailed logging for chunk and part processing for better debugging This ensures reliable data loading for large datasets with multiple chunks and parts by preventing credential timeout errors and enabling proper per-file credential refresh.
-
hit error due to ?s in column names looking at clickhouse-go's bind.go, ? can be escaped as `\?` but this doesn't apply when no params passed, in which case we need to not prepend `\` also `$1` can't be escaped so queries which mix user supplied identifiers & params are never safe conclusion: never use params, we already handle escaping strings ourselves in most cases on handling escaping ourselves: cleanup code to call Quote functions more diligently --------- Co-authored-by: Amogh-Bharadwaj <amogh@peerdb.io>
Commits on Jun 20, 2025
-
feat(ui): pull and show column mapping from source (#3074)
a modal shows up when we click on the table name <img width="1895" alt="image" src="https://github.com/user-attachments/assets/95f8d20b-76f0-4742-ae4b-8b11f220b124" />
-
-
Commits on Jun 22, 2025
-
mysql: update checkpoint in catalog once an hour while waiting for re…
…cords (#3070) avoids pause on inactive mirror resuming with invalid binlog errors
Commits on Jun 23, 2025
-
-
-
mysql: more accurate bean counting (#3080)
cdc: only count row events which aren't ignored qrep: meter connection rather than making terrible estimate based on decoded row values based on talking upstream (go-mysql-org/go-mysql#1044) this was the recommended approach
-
chore(deps): update temporalio/ui docker tag to v2.38.3 (#3081)
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [temporalio/ui](https://redirect.github.com/temporalio/ui-server) | patch | `v2.38.0` -> `v2.38.3` | --- ### Release Notes <details> <summary>temporalio/ui-server (temporalio/ui)</summary> ### [`v2.38.3`](https://redirect.github.com/temporalio/ui-server/releases/tag/v2.38.3) [Compare Source](https://redirect.github.com/temporalio/ui-server/compare/v2.38.2...v2.38.3) Release synced from [ui@v2.38.3](https://redirect.github.com/temporalio/ui/releases/tag/v2.38.3) **Full Changelog**: temporalio/ui-server@v2.38.2...v2.38.3 ### [`v2.38.2`](https://redirect.github.com/temporalio/ui-server/releases/tag/v2.38.2) [Compare Source](https://redirect.github.com/temporalio/ui-server/compare/v2.38.1...v2.38.2) Release synced from [ui@v2.38.2](https://redirect.github.com/temporalio/ui/releases/tag/v2.38.2) **Full Changelog**: temporalio/ui-server@v2.38.1...v2.38.2 ### [`v2.38.1`](https://redirect.github.com/temporalio/ui-server/releases/tag/v2.38.1) [Compare Source](https://redirect.github.com/temporalio/ui-server/compare/v2.38.0...v2.38.1) Release synced from [ui@v2.38.1](https://redirect.github.com/temporalio/ui/releases/tag/v2.38.1) **Full Changelog**: temporalio/ui-server@v2.38.0...v2.38.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC42Mi4xIiwidXBkYXRlZEluVmVyIjoiNDAuNjIuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
-
chore(deps): update docker/dockerfile docker tag to v1.17 (#3082)
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [docker/dockerfile](https://redirect.github.com/moby/buildkit) | syntax | minor | `1.16` -> `1.17` | --- ### Configuration 📅 **Schedule**: Branch creation - "after 5pm on monday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/PeerDB-io/peerdb). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC42Mi4xIiwidXBkYXRlZEluVmVyIjoiNDAuNjIuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
-
chore: update dependencies (#3079)
new entries in go.mod are related to listing pgvector testing dependencies, these aren't downloaded in normal operation see golang/go#26955
-
TEMPORAL_CLIENT_CERT_PATH, TEMPORAL_CLIENT_KEY_PATH (#3064)
for when you don't want credentials in your environment variables, or you want to be able to refresh client certs without reconnecting --------- Co-authored-by: Kunal Gupta <39487888+iamKunalGupta@users.noreply.github.com>
Commits on Jun 24, 2025
-
1. quote identifiers in rename 2. don't use sql.Null for non-nullable results
-
introduce mirror versioning, put pgvector support behind version gate (…
…#3066) this avoids breaking existing setups build on pgvector types coming through as strings this system can be used in the future to maintain a chronology of breaking changes & not apply new behavior to old mirrors it's meant to be a single dimension, so there isn't the exponential explosion of feature combinations we have with the dynconf system We've previously tried to leverage by having breaking changes only be introduced as optional features, but this hits two problems: old behavior has to be default, & it's a complicated mechanism designed for flexible configuration, rather than a means to gate backwards incompatible changes from existing mirrors this represents a point in PeerDB's life where we have people who just want the thing they tested (+ security fixes, etc) infra for managing this system will be future work, outlined in #3095
-
Commits on Jun 25, 2025
-
Remove custom column name regex (#3097)
This PR removes the regex constraint on source column names for mirrors, as unusual column names are supported and tested
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff v0.30.3...v0.30.4