[WIP] parquet-mimir rebased main d29961bf x theirs #11421
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Changed error handling for LabelValues and LabelNames in `blocks_store_queryable.go` to distinguish retriable and non-retriable errors. This matches the logic for series calls https://github.com/grafana/mimir/blob/0676ed782eb07d5383093b822683923cb33bdca9/pkg/querier/blocks_store_queryable.go#L781-L788 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Adding some observability on grpc connections: grafana/dskit#685
… runtime config is disabled (#11359) * Fix tenant config mapping when runtime config service is disabled * Fix comment location
* Add ingester HPA trigger targeting max owned series per pod * Update CHANGELOG * make format-jsonnet-manifests * Update CHANGELOG.md Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> --------- Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com>
* Upgrade to gRPC v1.71.1 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Charles Korn <charles.korn@grafana.com>
#11380) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…1378) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…1.79.3 (#11383) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…nants (#11309) * Alertmanager: Make SMTP From configurable for Grafana Alertmanager tenants * update and fix tests * use default globals instead of empty globals * trigger build * make check-protos
…e/azblob to v1.6.1 (#11384) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…to v1.28 (#11386) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
* update otel resource attribute promotion docs * update docs based on feedback
Add a timing histogram for job completions.
* feature(queries): enable duration arithmetic * Add changelog * Enable engine upstream tests * Add about-versioning. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
* ingest: Clarify we expect ordering between samples based on 11286 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Change order in requests Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
this helps with flaky tests related to XXX Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
…11396) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* querier: add support dynamic replication in consistency checks the querier today retries only 3 times each block. This works ok with replication factor=3. But with dynamic replication it feels like we’re giving up too early. There might be other store-gateways that can serve the block, but we don’t even attempt them. This PR changes the blockStoreQuerier to retry until the maximum replication factor. This doesn't add additional overhead for non-replicated blocks because `blocksStoreReplicationSet.GetClientsFor` would return an error after the third retry anyway and we'd short-circuit here Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Add CHANGELOG.md entry Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Use MaxReplicationFactor from DynamicReplication instead of BlocksConsistency(Tracker) Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Update CHANGELOG.md Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> * Fix panic in test Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Fix tests in blocks_store_queryable_test.go Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com>
…lientWhenDone (#11397) Signed-off-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Vladimir Varankin <vladimir.varankin@grafana.com>
* querier: fix NopDynamicReplication.MaxReplicationFactor follow-up of 11354 This method should be returning the full replication factor including the factors from the sharding ring and the dynamic replication config. Without that retries here will not work properly when dynamic replication is off https://github.com/grafana/mimir/blob/1140f7c6191b08a3f9078f6e45af166a1cf57e77/pkg/querier/blocks_store_queryable.go#L571 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Update CHANGELOG.md entry Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Fix TestShuffleShardingStrategy Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Fix TestBlocksStoreReplicationSet_GetClientsFor_ShouldSupportRandomLoadBalancingStrategy Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Fix TestBlocksStoreReplicationSet_GetClientsFor Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
…pportWriteBothToIngestersAndPartitions and TestDistributor_Push_ShouldSupportIngestStorage (#11399) Signed-off-by: Marco Pracucci <marco@pracucci.com>
Enrich alerts for invalid cluster validation labels with an additional label: protocol. Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* Parquet converter Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Co-authored-by: Nico Pazos <nicolas.pazos-mendez@grafana.com> Co-authored-by: Franco Posa <franco.posa@grafana.com> * move readTsdb to parquet storage sub-package; rename method & vars for clarity & testability (#11222) * move readTsdb to parquet storage sub-package; rename method & vars for clarity * add license header * Parquet: Initialize parquet converter module (#11226) * Initialize parquet compactor module Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> * Rename to parquet converter Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> * Implement allowlist * fix reader tests * Update docs * also skip disallowed tenants when converting blocks * fix initialization of allowedTenants * various name changes --------- Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Co-authored-by: Nicolás Pazos <npazosmendez@gmail.com> --------- Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Co-authored-by: Nico Pazos <nicolas.pazos-mendez@grafana.com> Co-authored-by: Franco Posa <franco.posa@grafana.com> Co-authored-by: Franco Posa <franco@francoposa.io> Co-authored-by: Nicolás Pazos <npazosmendez@gmail.com>
* fix lints * fix license header * update parquet-converter intialization to match compactor; introduce extra metrics prefix for bucket index loader to avoid duplicate registration; tweak integration test ring metric values for parquet-converter presence * clean up logger usages
Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>
* add in parquet-common files * fix imports * fix license headers * fix error comparison lints * fix lints * fix go.mod * fix parquet/convert/writer_test.go * quarantine existing queryable in subpackage * add in latest search constraint improvements
|
💻 Deploy preview deleted. |
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does
Which issue(s) this PR fixes or relates to
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX].about-versioning.mdupdated with experimental features.