8000 feat(schema): add backend side migrations for query schema by thmsobrmlr · Pull Request #31554 · PostHog/posthog · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat(schema): add backend side migrations for query schema #31554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 64 commits into from
Jun 12, 2025

Conversation

thmsobrmlr
Copy link
Contributor
@thmsobrmlr thmsobrmlr commented Apr 24, 2025

Problem

https://github.com/PostHog/meta/blob/query-migrations/requests-for-comments/2025-02-25-query-migrations.md

Changes

posthog/
└── schema_migrations/
    ├── __init__.py           # auto-discovery of MIGRATIONS and LATEST_VERSIONS
    ├── base.py               # migration base class
    ├── upgrade.py            # upgrade method that recursively upgrades queries
    ├── upgrade_manager.py    # upgrade_insight context to upgrade insights in place
    ├── validate.py           # validate linear migrations
    ├── 0001_retention_query_show_mean.py # concrete migrations
  • adds a version field to each query node
  • implements the core migration engine, i.e.:
    • an abstract SchemaMigration class, that bumps the version of a node and applies the upgrade given by the migration subclass
    • auto-registry of migrations and the latest version for each node kind
    • an upgrade method that recursively upgrades a query
    • an context wrapper for this upgrade method that upgrades insights in place
  • upgrades queries in all required places:
    • replaces the existing conversion_to_query_based context manager with the new context manager that also upgrade queries. this should cover all places where we read an insight's query.
    • in the insight serializer (to_representation).
    • in the legacy calculate_trends_hogql and calculate_funnel_hogql endpoints.
    • in query and query_awaited endpoints and process_query_dict.
  • adds a sample migration for retentionFilter.showMean -> retentionFilter.meanRetentionCalculation
  • validates that migrations are linear

How did you test this code?

Added tests and manual testing

@PostHog PostHog deleted a comment from posthog-bot May 2, 2025
@thmsobrmlr thmsobrmlr removed the stale label May 2, 2025
# Conflicts:
#	frontend/src/queries/schema.json
#	posthog/schema.py
@thmsobrmlr thmsobrmlr force-pushed the query-schema-migrations branch from 95dce02 to 655b0c1 Compare May 8, 2025 07:53
Copy link
Contributor
@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Major UI and feature enhancements across file system management and navigation, along with infrastructure improvements for project organization and quota management.

  • Added new MoveTo and PinnedFolder components with complete file system management capabilities in frontend/src/lib/components/MoveTo/MoveTo.tsx and related files
  • Introduced quota limiting improvements in ee/billing/quota_limiting.py with region-specific monitoring and better error handling
  • Enhanced session recording analysis in ee/session_recordings/session_summary/ with granular error categorization (abandonment, confusion, exceptions)
  • Implemented pre-aggregated web analytics with new Dagster assets in dags/web_preaggregated_internal.py for efficient data processing
  • Refactored project tree navigation with new protocol-based file system (project://) and expanded panel types in frontend/src/layout/panel-layout/

113 files reviewed, 1 comment
Edit PR Review Bot Settings | Greptile

@thmsobrmlr thmsobrmlr removed the stale label Jun 10, 2025
# Conflicts:
#	posthog/hogql_queries/legacy_compatibility/filter_to_query.py
#	posthog/schema.py
#	posthog/tasks/exports/image_exporter.py
@PostHog PostHog deleted a comment from posthog-bot Jun 10, 2025
@PostHog PostHog deleted a comment from posthog-bot Jun 10, 2025
@thmsobrmlr thmsobrmlr enabled auto-merge (squash) June 11, 2025 10:14
@thmsobrmlr thmsobrmlr disabled auto-merge June 11, 2025 15:16
@thmsobrmlr thmsobrmlr merged commit ea32b2a into master Jun 12, 2025
98 checks passed
@thmsobrmlr thmsobrmlr deleted the query-schema-migrations branch June 12, 2025 06:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0