8000 [BUGFIX] support YAML extension in database configuration by ibakshay · Pull Request #2372 · perses/perses · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[BUGFIX] support YAML extension in database configuration #2372

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 25, 2024
Merged

[BUGFIX] support YAML extension in database configuration #2372

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c8f53fa
[BUGFIX] parse files with extension: "yaml"
ibakshay Oct 24, 2024
274c6a8
[BUGFIX] include project display name in provisioning
ibakshay Oct 24, 2024
6232c10
[ENHANCEMENT] change hardcoded yaml value
ibakshay Oct 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Fail 8000 ed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions dev/data/6-project.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,33 @@
"kind": "Project",
"metadata": {
"name": "perses"
},
"spec": {
"display": {
"name": "perses"
}
Comment on lines +6 to +10
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added spec.display.name since gopkg.in/yaml.v3 and encoding/json marshalers handle nil structs differently. Without this initialization, gopkg.in/yaml.v3 would omit the entire 'spec' field while encoding/json keeps it as an empty spec object.
You can see this behavior here: https://go.dev/play/p/kv0gT5GZt6M

Without a spec object the UI fails to load.
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm it's weird because the UI is getting a JSON not a YAML format so spec should be initialised in the API response.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is happening much before actually. The issue is happening during the provisioning of file database during the application start up.

So, when the config.database.file.extension is "yaml" extension, a project manifest in local_db/projects folder looks like the below snippet.

kind: Project
metadata:
    name: perses
    createdAt: 2024-10-24T18:48:14.174051Z
    updatedAt: 2024-10-24T18:48:14.174051Z
    version: 0

However, when the config.database.file.extension is "json", a project manifest in local_db/projects looks like the below snippet.

{
  "kind": "Project",
  "metadata": {
    "name": "incredibleproject",
    "createdAt": "2024-10-24T18:49:56.921669Z",
    "updatedAt": "2024-10-24T18:49:56.921669Z",
    "version": 0
  },
  "spec": {}
}

The marshalling happens in this line

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is it shouldn't happen actually.

Because when the frontend is requesting the backend for anything, the response will be marshalled in JSON. So whatever the format is in the database, once it is sent to the frontend, the format is JSON.

So probably there is another thing to fix there. But it's not that urgent.

}
},
{
"kind": "Project",
"metadata": {
"name": "IncredibleProject"
},
"spec": {
"display": {
"name": "Incredible Project"
}
}
},
{
"kind": "Project",
"metadata": {
"name": "testing"
},
"spec": {
"display": {
"name": "Testing"
}
}
}
]
20 changes: 19 additions & 1 deletion internal/api/database/file/file.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,25 @@ func (d *DAO) RawQuery(query databaseModel.Query) ([]json.RawMessage, error) {
if readErr != nil {
return nil, readErr
}
result = append(result, data)

// If it's YAML, we need to convert to JSON first
if d.Extension == config.YAMLExtension {
// Parse the YAML content
var yamlData interface{}
if err := yaml.Unmarshal(data, &yamlData); err != nil {
return nil, fmt.Errorf("failed to parse YAML from %s: %w", file, err)
}

// Convert YAML to JSON
jsonData, err := json.Marshal(yamlData)
if err != nil {
return nil, fmt.Errorf("failed to convert to JSON from %s: %w", file, err)
}
result = append(result, json.RawMessage(jsonData))
} else {
// For JSON files, we can append directly
result = append(result, json.RawMessage(data))
}
}
return result, nil
}
Expand Down
Loading
0