document how to use Kiali diagnostics for measuring performance #8449
Comments
|
Here's what I think would be very helpful to have. For a given request, like should show this. |
We'd want to also document for the user what this request-id is and, more importantly, how to get one (they would need to look at the logs, find the request-ids somehow and pick an interesting one - one related to the graph generation, for example). So we'd want to document that portion too - it won't be enough just to say "grep for a request-id" because the first question they will ask is, "what request-id do I search for?" One way I am thinking of documenting this is to have them look for kubectl logs -n istio-system deployments/kiali | jq -R 'fromjson? | select(.route == "GraphNamespaces") | .["request-id"]' | sort -uwill output all the request-ids that requested a graph. If the logs are in text (which is our default), then this does the same thing: kubectl logs -n istio-system deployments/kiali | grep 'route=GraphNamespaces' | sed -n 's/.*request-id=\([^ ]*\).*/\1/p' | sort -uThey will return a list like this: Then people are going to ask "what are the different routes I can look at?"... here's how you can get those: JSON: text: That will return a list like this: |
|
Just a note that you can also look at the logs IN Kiali. I think we should encourage users to inspect Kiali from the Kiali workload itself. The logs tab has nice filtering and highlighting. Although, has anyone checked to see if the new structured logging looks decent? |
|
I forgot all about this docs page - we can just add to this rather than create a new one: |
Can we log the actual route like |
Maybe that's what the URL handler is for and we can add that if the log level == trace. We'd probably want to exclude URLs for certain routes like the auth callback handlers if possible. |
Those names are the actual Route names themselves and is how we (devs) can correlate back that log message to the handler, e.g. https://github.com/kiali/kiali/blob/v2.10.0/routing/routes.go#L680 They get set here: https://github.com/kiali/kiali/blob/v2.10.0/routing/router.go#L370 We could add "route-pattern" that logs the Route.Pattern as defined here: https://github.com/kiali/kiali/blob/v2.10.0/routing/routes.go#L24 , e.g. c = c.append(hlog.NewHandler(zerolog.With().Str("route", route.Name).Str("route-pattern", route.Pattern).Logger()))These patterns (some of them anyway) have placeholders, so they will look something like this: I put that in my last PR that is in flight: #8425 I just tested it - things would look like this: Note that we "could" log the actual URL - but the point was made earlier that this might have sensitive information, so we don't really want to log that. I think the Route.Pattern gets the user what he needs for the most part. Maybe we can consider logging the actual URL if, say, the logger has trace level enabled (that doesn't get around the "sensitive information" problem so we probably don't want to do that either). |
We already have it - in the Debugging Kiali page (which is where this PR is adding its stuff).
It's in this Debugging Kiali page.
I'm just adding to this Debugging Kiali page - so these decisions were already made for me :) |
We should have a page on kiali.io that provides information on how a user can use Kiali diagnostics to help figure out performance issues. I'm thinking document things like:
There might be other stuff - comments welcome on what we should have in these docs.
Not sure what the title of this doc page should be or where it should be under kiali.io. Suggestions welcome.
The text was updated successfully, but these errors were encountered: