Description
In Istio release 1.23, we were able to see retry logs at the egress gateway pod without needing to adjust the log level. However, in releases 1.24 and 1.25, retry logs are visible when the log level is increased, for instance , to debug level. This change in behavior has impacted our ability to monitor and troubleshoot retry attempts effectively. So, for example , when the attempts was set to 2 , we were able to see the logs of retry( attempts+1) times.
- For Istio 1.23 , we get these logs with the following log message :
NOTE : this is a POST request but we are getting the same for GET as well, according to the number of retries configured in our Virtual Service.
- But for Istio 1.24 or 1.25 , we get these as logs :
No retry logs . Also we have tried this with different status code ( 404, 504 , 5xx ) , but it did not work.
On increasing the log level to debug , we get logs something like this but the log message is also different from the one we get in Istio 1.23.
Is there any change at Istio level for such behavior of logs?
Steps to Reproduce:
- Deploy Istio 1.24 or 1.25.
- Configure a VirtualService with retry policies.
- Generate traffic that triggers retries.
- Observe the logs of the egress gateway pod at the default log level.
- Increase the log level to debug and observe the logs again
Expected Behavior: Retry logs should be visible at the default log level, as they were in Istio 1.23.
Actual Behavior: Retry logs are only visible when the log level is increased to debug.
Please provide any additional information or context that might be relevant to this issue.