Closed
Description
Hi, there was an issue with the message lag when our consumer app was currently having high CPU load on other processes that sometimes take up to 100% for a long period. The consumers stop consuming the message, and the message lag has been keep increasing until we have to restart the app. The consumers in our app are critical and need to be highly prioritized.
Could we make the consumers' thread have the highest priority than the other processes?
Our setups:
- ASP.NET Core 6.0
- DotNetCore.CAP 7.2.0
- Redpanda 24.1.9 as Kafka broker
- PostgreSQL 16
Reproduceable repo: https://github.com/kl-tie/dotnetcore-cap-test
Steps to reproduce:
- Clone the repo.
- Build the Docker image using the script inside
docker-build.bat. - Run Docker Compose file using
docker-compose.yml. - Open http://localhost:8080/topics/transaction?p=-1&s=50&o=-1#consumers for monitoring the message lag.
- Open http://localhost:1234/swagger/index.html and hit
/api/stress-test/cpuendpoint to simulate the high CPU usage. - The app will have 100% CPU utilization and all incoming messages will be stop processed. You can use the included
DotNetCap.Publisherconsole app to produce a message.