8000 Resiliency Policy doesn't print expected output based on tutorial · Issue #75 · dapr/dapr-agents · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Resiliency Policy doesn't print expected output based on tutorial #75

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub 8000

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
kendallroden opened this issue Apr 8, 2025 · 1 comment
Open

Resiliency Policy doesn't print expected output based on tutorial #75

kendallroden opened this issue Apr 8, 2025 · 1 comment
Assignees
@yaron2

Comments

@kendallroden
Copy link
kendallroden commented Apr 8, 2025 •
edited
Loading

I ran the quickstart 02_llm_call_dapr using the README.md instructions.

On the AWS bedrock resiliency test here: https://github.com/dapr/dapr-agents/tree/main/quickstarts/02_llm_call_dapr#3-simulating-failure-with-aws-bedrock, the output did not match the expected output, which should be:

WARN[0002] Error processing operation component[awsbedrock] output. Retrying in 1s...
== APP == 	details = "...exceeded maximum number of attempts, 3, https response error ... connect: connection refused"

You can see the component and resiliency policy are loaded

INFO[0000] Loading Resiliency configuration: awsbedrock-resiliency  app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] standalone mode configured                    app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] app id: dapr-llm                              app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] Dapr trace sampler initialized: ParentBased{root:AlwaysOnSampler,remoteParentSampled:AlwaysOnSampler,remoteParentNotSampled:AlwaysOffSampler,localParentSampled:AlwaysOnSampler,localParentNotSampled:AlwaysOffSampler}  app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] metrics server started on 0.0.0.0:50983/      app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] local service entry announced: dapr-llm -> 192.168.1.65:50984  app_id=dapr-llm component="nr (mdns/v1)" instance=Kendalls-MBP.attlocal.net scope=dapr.contrib type=log ver=1.15.4
INFO[0000] Initialized name resolution to mdns           app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] Loading components…                           app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
INFO[0000] Received updated scheduler hosts addresses: [localhost:50006]  app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime.scheduler.watchhosts type=log ver=1.15.4
INFO[0000] Component loaded: awsbedrock (conversation.aws.bedrock)  app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime.processor type=log ver=1.15.4

You can see the initial warn message but nothing in the app that indicates the 3 retries occurred as laid out in the resiliency policy.

WARN[0003] Error processing operation component[awsbedrock] output. Retrying in 1s…  app_id=dapr-llm instance=Kendalls-MBP.attlocal.net scope=dapr.runtime type=log ver=1.15.4
== APP == An error occurred during the Dapr Conversation API call: <_InactiveRpcError of RPC that terminated with:
== APP ==       status = StatusCode.INTERNAL
== APP ==       details = "failed conversing with component awsbedrock: context deadline exceeded"
== APP ==       debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:50907 {created_time:"2025-04-08T16:52:27.391578-05:00", grpc_status:13, grpc_message:"failed conversing with component awsbedrock: context deadline exceeded"}"
== APP == >
== APP == Traceback (most recent call last):
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/dapr/clients/grpc/client.py", line 1765, in converse_alpha1
== APP ==     response, call = self.retry_policy.run_rpc(self._stub.ConverseAlpha1.with_call, request)
== APP ==                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/dapr/clients/retry.py", line 75, in run_rpc
== APP ==     return func(*args, **kwargs)
== APP ==            ^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 343, in with_call
== APP ==     return self._with_call(
== APP ==            ^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 332, in _with_call
== APP ==     return call.result(), call
== APP ==            ^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 440, in result
== APP ==     raise self
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_interceptor.py", line 315, in continuation
== APP ==     response, call = self._thunk(new_method).with_call(
== APP ==                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1198, in with_call
== APP ==     return _end_unary_response_blocking(state, call, True, None)
== APP ==            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
== APP ==     raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
== APP ==     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP == grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
== APP ==       status = StatusCode.INTERNAL
== APP ==       details = "failed conversing with component awsbedrock: context deadline exceeded"
== APP ==       debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:50907 {created_time:"2025-04-08T16:52:27.391578-05:00", grpc_status:13, grpc_message:"failed conversing with component awsbedrock: context deadline exceeded"}"
== APP == >
== APP == 
== APP == The above exception was the direct cause of the following exception:
== APP == 
== APP == Traceback (most recent call last):
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/02_llm_call_dapr/text_completion.py", line 10, in <module>
== APP ==     response = llm.generate("Name a famous dog!")
== APP ==                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/dapr_agents/llm/dapr/chat.py", line 157, in generate
== APP ==     response = self.client.chat_completion(llm=llm_component or self._llm_component, conversation_inputs=inputs, scrub_pii=scrubPII, temperature=temperature)
== APP ==                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/dapr_agents/llm/dapr/client.py", line 30, in chat_completion
== APP ==     response = self.dapr_client.converse_alpha1(name=llm, inputs=conversation_inputs, scrub_pii=scrub_pii, temperature=temperature)
== APP ==                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
== APP ==   File "/Users/kendallroden/repos/dapr-agents/quickstarts/.venv/lib/python3.12/site-packages/dapr/clients/grpc/client.py", line 1774, in converse_alpha1
== APP ==     raise DaprGrpcError(err) from err
== APP == dapr.clients.exceptions.DaprGrpcError: <_InactiveRpcError of RPC that terminated with:
== APP ==       status = StatusCode.INTERNAL
== APP ==       details = "failed conversing with component awsbedrock: context deadline exceeded"
== APP ==       debug_error_string = "UNKNOWN:Error received from peer ipv4:127.0.0.1:50907 {created_time:"2025-04-08T16:52:27.391578-05:00", grpc_status:13, grpc_message:"failed conversing with component awsbedrock: context deadline exceeded"}"
== APP == >
❌  The App process exited with error code: exit status 1
ℹ️  
terminated signal received: shutting down
✅  Exited Dapr successfully
❌  Error exiting App: exit status 1

When I commented out the timeout or when I changed the timeout to something slower like 5s, I saw this output which was closer to the original suggested output:
Image
@yaron2 yaron2 self-assigned this Apr 9, 2025
@yaron2
Copy link
Member
yaron2 commented Apr 9, 2025

Identified root cause

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yaron2 yaron2

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yaron2 @kendallroden
0