8000 [serve] Remove hardcoded urls from serve microbenchmarks by akyang-anyscale · Pull Request #54026 · ray-project/ray · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[serve] Remove hardcoded urls from serve microbenchmarks #54026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jun 24, 2025
Merged

[serve] Remove hardcoded urls from serve microbenchmarks #54026

merged 5 commits into from
Jun 24, 2025

Conversation

akyang-anyscale
Copy link
Contributor
@akyang-anyscale akyang-anyscale commented Jun 24, 2025 •
edited
Loading

Why are these changes needed?

We should migrate things to use get_application_url to get the target url.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@akyang-anyscale
use get_application_url
46ea359 
Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
@akyang-anyscale
fix
c374aab 
Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
@akyang-anyscale
fix
f3a60ca 
Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
@akyang-anyscale
lint
4efd3c7 
Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
@akyang-anyscale
Copy link
Contributor Author
akyang-anyscale commented Jun 24, 2025 •
edited
Loading

results:

Perf metrics:
 [
    {
        "perf_metric_name": "http_p50_latency",
        "perf_metric_value": 3.9298470000019847,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_p90_latency",
        "perf_metric_value": 4.165507100000809,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_p95_latency",
        "perf_metric_value": 4.2539253000029476,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_p99_latency",
        "perf_metric_value": 7.885859510001083,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_1mb_p50_latency",
        "perf_metric_value": 4.410359999994284,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_1mb_p90_latency",
        "perf_metric_value": 4.722198399990418,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_1mb_p95_latency",
        "perf_metric_value": 4.809983050000709,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_1mb_p99_latency",
        "perf_metric_value": 9.024998560003183,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_10mb_p50_latency",
        "perf_metric_value": 9.679017000003398,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_10mb_p90_latency",
        "perf_metric_value": 11.08750160000369,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_10mb_p95_latency",
        "perf_metric_value": 11.374391650000602,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_10mb_p99_latency",
        "perf_metric_value": 15.059007039998983,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_avg_rps",
        "perf_metric_value": 315.46,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_throughput_std",
        "perf_metric_value": 44.57,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_100_max_ongoing_requests_avg_rps",
        "perf_metric_value": 406.86,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_100_max_ongoing_requests_throughput_std",
        "perf_metric_value": 82.17,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_streaming_avg_tps",
        "perf_metric_value": 32968.52,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_streaming_throughput_std",
        "perf_metric_value": 456.8,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_streaming_p50_latency",
        "perf_metric_value": 14845.410244999983,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_streaming_p90_latency",
        "perf_metric_value": 15115.612243100031,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_streaming_p95_latency",
        "perf_metric_value": 15198.525187900063,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_streaming_p99_latency",
        "perf_metric_value": 15300.549365920024,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_avg_tps",
        "perf_metric_value": 3053.35,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_throughput_std",
        "perf_metric_value": 13.61,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_p50_latency",
        "perf_metric_value": 49007.43956999997,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_p90_latency",
        "perf_metric_value": 49364.39536730004,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_p95_latency",
        "perf_metric_value": 49439.29105454996,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "http_intermediate_streaming_p99_latency",
        "perf_metric_value": 49476.66284349011,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_p50_latency",
        "perf_metric_value": 2.993550000041978,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_p90_latency",
        "perf_metric_value": 3.1493057999341545,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_p95_latency",
        "perf_metric_value": 3.2038342999499037,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_p99_latency",
        "perf_metric_value": 3.4646879000274557,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_1mb_p50_latency",
        "perf_metric_value": 8.693679999964843,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_1mb_p90_latency",
        "perf_metric_value": 9.150017400065735,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_1mb_p95_latency",
        "perf_metric_value": 9.340735599857908,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_1mb_p99_latency",
        "perf_metric_value": 10.199990840021655,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_10mb_p50_latency",
        "perf_metric_value": 56.29447349997463,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_10mb_p90_latency",
        "perf_metric_value": 57.739734899951145,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_10mb_p95_latency",
        "perf_metric_value": 58.34309029985433,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_10mb_p99_latency",
        "perf_metric_value": 59.538128699964545,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "grpc_avg_rps",
        "perf_metric_value": 492.64,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "grpc_throughput_std",
        "perf_metric_value": 6.83,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "grpc_100_max_ongoing_requests_avg_rps",
        "perf_metric_value": 530.07,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "grpc_100_max_ongoing_requests_throughput_std",
        "perf_metric_value": 5.58,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_p50_latency",
        "perf_metric_value": 1.7584859999715263,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_p90_latency",
        "perf_metric_value": 1.8979340999749184,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_p95_latency",
        "perf_metric_value": 2.0001751500331006,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_p99_latency",
        "perf_metric_value": 2.2818287501422674,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_1mb_p50_latency",
        "perf_metric_value": 3.0682545000217942,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_1mb_p90_latency",
        "perf_metric_value": 3.2987369999318616,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_1mb_p95_latency",
        "perf_metric_value": 3.392758350025815,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_1mb_p99_latency",
        "perf_metric_value": 3.8739288000374454,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_10mb_p50_latency",
        "perf_metric_value": 12.469309000039175,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_10mb_p90_latency",
        "perf_metric_value": 13.038207599993257,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_10mb_p95_latency",
        "perf_metric_value": 13.584721099971373,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_10mb_p99_latency",
        "perf_metric_value": 19.828941809919343,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_avg_rps",
        "perf_metric_value": 545.59,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_throughput_std",
        "perf_metric_value": 15.54,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_100_max_ongoing_requests_avg_rps",
        "perf_metric_value": 931.82,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_100_max_ongoing_requests_throughput_std",
        "perf_metric_value": 13.7,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_streaming_avg_tps",
        "perf_metric_value": 7009.08,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_streaming_throughput_std",
        "perf_metric_value": 34.34,
        "perf_metric_type": "THROUGHPUT"
    },
    {
        "perf_metric_name": "handle_streaming_p50_latency",
        "perf_metric_value": 21357.04021799984,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_streaming_p90_latency",
        "perf_metric_value": 21464.601222200297,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_streaming_p95_latency",
        "perf_metric_value": 21648.01590624968,
        "perf_metric_type": "LATENCY"
    },
    {
        "perf_metric_name": "handle_streaming_p99_latency",
        "perf_metric_value": 21658.858575770086,
        "perf_metric_type": "LATENCY"
    }
]

@akyang-anyscale
slash
fb23ce2 
Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
@akyang-anyscale akyang-anyscale added the go add ONLY when ready to merge, run all tests label Jun 24, 2025
@zcin zcin merged commit 9eb1dbf into ray-project:master Jun 24, 2025
5 of 6 checks passed
minerharry pushed a commit to minerharry/ray that referenced this pull request Jun 27, 2025
@akyang-anyscale @minerharry
[serve] Remove hardcoded urls from serve microbenchmarks (ray-project…
8de79fe 
…#54026)

## Why are these changes needed?

We should migrate things to use `get_application_url` to get the target
url.



---------

Signed-off-by: akyang-anyscale <alexyang@anyscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zcin zcin zcin approved these changes

@abrarsheikh abrarsheikh Awaiting requested review from abrarsheikh

Assignees
No one assigned
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@akyang-anyscale @zcin
0