Telemetry on WASM runtime #362

dotansimha · 2024-01-24T08:02:45Z

Closes #354

github-actions · 2024-01-24T08:03:09Z

🚨 Rust Panic Audit: 86 Potential Panic Points Detected 🚨

Crate: `federation_query_planner`

📊 Total Usages: 50

🎁 unwrap usages: 29
🚨 panic usages: 3
🔎 expect usages: 8
🔢 array_index usages: 10

Crate: `engine`

📊 Total Usages: 11

🔎 expect usages: 1
🚨 panic usages: 3
🎁 unwrap usages: 7

Crate: `telemetry`

📊 Total Usages: 9

🎁 unwrap usages: 4
🔢 array_index usages: 5

Crate: `cloudflare_worker`

📊 Total Usages: 6

🎁 unwrap usages: 5
🚨 panic usages: 1

Crate: `tracing`

📊 Total Usages: 6

🔎 expect usages: 1
🎁 unwrap usages: 5

Crate: `conductor`

📊 Total Usages: 3

🔎 expect usages: 1
🎁 unwrap usages: 1
🚨 panic usages: 1

Crate: `config`

📊 Total Usages: 1

🚨 panic usages: 1

📌 Expected Annotations

Crate: `jwt_auth`

📊 Total Expected Usages: 1

expand details

Reason: "if initiating an http client fails, then we have to exit."

Code: let client = wasm_polyfills::create_http_client().build().unwrap();
Location: ./plugins/jwt_auth/src/jwks_provider.rs:49

Crate: `napi`

📊 Total Expected Usages: 1

expand details

Reason: "we need this"

Code: panic!("Exited process!")
Location: ./libs/napi/src/lib.rs:18

Crate: `conductor`

📊 Total Expected Usages: 2

expand details

Reason: "we need to exit the process, if the logger can't be correctly set."

Code: tracing::subscriber::set_global_default(subscriber).expect("failed to set up tracing");
Location: ./bin/conductor/src/lib.rs:37

Reason: "we need to exit the process, if the provided configuration file is incorrect."

Code: panic!("Failed to initialize gateway: {:?}", e);
Location: ./bin/conductor/src/lib.rs:75

Crate: `vrl`

📊 Total Expected Usages: 2

expand details

Reason: "if the provided VRL code in the config file can't compile, we have to exit."

Code: panic!("failed to compile vrl program");
Location: ./plugins/vrl/src/plugin.rs:122

Reason: "states is a non-user provided variable"

Code: .expect("can't merge states when states is an empty vector!")
Location: ./plugins/vrl/src/plugin.rs:139

Crate: `common`

📊 Total Expected Usages: 1

expand details

Reason: "we're parsing a statically defined constant, we know it works ;)"

Code: .unwrap()
Location: ./libs/common/src/graphql.rs:23

Crate: `engine`

📊 Total Expected Usages: 5

expand details

Reason: "if we are unable to construct the endpoints and attach them onto the gateway's http server, we have to exit"

Code: Err(e) => panic!("failed to construct endpoint: {:?}", e),
Location: ./libs/engine/src/gateway.rs:145

Reason: "we can safely index here, it's inside a test with constant defined fixtures."

Code: ConductorGateway::execute(request, &gw.routes[0].route_data).await
Location: ./libs/engine/src/gateway.rs:177

Reason: "without a fetcher, there's no executor, without an executor, there's no gateway."

Code: panic!("Failed while initializing the executor's fetcher for Federation source");
Location: ./libs/engine/src/source/federation_source.rs:109

Reason: "without a fetcher, there's no executor, without an executor, there's no gateway."

Code: panic!("Failed while initializing the executor's fetcher for Federation source");
Location: ./libs/engine/src/source/federation_source.rs:140

Reason: "without a fetcher, there's no executor, without an executor, there's no gateway."

Code: panic!(
Location: ./libs/engine/src/source/graphql_source.rs:30

Crate: `disable_introspection`

📊 Total Expected Usages: 1

expand details

Reason: "we need to exit the process if our provided VRL condition has incorrect syntax."

Code: panic!("failed to compile vrl program for disable_introspection plugin");
Location: ./plugins/disable_introspection/src/plugin.rs:33

Crate: `cloudflare_worker`

📊 Total Expected Usages: 4

expand details

Reason: "it panics only if the header name is not valid, and we know it is."

Code: .unwrap()
Location: ./bin/cloudflare_worker/src/http_tracing.rs:20

Reason: "it panics only if the URL source is not valid, and it's already validated before."

Code: let url = req.url().unwrap();
Location: ./bin/cloudflare_worker/src/http_tracing.rs:23

Reason: "it only panics if we are not running in a CF context, should be safe."

Code: let cf_info = req.cf().unwrap();
Location: ./bin/cloudflare_worker/src/http_tracing.rs:27

Reason: "unwraps only in special cases where "data:text" is used."

Code: let http_host = url.host().unwrap().to_string();
Location: ./bin/cloudflare_worker/src/http_tracing.rs:36

Crate: `config`

📊 Total Expected Usages: 9

expand details

Reason: "statically defined regex pattern, we know it works ;)"

Code: .unwrap();
Location: ./libs/config/src/interpolate.rs:18

Reason: "👇"

Code: let raw_contents = read_to_string(file_path)
Location: ./libs/config/src/lib.rs:576

Reason: "👇"

Code: panic!("Failed to interpolate config file, please resolve the above errors");
Location: ./libs/config/src/lib.rs:608

Reason: "👇"

Code: parse_config_from_json(&config_string).expect("Failed to parse JSON config file")
Location: ./libs/config/src/lib.rs:615

Reason: "👇"

Code: parse_config_from_yaml(&config_string).expect("Failed to parse YAML config file")
Location: ./libs/config/src/lib.rs:619

Reason: "👇"

Code: _ => panic!("Unsupported config file extension"),
Location: ./libs/config/src/lib.rs:636

Reason: "👇"

Code: None => panic!("Config file has no extension"),
Location: ./libs/config/src/lib.rs:639

Reason: "part of development docgen CLI"

Code: .expect("Failed to serialize json schema for config file!");
Location: ./libs/config/src/generate-json-schema.rs:50

Reason: "part of development docgen CLI"

Code: .expect("Failed to write the json schema to the file system!");
Location: ./libs/config/src/generate-json-schema.rs:54

github-actions · 2024-01-24T08:11:16Z

🐋 This PR was built and pushed to the following Docker images:

Docker Bake metadata

{
"conductor": {
  "containerimage.config.digest": "sha256:0e71789f2cca121c48dc7451f85aa9ae02ca6abb90d5b34268b63558d2bef1d2",
  "containerimage.descriptor": {
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "digest": "sha256:547e096eda6499fd5129498770012bc070fd8844d1ca09f1583f9850ade57ddc",
    "size": 902,
    "platform": {
      "architecture": "amd64",
      "os": "linux"
    }
  },
  "containerimage.digest": "sha256:547e096eda6499fd5129498770012bc070fd8844d1ca09f1583f9850ade57ddc",
  "image.name": "ghcr.io/the-guild-org/conductor/conductor:047a8dd54b5da7f15fabebaa42f95322ac7f3e04"
}
}

github-actions · 2024-01-24T08:11:51Z

✅ Benchmark Results

     data_received..................: 13 MB   221 kB/s
     data_sent......................: 22 MB   363 kB/s
     http_req_blocked...............: min=1.07µs   avg=3.03µs   med=2.34µs   max=821.51µs p(95)=3.41µs   p(99)=5.91µs  
     http_req_connecting............: min=0s       avg=355ns    med=0s       max=456.76µs p(95)=0s       p(99)=0s      
     http_req_duration..............: min=311.93µs avg=411.7µs  med=386.42µs max=18.26ms  p(95)=486.77µs p(99)=578.83µs
       { expected_response:true }...: min=311.93µs avg=411.7µs  med=386.42µs max=18.26ms  p(95)=486.77µs p(99)=578.83µs
     ✓ { scenario:rps_1000 }........: min=311.93µs avg=411.7µs  med=386.42µs max=18.26ms  p(95)=486.77µs p(99)=578.83µs
     http_req_failed................: 0.00%   ✓ 0           ✗ 60001
     ✓ { scenario:rps_1000 }........: 0.00%   ✓ 0           ✗ 60001
     http_req_receiving.............: min=9.72µs   avg=26.86µs  med=25.9µs   max=1.71ms   p(95)=36.64µs  p(99)=47.1µs  
     http_req_sending...............: min=6.75µs   avg=15.62µs  med=14.48µs  max=1.87ms   p(95)=22.7µs   p(99)=32.48µs 
     http_req_tls_handshaking.......: min=0s       avg=0s       med=0s       max=0s       p(95)=0s       p(99)=0s      
     http_req_waiting...............: min=279.14µs avg=369.21µs med=344.77µs max=18.21ms  p(95)=442.7µs  p(99)=526.87µs
     http_reqs......................: 60001   1000.003642/s
     ✓ { scenario:rps_1000 }........: 60001   1000.003642/s
     iteration_duration.............: min=398.61µs avg=505.57µs med=478.59µs max=18.58ms  p(95)=586.17µs p(99)=748.36µs
     iterations.....................: 60001   1000.003642/s
     ✓ { scenario:rps_1000 }........: 60001   1000.003642/s
     valid_graphql_response.........: 100.00% ✓ 60001       ✗ 0    
     ✓ { scenario:rps_1000 }........: 100.00% ✓ 60001       ✗ 0    
     valid_http_code................: 100.00% ✓ 60001       ✗ 0    
     ✓ { scenario:rps_1000 }........: 100.00% ✓ 60001       ✗ 0    
     vus............................: 0       min=0         max=2  
     vus_max........................: 200     min=200       max=200

github-actions · 2024-02-09T14:04:03Z

💻 Website Preview

The latest changes are available as preview in: https://724b11a7.conductor-t2.pages.dev

ok ok ok ok almost there getting there ok maybe like this ok ok almost there cleanup ok wip ok ok ok ok ok ok datadog works ok ok apply fixes deps cleanup fix tests getting there ok it works ok

dotansimha force-pushed the tracing-wasm branch 4 times, most recently from 5a55058 to 7e4114c Compare January 24, 2024 12:09

dotansimha force-pushed the tracing-wasm branch 9 times, most recently from b0902bb to 880dc3d Compare February 9, 2024 14:02

dotansimha force-pushed the tracing-wasm branch from 880dc3d to b1b1e45 Compare February 9, 2024 14:04

Telemetry on WASM runtime

123d44a

ok ok ok ok almost there getting there ok maybe like this ok ok almost there cleanup ok wip ok ok ok ok ok ok datadog works ok ok apply fixes deps cleanup fix tests getting there ok it works ok

dotansimha force-pushed the tracing-wasm branch from b1b1e45 to 123d44a Compare February 11, 2024 07:57

dotansimha added 3 commits February 11, 2024 10:46

ok that's better

d1d43bb

fix

9d9ca5e

fixes

047a8dd

dotansimha marked this pull request as ready for review February 11, 2024 09:31

dotansimha merged commit 61b99c8 into master Feb 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Telemetry on WASM runtime #362

Telemetry on WASM runtime #362

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Telemetry on WASM runtime #362

Telemetry on WASM runtime #362

Uh oh!

Conversation

Uh oh!

Uh oh!

Uh oh!

🚨 Rust Panic Audit: 86 Potential Panic Points Detected 🚨

Crate: federation_query_planner

Crate: engine

Crate: telemetry

Crate: cloudflare_worker

Crate: tracing

Crate: conductor

Crate: config

📌 Expected Annotations

Crate: jwt_auth

Crate: napi

Crate: conductor

Crate: vrl

Crate: common

Crate: engine

Crate: disable_introspection

Crate: cloudflare_worker

Crate: config

Uh oh!

Uh oh!

Uh oh!

Uh oh!

✅ Benchmark Results

Uh oh!

Uh oh!

💻 Website Preview

Uh oh!

Uh oh!

Crate: `federation_query_planner`

Crate: `engine`

Crate: `telemetry`

Crate: `cloudflare_worker`

Crate: `tracing`

Crate: `conductor`

Crate: `config`

Crate: `jwt_auth`

Crate: `napi`

Crate: `conductor`

Crate: `vrl`

Crate: `common`

Crate: `engine`

Crate: `disable_introspection`

Crate: `cloudflare_worker`

Crate: `config`