This Rust project implements a gRPC server for interacting with the Mighty Inference Server. Currently, the Mighty Inference services are only available through its REST API interface, so the communication is limited to a REST client.
This library, however, provides a consistent interface which could be leveraged to make direct binary calls into the library while maintaining the same gRPC
server interface. Presently, the BinaryClient implementation has not been completed as direct binary calls are not yet supported.
-
Clone the repository:
git clone https://github.com/mbarth/mighty-grpc.git
-
Change to the project directory:
cd mighty-grpc -
Build the project:
cargo build --release
-
Start the Mighty Inference Server in a separate terminal. Note the port number used as you'll update your configuration using this port in the next step.
-
The application settings are defined in a configuration file named
config.toml. These settings include the gRPC server address and port, Mighty server configuration, and logging level.Example
config.tomlfile:[grpc_server] address = "127.0.0.1" port = 50051 [mighty_server] base_url = "http://localhost:5050" [logging] level = "debug"
Update these values to match your environment in terms of available ports for the gRPC server and the URL used to access the Mighty server.
-
Start the gRPC server in another terminal using:
cargo run --release --bin grpc
-
Access the gRPC server through your preferred client. Here is an example using gRPCurl:
# Server supports reflection grpcurl -plaintext localhost:50051 describe mighty_inference_server.MightyInference # Using proto sources grpcurl -plaintext -import-path ./src/proto -proto mighty_inference.proto -d '{}' localhost:50051 mighty_inference_server.MightyInference.Metadata # Using reflection grpcurl -plaintext -d '{}' localhost:50051 mighty_inference_server.MightyInference.HealthCheck
- There are two different client implementations examples available:
graph TD;
subgraph Clients
A[JavaScript Client] --> B[gRPC Server]
C[Python Client] --> B[gRPC Server]
E[Other Proto Client] --> B[gRPC Server]
end
B[gRPC Server] --> |REST Client| D[Mighty Inference Server]
B[gRPC Server] -.-> |Direct Binary Calls<br>*Not Implemented*| D[Mighty Inference Server]
style B fill:#ffcccc,stroke:#333,stroke-width:2px,color:#000;
style D fill:#cce5ff,stroke:#333,stroke-width:2px,color:#000;
style A fill:#e6fff7,stroke:#333,stroke-width:1px,color:#000;
style C fill:#e6fff7,stroke:#333,stroke-width:1px,color:#000;
style E fill:#e6fff7,stroke:#333,stroke-width:1px,color:#000;
This project is licensed under the Unlicense. See the LICENSE file for details.