Dartion is a CLI server for running the Danbooru Tags Tranformer models. It is implemented in Rust + Candle thanks to dartrs.
$ dartion --model moe-sft-v2
Loading model: p1atdev/dart-v2-moe-sft
Using CPU (CUDA feature not enabled)
Model loaded. Starting stdio server...
Listening on stdin...
{"rating": "general", "aspectRatio": "tall", "length": "long", "identity": "free", "character": "hatsune miku", "copyright": "vocaloid", "prompt": "blue hair, smiling"}
Received request: GenerationRequest { copyright: Some("vocaloid"), character: Some("hatsune miku"), rating: RequestRatingTag("general"), aspect_ratio: RequestAspectRatioTag("tall"), length: RequestLengthTag("long"), identity: Some("free"), prompt: "blue hair, smiling", do_completion: None, max_length: None, temperature: None, top_k: None }
blue nails, blue necktie, bridge, cloud, cloudy sky, day, dog, dutch angle, earrings, fence, from below, green eyes, hair ornament, hand in pocket, japanese clothes, jewelry, kimono, kimono skirt, looking at another, multicolored hair, necktie, outdoors, pet walking, plant, skirt, sky, smile, summer, thighhighs, tree, twintails, zettai ryouiki
Successfully generated and sent response in 1.3051377s.
^C
$ dartion -h
Dartion is a CLI server for running the Danbooru Tags Tranformer models
Usage: dartion.exe [OPTIONS] --model <MODEL>
Options:
-m, --model <MODEL> [possible values: moe-sft-v2, moe-base-v2, sft-v2, base-v2, sft-v1, base-v1]
--dtype <DTYPE> [default: fp32] [possible values: bf16, fp16, fp32]
--max-length <MAX_LENGTH> Max generation length (default 256)
--temperature <TEMPERATURE> Temperature (default 1.0)
--top-k <TOP_K> Top-k sampling (default 100)
-h, --help Print help
-V, --version Print versionTo run with CUDA, compile with the cuda feature and pass the --cuda flag. Use --device-id to specify your CUDA device ID.