Targon (Bittensor Subnet 4) is a deterministic verification mechanism that is used to incentivize miners to run openai compliant endpoints and serve synthetic and organic queries.
NOTICE: Using this software, you must agree to the Terms and Agreements provided in the terms and conditions document. By downloading and running this software, you implicitly agree to these terms and conditions.
- Compute Requirements
- Installation
- How to Run Targon
- What is a Deterministic Verification Network?
- Features of Targon
- How to Contribute
The following table shows the suggested compute providers for running a validator or miner.
| Provider | Cost | Location | Machine Type | Rating |
|---|---|---|---|---|
| TensorDock | Low | Global | VM & Container | 4/5 |
| Latitude | Medium | Global | Bare Metal | 5/5 |
| Paperspace | High | Global | VM & Bare Metal | 4.5/5 |
| GCP | High | Global | VM & Bare Metal | 3/5 |
| Azure | High | Global | VM & Bare Metal | 3/5 |
| AWS | High | Global | VM & Bare Metal | 3/5 |
| Runpod | Low | Global | VM & Container | 5/5 |
- VRAM: 80 GB
- Storage: 200 GB
- RAM: 16 GB
- CPU: 4
In order to run Targon, you will need to install PM2 and the Targon package. The following instructions apply only to Ubuntu OSes. For your specific OS, please refer to the official documentation.
To install or update nvm, you should run the install script. To do that, you may either download and run the script manually, or use the following cURL or Wget command:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bashRunning either of the above commands downloads a script and runs it. The script
clones the nvm repository to /.nvm, and attempts to add the source lines from
the snippet below to the correct profile file (/.bash_profile, ~/.zshrc,
~/.profile, or ~/.bashrc).
export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvmnvm install nodenpm install pm2@latest -gYou have now installed PM2.
git clone https://github.com/manifold-inc/targon.git
cd targonpython3 -m pip install -e .You have now installed Targon. You can now run a validator or a miner.
In order to run Targon, you must have the manifold vLLM fork instance that includes powv tokens up and running. The easiest way is to run the vllm fork via docker. It is recommended to add an environment variable for the api key.
NOTE: The api key passed to vllm is your made up api key that you will also pass to your miner/validator. This ensures that no one else can access your vllm instance
NOTE Some features like --tensor-parallel-size might not work without manual changes to vllm in coordination with powv. We suggest validators run the fork with minimal to no extra paramaters.
Install docker and run
docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "VLLM_API_KEY=<make up your own>" \
-p 8000:8000 \
--ipc=host \
manifoldlabs/vllm-openai:powv \
--model NousResearch/Meta-Llama-3.1-8B-Instructhttps://hub.docker.com/r/manifoldlabs/vllm-openai
Running without docker currently requires a dev installation of the vllm fork.
git clone https://github.com/manifold-inc/vllm-powv.git
cd vllm-powv
pip install jsonschema
# If this fails, you may need to set the path for cuda. This may be different per
# system.
#
# CUDACXX=/usr/local/cuda-12/bin/nvcc pip install -e .
pip install -e .Now you are ready to server your VLLM instance to PM2
pm2 start vllm --name vllm-serve --interpreter python3 -- serve NousResearch/Meta-Llama-3.1-8B-Instruct --dtype auto --api-key [some-secret-you-also-pass-to-validator] --port 8000The --neuron.model_endpoint for miner / vali using this vllm instance on the
same machine would be http://localhost:8000/v1. Make sure to include the
/v1 to the end of the URL.
Also note again that --api-key is defined by you. Set it to something hard
to guess, and probably random. Whatever you decide, pass it in to both
--api-key in vllm, and --neuron.api_key on the miner / vali
Running a miner through PM2 will require the vLLM instance to be running.
pm2 start neurons/miner.py --name miner --interperter python3 -- --wallet.name [WALLET_NAME] --netuid 4 --wallet.hotkey [WALLET_HOTKEY] --subtensor.network finney --neuron.model_endpoint [MODEL_ENDPOINT] --neuron.api_key [NEURON_API_KEY] --axon.port [AXON_PORT] --logging.tracePlease replace the following with your specific configuration:
- [WALLET_NAME]
- [WALLET_HOTKEY]
- [MODEL_ENDPOINT]
- [NEURON_API_KEY]
- [AXON_PORT]
NOTE: Trace logging is very verbose. You can use --logging.info instead for
less log bloat.
Additionally:
--blacklist.force_validator_permit [TRUE/FALSE]
is defaulted to true to force incoming requests to have a permit.
Validators are simply run through pm2, enabling auto restarts and auto updates.
No vllm instance needed
Validator Instance:
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME]
Please replace the following with your specific configuration:
- [WALLET_NAME]
The goal of the hub is to give validators a simple way to directly generate revenue off of their bittensor bandwidth. This is designed as a template for validators to take and create their own branded hubs with, however pull requests are still encouraged.
If you are interested in running your own instance of Targon Hub, you will need to add an additional flag to save the records of miners' responses to a PostgreSQL DB.
NOTE: No flag means no database!
--database.url [DB_CONNECTION_STRING]
Please replace the following with your specific connection URL:
- [DB_CONNECTION_STRING]
Below are steps to create a Supabase connection string to utilze this feature:
- Either create an account or log in to Supabase
- You might be asked to create an organization. In which case, choose the options best suited for your use case.
- Once completed, create a new project with a secure password and location of your choosing. Save your password, you will need it later. Your project will then take a few minutes to be provisioned.
- Once the project has been created, click on the green
Connectbutton near the top right of the screen - A modal should open up. Click on connection string, URI, and change the mode
from
transactiontosessionin the dropdown - Copy the connection string shown and insert your password
- Clone Targon Hub and follow its setup instructions
- Launch the validator with new
--database.urlflag and connection string
Please reach out to the SN4 team for help setting up targon hub in sn4 chat or our discord
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --database.url [DB_CONNECTION_STRING]As your validator runs, you will start seeing records being added into your Supabase database. This will be directly what your Targon Hub will query.
Autoupdate is implemented in targon/utils.py. This is to ensure that your codebase matches the latest version on Main of the Targon Github Repository.
Validator Autoupdate is implemented and defaulted to run once weights have been set. To disable, please add the flag to your command line build:
pm2 start neurons/validator.py --name validator --interperter python3 -- --wallet.name [WALLET_NAME] --autoupdate_offMiner Autoupdate is not implemented. Miners will need to check the Targon repository and update themselves as new versions are released. If interested in utilizing the autoupdate feature that Validators use, please follow the steps below:
NOTE: This will not be maintained by the Manifold Labs Team.
- Import the autoupdate function into your miner script (neurons/miner.py) at the top of the file.
from targon.updater import autoupdate- Call the function at a place of your choosing.
if self.config.autoupdate:
autoupdate(branch="main")- Relaunch your miner with the changes.
- --netuid ==> Subnet Netuid. Defaults to 4
- --neuron.epoch_length ==> Default epoch length (how often we set weights, measured in 12 second blocks). Defaults to 360
- --mock ==> Mock neuron and all network components. Defaults to False
- --neuron.model_name ==> Name of the model used for completion. Defaults to "NousResearch/Meta-Llama-3.1-8B-Instruct"
- --neuron.name ==> Trials for this neuron go in neuron.root/ (wallet-cold - wallet-hot) / neuron.name. Defaults to miner
- --blacklist.force_validator.permit ==> If set, forces incoming requests to have a permit. Defaults to True
- --neuron.model_endpoint ==> Endpoint to use for the OpenAi CompatibleClient. Defaults to "http://127.0.0.1:8000/v1"
- --neuron.api_key ==> API key for OpenAi Compatible API. Defaults to "12345"
- --neuron.name ==> Trials for this neuron go in neuron.root/ (wallet-cold - wallet-hot) / neuron.name. Defaults to validator
- --neuron.timeout ==> The timeout for each forward call in seconds. Defaults to 8
- --neuron.sample_size ==> The number of miners to query in a single step. Defaults to 48
- --nueron.vpermit_tao_limit ==> The maximum number of TAO allowed to query a validator with a permit. Defaults to 4096
- --nueron.cache_file ==> Pickle file to save score cache to. Defaults to cache.pickle
- --database.url ==> Database URL to save Miner Data to Targon Hub.
- --autoupdate_off ==> Disable automatic updates to Targon on latest version on Main if set. Defaults to True
Targon uses a novel, in-house proof-of-work value to verify model responses. These are generated just like log-probs, and can be used to verify with 99.99% accuracy that a response was generated with the model requested. Verified responses are scored by speed and consistency of verification.
A miner is a node that is responsible for generating a output from a query, both organic and synthetic.
A validator is a node that is responsible for verifying a miner's output. The validator will send an openai compliant request to a miner with. The miner will then send back a response with the output. The validator will then use the proof of work values of the response to verify that each miners response is accurate. Validators will keep score of each miners response time and use their averages to assign scores each epoch.
Validators can query miners directly using any openai package, and Epistula headers. Below is boilerplate for querying a miner in python.
miner = openai.AsyncOpenAI(
base_url=f"http://{axon.ip}:{axon.port}/v1",
api_key="sn4",
max_retries=0,
timeout=Timeout(12, connect=5, read=5),
http_client=openai.DefaultAsyncHttpxClient(
event_hooks={
"request": [
# This injects Epistula headers right before the request is sent.
# wallet.hotkey is the public / private keypair
create_header_hook(wallet.hotkey, axon.hotkey_ss58)
]
}
),
)Project maintainers reserve the right to weigh the opinions of peer reviewers using common sense judgement and may also weigh based on merit. Reviewers that have demonstrated a deeper commitment and understanding of the project over time or who have clear domain expertise may naturally have more weight, as one would expect in all walks of life. Where a patch set affects consensus-critical code, the bar will be much higher in terms of discussion and peer review requirements, keeping in mind that mistakes could be very costly to the wider community. This includes refactoring of consensus-critical code. Where a patch set proposes to change the Targon subnet, it must have been discussed extensively on the discord server and other channels, be accompanied by a widely discussed BIP and have a generally widely perceived technical consensus of being a worthwhile change based on the judgement of the maintainers. That being said, Manifold welcomes all PR's for the betterment of the subnet and Bittensor as a whole. We are striving for improvement at every interval and believe through open communication and sharing of ideas will success be attainable.