The MxAccl repository provides the open-source code for both the mx_accl runtime library and the acclBench benchmarking tool. These components enable seamless integration and performance measurement of C++ applications using the MemryX MX3 accelerator.
The mx_accl library is designed to efficiently handle multi-model and multi-input streams. It offers:
- Auto-threading Mode: Where the library manages send and receive threads automatically, simplifying usage.
- Manual-threading Mode: Where the user manually creates and manages threads, providing greater control and customization.
This architecture helps maximize the performance of the MX3’s pipelined dataflow system, while also allowing users to manage resources and threading as needed.
The acclBench command line interface tool provides an easy way to benchmark model performance on the MX3 accelerator. It measures the latency and FPS of inference operations, capturing the performance from the host side, which includes driver and interface time. acclBench supports both single and multi-stream scenarios and is built using the high-performance MxAccl C++ API.
For detailed information on using the mx_accl library and acclBench tool, please visit the MemryX Developer Hub:
Note: If you are looking for the Python API for the MemryX MX3 accelerator, please visit the Developer Hub for documentation and usage examples. This repository only contains the C++ version of the library.
This repository contains the source code for the core mx_accl library and associated tools. These components are typically pre-built and packaged as memx-accl within the MemryX SDK.
| Folder | Description |
|---|---|
mx_accl |
Core MxAccl runtime library code |
mx_accl/tests |
Unit tests for MxAccl |
mx_server |
MXA-Manager daemon and config files |
tools |
Utilities like acclBench |
IMPORTANT: For most users, we highly recommend using the prebuilt
memx-acclpackage provided by the MemryX SDK, as it simplifies development and ensures all dependencies are properly managed.
To simplify development and avoid building from source, install the memx-accl package through the MemryX SDK. The SDK includes precompiled libraries, drivers, and tools optimized for MemryX accelerators. Follow the installation guide for step-by-step instructions.
The MemryX Developer Hub provides comprehensive documentation, tutorials, and examples to get you started quickly.
For advanced users who prefer to build the MxAccl library from source, follow these instructions:
git clone https://github.com/memryx/MxAccl.gitOutside your MxAccl folder, we need to build a static library version of gRPC.
# clone and make build dirs
git clone --recurse-submodules -b v1.68.2 --depth 1 --shallow-submodules https://github.com/grpc/grpc
cd grpc
mkdir cmake/build
cd cmake/build
# configure gRPC and set it to install to $HOME/grpc_inst
cmake -DgRPC_INSTALL=ON -DgRPC_BUILD_TESTS=OFF -DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF -DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF \
-DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF -DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF -DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF \
-DCMAKE_INSTALL_PREFIX=$HOME/grpc_inst \
-DCMAKE_BUILD_TYPE=Release ../..
# build (will take a while)
make -j$(nproc)
# install to $HOME/grpc_inst
make install
If you plan on using Shared mode, you will have to build gRPC above and then use Step 3a to build MxAccl.
If you're definitely only going to use one application at a time in Local mode, such as on Yocto systems, you can skip the dependency on mxa-manager (and thus gRPC) entirely with Step 3b.
Hint: if you don't care about running unit tests, use the -DBUILD_TYPE=Packaging argument!
mkdir build && cd build
# must include the path to grpc_inst here
PATH="$HOME/grpc_inst:$PATH" cmake [-DBUILD_TYPE=[Debug | Release | Packaging]] ..
make -jmkdir build && cd build
cmake -DDISABLE_DAEMON=1 [-DBUILD_TYPE=[Debug | Release | Packaging]] ..
make -j-
Gtest: Required for running the unit test suite. You can find it here.
-
Test Models: Download the DFPs and source models for the unit tests from this link and extract them into the mx_accl/tests/models/ folder.
The typical use of MxAccl is to integrate it directly into your C++ application to manage model inference on the MemryX MX3 accelerator. For complete documentation and detailed integration tutorials, visit the MxAccl Documentation and Tutorials Page.
Starting with the 1.1 release, the mxa_manager process is used, even if you're not using Shared mode, for the daemon to manage locks/access to connected MX3 devices.
But, if you compile with -DDISABLE_DAEMON=1 like in Step 3b above, it can be bypassed by your applications. This is not recommended unless you have no alternative way to build gRPC for your platform.
So before starting any tests or example apps, run the mxa_manager command in a separate terminal and leave it going.
./mx_server/mxa_managerNow to verify your setup, you can run the unit tests with the following command:
cd build
ctestTo measure the performance (latency and FPS) of a model on the MemryX MX3 accelerator, use the acclBench command line tool. Here's how to get started:
You can download a precompiled MobileNet DFP file from the link below:
For more information on creating and using DFP files, refer to the Hello MXA Tutorial.
Once you have the DFP file and have successfully installed the MemryX SDK, drivers, and runtime libraries, navigate to the directory where you downloaded the MobileNet DFP file and run:
acclBench -d mobilenet.dfp -f 1000-d mobilenet.dfp: Specifies the DFP model file to be used for benchmarking.-f 1000: Sets the number of frames for testing inference performance. The default is 1000 frames.
For Onnx, Tensorflow, and TFLite pre/post plugins, refer to the MxUtils repository. These plugins are packaged separately to minimize dependencies for the core MxAccl library and are only required if pre/post models are used.
MxAccl is open-source software under the permissive MIT license.
Enhance your experience with MemryX solutions by exploring the following resources:
- Developer Hub: Access comprehensive documentation for MemryX hardware and software.
- MemryX SDK Installation Guide: Learn how to set up essential tools and drivers to start using MemryX accelerators.
- Tutorials: Follow detailed, step-by-step instructions for various use cases and applications.
- Model Explorer: Discover and explore models that have been compiled and optimized for MemryX accelerators.
- Examples: Explore a collection of end-to-end AI applications powered by MemryX hardware and software.