ASUS UGen300 USB AI accelerator features a 40 TOPS Hailo-10H AI processor and 8GB of LPDDR4 memory to support large language models (LLMs), vision language models (VLMs), and AI vision workloads while consuming less than 2.5 Watts.
In some ways, the hardware is similar to the Raspberry Pi AI HAT+ 2 , but in a slim dongle form factor (105 x 50 x 18mm) and a 10 Gbps USB 3.2 Gen1 USB-C Type-C port instead of a 5 or 8 GT/s PCIe interface.
ASUS UGen300 USB AI accelerator (UGen300-USB-8G) specifications:
AI accelerator – Hailo Hailo-10H delivering up to 40 TOPS (INT4) or 20 TOPS (INT8) inferencing performance
Host interface – USB 3.1 Gen2 (10 Gbps) Type-C port
Fanless device with a large internal heatsink cooling the Hailo-10H chip
ASUS says the USB AI accelerator works with Windows, Linux, and Android operating systems running on x86 or Arm hosts. It supports both vision and GenAI models, including 150+ pre-trained models, using AI frameworks such as Keras, TensorFlow, TensorFlowLite, PyTorch, and ONNX. ASUS documentation shows how to install the HailoRT SDK on Ubuntu and provides Python tutorials for LLM, VLM, and speed-to-text.
As noted in our review of the Raspberry Pi AI HAT+ 2 , the Hailo-10H will not necessarily accelerate LLM workloads, but it will help offload the work to the USB adapter, leaving more memory and CPU/GPU resources on the host, and potentially do so at lower power. So in that respect it should be more of an LLM offloader than an accelerator. However, it will accelerate AI vision workloads and may also help improve performance somewhat with VLMs.