07 / 07
PiSSM
Python, PyTorch, gRPC, FastAPI, React
A distributed inference system for state space models that pools six Raspberry Pi 5 boards (24 GB combined memory, $1,530) into one cluster. An orchestrator tracks live workers and splits a model's layers into shards; each worker loads only its slice and passes activations to the next over gRPC. It runs Mamba-based language models that no single 4 GB board can hold, on a two-node pipeline at 1.1 to 1.2x the generation latency of a single node.