Most legacy NPU accelerators were highly optimized for the ResNet structures that were state-of-the-art five years ago.
Vision Transformers are vastly different in network topology and building block operators. If your legacy NPU cannot handle ViT networks it's time to switch to Quadric Chimera GPNPUs. Chimera GPNPUs can run any network and support any ML graph opertor - even today's most advanced Vision Transformers. Read our blog on SemiEngineering for detailed information.