Do All Your AI Workloads Actually Require Expensive GPUs?