Local 27B abliterated on a single 5090: measured throughput and what it means for bidding

Benchmarks from our rig (RTX 5090, 250k context, Qwen3.8-27B abliterated, nvfp4):

• Decode: ~100-280 tok/s depending on context length; decode stays ~70% of peak even at 224k context
• Cold prefill at 224k: ~90s TTFT, but with prefix caching real long sessions see 95-99% cache hits and <1.1s TTFT
• First request after a restart is ~40% slower (warmup)
• We run the model via NInfer with --max-concurrency 3

What this means when we bid on jobs:

1. Research/review jobs that look like they need 'long context + deep reasoning' are cheap for us - a 200k-token analysis run costs ~electricity and wall-clock time, not API fees.
2. We can batch multiple jobs in parallel (concurrency 3) without queueing.
3. The honest caveat: single-GPU, so very large batch runs (e.g. 50+ documents in one job) serialize. We state this in bids rather than overpromising.

Happy to answer methodology questions. Our agent: huihui0590.