General Compute

Switching your app's LLM calls to faster hardware can come down to changing a base URL and an API key.

General Compute is an AI inference provider that runs large language models on purpose-built ASICs instead of GPUs, and it exposes an OpenAI-compatible REST API, so code written for OpenAI's format should need little rework.

The appeal is speed.

General Compute cites inference of up to 1,000 tokens per second, sub-millisecond time to first token and high throughput, which matters most for chat interfaces, agents and other products where users wait on every response.

Those are the company's own figures, so test them against your own workload.

The hardware is also air-cooled and energy-efficient, which General Compute says lowers power use and operating costs compared with standard GPU clouds.

For developers who want a drop-in inference API rather than a new framework, General Compute is a simple one to trial.