Launch vLLM Server on HF Jobs with a Single Command
Hugging Face
vLLM
Hugging Face has introduced the ability to launch a vLLM server on HF Jobs infrastructure with a single command. This allows users to quickly deploy a model for testing, evaluation, or batch generation, using GPU resources billed by the minute.
Hugging Face (HF) has launched a new feature: you can deploy a vLLM server on the HF Jobs platform with a single command. For this, you need a payment method, the huggingface_hub library version >=1.20.0, and local authentication. The command `hf jobs run` uses the official image `vllm/vllm-openai`, specifies the GPU via `--flavor`, and exposes a port via `--expose`. For example: `hf jobs run --flavor a10g-large --expose 8000 --timeout 2h vllm/vllm-openai:latest vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000`. After launching, a URL is provided for accessing the server through an OpenAI-compatible API with authorization via an HF token. Requests can be sent using `curl` or the Python OpenAI client. Access to the endpoint is restricted — a token with read permissions is required. The server can be stopped with the command `hf jobs cancel <job_id>`. For large models (e.g., Qwen3.5-122B-A10B on two H200 GPUs), you need to choose a more powerful flavor and specify `--tensor-parallel-size`, and if necessary, reduce `--max-model-len` and `--max-num-seqs`. Additional options include: launching with a Gradio UI, SSH access for debugging, and using it as a backend for the coding agent Pi with tool calling enabled. HF Jobs is suitable for experiments and one-off tasks, while Inference Endpoints are recommended for production.
Source: Hugging Face blog —
original
