Run AI Workloads on Any Cloud with Hugging Face Storage: Zero Egress via SkyPilot

Hugging Face, in collaboration with the SkyPilot project, has introduced an integration that allows models and datasets to be stored on Hugging Face Hub while running compute tasks (training, fine-tuning, inference) on any cloud or cluster via SkyPilot. Data is accessed using the hf:// scheme, with no egress charges: Hugging Face Storage does not charge for outgoing traffic or CDN usage. This eliminates the need for data duplication across cloud providers and the associated costs. The storage is based on Xet technology with content-level deduplication: incremental checkpoints only transfer modified fragments. The integration supports both mounting (MOUNT) via FUSE and copying (COPY). Authentication uses the HF_TOKEN. In a benchmark for fine-tuning Qwen/Qwen3.5-4B on AWS, GCP, and Lambda, the model loaded in approximately 30 seconds, and checkpoints were written at speeds between 112 and 168 MB/s. The solution is fully open source: SkyPilot, hf-mount, and huggingface_hub are all open source.