Hugging Face has introduced a command to deploy private, OpenAI-compatible Large Language Model (LLM) endpoints directly on its infrastructure via the HF Jobs platform. Users can provision server instances using a single command, benefiting from a per-second billing model that eliminates the need for managing Kubernetes or manual server provisioning. This capability is designed for developers seeking a flexible environment for testing, model evaluations, or batch generation tasks.

To launch a server, users must ensure their local environment is configured with huggingface_hub version 1.20.0 or higher. By executing a command structured similarly to a Docker container launch, users can specify hardware resources, such as an a10g-large GPU, and expose specific network ports to reach the model via a public proxy. Once the application completes its startup sequence, the server functions as a standard OpenAI-compatible API endpoint. Requests require an HF token with read access to the job's namespace, which acts as the primary authentication gateway for private usage.

The platform also supports scaling for larger models by enabling higher-tier hardware flavors and configuring tensor parallelism (partitioning large models across multiple GPUs). For instance, deploying a 122B parameter model requires using h200x2 hardware and adjusting parameters such as --max-model-len to stay within memory limits. Explicitly cancelling the job using the job ID stops the billing cycle, providing cost control for experimental workloads. For production-ready requirements, such as auto-scaling or granular access management, the platform recommends using its dedicated Inference Endpoints service.

Advanced features include direct SSH access into the running job container for debugging or monitoring, and integration with terminal-based coding agents like Pi. By relaunching the server with tool-calling flags, users can enable agentic capabilities for automated code generation and interaction. The service supports a range of serving backends beyond vLLM, including llama.cpp for GGUF formats and SGLang, offering a versatile toolset for developers needing immediate, transient access to custom inference hardware.

Hugging Face has introduced a command to deploy private, OpenAI-compatible Large Language Model (LLM) endpoints directly on its infrastructure via the HF Jobs platform. Users can provision server instances using a single command, benefiting from a per-second billing model that eliminates the need for managing Kubernetes or manual server provisioning. This capability is designed for developers seeking a flexible environment for testing, model evaluations, or batch generation tasks.

To launch a server, users must ensure their local environment is configured with huggingface_hub version 1.20.0 or higher. By executing a command structured similarly to a Docker container launch, users can specify hardware resources, such as an a10g-large GPU, and expose specific network ports to reach the model via a public proxy. Once the application completes its startup sequence, the server functions as a standard OpenAI-compatible API endpoint. Requests require an HF token with read access to the job's namespace, which acts as the primary authentication gateway for private usage.

The platform also supports scaling for larger models by enabling higher-tier hardware flavors and configuring tensor parallelism (partitioning large models across multiple GPUs). For instance, deploying a 122B parameter model requires using h200x2 hardware and adjusting parameters such as --max-model-len to stay within memory limits. Explicitly cancelling the job using the job ID stops the billing cycle, providing cost control for experimental workloads. For production-ready requirements, such as auto-scaling or granular access management, the platform recommends using its dedicated Inference Endpoints service.

Advanced features include direct SSH access into the running job container for debugging or monitoring, and integration with terminal-based coding agents like Pi. By relaunching the server with tool-calling flags, users can enable agentic capabilities for automated code generation and interaction. The service supports a range of serving backends beyond vLLM, including llama.cpp for GGUF formats and SGLang, offering a versatile toolset for developers needing immediate, transient access to custom inference hardware.