loadModel( )

// Load new model
function loadModel(options: LoadModelOptions, rpcOptions?: RPCOptions): Promise<string>;

// Hot-reload config on an already-loaded model
function loadModel(options: ReloadConfigOptions, rpcOptions?: RPCOptions): Promise<string>;

Supports multiple model types: LLM, Whisper (speech recognition), Parakeet (NVIDIA NeMo transcription), embeddings, NMT (translation), TTS, and OCR. Handles local file paths, HTTP/HTTPS URLs, Hyperdrive URLs (pear://), and registry URLs.

When onProgress is provided, streaming is used for real-time download progress. Otherwise, a simple request-response pattern is used.

Parameters

LoadModelOptions

Common fields present in all variants:

Delegate

Optional delegation configuration for remote (P2P) inference:

ReloadConfigOptions

Hot-reload configuration on an already-loaded model without reloading the model weights. Currently supported for Whisper models only.

Model type variants

The modelType field determines which variant of modelConfig is accepted.

"llm"

All LLM-specific fields live inside modelConfig. See LLM modelConfig for the full reference.

"whisper"

All Whisper-specific fields live inside modelConfig. See Whisper modelConfig for the full reference.

"parakeet"

NVIDIA NeMo Parakeet models for speech recognition. modelConfig is required.

See Parakeet modelConfig for the full reference.

"embeddings"

All embeddings fields live inside modelConfig. See Embeddings modelConfig for the full reference.

"nmt"

modelConfig is required and is a discriminated union on engine. See NMT modelConfig for the full reference.

"tts"

modelConfig is required and is a discriminated union on ttsEngine:

Chatterbox engine (ttsEngine: "chatterbox"):

Supertonic engine (ttsEngine: "supertonic"):

"ocr"

All OCR-specific fields live inside modelConfig. See OCR modelConfig for the full reference.

Custom plugin

Any modelType string that is not a built-in type. modelConfig accepts Record<string, unknown>.

modelConfig reference

LLM modelConfig

Whisper modelConfig

Common fields:

Additional fields: n_threads, n_max_text_ctx, offset_ms, duration_ms, audio_ctx, no_context, no_timestamps, single_segment, print_special, print_progress, print_realtime, print_timestamps, token_timestamps, thold_pt, thold_ptsum, max_len, split_on_word, max_tokens, debug_mode, tdrz_enable, suppress_regex, suppress_blank, suppress_nst, length_penalty, temperature_inc, entropy_thold, logprob_thold, greedy_best_of, beam_search_beam_size. All optional. See whisperConfigSchema in the source for details.

Parakeet modelConfig

modelConfig is required. Parakeet models support three variants via modelType: "tdt" (default), "ctc", and "sortformer".

Runtime config:

Model sources (all string \| ModelDescriptor, all optional):

Embeddings modelConfig

NMT modelConfig

Discriminated union on engine. Common generation parameters (all optional):

Engine-specific:

• Opus: from/to accept "en" | "de" | "es" | "it" | "ru" | "ja"
• Bergamot: from/to accept 24 languages (en, ar, bg, ca, cs, de, es, et, fi, fr, hu, is, it, ja, ko, lt, lv, nl, pl, pt, ru, sk, sl, uk, zh). Additional fields: srcVocabSrc, dstVocabSrc, normalize, pivotModel
• IndicTrans: from/to accept 26 Indic language codes (e.g., "eng_Latn", "hin_Deva")

Bergamot pivotModel (optional) — for translation via an intermediate language:

Plus all common generation parameters above.

OCR modelConfig

ModelProgressUpdate

Returns

Promise<string> — Resolves to the model ID (used to reference the model in subsequent API calls).

Throws

Example

// Local file path
const modelId = await loadModel({
  modelSrc: "/home/user/models/llama-7b.gguf",
  modelType: "llm",
  modelConfig: { ctx_size: 2048 }
});

// Remote URL with progress tracking
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../model.gguf",
  modelType: "llm",
  onProgress: (progress) => {
    console.log(`Downloaded: ${progress.percentage}%`);
  }
});

// Hyperdrive URL
const modelId = await loadModel({
  modelSrc: "pear://<hyperdrive-key>/llama-7b.gguf",
  modelType: "llm",
  modelConfig: { ctx_size: 2048 }
});

// Multimodal model with projection
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../main-model.gguf",
  modelType: "llm",
  modelConfig: {
    ctx_size: 512,
    projectionModelSrc: "https://huggingface.co/.../projection-model.gguf"
  }
});

// Whisper with VAD model
const modelId = await loadModel({
  modelSrc: "https://huggingface.co/.../whisper-model.gguf",
  modelType: "whisper",
  modelConfig: {
    language: "en",
    strategy: "greedy",
    vadModelSrc: "https://huggingface.co/.../vad-model.bin"
  }
});

// With logger forwarding
import { getLogger } from "@qvac/sdk";
const logger = getLogger("my-app");

const modelId = await loadModel({
  modelSrc: "/path/to/model.gguf",
  modelType: "llm",
  logger
});