Skip to content

vox-plugin-mens-candle-cuda API Reference (generated)

Crate: vox-plugin-mens-candle-cuda · Version: 0.6.0 · Target: Library (lib.rs)

ML training backend plugin: Candle + CUDA. Implements MlBackend.

vox-plugin-mens-candle-cuda — Candle + CUDA ML backend plugin.

Implements MlBackend from vox-plugin-api. The model handle stores a Box<CandleModel>; methods recover it via the opaque pointer field.

SP3 sub-batch C — QLoRA training fully ported from vox-populi

Section titled “SP3 sub-batch C — QLoRA training fully ported from vox-populi”

candle_qlora_train/ contains the full training loop extracted from vox-populi/src/mens/tensor/candle_qlora_train/. training.rs now wires through to candle_qlora_train::run_candle_qlora_train via a TrainRequest JSON envelope.

See: docs/src/architecture/plugin-system-redesign-2026.md

FeatureDefault / Enables
cudacandle-core/cuda, candle-nn/cuda, vox-plugin-mens-candle-core/cuda
default—
metalcandle-core/metal, candle-nn/metal, vox-plugin-mens-candle-core/metal
DependencyKind
vox-compilerNormal
vox-configNormal
vox-evalNormal
vox-hf-layoutNormal
vox-plugin-apiNormal
vox-plugin-mens-candle-coreNormal
vox-secretsNormal
vox-tensorNormal
ModuleDescription
candle_qlora_train—
device—
inference—
Dependency Topology, diagram 19 nodes, 8 edges: vox-plugin-mens-candle-… → vox-compiler; vox-plugin-mens-candle-… → vox-config; vox-plugin-mens-candle-… → vox-eval; vox-plugin-mens-candle-… → vox-hf-layout; vox-plugin-mens-candle-… → vox-plugin-api; vox-plugin-mens-candle-… → vox-plugin-mens-candle-…; vox-plugin-mens-candle-… →…vox-plugin-mens-candle-cudavox-compilervox-configvox-evalvox-hf-layoutvox-plugin-apivox-plugin-mens-candle-corevox-secretsvox-tensor
Diagram source
graph {
direction: right
node vox_plugin_mens_candle_cuda: "vox-plugin-mens-candle-cuda"
node vox_compiler: "vox-compiler"
vox_plugin_mens_candle_cuda -> vox_compiler
node vox_config: "vox-config"
vox_plugin_mens_candle_cuda -> vox_config
node vox_eval: "vox-eval"
vox_plugin_mens_candle_cuda -> vox_eval
node vox_hf_layout: "vox-hf-layout"
vox_plugin_mens_candle_cuda -> vox_hf_layout
node vox_plugin_api: "vox-plugin-api"
vox_plugin_mens_candle_cuda -> vox_plugin_api
node vox_plugin_mens_candle_core: "vox-plugin-mens-candle-core"
vox_plugin_mens_candle_cuda -> vox_plugin_mens_candle_core
node vox_secrets: "vox-secrets"
vox_plugin_mens_candle_cuda -> vox_secrets
node vox_tensor: "vox-tensor"
vox_plugin_mens_candle_cuda -> vox_tensor
}
KindSymbolSignatureSourceDescription
fncandle_qlora_train::forwardpub fn forward(&self, input_ids: &Tensor) -> anyhow::Result<Tensor>crates/vox-plugin-mens-candle-cuda/src/candle_qlora_train/mod.rs#L93-L97—
fncandle_qlora_train::forward_checkpointedpub fn forward_checkpointed(crates/vox-plugin-mens-candle-cuda/src/candle_qlora_train/mod.rs#L101Checkpointed forward (activation/gradient checkpointing). See [crate::model::Qwen35Model::forward_checkpointed].
fncandle_qlora_train::recompute_segmentpub fn recompute_segment(crates/vox-plugin-mens-candle-cuda/src/candle_qlora_train/mod.rs#L113Recompute one checkpoint segment. See [crate::model::Qwen35Model::recompute_segment].
structcandle_qlora_train::TrainRequestpub struct TrainRequestcrates/vox-plugin-mens-candle-cuda/src/candle_qlora_train/mod.rs#L174-L184JSON wire format for run_candle_qlora_train — all parameters needed by the plugin. The caller (vox-populi or CLI) serializes this; the plugin deserializes it.
fncandle_qlora_train::run_candle_qlora_trainpub fn run_candle_qlora_train(crates/vox-plugin-mens-candle-cuda/src/candle_qlora_train/mod.rs#L191Main entry point — called from crate::training::run_full_training.
fndevice::probe_gpupub fn probe_gpu() -> GpuInfocrates/vox-plugin-mens-candle-cuda/src/device.rs#L14-L30Minimal GPU probe — returns unknown vendor when no probe is possible. SP3-C stub: hardware registry is a vox-populi concern; reconnect via host capability in sub-batch D.
fndevice::device_mem_used_total_mbpub fn device_mem_used_total_mb() -> Option<(u64, u64)>crates/vox-plugin-mens-candle-cuda/src/device.rs#L73-L88Current device memory usage as (used_mb, total_mb), via cuMemGetInfo. Returns None when there is no live CUDA context yet (before the first GPU op) or the driver call fails. used = total - free includes all processes’ usage, which on a training box is dominated by this process once weights are resident.
fndevice::trim_default_poolpub fn trim_default_pool(retain_bytes: u64) -> Result<(), String>crates/vox-plugin-mens-candle-cuda/src/device.rs#L95-L117Trim the current context’s default memory pool, retaining at most retain_bytes. Caller must already hold a live CUDA context (any candle CUDA tensor op establishes one). Returns a String error rather than a typed error: this is best-effort housekeeping and the training loop logs + continues on failure.
structinference::InferenceEnginepub struct InferenceEnginecrates/vox-plugin-mens-candle-cuda/src/inference.rs#L26-L33—
enuminference::InferenceModelpub enum InferenceModelcrates/vox-plugin-mens-candle-cuda/src/inference.rs#L35-L37—
fninference::weight_sourcespub fn weight_sources(model_dir: &Path, base_shards: &[PathBuf]) -> Vec<PathBuf>crates/vox-plugin-mens-candle-cuda/src/inference.rs#L94-L106Every safetensors file whose tensors feed the inference weight map, in search order: adapter before merged before base shards. The adapter file contributes no base keys (its tensors are named <logical>.lora_a.weight / .lora_b.weight), so it never shadows a base weight here; the trained delta is folded in separately by merge::lora_factors_by_base_key, gated on adapter_deltas_must_be_folded.
fninference::loadpub fn load(model_dir: &Path, device_kind: &crate::device::DeviceKind) -> Result<Self>crates/vox-plugin-mens-candle-cuda/src/inference.rs#L116—
fninference::generate_from_jsonpub fn generate_from_json(&mut self, prompt_json: &str) -> Result<String>crates/vox-plugin-mens-candle-cuda/src/inference.rs#L461-L515Autoregressive generation loop (unconstrained). prompt_json is a JSON object with fields: - "prompt": string — the prompt text - "system": string — optional trained system prompt - "max_tokens": integer (default 256) - "temperature": float (default 1.0; <= 0 forces greedy) - "top_k": integer (default 0 = full vocabulary; 1 forces greedy) - "top_p": float nucleus mass (default absent = 1.0, no filtering) - "seed": integer; sampling draws from StdRng::seed_from_u64(seed) so equal seeds over equal logits give equal tokens (default absent = entropy-seeded) - "output_mode": optional structured-output label — see check_output_mode for what this plugin does and does not enforce. - "lane": optional serving lane — codegen (default), hub or agent; see resolve_lane.
fninference::runpub fn run(model_dir: &str, prompt_json: &str) -> Result<String>crates/vox-plugin-mens-candle-cuda/src/inference.rs#L884-L888Entry point called from backend.rs run_inference. model_dir_json is a JSON string with a "model_dir" field pointing to the local model directory, plus the prompt fields consumed by generate_from_json.