Skip to content

vox-plugin-mens-candle-metal API Reference (generated)

Crate: vox-plugin-mens-candle-metal · Version: 0.6.0 · Target: Library (lib.rs)

ML training backend plugin: Candle + Metal. Implements MlBackend.

vox-plugin-mens-candle-metal — Candle + Metal ML backend plugin.

Implements MlBackend from vox-plugin-api. The model handle stores a Box<CandleModel>; methods recover it via the opaque pointer field.

SP3 sub-batch C — QLoRA training fully ported from vox-populi

Section titled “SP3 sub-batch C — QLoRA training fully ported from vox-populi”

candle_qlora_train/ contains the full training loop extracted from vox-populi/src/mens/tensor/candle_qlora_train/. training.rs now wires through to candle_qlora_train::run_candle_qlora_train via a TrainRequest JSON envelope.

See: docs/src/architecture/plugin-system-redesign-2026.md

FeatureDefault / Enables
default—
metalcandle-core/metal, candle-nn/metal, vox-plugin-mens-candle-core/metal
DependencyKind
vox-compilerNormal
vox-configNormal
vox-evalNormal
vox-hf-layoutNormal
vox-plugin-apiNormal
vox-plugin-mens-candle-coreNormal
vox-secretsNormal
vox-tensorNormal
vox-plugin-mens-candle-coreNormal
ModuleDescription
candle_qlora_train—
device—
inference—
Dependency Topology, diagram 19 nodes, 8 edges: vox-plugin-mens-candle-… → vox-compiler; vox-plugin-mens-candle-… → vox-config; vox-plugin-mens-candle-… → vox-eval; vox-plugin-mens-candle-… → vox-hf-layout; vox-plugin-mens-candle-… → vox-plugin-api; vox-plugin-mens-candle-… → vox-plugin-mens-candle-…; vox-plugin-mens-candle-… →…vox-plugin-mens-candle-metalvox-compilervox-configvox-evalvox-hf-layoutvox-plugin-apivox-plugin-mens-candle-corevox-secretsvox-tensor
Diagram source
graph {
direction: right
node vox_plugin_mens_candle_metal: "vox-plugin-mens-candle-metal"
node vox_compiler: "vox-compiler"
vox_plugin_mens_candle_metal -> vox_compiler
node vox_config: "vox-config"
vox_plugin_mens_candle_metal -> vox_config
node vox_eval: "vox-eval"
vox_plugin_mens_candle_metal -> vox_eval
node vox_hf_layout: "vox-hf-layout"
vox_plugin_mens_candle_metal -> vox_hf_layout
node vox_plugin_api: "vox-plugin-api"
vox_plugin_mens_candle_metal -> vox_plugin_api
node vox_plugin_mens_candle_core: "vox-plugin-mens-candle-core"
vox_plugin_mens_candle_metal -> vox_plugin_mens_candle_core
node vox_secrets: "vox-secrets"
vox_plugin_mens_candle_metal -> vox_secrets
node vox_tensor: "vox-tensor"
vox_plugin_mens_candle_metal -> vox_tensor
}
KindSymbolSignatureSourceDescription
structcandle_qlora_train::PeakSamplerpub struct PeakSamplercrates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L47-L51Samples a training device’s currently-allocated bytes on a background thread, retaining the maximum observed value. This is what turns a CalibrationRecord::peak_bytes (see vox_populi::mens::tensor::calibration) into a real measurement of an actual candle-metal run instead of a predicted estimate — no external tooling, no Python probe.
fncandle_qlora_train::startpub fn start(device: &Device, interval: Duration) -> Selfcrates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L60-L70Starts sampling device every interval, from before the first training step. A non-Metal device (CPU, CUDA, or a build without the metal feature) reports a constant 0 for the life of the sampler — an honest “unmeasured”, not a fabricated number (same convention as vox_populi::mens::tensor::accel_budget::query_accel_budget on non-macOS).
fncandle_qlora_train::observed_peak_bytespub fn observed_peak_bytes(&self) -> u64crates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L74-L76The largest allocated-bytes value observed since start. Read this after the last training step.
fncandle_qlora_train::forwardpub fn forward(&self, input_ids: &Tensor) -> anyhow::Result<Tensor>crates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L159-L163—
fncandle_qlora_train::forward_checkpointedpub fn forward_checkpointed(crates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L167Checkpointed forward (activation/gradient checkpointing). See [crate::model::Qwen35Model::forward_checkpointed].
fncandle_qlora_train::recompute_segmentpub fn recompute_segment(crates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L179Recompute one checkpoint segment. See [crate::model::Qwen35Model::recompute_segment].
structcandle_qlora_train::TrainRequestpub struct TrainRequestcrates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L229-L239JSON wire format for run_candle_qlora_train — all parameters needed by the plugin. The caller (vox-populi or CLI) serializes this; the plugin deserializes it.
fncandle_qlora_train::run_candle_qlora_trainpub fn run_candle_qlora_train(crates/vox-plugin-mens-candle-metal/src/candle_qlora_train/mod.rs#L275Main entry point — called from crate::training::run_full_training.
fndevice::probe_gpupub fn probe_gpu() -> GpuInfocrates/vox-plugin-mens-candle-metal/src/device.rs#L15-L21Minimal GPU probe — returns unknown vendor when no probe is possible. SP3-C stub: hardware registry is a vox-populi concern; reconnect via host capability in sub-batch D.
structinference::InferenceEnginepub struct InferenceEnginecrates/vox-plugin-mens-candle-metal/src/inference.rs#L28-L35—
enuminference::InferenceModelpub enum InferenceModelcrates/vox-plugin-mens-candle-metal/src/inference.rs#L37-L39—
fninference::forward_with_cachepub fn forward_with_cache(crates/vox-plugin-mens-candle-metal/src/inference.rs#L42—
fninference::forward_with_cache_optpub fn forward_with_cache_opt(crates/vox-plugin-mens-candle-metal/src/inference.rs#L53—
fninference::weight_sourcespub fn weight_sources(model_dir: &Path, base_shards: &[PathBuf]) -> Vec<PathBuf>crates/vox-plugin-mens-candle-metal/src/inference.rs#L162-L174Every safetensors file whose tensors feed the inference weight map, in search order: adapter before merged before base shards. The adapter file contributes no base keys (its tensors are named <logical>.lora_a.weight / .lora_b.weight), so it never shadows a base weight here; the trained delta is folded in separately by merge::lora_factors_by_base_key, gated on adapter_deltas_must_be_folded.
fninference::loadpub fn load(model_dir: &Path, device_kind: &crate::device::DeviceKind) -> Result<Self>crates/vox-plugin-mens-candle-metal/src/inference.rs#L177—
fninference::generate_from_jsonpub fn generate_from_json(&mut self, prompt_json: &str) -> Result<String>crates/vox-plugin-mens-candle-metal/src/inference.rs#L654-L708Autoregressive generation loop (unconstrained). prompt_json is a JSON object with fields: - "prompt": string — the prompt text - "system": string — optional trained system prompt - "max_tokens": integer (default 256) - "temperature": float (default 1.0; <= 0 forces greedy) - "top_k": integer (default 0 = full vocabulary; 1 forces greedy) - "top_p": float nucleus mass (default absent = 1.0, no filtering) - "seed": integer; sampling draws from StdRng::seed_from_u64(seed) so equal seeds over equal logits give equal tokens (default absent = entropy-seeded) - "output_mode": optional structured-output label — see check_output_mode for what this plugin does and does not enforce. - "lane": optional serving lane — codegen (default), hub or agent; see resolve_lane.
fninference::runpub fn run(model_dir: &str, prompt_json: &str) -> Result<String>crates/vox-plugin-mens-candle-metal/src/inference.rs#L1096-L1100model_dir_json is a JSON string with a "model_dir" field pointing to the local model directory, plus the prompt fields consumed by generate_from_json.