Trait chat_flame_backend::llm::sampler::Sampler

pub trait Sampler {
    // Required method
    fn sample(&mut self, logits: &Tensor) -> Result<u32>;
}

A trait for sampling a token based on logits output.

This trait defines a method for sampling a single token from a distribution represented by logits.

Required Methods

fn sample(&mut self, logits: &Tensor) -> Result<u32>

Samples a token based on provided logits.

Arguments

• logits - A reference to a tensor containing logits output from the model.

Returns

Returns a Result containing the sampled token’s ID.

Implementations on Foreign Types

impl Sampler for LogitsProcessor

Implementation of Sampler for the LogitsProcessor from candle_transformers.

fn sample(&mut self, logits: &Tensor) -> Result<u32>

Implementors

impl Sampler for DummySampler

Implementation of Sampler for DummySampler.