Trait chat_flame_backend::llm::token_generator::TokenGeneratorTrait

source · 
pub trait TokenGeneratorTrait: Send {
    // Required methods
    fn init(&mut self, prompt_tokens: Vec<u32>) -> Result<()>;
    fn next(&mut self) -> Result<TokenGeneratorResult>;
}
Expand description

A trait defining the behavior of a token generator.

This trait is implemented by objects that can generate tokens based on some internal logic. The trait provides methods to initialize the generator and to retrieve the next token in the sequence.

Required Methods§

source

fn init(&mut self, prompt_tokens: Vec<u32>) -> Result<()>

Initializes the token generator with a given set of prompt tokens.

Arguments
  • prompt_tokens - A vector of initial tokens used to start the token generation process.
Returns

A Result indicating the success or failure of the initialization.

source

fn next(&mut self) -> Result<TokenGeneratorResult>

Retrieves the next token from the generator.

Returns

A Result containing the TokenGeneratorResult, which can be either a token or a signal to finish generation.

Implementors§