Function chat_flame_backend::api::routes::model::generate_model_handler

source · 
pub async fn generate_model_handler(
    __arg0: Path<Models>,
    app_state: State<AppState>,
    __arg2: Json<CompatGenerateRequest>
) -> Result<Response, (StatusCode, Json<ErrorResponse>)>
Expand description

Handler for generating text tokens.

This endpoint accepts a CompatGenerateRequest and returns a stream of generated text or a single text response based on the stream field in the request. If stream is true, it returns a stream of StreamResponse. If stream is false, it returns GenerateResponse.

Arguments

  • config - State containing the application configuration.
  • payload - JSON payload containing the input text and optional parameters.

Responses

  • 200 OK - Successful generation of text.
  • 501 Not Implemented - Returned if streaming is not implemented.