formatron.integrations.vllm Namespace Reference

This module integrates the vllm library by providing convenience utilities. More...

Classes
class	FormattersLogitsProcessor
	Logit processor that uses formatters to mask batch logits. More...

Functions
kbnf.Vocabulary	create_engine_vocabulary (LLM llm)
	Create a vocabulary for the KBNF engine.
FormattersLogitsProcessor	create_formatters_logits_processor (LLM llm, typing.Sequence[FormatterBuilder|None]|FormatterBuilder formatter_builders, typing.Sequence[EngineGenerationConfig] configs=None)
	Create a formatter logits processor.

Detailed Description

This module integrates the vllm library by providing convenience utilities.

Function Documentation

create_engine_vocabulary()

kbnf.Vocabulary formatron.integrations.vllm.create_engine_vocabulary

(

LLM

llm

)

Create a vocabulary for the KBNF engine.

Definition at line 113 of file vllm.py.

create_formatters_logits_processor()

FormattersLogitsProcessor formatron.integrations.vllm.create_formatters_logits_processor	(	LLM	llm,
		typing.Sequence[FormatterBuilder | None] | FormatterBuilder	formatter_builders,
		typing.Sequence[EngineGenerationConfig]	configs = None )

Create a formatter logits processor.

Definition at line 124 of file vllm.py.