vllm.py File Reference

Go to the source code of this file.

Classes
class	formatron.integrations.vllm.FormattersLogitsProcessor
	Logit processor that uses formatters to mask batch logits. More...

Namespaces
namespace	formatron
namespace	formatron.integrations
	This subpackage contains integrations with other frameworks and libraries.
namespace	formatron.integrations.vllm
	This module integrates the vllm library by providing convenience utilities.

Functions
kbnf.Vocabulary	formatron.integrations.vllm.create_engine_vocabulary (LLM llm)
	Create a vocabulary for the KBNF engine.
FormattersLogitsProcessor	formatron.integrations.vllm.create_formatters_logits_processor (LLM llm, typing.Sequence[FormatterBuilder|None]|FormatterBuilder formatter_builders, typing.Sequence[EngineGenerationConfig] configs=None)
	Create a formatter logits processor.