transformers.py File Reference

Go to the source code of this file.

Classes
class	formatron.integrations.transformers.FormattersLogitsProcessor
	Logit processor that uses formatters to mask batch logits. More...

Namespaces
namespace	formatron
namespace	formatron.integrations
	This subpackage contains integrations with other frameworks and libraries.
namespace	formatron.integrations.transformers
	This module integrates the transformers library by providing convenience utilities.

Functions
kbnf.Vocabulary	formatron.integrations.transformers.create_engine_vocabulary (PreTrainedTokenizerBase tokenizer, typing.Optional[list[typing.Callable]] vocab_processors=None)
	Create a vocabulary for the KBNF engine.
LogitsProcessor	formatron.integrations.transformers.create_formatter_logits_processor (PreTrainedTokenizerBase tokenizer, typing.Sequence[FormatterBuilder|None]|FormatterBuilder formatter_builders, typing.Sequence[EngineGenerationConfig] configs=None, typing.Optional[list[typing.Callable]] vocab_processors=None)
	Create a formatter logits processor.
LogitsProcessorList	formatron.integrations.transformers.create_formatter_logits_processor_list (PreTrainedTokenizerBase tokenizer, typing.Sequence[FormatterBuilder|None]|FormatterBuilder formatter_builders, typing.Sequence[EngineGenerationConfig] configs=None, typing.Optional[list[typing.Callable]] vocab_processors=None)
	Create a formatter logits processor list.