exllamav2.py File Reference

Go to the source code of this file.

Classes
class	formatron.integrations.exllamav2.FormatterFilter
	ExLlamaV2Filter that uses a formatter to mask logits. More...

Namespaces
namespace	formatron
namespace	formatron.integrations
	This subpackage contains integrations with other frameworks and libraries.
namespace	formatron.integrations.exllamav2
	This module integrates the ExLlamaV2 library by providing convenience utilities.

Functions
kbnf.Vocabulary	formatron.integrations.exllamav2.create_engine_vocabulary (ExLlamaV2Tokenizer tokenizer)
	Create a vocabulary for the KBNF engine.
ExLlamaV2Filter	formatron.integrations.exllamav2.create_formatter_filter (ExLlamaV2 model, ExLlamaV2Tokenizer tokenizer, FormatterBuilder formatter_builder, EngineGenerationConfig engine_config=None)
	Create a formatter filter for the ExLlamaV2 engine.