Formatron v0.4.2
Formatron empowers everyone to control the output format of language models with minimal overhead.
|
This module integrates the ExLlamaV2 library by providing convenience utilities. More...
Classes | |
class | FormatterFilter |
ExLlamaV2Filter that uses a formatter to mask logits. More... | |
Functions | |
kbnf.Vocabulary | create_engine_vocabulary (ExLlamaV2Tokenizer tokenizer) |
Create a vocabulary for the KBNF engine. | |
ExLlamaV2Filter | create_formatter_filter (ExLlamaV2 model, ExLlamaV2Tokenizer tokenizer, FormatterBuilder formatter_builder, EngineGenerationConfig engine_config=None) |
Create a formatter filter for the ExLlamaV2 engine. | |
This module integrates the ExLlamaV2 library by providing convenience utilities.
kbnf.Vocabulary formatron.integrations.exllamav2.create_engine_vocabulary | ( | ExLlamaV2Tokenizer | tokenizer | ) |
Create a vocabulary for the KBNF engine.
Definition at line 20 of file exllamav2.py.
ExLlamaV2Filter formatron.integrations.exllamav2.create_formatter_filter | ( | ExLlamaV2 | model, |
ExLlamaV2Tokenizer | tokenizer, | ||
FormatterBuilder | formatter_builder, | ||
EngineGenerationConfig | engine_config = None ) |
Create a formatter filter for the ExLlamaV2 engine.
Definition at line 34 of file exllamav2.py.