Formatron v0.4.2
Formatron empowers everyone to control the output format of language models with minimal overhead.
|
A builder for creating a Formatter. More...
Public Member Functions | |
__init__ (self) | |
Initialize the formatter builder. | |
None | append_line (self, str line) |
Append a line to the format. | |
None | append_multiline_str (self, str lines) |
Appends a multiline string to the format, preserving the first line's leading whitespaces and remove any common leading whitespaces from subsequent lines. | |
None | append_str (self, str string) |
Append a string to the format without any post-processing. | |
ChoiceExtractor | choose (self, *Extractor|str extractors, str capture_name=None) |
Create a choice extractor. | |
Extractor | extractor (self, typing.Callable[[str], Extractor] create_extractor) |
Create a custom extractor. | |
JsonExtractor | json (self, Schema schema, *, str capture_name=None) |
Create a JSON extractor. | |
RegexExtractor | regex (self, str regex, *, str capture_name=None) |
Create a regex extractor. | |
Extractor | str (self, *, typing.Union[str, list[str]] stop=None, typing.Optional[str] capture_name=None) |
Create a string extractor. | |
Extractor | substr (self, str string, *, str capture_name=None, bool extract_empty_substring=False) |
Create a substring extractor. | |
Formatter | build (self, kbnf.Vocabulary vocabulary, typing.Callable[[list[int]], str] decode, kbnf.Config engine_config=None) |
Build a formatter from the builder. | |
Protected Member Functions | |
_assert_capture_name_valid (self, str capture_name) | |
str | _create_nonterminal (self, str name) |
None | _add_capture_name (self, NonterminalExtractor extractor) |
_add_extractor (self, str extractor_type, typing.Callable[[str], Extractor] create_extractor) | |
Protected Attributes | |
_counter | |
_main_rule | |
_rules | |
_capture_names | |
_nonterminal_to_extractor | |
_extractors | |
_instance_id | |
Static Protected Attributes | |
int | _formatter_builder_counter |
A builder for creating a Formatter.
Definition at line 238 of file formatter.py.
formatron.formatter.FormatterBuilder.__init__ | ( | self | ) |
Initialize the formatter builder.
Definition at line 247 of file formatter.py.
|
protected |
Definition at line 333 of file formatter.py.
|
protected |
Definition at line 359 of file formatter.py.
|
protected |
Definition at line 258 of file formatter.py.
Definition at line 328 of file formatter.py.
None formatron.formatter.FormatterBuilder.append_line | ( | self, | |
str | line ) |
Append a line to the format.
Specifically, a newline character is appended to the input.
Note that if you need a literal `$`, you need to escape it by adding a backslash: `\\$`.
Definition at line 269 of file formatter.py.
None formatron.formatter.FormatterBuilder.append_multiline_str | ( | self, | |
str | lines ) |
Appends a multiline string to the format, preserving the first line's leading whitespaces and remove any common leading whitespaces from subsequent lines.
Note that tabs and spaces are both treated as whitespace, but they are not equal: the lines " hello" and "\\thello" are considered to have no common leading whitespace.
Entirely blank lines are normalized to a newline character.
Note that if you need a literal $
, you need to escape it by adding a backslash: \\$
.
Definition at line 283 of file formatter.py.
None formatron.formatter.FormatterBuilder.append_str | ( | self, | |
str | string ) |
Append a string to the format without any post-processing.
Note that if you need a literal `$`, you need to escape it by adding a backslash: `\\$`.
Definition at line 292 of file formatter.py.
Formatter formatron.formatter.FormatterBuilder.build | ( | self, | |
kbnf.Vocabulary | vocabulary, | ||
typing.Callable[[list[int]], str] | decode, | ||
kbnf.Config | engine_config = None ) |
Build a formatter from the builder.
The builder will not be consumed and can be used again.
vocabulary | The KBNF engine vocabulary for the formatter. |
decode | The callback to decode the token IDs to a string. |
engine_config | The KBNF engine configuration. |
Definition at line 466 of file formatter.py.
ChoiceExtractor formatron.formatter.FormatterBuilder.choose | ( | self, | |
*Extractor | str | extractors, | ||
str | capture_name = None ) |
Create a choice extractor.
Check out the ChoiceExtractor docs for more details.
extractors | The extractors to choose from. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
Definition at line 349 of file formatter.py.
Extractor formatron.formatter.FormatterBuilder.extractor | ( | self, | |
typing.Callable[[str], Extractor] | create_extractor ) |
Create a custom extractor.
create_extractor | callable with signature (extractor_nonterminal: str)->Extractor that create the extractor. extractor_nonterminal is the auto-generated nonterminal reference for the extractor. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
Definition at line 376 of file formatter.py.
JsonExtractor formatron.formatter.FormatterBuilder.json | ( | self, | |
Schema | schema, | ||
* | , | ||
str | capture_name = None ) |
Create a JSON extractor.
Check out the JsonExtractor docs for more details.
schema | The schema for extraction. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
Definition at line 388 of file formatter.py.
RegexExtractor formatron.formatter.FormatterBuilder.regex | ( | self, | |
str | regex, | ||
* | , | ||
str | capture_name = None ) |
Create a regex extractor.
Check out the RegexExtractor docs for more details.
regex | The regular expression for extraction. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
Definition at line 408 of file formatter.py.
Extractor formatron.formatter.FormatterBuilder.str | ( | self, | |
* | , | ||
typing.Union[str, list[str]] | stop = None, | ||
typing.Optional[str] | capture_name = None ) |
Create a string extractor.
The extractor will extract all text until(inclusive) one of the stop strings is encountered.
stop | The strings for the extractors to stop at. They will be included in text generation and extraction. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
Definition at line 423 of file formatter.py.
Extractor formatron.formatter.FormatterBuilder.substr | ( | self, | |
str | string, | ||
* | , | ||
str | capture_name = None, | ||
bool | extract_empty_substring = False ) |
Create a substring extractor.
The extractor will extract a substring of the input string.
string | The string to extract. |
capture_name | The capture name of the extractor, or None if the extractor does not capture. |
extract_empty_substring | Whether to extract an empty substring as a valid substring. |
Definition at line 451 of file formatter.py.
|
protected |
Definition at line 251 of file formatter.py.
|
protected |
Definition at line 248 of file formatter.py.
|
protected |
Definition at line 253 of file formatter.py.
|
staticprotected |
Definition at line 242 of file formatter.py.
|
protected |
Definition at line 254 of file formatter.py.
|
protected |
Definition at line 249 of file formatter.py.
|
protected |
Definition at line 252 of file formatter.py.
|
protected |
Definition at line 250 of file formatter.py.