 | TokenizerBase Properties |
The TokenizerBase type exposes the following members.
Properties| Name | Description | |
|---|---|---|
 | InputReplacers |
Set of replacement rules to be applied on a text, before splitting the text into tokens
|
| LowerCase | |
| MinLength | |
| tokenSelector | |
| TokenSplitterChars |
Characters that are going to be used to split the text into tokens
|
See Also