Tokenizer Class

Inheritance Hierarchy

SystemObject
BibtexLibrary.TokenizerTokenizer

Namespace: BibtexLibrary.Tokenizer
Assembly: BibtexLibrary.Standard (in BibtexLibrary.Standard.dll) Version: 1.4.4

Syntax

public class Tokenizer

The Tokenizer type exposes the following members.

Constructors

	Name	Description
	Tokenizer	Initializes a new instance of the Tokenizer class

Properties

	Name	Description
	EndOfInput	Checks whether the scanner is at the end of the input.

Methods

	Name	Description
	Equals	Determines whether the specified object is equal to the current object. (Inherited from Object.)
	Finalize	Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (Inherited from Object.)
	GetAllTokens	Return all the tokens in the inputstring.
	GetHashCode	Serves as the default hash function. (Inherited from Object.)
	GetPreviousCharacters	Returns the previous n characters.
	GetType	Gets the Type of the current instance. (Inherited from Object.)
	MemberwiseClone	Creates a shallow copy of the current Object. (Inherited from Object.)
	NextToken
	Peek
	ToString	Returns a string that represents the current object. (Inherited from Object.)