com.Ostermiller.Syntax.Lexer
Class SQLToken

java.lang.Object
  com.Ostermiller.Syntax.Lexer.Token
      com.Ostermiller.Syntax.Lexer.SQLToken

public class SQLToken
extends Token

A SQLToken is a token that is returned by a lexer that is lexing an SQL source file. It has several attributes describing the token: The type of token, the text of the token, the line number on which it occurred, the number of characters into the input at which it started, and similarly, the number of characters into the input at which it ended.

Field Summary

static int	COMMENT_END_OF_LINE a -- to end of line comment.
static int	COMMENT_TRADITIONAL C style comment, (except possibly nested)
static int	ERROR An error
static int	ERROR_BAD_BIT_STRING An comment start embedded in an operator
static int	ERROR_UNCLOSED_BIT_STRING An comment start embedded in an operator
static int	ERROR_UNCLOSED_COMMENT An comment start embedded in an operator
static int	ERROR_UNCLOSED_STRING An comment start embedded in an operator
static int	IDENTIFIER A variable, name, or other identifier
static int	LITERAL_BIT_STRING A bit-string
static int	LITERAL_FLOAT A floating point
static int	LITERAL_INTEGER An integer
static int	LITERAL_STRING A string literal
static int	OPERATOR An operator
static int	RESERVED_WORD A reserved word (keyword)
static int	SEPARATOR A separator
static int	WHITE_SPACE White space

Fields inherited from class com.Ostermiller.Syntax.Lexer.Token

INITIAL_STATE, UNDEFINED_STATE

Constructor Summary

SQLToken(int ID, String contents, int lineNumber, int charBegin, int charEnd)
Create a new token.

SQLToken(int ID, String contents, int lineNumber, int charBegin, int charEnd, int state)
Create a new token.

Method Summary

String	errorString() get a String that explains the error, if this token is an error.
int	getCharBegin() get the offset into the input in characters at which this token started
int	getCharEnd() get the offset into the input in characters at which this token ended
String	getContents() get the contents of this token
String	getDescription() A description of this token.
int	getID() get the ID number of this token
int	getLineNumber() get the line number of the input on which this token started
int	getState() Get an integer representing the state the tokenizer is in after returning this token.
boolean	isComment() Checks this token to see if it is a comment.
boolean	isError() Checks this token to see if it is an Error.
boolean	isIdentifier() Checks this token to see if it is an identifier.
boolean	isLiteral() Checks this token to see if it is a literal.
boolean	isOperator() Checks this token to see if it is a Operator.
boolean	isReservedWord() Checks this token to see if it is a reserved word.
boolean	isSeparator() Checks this token to see if it is a Separator.
boolean	isWhiteSpace() Checks this token to see if it is White Space.
String	toString() get a representation of this token as a human readable string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

RESERVED_WORD

public static final int RESERVED_WORD

A reserved word (keyword)

See Also:

Constant Field Values

IDENTIFIER

public static final int IDENTIFIER

A variable, name, or other identifier

See Also:

Constant Field Values

LITERAL_STRING

public static final int LITERAL_STRING

A string literal

See Also:

Constant Field Values

LITERAL_BIT_STRING

public static final int LITERAL_BIT_STRING

A bit-string

See Also:

Constant Field Values

LITERAL_INTEGER

public static final int LITERAL_INTEGER

An integer

See Also:

Constant Field Values

LITERAL_FLOAT

public static final int LITERAL_FLOAT

A floating point

See Also:

Constant Field Values

SEPARATOR

public static final int SEPARATOR

A separator

See Also:

Constant Field Values

OPERATOR

public static final int OPERATOR

An operator

See Also:

Constant Field Values

COMMENT_TRADITIONAL

public static final int COMMENT_TRADITIONAL

C style comment, (except possibly nested)

See Also:

Constant Field Values

COMMENT_END_OF_LINE

public static final int COMMENT_END_OF_LINE

a -- to end of line comment.

See Also:

Constant Field Values

WHITE_SPACE

public static final int WHITE_SPACE

White space

See Also:

Constant Field Values

ERROR

public static final int ERROR

An error

See Also:

Constant Field Values

ERROR_UNCLOSED_COMMENT

public static final int ERROR_UNCLOSED_COMMENT

An comment start embedded in an operator

See Also:

Constant Field Values

ERROR_UNCLOSED_STRING

public static final int ERROR_UNCLOSED_STRING

An comment start embedded in an operator

See Also:

Constant Field Values

ERROR_UNCLOSED_BIT_STRING

public static final int ERROR_UNCLOSED_BIT_STRING

An comment start embedded in an operator

See Also:

Constant Field Values

ERROR_BAD_BIT_STRING

public static final int ERROR_BAD_BIT_STRING

An comment start embedded in an operator

See Also:

Constant Field Values

Constructor Detail

SQLToken

public SQLToken(int ID,
                String contents,
                int lineNumber,
                int charBegin,
                int charEnd)

Create a new token. The constructor is typically called by the lexer

Parameters:

ID - the id number of the token

contents - A string representing the text of the token

lineNumber - the line number of the input on which this token started

charBegin - the offset into the input in characters at which this token started

charEnd - the offset into the input in characters at which this token ended

SQLToken

public SQLToken(int ID,
                String contents,
                int lineNumber,
                int charBegin,
                int charEnd,
                int state)

Create a new token. The constructor is typically called by the lexer

Parameters:

ID - the id number of the token

contents - A string representing the text of the token

lineNumber - the line number of the input on which this token started

charBegin - the offset into the input in characters at which this token started

charEnd - the offset into the input in characters at which this token ended

state - the state the tokenizer is in after returning this token.

Method Detail

getState

public int getState()

Get an integer representing the state the tokenizer is in after returning this token. Those who are interested in incremental tokenizing for performance reasons will want to use this method to figure out where the tokenizer may be restarted. The tokenizer starts in Token.INITIAL_STATE, so any time that it reports that it has returned to this state, the tokenizer may be restarted from there.

Specified by:

getState in class Token

getID

public int getID()

get the ID number of this token

Specified by:

getID in class Token

Returns:

the id number of the token

getContents

public String getContents()

get the contents of this token

Specified by:

getContents in class Token

Returns:

A string representing the text of the token

getLineNumber

public int getLineNumber()

get the line number of the input on which this token started

Specified by:

getLineNumber in class Token

Returns:

the line number of the input on which this token started

getCharBegin

public int getCharBegin()

get the offset into the input in characters at which this token started

Specified by:

getCharBegin in class Token

Returns:

the offset into the input in characters at which this token started

getCharEnd

public int getCharEnd()

get the offset into the input in characters at which this token ended

Specified by:

getCharEnd in class Token

Returns:

the offset into the input in characters at which this token ended

isReservedWord

public boolean isReservedWord()

Checks this token to see if it is a reserved word. Reserved words are explained in Java Language Specification.

Returns:

true if this token is a reserved word, false otherwise

isIdentifier

public boolean isIdentifier()

Checks this token to see if it is an identifier. Identifiers are explained in Java Language Specification.

Returns:

true if this token is an identifier, false otherwise

isLiteral

public boolean isLiteral()

Checks this token to see if it is a literal. Literals are explained in Java Language Specification.

Returns:

true if this token is a literal, false otherwise

isSeparator

public boolean isSeparator()

Checks this token to see if it is a Separator. Separators are explained in Java Language Specification.

Returns:

true if this token is a Separator, false otherwise

isOperator

public boolean isOperator()

Checks this token to see if it is a Operator. Operators are explained in Java Language Specification.

Returns:

true if this token is a Operator, false otherwise

isComment

public boolean isComment()

Checks this token to see if it is a comment.

Specified by:

isComment in class Token

Returns:

true if this token is a comment, false otherwise

isWhiteSpace

public boolean isWhiteSpace()

Checks this token to see if it is White Space. Usually tabs, line breaks, form feed, spaces, etc.

Specified by:

isWhiteSpace in class Token

Returns:

true if this token is White Space, false otherwise

isError

public boolean isError()

Checks this token to see if it is an Error. Unfinished comments, numbers that are too big, unclosed strings, etc.

Specified by:

isError in class Token

Returns:

true if this token is an Error, false otherwise

getDescription

public String getDescription()

A description of this token. The description should be appropriate for syntax highlighting. For example "comment" is returned for a comment.

Specified by:

getDescription in class Token

Returns:

a description of this token.

errorString

public String errorString()

get a String that explains the error, if this token is an error.

Specified by:

errorString in class Token

Returns:

a String that explains the error, if this token is an error, null otherwise.

toString

public String toString()

get a representation of this token as a human readable string. The format of this string is subject to change and should only be used for debugging purposes.

Overrides:

toString in class Object

Returns:

a string representation of this token