com.Ostermiller.Syntax.Lexer
Class JavaToken

java.lang.Object
  extended by com.Ostermiller.Syntax.Lexer.Token
      extended by com.Ostermiller.Syntax.Lexer.JavaToken

public class JavaToken
extends Token

A JavaToken is a token that is returned by a lexer that is lexing a java source file. It has several attributes describing the token: The type of token, the text of the token, the line number on which it occurred, the number of characters into the input at which it started, and similarly, the number of characters into the input at which it ended.
The tokens should comply with the Java Language Specification.


Field Summary
static int COMMENT_DOCUMENTATION
          The Java Language Specification explains this token
static int COMMENT_END_OF_LINE
          The Java Language Specification explains this token
static int COMMENT_TRADITIONAL
          The Java Language Specification explains this token
static int ERROR_DOUBLE_SIZE
          The Java Language Specification explains this token
static int ERROR_FLOAT
          The Java Language Specification explains this token
static int ERROR_FLOAT_SIZE
          The Java Language Specification explains this token
static int ERROR_IDENTIFIER
          The Java Language Specification explains this token
static int ERROR_INTEGER_DECIMIAL_SIZE
          The Java Language Specification explains this token
static int ERROR_INTEGER_HEXIDECIMAL_SIZE
          The Java Language Specification explains this token
static int ERROR_INTEGER_OCTAL_SIZE
          The Java Language Specification explains this token
static int ERROR_LONG_DECIMIAL_SIZE
          The Java Language Specification explains this token
static int ERROR_LONG_HEXIDECIMAL_SIZE
          The Java Language Specification explains this token
static int ERROR_LONG_OCTAL_SIZE
          The Java Language Specification explains this token
static int ERROR_MALFORMED_CHARACTER
          The Java Language Specification explains this token
static int ERROR_MALFORMED_STRING
          The Java Language Specification explains this token
static int ERROR_MALFORMED_UNCLOSED_CHARACTER
          The Java Language Specification explains this token
static int ERROR_MALFORMED_UNCLOSED_STRING
          The Java Language Specification explains this token
static int ERROR_UNCLOSED_CHARACTER
          The Java Language Specification explains this token
static int ERROR_UNCLOSED_COMMENT
          The Java Language Specification explains this token
static int ERROR_UNCLOSED_STRING
          The Java Language Specification explains this token
static int IDENTIFIER
          The Java Language Specification explains this token
static int LITERAL_BOOLEAN
          The Java Language Specification explains this token
static int LITERAL_CHARACTER
          The Java Language Specification explains this token
static int LITERAL_DOUBLE
          The Java Language Specification explains this token
static int LITERAL_FLOATING_POINT
          The Java Language Specification explains this token
static int LITERAL_INTEGER_DECIMAL
          The Java Language Specification explains this token
static int LITERAL_INTEGER_HEXIDECIMAL
          The Java Language Specification explains this token
static int LITERAL_INTEGER_OCTAL
          The Java Language Specification explains this token
static int LITERAL_LONG_DECIMAL
          The Java Language Specification explains this token
static int LITERAL_LONG_HEXIDECIMAL
          The Java Language Specification explains this token
static int LITERAL_LONG_OCTAL
          The Java Language Specification explains this token
static int LITERAL_NULL
          The Java Language Specification explains this token
static int LITERAL_STRING
          The Java Language Specification explains this token
static int OPERATOR_ADD
          The Java Language Specification explains this token
static int OPERATOR_ADD_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_AND
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_AND_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_COMPLIMENT
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_OR
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_OR_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_XOR
          The Java Language Specification explains this token
static int OPERATOR_BITWISE_XOR_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_COLON
          The Java Language Specification explains this token
static int OPERATOR_DECREMENT
          The Java Language Specification explains this token
static int OPERATOR_DIVIDE
          The Java Language Specification explains this token
static int OPERATOR_DIVIDE_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_EQUAL
          The Java Language Specification explains this token
static int OPERATOR_GREATER_THAN
          The Java Language Specification explains this token
static int OPERATOR_GREATER_THAN_OR_EQUAL
          The Java Language Specification explains this token
static int OPERATOR_INCREMENT
          The Java Language Specification explains this token
static int OPERATOR_LESS_THAN
          The Java Language Specification explains this token
static int OPERATOR_LESS_THAN_OR_EQUAL
          The Java Language Specification explains this token
static int OPERATOR_LOGICAL_AND
          The Java Language Specification explains this token
static int OPERATOR_LOGICAL_NOT
          The Java Language Specification explains this token
static int OPERATOR_LOGICAL_OR
          The Java Language Specification explains this token
static int OPERATOR_MOD
          The Java Language Specification explains this token
static int OPERATOR_MOD_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_MULTIPLY
          The Java Language Specification explains this token
static int OPERATOR_MULTIPLY_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_NOT_EQUAL
          The Java Language Specification explains this token
static int OPERATOR_QUESTION
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_LEFT
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_LEFT_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_RIGHT
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_RIGHT_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_RIGHT_UNSIGNED
          The Java Language Specification explains this token
static int OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN
          The Java Language Specification explains this token
static int OPERATOR_SUBTRACT
          The Java Language Specification explains this token
static int OPERATOR_SUBTRACT_ASSIGN
          The Java Language Specification explains this token
static int RESERVED_WORD_ABSTRACT
          The Java Language Specification explains this token
static int RESERVED_WORD_BOOLEAN
          The Java Language Specification explains this token
static int RESERVED_WORD_BREAK
          The Java Language Specification explains this token
static int RESERVED_WORD_BYTE
          The Java Language Specification explains this token
static int RESERVED_WORD_CASE
          The Java Language Specification explains this token
static int RESERVED_WORD_CATCH
          The Java Language Specification explains this token
static int RESERVED_WORD_CHAR
          The Java Language Specification explains this token
static int RESERVED_WORD_CLASS
          The Java Language Specification explains this token
static int RESERVED_WORD_CONST
          The Java Language Specification explains this token
static int RESERVED_WORD_CONTINUE
          The Java Language Specification explains this token
static int RESERVED_WORD_DEFAULT
          The Java Language Specification explains this token
static int RESERVED_WORD_DO
          The Java Language Specification explains this token
static int RESERVED_WORD_DOUBLE
          The Java Language Specification explains this token
static int RESERVED_WORD_ELSE
          The Java Language Specification explains this token
static int RESERVED_WORD_EXTENDS
          The Java Language Specification explains this token
static int RESERVED_WORD_FINAL
          The Java Language Specification explains this token
static int RESERVED_WORD_FINALLY
          The Java Language Specification explains this token
static int RESERVED_WORD_FLOAT
          The Java Language Specification explains this token
static int RESERVED_WORD_FOR
          The Java Language Specification explains this token
static int RESERVED_WORD_GOTO
          The Java Language Specification explains this token
static int RESERVED_WORD_IF
          The Java Language Specification explains this token
static int RESERVED_WORD_IMPLEMENTS
          The Java Language Specification explains this token
static int RESERVED_WORD_IMPORT
          The Java Language Specification explains this token
static int RESERVED_WORD_INSTANCEOF
          The Java Language Specification explains this token
static int RESERVED_WORD_INT
          The Java Language Specification explains this token
static int RESERVED_WORD_INTERFACE
          The Java Language Specification explains this token
static int RESERVED_WORD_LONG
          The Java Language Specification explains this token
static int RESERVED_WORD_NATIVE
          The Java Language Specification explains this token
static int RESERVED_WORD_NEW
          The Java Language Specification explains this token
static int RESERVED_WORD_PACKAGE
          The Java Language Specification explains this token
static int RESERVED_WORD_PRIVATE
          The Java Language Specification explains this token
static int RESERVED_WORD_PROTECTED
          The Java Language Specification explains this token
static int RESERVED_WORD_PUBLIC
          The Java Language Specification explains this token
static int RESERVED_WORD_RETURN
          The Java Language Specification explains this token
static int RESERVED_WORD_SHORT
          The Java Language Specification explains this token
static int RESERVED_WORD_STATIC
          The Java Language Specification explains this token
static int RESERVED_WORD_STRICTFP
          The Java Language Specification explains this token
static int RESERVED_WORD_SUPER
          The Java Language Specification explains this token
static int RESERVED_WORD_SWITCH
          The Java Language Specification explains this token
static int RESERVED_WORD_SYNCHRONIZED
          The Java Language Specification explains this token
static int RESERVED_WORD_THIS
          The Java Language Specification explains this token
static int RESERVED_WORD_THROW
          The Java Language Specification explains this token
static int RESERVED_WORD_THROWS
          The Java Language Specification explains this token
static int RESERVED_WORD_TRANSIENT
          The Java Language Specification explains this token
static int RESERVED_WORD_TRY
          The Java Language Specification explains this token
static int RESERVED_WORD_VOID
          The Java Language Specification explains this token
static int RESERVED_WORD_VOLATILE
          The Java Language Specification explains this token
static int RESERVED_WORD_WHILE
          The Java Language Specification explains this token
static int SEPARATOR_COMMA
          The Java Language Specification explains this token
static int SEPARATOR_LBRACE
          The Java Language Specification explains this token
static int SEPARATOR_LBRACKET
          The Java Language Specification explains this token
static int SEPARATOR_LPAREN
          The Java Language Specification explains this token
static int SEPARATOR_PERIOD
          The Java Language Specification explains this token
static int SEPARATOR_RBRACE
          The Java Language Specification explains this token
static int SEPARATOR_RBRACKET
          The Java Language Specification explains this token
static int SEPARATOR_RPAREN
          The Java Language Specification explains this token
static int SEPARATOR_SEMICOLON
          The Java Language Specification explains this token
static int WHITE_SPACE
          The Java Language Specification explains this token
 
Fields inherited from class com.Ostermiller.Syntax.Lexer.Token
INITIAL_STATE, UNDEFINED_STATE
 
Constructor Summary
JavaToken(int ID, String contents, int lineNumber, int charBegin, int charEnd)
          Create a new token.
JavaToken(int ID, String contents, int lineNumber, int charBegin, int charEnd, int state)
          Create a new token.
 
Method Summary
 String errorString()
          get a String that explains the error, if this token is an error.
 int getCharBegin()
          get the offset into the input in characters at which this token started
 int getCharEnd()
          get the offset into the input in characters at which this token ended
 String getContents()
          get the contents of this token
 String getDescription()
          A description of this token.
 int getID()
          get the ID number of this token
 int getLineNumber()
          get the line number of the input on which this token started
 int getState()
          Get an integer representing the state the tokenizer is in after returning this token.
 boolean isComment()
          Checks this token to see if it is a comment.
 boolean isError()
          Checks this token to see if it is an Error.
 boolean isIdentifier()
          Checks this token to see if it is an identifier.
 boolean isLiteral()
          Checks this token to see if it is a literal.
 boolean isOperator()
          Checks this token to see if it is a Operator.
 boolean isReservedWord()
          Checks this token to see if it is a reserved word.
 boolean isSeparator()
          Checks this token to see if it is a Separator.
 boolean isWhiteSpace()
          Checks this token to see if it is White Space.
 String toString()
          get a representation of this token as a human readable string.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

RESERVED_WORD_ABSTRACT

public static final int RESERVED_WORD_ABSTRACT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_BOOLEAN

public static final int RESERVED_WORD_BOOLEAN
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_BREAK

public static final int RESERVED_WORD_BREAK
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_BYTE

public static final int RESERVED_WORD_BYTE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CASE

public static final int RESERVED_WORD_CASE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CATCH

public static final int RESERVED_WORD_CATCH
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CHAR

public static final int RESERVED_WORD_CHAR
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CLASS

public static final int RESERVED_WORD_CLASS
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CONST

public static final int RESERVED_WORD_CONST
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_CONTINUE

public static final int RESERVED_WORD_CONTINUE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_DEFAULT

public static final int RESERVED_WORD_DEFAULT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_DO

public static final int RESERVED_WORD_DO
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_DOUBLE

public static final int RESERVED_WORD_DOUBLE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_ELSE

public static final int RESERVED_WORD_ELSE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_EXTENDS

public static final int RESERVED_WORD_EXTENDS
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_FINAL

public static final int RESERVED_WORD_FINAL
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_FINALLY

public static final int RESERVED_WORD_FINALLY
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_FLOAT

public static final int RESERVED_WORD_FLOAT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_FOR

public static final int RESERVED_WORD_FOR
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_GOTO

public static final int RESERVED_WORD_GOTO
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_IF

public static final int RESERVED_WORD_IF
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_IMPLEMENTS

public static final int RESERVED_WORD_IMPLEMENTS
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_IMPORT

public static final int RESERVED_WORD_IMPORT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_INSTANCEOF

public static final int RESERVED_WORD_INSTANCEOF
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_INT

public static final int RESERVED_WORD_INT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_INTERFACE

public static final int RESERVED_WORD_INTERFACE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_LONG

public static final int RESERVED_WORD_LONG
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_NATIVE

public static final int RESERVED_WORD_NATIVE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_NEW

public static final int RESERVED_WORD_NEW
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_PACKAGE

public static final int RESERVED_WORD_PACKAGE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_PRIVATE

public static final int RESERVED_WORD_PRIVATE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_PROTECTED

public static final int RESERVED_WORD_PROTECTED
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_PUBLIC

public static final int RESERVED_WORD_PUBLIC
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_RETURN

public static final int RESERVED_WORD_RETURN
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_SHORT

public static final int RESERVED_WORD_SHORT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_STATIC

public static final int RESERVED_WORD_STATIC
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_STRICTFP

public static final int RESERVED_WORD_STRICTFP
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_SUPER

public static final int RESERVED_WORD_SUPER
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_SWITCH

public static final int RESERVED_WORD_SWITCH
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_SYNCHRONIZED

public static final int RESERVED_WORD_SYNCHRONIZED
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_THIS

public static final int RESERVED_WORD_THIS
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_THROW

public static final int RESERVED_WORD_THROW
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_THROWS

public static final int RESERVED_WORD_THROWS
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_TRANSIENT

public static final int RESERVED_WORD_TRANSIENT
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_TRY

public static final int RESERVED_WORD_TRY
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_VOID

public static final int RESERVED_WORD_VOID
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_VOLATILE

public static final int RESERVED_WORD_VOLATILE
The Java Language Specification explains this token

See Also:
Constant Field Values

RESERVED_WORD_WHILE

public static final int RESERVED_WORD_WHILE
The Java Language Specification explains this token

See Also:
Constant Field Values

IDENTIFIER

public static final int IDENTIFIER
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_BOOLEAN

public static final int LITERAL_BOOLEAN
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_INTEGER_DECIMAL

public static final int LITERAL_INTEGER_DECIMAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_INTEGER_OCTAL

public static final int LITERAL_INTEGER_OCTAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_INTEGER_HEXIDECIMAL

public static final int LITERAL_INTEGER_HEXIDECIMAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_LONG_DECIMAL

public static final int LITERAL_LONG_DECIMAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_LONG_OCTAL

public static final int LITERAL_LONG_OCTAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_LONG_HEXIDECIMAL

public static final int LITERAL_LONG_HEXIDECIMAL
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_FLOATING_POINT

public static final int LITERAL_FLOATING_POINT
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_DOUBLE

public static final int LITERAL_DOUBLE
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_CHARACTER

public static final int LITERAL_CHARACTER
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_STRING

public static final int LITERAL_STRING
The Java Language Specification explains this token

See Also:
Constant Field Values

LITERAL_NULL

public static final int LITERAL_NULL
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_LPAREN

public static final int SEPARATOR_LPAREN
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_RPAREN

public static final int SEPARATOR_RPAREN
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_LBRACE

public static final int SEPARATOR_LBRACE
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_RBRACE

public static final int SEPARATOR_RBRACE
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_LBRACKET

public static final int SEPARATOR_LBRACKET
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_RBRACKET

public static final int SEPARATOR_RBRACKET
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_SEMICOLON

public static final int SEPARATOR_SEMICOLON
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_COMMA

public static final int SEPARATOR_COMMA
The Java Language Specification explains this token

See Also:
Constant Field Values

SEPARATOR_PERIOD

public static final int SEPARATOR_PERIOD
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_GREATER_THAN

public static final int OPERATOR_GREATER_THAN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_LESS_THAN

public static final int OPERATOR_LESS_THAN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_LESS_THAN_OR_EQUAL

public static final int OPERATOR_LESS_THAN_OR_EQUAL
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_GREATER_THAN_OR_EQUAL

public static final int OPERATOR_GREATER_THAN_OR_EQUAL
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_EQUAL

public static final int OPERATOR_EQUAL
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_NOT_EQUAL

public static final int OPERATOR_NOT_EQUAL
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_LOGICAL_NOT

public static final int OPERATOR_LOGICAL_NOT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_LOGICAL_AND

public static final int OPERATOR_LOGICAL_AND
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_LOGICAL_OR

public static final int OPERATOR_LOGICAL_OR
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_ADD

public static final int OPERATOR_ADD
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SUBTRACT

public static final int OPERATOR_SUBTRACT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_MULTIPLY

public static final int OPERATOR_MULTIPLY
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_DIVIDE

public static final int OPERATOR_DIVIDE
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_MOD

public static final int OPERATOR_MOD
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_COMPLIMENT

public static final int OPERATOR_BITWISE_COMPLIMENT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_AND

public static final int OPERATOR_BITWISE_AND
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_OR

public static final int OPERATOR_BITWISE_OR
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_XOR

public static final int OPERATOR_BITWISE_XOR
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_LEFT

public static final int OPERATOR_SHIFT_LEFT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_RIGHT

public static final int OPERATOR_SHIFT_RIGHT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_RIGHT_UNSIGNED

public static final int OPERATOR_SHIFT_RIGHT_UNSIGNED
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_ASSIGN

public static final int OPERATOR_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_ADD_ASSIGN

public static final int OPERATOR_ADD_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SUBTRACT_ASSIGN

public static final int OPERATOR_SUBTRACT_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_MULTIPLY_ASSIGN

public static final int OPERATOR_MULTIPLY_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_DIVIDE_ASSIGN

public static final int OPERATOR_DIVIDE_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_MOD_ASSIGN

public static final int OPERATOR_MOD_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_AND_ASSIGN

public static final int OPERATOR_BITWISE_AND_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_OR_ASSIGN

public static final int OPERATOR_BITWISE_OR_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_BITWISE_XOR_ASSIGN

public static final int OPERATOR_BITWISE_XOR_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_LEFT_ASSIGN

public static final int OPERATOR_SHIFT_LEFT_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_RIGHT_ASSIGN

public static final int OPERATOR_SHIFT_RIGHT_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN

public static final int OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_INCREMENT

public static final int OPERATOR_INCREMENT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_DECREMENT

public static final int OPERATOR_DECREMENT
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_QUESTION

public static final int OPERATOR_QUESTION
The Java Language Specification explains this token

See Also:
Constant Field Values

OPERATOR_COLON

public static final int OPERATOR_COLON
The Java Language Specification explains this token

See Also:
Constant Field Values

COMMENT_TRADITIONAL

public static final int COMMENT_TRADITIONAL
The Java Language Specification explains this token

See Also:
Constant Field Values

COMMENT_END_OF_LINE

public static final int COMMENT_END_OF_LINE
The Java Language Specification explains this token

See Also:
Constant Field Values

COMMENT_DOCUMENTATION

public static final int COMMENT_DOCUMENTATION
The Java Language Specification explains this token

See Also:
Constant Field Values

WHITE_SPACE

public static final int WHITE_SPACE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_IDENTIFIER

public static final int ERROR_IDENTIFIER
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_UNCLOSED_STRING

public static final int ERROR_UNCLOSED_STRING
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_MALFORMED_STRING

public static final int ERROR_MALFORMED_STRING
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_MALFORMED_UNCLOSED_STRING

public static final int ERROR_MALFORMED_UNCLOSED_STRING
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_UNCLOSED_CHARACTER

public static final int ERROR_UNCLOSED_CHARACTER
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_MALFORMED_CHARACTER

public static final int ERROR_MALFORMED_CHARACTER
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_MALFORMED_UNCLOSED_CHARACTER

public static final int ERROR_MALFORMED_UNCLOSED_CHARACTER
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_INTEGER_DECIMIAL_SIZE

public static final int ERROR_INTEGER_DECIMIAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_INTEGER_OCTAL_SIZE

public static final int ERROR_INTEGER_OCTAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_INTEGER_HEXIDECIMAL_SIZE

public static final int ERROR_INTEGER_HEXIDECIMAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_LONG_DECIMIAL_SIZE

public static final int ERROR_LONG_DECIMIAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_LONG_OCTAL_SIZE

public static final int ERROR_LONG_OCTAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_LONG_HEXIDECIMAL_SIZE

public static final int ERROR_LONG_HEXIDECIMAL_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_FLOAT_SIZE

public static final int ERROR_FLOAT_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_DOUBLE_SIZE

public static final int ERROR_DOUBLE_SIZE
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_FLOAT

public static final int ERROR_FLOAT
The Java Language Specification explains this token

See Also:
Constant Field Values

ERROR_UNCLOSED_COMMENT

public static final int ERROR_UNCLOSED_COMMENT
The Java Language Specification explains this token

See Also:
Constant Field Values
Constructor Detail

JavaToken

public JavaToken(int ID,
                 String contents,
                 int lineNumber,
                 int charBegin,
                 int charEnd)
Create a new token. The constructor is typically called by the lexer

Parameters:
ID - the id number of the token
contents - A string representing the text of the token
lineNumber - the line number of the input on which this token started
charBegin - the offset into the input in characters at which this token started
charEnd - the offset into the input in characters at which this token ended

JavaToken

public JavaToken(int ID,
                 String contents,
                 int lineNumber,
                 int charBegin,
                 int charEnd,
                 int state)
Create a new token. The constructor is typically called by the lexer

Parameters:
ID - the id number of the token
contents - A string representing the text of the token
lineNumber - the line number of the input on which this token started
charBegin - the offset into the input in characters at which this token started
charEnd - the offset into the input in characters at which this token ended
state - the state the tokenizer is in after returning this token.
Method Detail

getState

public int getState()
Get an integer representing the state the tokenizer is in after returning this token. Those who are interested in incremental tokenizing for performance reasons will want to use this method to figure out where the tokenizer may be restarted. The tokenizer starts in Token.INITIAL_STATE, so any time that it reports that it has returned to this state, the tokenizer may be restarted from there.

Specified by:
getState in class Token

getID

public int getID()
get the ID number of this token

Specified by:
getID in class Token
Returns:
the id number of the token

getContents

public String getContents()
get the contents of this token

Specified by:
getContents in class Token
Returns:
A string representing the text of the token

getLineNumber

public int getLineNumber()
get the line number of the input on which this token started

Specified by:
getLineNumber in class Token
Returns:
the line number of the input on which this token started

getCharBegin

public int getCharBegin()
get the offset into the input in characters at which this token started

Specified by:
getCharBegin in class Token
Returns:
the offset into the input in characters at which this token started

getCharEnd

public int getCharEnd()
get the offset into the input in characters at which this token ended

Specified by:
getCharEnd in class Token
Returns:
the offset into the input in characters at which this token ended

isReservedWord

public boolean isReservedWord()
Checks this token to see if it is a reserved word. Reserved words are explained in Java Language Specification.

Returns:
true if this token is a reserved word, false otherwise

isIdentifier

public boolean isIdentifier()
Checks this token to see if it is an identifier. Identifiers are explained in Java Language Specification.

Returns:
true if this token is an identifier, false otherwise

isLiteral

public boolean isLiteral()
Checks this token to see if it is a literal. Literals are explained in Java Language Specification.

Returns:
true if this token is a literal, false otherwise

isSeparator

public boolean isSeparator()
Checks this token to see if it is a Separator. Separators are explained in Java Language Specification.

Returns:
true if this token is a Separator, false otherwise

isOperator

public boolean isOperator()
Checks this token to see if it is a Operator. Operators are explained in Java Language Specification.

Returns:
true if this token is a Operator, false otherwise

isComment

public boolean isComment()
Checks this token to see if it is a comment.

Specified by:
isComment in class Token
Returns:
true if this token is a comment, false otherwise

isWhiteSpace

public boolean isWhiteSpace()
Checks this token to see if it is White Space. Usually tabs, line breaks, form feed, spaces, etc.

Specified by:
isWhiteSpace in class Token
Returns:
true if this token is White Space, false otherwise

isError

public boolean isError()
Checks this token to see if it is an Error. Unfinished comments, numbers that are too big, unclosed strings, etc.

Specified by:
isError in class Token
Returns:
true if this token is an Error, false otherwise

getDescription

public String getDescription()
A description of this token. The description should be appropriate for syntax highlighting. For example "comment" is returned for a comment.

Specified by:
getDescription in class Token
Returns:
a description of this token.

errorString

public String errorString()
get a String that explains the error, if this token is an error.

Specified by:
errorString in class Token
Returns:
a String that explains the error, if this token is an error, null otherwise.

toString

public String toString()
get a representation of this token as a human readable string. The format of this string is subject to change and should only be used for debugging purposes.

Overrides:
toString in class Object
Returns:
a string representation of this token