com.Ostermiller.Syntax.Lexer
Class JavaToken
java.lang.Object
com.Ostermiller.Syntax.Lexer.Token
com.Ostermiller.Syntax.Lexer.JavaToken
public class JavaToken
- extends Token
A JavaToken is a token that is returned by a lexer that is lexing a java
source file. It has several attributes describing the token:
The type of token, the text of the token, the line number on which it
occurred, the number of characters into the input at which it started, and
similarly, the number of characters into the input at which it ended.
The tokens should comply with the
Java
Language Specification.
|
Constructor Summary |
JavaToken(int ID,
String contents,
int lineNumber,
int charBegin,
int charEnd)
Create a new token. |
JavaToken(int ID,
String contents,
int lineNumber,
int charBegin,
int charEnd,
int state)
Create a new token. |
|
Method Summary |
String |
errorString()
get a String that explains the error, if this token is an error. |
int |
getCharBegin()
get the offset into the input in characters at which this token started |
int |
getCharEnd()
get the offset into the input in characters at which this token ended |
String |
getContents()
get the contents of this token |
String |
getDescription()
A description of this token. |
int |
getID()
get the ID number of this token |
int |
getLineNumber()
get the line number of the input on which this token started |
int |
getState()
Get an integer representing the state the tokenizer is in after
returning this token. |
boolean |
isComment()
Checks this token to see if it is a comment. |
boolean |
isError()
Checks this token to see if it is an Error. |
boolean |
isIdentifier()
Checks this token to see if it is an identifier. |
boolean |
isLiteral()
Checks this token to see if it is a literal. |
boolean |
isOperator()
Checks this token to see if it is a Operator. |
boolean |
isReservedWord()
Checks this token to see if it is a reserved word. |
boolean |
isSeparator()
Checks this token to see if it is a Separator. |
boolean |
isWhiteSpace()
Checks this token to see if it is White Space. |
String |
toString()
get a representation of this token as a human readable string. |
RESERVED_WORD_ABSTRACT
public static final int RESERVED_WORD_ABSTRACT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_BOOLEAN
public static final int RESERVED_WORD_BOOLEAN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_BREAK
public static final int RESERVED_WORD_BREAK
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_BYTE
public static final int RESERVED_WORD_BYTE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CASE
public static final int RESERVED_WORD_CASE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CATCH
public static final int RESERVED_WORD_CATCH
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CHAR
public static final int RESERVED_WORD_CHAR
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CLASS
public static final int RESERVED_WORD_CLASS
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CONST
public static final int RESERVED_WORD_CONST
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_CONTINUE
public static final int RESERVED_WORD_CONTINUE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_DEFAULT
public static final int RESERVED_WORD_DEFAULT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_DO
public static final int RESERVED_WORD_DO
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_DOUBLE
public static final int RESERVED_WORD_DOUBLE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_ELSE
public static final int RESERVED_WORD_ELSE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_EXTENDS
public static final int RESERVED_WORD_EXTENDS
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_FINAL
public static final int RESERVED_WORD_FINAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_FINALLY
public static final int RESERVED_WORD_FINALLY
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_FLOAT
public static final int RESERVED_WORD_FLOAT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_FOR
public static final int RESERVED_WORD_FOR
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_GOTO
public static final int RESERVED_WORD_GOTO
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_IF
public static final int RESERVED_WORD_IF
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_IMPLEMENTS
public static final int RESERVED_WORD_IMPLEMENTS
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_IMPORT
public static final int RESERVED_WORD_IMPORT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_INSTANCEOF
public static final int RESERVED_WORD_INSTANCEOF
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_INT
public static final int RESERVED_WORD_INT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_INTERFACE
public static final int RESERVED_WORD_INTERFACE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_LONG
public static final int RESERVED_WORD_LONG
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_NATIVE
public static final int RESERVED_WORD_NATIVE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_NEW
public static final int RESERVED_WORD_NEW
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_PACKAGE
public static final int RESERVED_WORD_PACKAGE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_PRIVATE
public static final int RESERVED_WORD_PRIVATE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_PROTECTED
public static final int RESERVED_WORD_PROTECTED
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_PUBLIC
public static final int RESERVED_WORD_PUBLIC
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_RETURN
public static final int RESERVED_WORD_RETURN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_SHORT
public static final int RESERVED_WORD_SHORT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_STATIC
public static final int RESERVED_WORD_STATIC
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_STRICTFP
public static final int RESERVED_WORD_STRICTFP
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_SUPER
public static final int RESERVED_WORD_SUPER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_SWITCH
public static final int RESERVED_WORD_SWITCH
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_SYNCHRONIZED
public static final int RESERVED_WORD_SYNCHRONIZED
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_THIS
public static final int RESERVED_WORD_THIS
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_THROW
public static final int RESERVED_WORD_THROW
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_THROWS
public static final int RESERVED_WORD_THROWS
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_TRANSIENT
public static final int RESERVED_WORD_TRANSIENT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_TRY
public static final int RESERVED_WORD_TRY
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_VOID
public static final int RESERVED_WORD_VOID
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_VOLATILE
public static final int RESERVED_WORD_VOLATILE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
RESERVED_WORD_WHILE
public static final int RESERVED_WORD_WHILE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
IDENTIFIER
public static final int IDENTIFIER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_BOOLEAN
public static final int LITERAL_BOOLEAN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_INTEGER_DECIMAL
public static final int LITERAL_INTEGER_DECIMAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_INTEGER_OCTAL
public static final int LITERAL_INTEGER_OCTAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_INTEGER_HEXIDECIMAL
public static final int LITERAL_INTEGER_HEXIDECIMAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_LONG_DECIMAL
public static final int LITERAL_LONG_DECIMAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_LONG_OCTAL
public static final int LITERAL_LONG_OCTAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_LONG_HEXIDECIMAL
public static final int LITERAL_LONG_HEXIDECIMAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_FLOATING_POINT
public static final int LITERAL_FLOATING_POINT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_DOUBLE
public static final int LITERAL_DOUBLE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_CHARACTER
public static final int LITERAL_CHARACTER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_STRING
public static final int LITERAL_STRING
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
LITERAL_NULL
public static final int LITERAL_NULL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_LPAREN
public static final int SEPARATOR_LPAREN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_RPAREN
public static final int SEPARATOR_RPAREN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_LBRACE
public static final int SEPARATOR_LBRACE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_RBRACE
public static final int SEPARATOR_RBRACE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_LBRACKET
public static final int SEPARATOR_LBRACKET
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_RBRACKET
public static final int SEPARATOR_RBRACKET
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_SEMICOLON
public static final int SEPARATOR_SEMICOLON
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_COMMA
public static final int SEPARATOR_COMMA
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
SEPARATOR_PERIOD
public static final int SEPARATOR_PERIOD
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_GREATER_THAN
public static final int OPERATOR_GREATER_THAN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_LESS_THAN
public static final int OPERATOR_LESS_THAN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_LESS_THAN_OR_EQUAL
public static final int OPERATOR_LESS_THAN_OR_EQUAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_GREATER_THAN_OR_EQUAL
public static final int OPERATOR_GREATER_THAN_OR_EQUAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_EQUAL
public static final int OPERATOR_EQUAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_NOT_EQUAL
public static final int OPERATOR_NOT_EQUAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_LOGICAL_NOT
public static final int OPERATOR_LOGICAL_NOT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_LOGICAL_AND
public static final int OPERATOR_LOGICAL_AND
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_LOGICAL_OR
public static final int OPERATOR_LOGICAL_OR
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_ADD
public static final int OPERATOR_ADD
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SUBTRACT
public static final int OPERATOR_SUBTRACT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_MULTIPLY
public static final int OPERATOR_MULTIPLY
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_DIVIDE
public static final int OPERATOR_DIVIDE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_MOD
public static final int OPERATOR_MOD
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_COMPLIMENT
public static final int OPERATOR_BITWISE_COMPLIMENT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_AND
public static final int OPERATOR_BITWISE_AND
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_OR
public static final int OPERATOR_BITWISE_OR
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_XOR
public static final int OPERATOR_BITWISE_XOR
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_LEFT
public static final int OPERATOR_SHIFT_LEFT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_RIGHT
public static final int OPERATOR_SHIFT_RIGHT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_RIGHT_UNSIGNED
public static final int OPERATOR_SHIFT_RIGHT_UNSIGNED
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_ASSIGN
public static final int OPERATOR_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_ADD_ASSIGN
public static final int OPERATOR_ADD_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SUBTRACT_ASSIGN
public static final int OPERATOR_SUBTRACT_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_MULTIPLY_ASSIGN
public static final int OPERATOR_MULTIPLY_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_DIVIDE_ASSIGN
public static final int OPERATOR_DIVIDE_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_MOD_ASSIGN
public static final int OPERATOR_MOD_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_AND_ASSIGN
public static final int OPERATOR_BITWISE_AND_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_OR_ASSIGN
public static final int OPERATOR_BITWISE_OR_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_BITWISE_XOR_ASSIGN
public static final int OPERATOR_BITWISE_XOR_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_LEFT_ASSIGN
public static final int OPERATOR_SHIFT_LEFT_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_RIGHT_ASSIGN
public static final int OPERATOR_SHIFT_RIGHT_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN
public static final int OPERATOR_SHIFT_RIGHT_UNSIGNED_ASSIGN
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_INCREMENT
public static final int OPERATOR_INCREMENT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_DECREMENT
public static final int OPERATOR_DECREMENT
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_QUESTION
public static final int OPERATOR_QUESTION
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
OPERATOR_COLON
public static final int OPERATOR_COLON
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
COMMENT_TRADITIONAL
public static final int COMMENT_TRADITIONAL
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
COMMENT_END_OF_LINE
public static final int COMMENT_END_OF_LINE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
COMMENT_DOCUMENTATION
public static final int COMMENT_DOCUMENTATION
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
WHITE_SPACE
public static final int WHITE_SPACE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_IDENTIFIER
public static final int ERROR_IDENTIFIER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_UNCLOSED_STRING
public static final int ERROR_UNCLOSED_STRING
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_MALFORMED_STRING
public static final int ERROR_MALFORMED_STRING
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_MALFORMED_UNCLOSED_STRING
public static final int ERROR_MALFORMED_UNCLOSED_STRING
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_UNCLOSED_CHARACTER
public static final int ERROR_UNCLOSED_CHARACTER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_MALFORMED_CHARACTER
public static final int ERROR_MALFORMED_CHARACTER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_MALFORMED_UNCLOSED_CHARACTER
public static final int ERROR_MALFORMED_UNCLOSED_CHARACTER
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_INTEGER_DECIMIAL_SIZE
public static final int ERROR_INTEGER_DECIMIAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_INTEGER_OCTAL_SIZE
public static final int ERROR_INTEGER_OCTAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_INTEGER_HEXIDECIMAL_SIZE
public static final int ERROR_INTEGER_HEXIDECIMAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_LONG_DECIMIAL_SIZE
public static final int ERROR_LONG_DECIMIAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_LONG_OCTAL_SIZE
public static final int ERROR_LONG_OCTAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_LONG_HEXIDECIMAL_SIZE
public static final int ERROR_LONG_HEXIDECIMAL_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_FLOAT_SIZE
public static final int ERROR_FLOAT_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_DOUBLE_SIZE
public static final int ERROR_DOUBLE_SIZE
- The Java
Language Specification explains this token
- See Also:
- Constant Field Values
ERROR_FLOAT
pu