protected static class WebcrawlerConnector.EvaluatorTokenStream extends Object
Modifier and Type | Field and Description |
---|---|
protected int |
pos |
protected String |
text |
protected WebcrawlerConnector.EvaluatorToken |
token |
Constructor and Description |
---|
WebcrawlerConnector.EvaluatorTokenStream(String text)
Constructor.
|
Modifier and Type | Method and Description |
---|---|
void |
advance()
Go on to next token.
|
protected WebcrawlerConnector.EvaluatorToken |
nextToken() |
WebcrawlerConnector.EvaluatorToken |
peek()
Get current token.
|
protected String text
protected int pos
protected WebcrawlerConnector.EvaluatorToken token
public WebcrawlerConnector.EvaluatorTokenStream(String text)
public WebcrawlerConnector.EvaluatorToken peek() throws ManifoldCFException
ManifoldCFException
public void advance()
protected WebcrawlerConnector.EvaluatorToken nextToken() throws ManifoldCFException
ManifoldCFException