Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV NEXT | FRAMES NO FRAMES

## Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream
org.apache.lucene.analysis | API and code to convert text into indexable tokens.
org.apache.lucene.analysis.de | Support for indexing and searching of German text.
org.apache.lucene.analysis.ru | Support for indexing and searching Russian text.
org.apache.lucene.analysis.standard | A grammar-based tokenizer constructed with JavaCC.
org.apache.lucene.search.highlight | The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis
`class` | `CharTokenizer` An abstract base class for simple, character-oriented tokenizers.
`class` | `LetterTokenizer` A LetterTokenizer is a tokenizer that divides text at non-letters.
`class` | `LowerCaseFilter` Normalizes token text to lower case.
`class` | `LowerCaseTokenizer` LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
`class` | `PorterStemFilter` Transforms the token stream as per the Porter stemming algorithm.
`class` | `StopFilter` Removes stop words from a token stream.
`class` | `TokenFilter` A TokenFilter is a TokenStream whose input is another token stream.
`class` | `Tokenizer` A Tokenizer is a TokenStream whose input is a Reader.
`class` | `WhitespaceTokenizer` A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

Fields in org.apache.lucene.analysis declared as TokenStream
`protected TokenStream` | `TokenFilter.input` The source of tokens for this filter.

Methods in org.apache.lucene.analysis that return TokenStream
`TokenStream` | `StopAnalyzer.tokenStream(String fieldName, Reader reader)` Filters LowerCaseTokenizer with StopFilter.
`TokenStream` | `SimpleAnalyzer.tokenStream(String fieldName, Reader reader)`
`TokenStream` | `PerFieldAnalyzerWrapper.tokenStream(String fieldName, Reader reader)`
`TokenStream` | `Analyzer.tokenStream(String fieldName, Reader reader)` Creates a TokenStream which tokenizes all the text in the provided Reader.
`TokenStream` | `Analyzer.tokenStream(Reader reader)` Deprecated. use tokenStream(String, Reader) instead.
`TokenStream` | `WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)`

Constructors in org.apache.lucene.analysis with parameters of type TokenStream
`StopFilter(TokenStream in, String[] stopWords)` Constructs a filter which removes words from the input TokenStream that are named in the array of words.
`StopFilter(TokenStream in, Hashtable stopTable)` Deprecated. Use `StopFilter.StopFilter(TokenStream, Set)` instead
`StopFilter(TokenStream in, Set stopWords)` Constructs a filter which removes words from the input TokenStream that are named in the Set.
`TokenFilter(TokenStream input)` Construct a token stream filtering the given input.
`PorterStemFilter(TokenStream in)`
`LowerCaseFilter(TokenStream in)`

Uses of TokenStream in org.apache.lucene.analysis.de

Subclasses of TokenStream in org.apache.lucene.analysis.de
`class` | `GermanStemFilter` A filter that stems German words.

Methods in org.apache.lucene.analysis.de that return TokenStream
`TokenStream` | `GermanAnalyzer.tokenStream(String fieldName, Reader reader)` Creates a TokenStream which tokenizes all the text in the provided Reader.

Constructors in org.apache.lucene.analysis.de with parameters of type TokenStream
`GermanStemFilter(TokenStream in)`
`GermanStemFilter(TokenStream in, Hashtable exclusiontable)` Deprecated. Use `GermanStemFilter.GermanStemFilter(org.apache.lucene.analysis.TokenStream, java.util.Set)` instead.
`GermanStemFilter(TokenStream in, Set exclusionSet)` Builds a GermanStemFilter that uses an exclusiontable.

Uses of TokenStream in org.apache.lucene.analysis.ru

Subclasses of TokenStream in org.apache.lucene.analysis.ru
`class` | `RussianLetterTokenizer` A RussianLetterTokenizer is a tokenizer that extends LetterTokenizer by additionally looking up letters in a given "russian charset".
`class` | `RussianLowerCaseFilter` Normalizes token text to lower case, analyzing given ("russian") charset.
`class` | `RussianStemFilter` A filter that stems Russian words.

Methods in org.apache.lucene.analysis.ru that return TokenStream
`TokenStream` | `RussianAnalyzer.tokenStream(String fieldName, Reader reader)` Creates a TokenStream which tokenizes all the text in the provided Reader.

Constructors in org.apache.lucene.analysis.ru with parameters of type TokenStream
`RussianStemFilter(TokenStream in, char[] charset)`
`RussianLowerCaseFilter(TokenStream in, char[] charset)`

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard
`class` | `StandardFilter` Normalizes tokens extracted with `StandardTokenizer`.
`class` | `StandardTokenizer` A grammar-based tokenizer constructed with JavaCC.

Methods in org.apache.lucene.analysis.standard that return TokenStream
`TokenStream` | `StandardAnalyzer.tokenStream(String fieldName, Reader reader)` Constructs a `StandardTokenizer` filtered by a `StandardFilter`, a `LowerCaseFilter` and a `StopFilter`.

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream
`StandardFilter(TokenStream in)` Construct filtering in.

Uses of TokenStream in org.apache.lucene.search.highlight

Methods in org.apache.lucene.search.highlight that return TokenStream
`static TokenStream` | `TokenSources.getAnyTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)` A convenience method that tries a number of approaches to getting a token stream.
`static TokenStream` | `TokenSources.getTokenStream(TermPositionVector tpv)`
`static TokenStream` | `TokenSources.getTokenStream(TermPositionVector tpv, boolean tokenPositionsGuaranteedContiguous)` Low level api.
`static TokenStream` | `TokenSources.getTokenStream(IndexReader reader, int docId, String field)`
`static TokenStream` | `TokenSources.getTokenStream(IndexReader reader, int docId, String field, Analyzer analyzer)`

Methods in org.apache.lucene.search.highlight with parameters of type TokenStream
`String` | `Highlighter.getBestFragment(TokenStream tokenStream, String text)` Highlights chosen terms in a text, extracting the most relevant section.
`String[]` | `Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments)` Highlights chosen terms in a text, extracting the most relevant sections.
`TextFragment[]` | `Highlighter.getBestTextFragments(TokenStream tokenStream, String text, boolean mergeContiguousFragments, int maxNumFragments)` Low level api to get the most relevant (formatted) sections of the document.
`String` | `Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator)` Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "...").

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV NEXT | FRAMES NO FRAMES

Copyright © 2000-2005 Apache Software Foundation. All Rights Reserved.