Lexical Transformer

The lexical transformer tokenizes the content of special marked elements of a SAX stream, by using a lexicon file.

• Name: lexer
• Class: org.apache.cocoon.transformation.LexicalTransformer
• Cacheable: yes - uses the last modification date of the lexicon document for validation.

The lexer parses the following elements from the SAX stream, and replaces them through generated documents.

<text xmlns="http://chaperon.sourceforge.net/schema/text/1.0">
 [Text, which should be parsed]
</text>

The lexical transformer will replace these elements by a list of lexemes (tokens).

<lexemes xmlns="http://chaperon.sourceforge.net/schema/lexemes/1.0">
 <lexeme symbol="..." text="..."/>
 <lexeme symbol="..." text="..."/>
 <lexeme symbol="..." text="..."/>
</lexemes>

A detailed explanation of function and the lexicon format can be found at Chaperon.