## net.sf.saxon.functions
Class Tokenize

    java.lang.Object
      net.sf.saxon.expr.Expression
          net.sf.saxon.expr.FunctionCall
              net.sf.saxon.functions.SystemFunction
                  net.sf.saxon.functions.Tokenize

All Implemented Interfaces:
  Serializable, SourceLocator, Container, EvaluableItem, SequenceIterable, InstructionInfoProvider

    public class Tokenize

extends SystemFunction

This class implements the tokenize() function for regular expression matching. This returns a sequence of strings representing the unmatched substrings: the separators which match the regular expression are not returned.

See Also:
  Serialized Form

Field Summary

Fields inherited from class net.sf.saxon.expr.Expression
`EVALUATE_METHOD, ITERATE_METHOD, PROCESS_METHOD`

Constructor Summary
`Tokenize()`

Method Summary
`RegularExpression` `getCompiledRegularExpression()` Get the compiled regular expression if available, otherwise return null
`SequenceIterator` `iterate(XPathContext c)` Iterate over the results of the function
`static void` `main(String[] args)` Simple command-line interface for testing.
`Expression` `simplify(StaticContext env)` Simplify and validate.

Methods inherited from class net.sf.saxon.functions.SystemFunction
`checkArguments, computeCardinality, computeSpecialProperties, display, getDetails, getErrorCodeForTypeErrors, getItemType, makeSystemFunction, optimize, setDetails`

Methods inherited from class net.sf.saxon.expr.FunctionCall
`getArguments, getDisplayName, getFunctionNameCode, getNumberOfArguments, iterateSubExpressions, preEvaluate, promote, replaceSubExpression, setArguments, setFunctionNameCode, typeCheck`

Methods inherited from class net.sf.saxon.expr.Expression

`adoptChildExpression, checkPermittedContents, computeDependencies, computeStaticProperties, doPromotion, effectiveBooleanValue, evaluateAsString, evaluateItem, getCardinality, getColumnNumber, getContainingProcedure, getDependencies, getExecutable, getHostLanguage, getImplementationMethod, getInstructionInfo, getIntrinsicDependencies, getLineNumber, getLocationId, getLocationProvider, getParentExpression, getPublicId, getSlotsUsed, getSpecialProperties, getSystemId, hasBadParentPointer, markTailFunctionCalls, process, resetStaticProperties, setLocationId, setParentExpression, setParentExpression, suppressValidation`

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### Tokenize

    public Tokenize()

Method Detail

### simplify

    public Expression simplify(StaticContext env)
                        throws XPathException

  Simplify and validate. This is a pure function so it can be simplified in advance if the arguments are known

  Overrides:
    `simplify` in class `FunctionCall`
  Returns:
    the simplified expression
  Throws:
    `XPathException` - if an error is discovered during expression rewriting

### getCompiledRegularExpression

    public RegularExpression getCompiledRegularExpression()

  Get the compiled regular expression if available, otherwise return null

### iterate

    public SequenceIterator iterate(XPathContext c)
                             throws XPathException

  Iterate over the results of the function

  Specified by:
    `iterate` in interface `SequenceIterable`
  Overrides:
    `iterate` in class `Expression`
  Parameters:
    `c` - supplies the context for evaluation
  Returns:
    a SequenceIterator that can be used to iterate over the result of the expression
  Throws:
    `XPathException` - if any dynamic error occurs evaluating the expression

### main

    public static void main(String[] args)
                     throws Exception

  Simple command-line interface for testing.

  Parameters:
    `args` - (1) the string to be tokenized (2) the regular expression
  Throws:
    `Exception`

Copyright (C) Michael H. Kay. All rights reserved.