Type Editor

---

Type Editor / @type-editor/changeset / tokenizer/tokenize-textNode / tokenizeTextNode

Function: tokenizeTextNode()

function tokenizeTextNode<T>(
  textNode,
  encoder,
  rangeStart,
  rangeEnd,
  nodeOffset,
  target,
): void;

Defined in: tokenizer/tokenize-textNode.ts:16

Tokenize a text node by encoding each character within the specified range.

Type Parameters

Type Parameter
T

Parameters

Parameter	Type	Description
textNode	Node_2	The text node to tokenize.
encoder	TokenEncoder<T>	The encoder to use for converting characters to tokens.
rangeStart	number	The start position in the document.
rangeEnd	number	The end position in the document.
nodeOffset	number	The offset of this node in the document.
target	T[]	The array to append tokens to.

Returns

void