Type Editor

---

Type Editor / @type-editor/model / content-parser/TokenStream / TokenStream

Class: TokenStream

Defined in: packages/model/src/content-parser/TokenStream.ts:7

Tokenizes and provides streaming access to a content expression string. Splits the expression into individual tokens for parsing.

Constructors

Constructor

new TokenStream(expression, nodeTypes): TokenStream;

Defined in: packages/model/src/content-parser/TokenStream.ts:26

Creates a new token stream from a content expression string.

Parameters

Parameter	Type	Description
expression	string	The content expression to tokenize
nodeTypes	Readonly<Record<string, NodeType>>	Map of available node types for name resolution

Returns

TokenStream

Accessors

inline

Get Signature

get inline(): boolean;

Defined in: packages/model/src/content-parser/TokenStream.ts:48

Tracks whether the expression contains inline content (true), block content (false), or hasn't been determined yet (null). Used to prevent mixing inline and block content in expressions.

Returns

boolean

Set Signature

set inline(isInline): void;

Defined in: packages/model/src/content-parser/TokenStream.ts:52

Parameters

Parameter	Type
isInline	boolean

Returns

void

---

next

Get Signature

get next(): string;

Defined in: packages/model/src/content-parser/TokenStream.ts:88

Gets the current token without consuming it.

Returns

string

The current token, or undefined if at end of stream

---

nodeTypes

Get Signature

get nodeTypes(): Readonly<Record<string, NodeType>>;

Defined in: packages/model/src/content-parser/TokenStream.ts:79

Map of available node types for name resolution

Returns

Readonly<Record<string, NodeType>>

---

pos

Get Signature

get pos(): number;

Defined in: packages/model/src/content-parser/TokenStream.ts:59

Current position in the token array

Returns

number

Set Signature

set pos(position): void;

Defined in: packages/model/src/content-parser/TokenStream.ts:69

Current position in the token array

Throws

If position is negative

Parameters

Parameter	Type
position	number

Returns

void

Methods

eat()

eat(tok): boolean;

Defined in: packages/model/src/content-parser/TokenStream.ts:99

Attempts to consume a specific token. If the current token matches, advances the position and returns true.

Parameters

Parameter	Type	Description
tok	string	The token to try to consume

Returns

boolean

true if the token was consumed, false otherwise

---

err()

err(str): never;

Defined in: packages/model/src/content-parser/TokenStream.ts:113

Throws a syntax error with context about the content expression.

Parameters

Parameter	Type	Description
str	string	The error message

Returns

never

Throws

Always throws with the provided message and expression context