tokens.tfile

source file

Classes
Summary
 

Objects
Summary
 

Functions
Summary
Details

Summary of Classes  

(none)

Summary of Global Objects  

cmdTokenizer

Summary of Global Functions  

concatTokens isWordToken

Global Functions  

concatTokens (a, b)

tokens.t[45]

Concatenate two tokens. This takes two token elements in the same format returned by Tokenizer.tokenize(), and returns a combined element in the same format. The result should be as though the original pair of tokens had been concatenated in the input string.

isWordToken (tok)

tokens.t[32]

Is the given token a word? This receives a token element in the same format returned by Tokenizer.tokenize(). Returns true if the token represents a word that could be looked up in the dictionary, nil if it’s something else (such as punctuation, a number, or a quoted literal).

[Required]

Adv3Lite Library Reference Manual
Generated on 15/03/2023 from adv3Lite version 1.6.1