Lhogho
0.0.028
|
Go to the source code of this file.
Tokenization methods | |
#define | TOKENIZE_DATA 0 |
tokenize as data More... | |
#define | TOKENIZE_COMMANDS 1 |
tokenize as commands More... | |
#define | TOKENIZE_READWORD 2 |
tokenize as for READWORD function More... | |
#define | TOKENIZE_READLIST 3 |
tokenize as for READLIST function More... | |
#define | ENBAR(ch) (((unsigned char)ch)>127 ? (ch) : enbar[(unsigned char)ch]) |
#define | DEBAR(ch) ((ch)>31 ? (ch) : debar[(unsigned char)ch]) |
char_t | enbar [128] |
table for a->|a| conversions More... | |
char_t | debar [32] |
table for |a|->a conversions More... | |
void | init_parser () |
initializes parser More... | |
atom_t | tokenize (atom_t input, int method) |
tokenizes into a list More... | |
atom_t | trim_shell_comment (atom_t word) |
trims shell comment (if any) More... | |
atom_t | purify (atom_t word) |
purifies a word More... | |
atom_t | build_syntax_tree (atom_t function) |
parses body of user-defined function More... | |
#define ENBAR | ( | ch) | (((unsigned char)ch)>127 ? (ch) : enbar[(unsigned char)ch]) |
#define DEBAR | ( | ch) | ((ch)>31 ? (ch) : debar[(unsigned char)ch]) |
void init_parser | ( | ) |
input | word, subword or list to be tokenized |
method | method of tokenization |
Tokenizes a word, a subword or a list into a list. If possible makes all words as subwords. Backslashes and bars in words are preserved (i.e. words are not purified). Comments and line continuations are ignored.
If the method is TOKENIZE_DATA then the input is tokenized as if it contains Logo data. If the method is TOKENIZE_COMMANDS then the input is tokenized as if it contains Logo commands. If the method is TOKENIZE_READWORD then the input is tokenized as expected by READWORD function.
If the input is a list then all its elements are tokenized one-by-one.
Return value is the tokenized list. In there is an error, returns an error atom which error code is ERROR_INCOMPLETE_PAIR, error position points the position in the word (0-based) and the error source is the word itself.
Definition at line 275 of file parser.c.
word | word containing source text |
Trims a shell comment from the beginning of the word. Shell comment can be only the first line if its first two characters are #!
. If a shell comment is trimmed, then the result is a subword from the first character on the second line, otherwise the input word is returned as is but with increased reference count.
Definition at line 1055 of file parser.c.
func | var atom for the parse context |
Parses completely a function. Its source is stored in its body as word, subword, data-tokenized list or command-tokenized list. Building algorithm:
TO
... ENDs
and create them as subfunctionsDefinition at line 2278 of file parser.c.
char_t enbar[128] |