Package COM.INFORMATIMAGO.COMMON-LISP.PARSER.SCANNER

An abstract scanner class.

A method to the SCAN-NEXT-TOKEN generic function needs to be provided.


License:

    AGPL3
    
    Copyright Pascal J. Bourguignon 2004 - 2015
    
    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.
    
    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.
    
    You should have received a copy of the GNU Affero General Public License
    along with this program.
    If not, see <http://www.gnu.org/licenses/>

ACCEPT

ADVANCE-LINE

BUFFERED-SCANNER

(define-scanner name &key (superclass (quote buffered-scanner)) terminals (alphanumerics abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz0123456789) (spaces |#()|) (skip-spaces t))
macro

DO:             This macro generates a simple scanner.  This defines
                subclass of BUFFERED-SCANNER named `NAME'.

RETURN:         NAME

SUPERCLASS:     The name of the superclass of the class NAME. Should
                be a subclass of SCANNER.

TOKEN-CLASS-NAME:
                The name of the class of tokens. Should TOKEN or a
                subclass of TOKEN.

TERMINALS:      A list of couples (name-of-terminals
                regexp-of-terminal) or strings containing the literal terminal..

ALPHANUMERICS:  Characters that may be present in identifiers and keywords,
                and as such, shall be eaten greedily when scanning for keywords.

SKIP-SPACES:    When NIL, the spaces are not eaten between token, but
                they must be dealt with by definiting a terminal
                covering them.

GENERATE-SCANNER

(getchar peek-stream)
generic-function
RETURN: The next character in the PEEK-STREAM, advancing.
(make-current-token scanner)
generic-function
Makes an instance of the TOKEN class or a subclass
thereof, filled with the current information in the scanner object.
(nextchar peek-stream &optional peek-type)
generic-function
DO:             Just like CL:PEEK-CHAR.

                If peek-type is not supplied or NIL, peek-char returns
                the next character to be read from input-stream,
                without actually removing it from input-stream. The
                next time input is done from input-stream, the
                character will still be there.

                If  peek-type is T, then peek-char skips over
                whitespace[2] characters, but not comments, and then
                performs the peeking operation on the next
                character. The last character examined, the one that
                starts an object, is not removed from  input-stream.

                If peek-type is a CHARACTER, then peek-char skips over
                input characters until a character that is char= to
                that character is found; that character is left in
                input-stream.

RETURN:         The same character that will be returned by the next
                (getchar self).

NOTE:           There's no conforming way to determine whether a
                character has the whitespace[2] syntax in the current
                *readtable*.  Therefore we use instead the
                PEEK-STREAM-SPACES method to get the list of spaces.

PRINT-PARSER-ERROR

PRINT-SCANNER-ERROR

(readline peek-stream)
generic-function
RETURN: A string containing the read line.
(scan-next-token scanner &optional parser-data)
generic-function
DO:           Scans a new token and store it into (scanner-current-token scanner)
PARSER-DATA:  Some parsers give information to the scanner.
RETURN:       (scanner-current-token scanner).
scanner
class
An abstract scanner.
Class precedence list: SCANNER STANDARD-OBJECT T
Class init args: SOURCE FILE LINE COLUMN STATE SPACES TAB-WIDTH TOKEN-KIND-PACKAGE

SCANNER-BUFFER

(scanner-column x)
generic-function
The number of the current column.

SCANNER-CURRENT-TEXT

(scanner-current-token x)
generic-function
The last token read.

SCANNER-END-OF-LINE-P

SCANNER-END-OF-SOURCE-P

scanner-error
condition
A scanner error.
Class precedence list: SCANNER-ERROR SIMPLE-ERROR SIMPLE-CONDITION ERROR SERIOUS-CONDITION CONDITION STANDARD-OBJECT T
Class init args: FORMAT-CONTROL FORMAT-ARGUMENTS FILE LINE COLUMN STATE CURRENT-TOKEN SCANNER FORMAT-CONTROL FORMAT-ARGUMENTS
(scanner-error-column x)
generic-function
The column on which the scanner error was detected.
(scanner-error-current-token x)
generic-function
The scanner token where error was detected.
(scanner-error-format-arguments x)
generic-function
The error message format control arguments.
(scanner-error-format-control x)
generic-function
The error message format control string.
scanner-error-invalid-character
condition
An invalid character scanner error.
Class precedence list: SCANNER-ERROR-INVALID-CHARACTER SCANNER-ERROR SIMPLE-ERROR SIMPLE-CONDITION ERROR SERIOUS-CONDITION CONDITION STANDARD-OBJECT T
Class init args: FORMAT-CONTROL FORMAT-ARGUMENTS FILE LINE COLUMN STATE CURRENT-TOKEN SCANNER FORMAT-CONTROL FORMAT-ARGUMENTS INVALID-CHARACTER
(scanner-error-invalid-character x)
generic-function
The invalid character that made the scanner error.
(scanner-error-line x)
generic-function
The line on which the scanner error was detected.
(scanner-error-scanner x)
generic-function
The scanner that detected the error.
(scanner-error-state x)
generic-function
The scanner state when error was detected.
(scanner-file x)
generic-function
The namestring of the current file being scanned.
(scanner-line x)
generic-function
The number of the current line.
(scanner-source x)
generic-function
The source can be a PEEK-STREAM, a STREAM, or a STRING.
(scanner-spaces x)
generic-function
A string containing the characters considered space by SKIP-SPACES.
(scanner-state x)
generic-function
The state of the scanner.
(scanner-tab-width x)
generic-function
TAB aligns to column number modulo TAB-WIDTH.

SCANNER-TOKEN-KIND-PACKAGE

(skip-spaces scanner)
generic-function
DO: Skips over the spaces in the input stream. Updates line and column slots.
RETURN: line; column
token
class
A syntactic element.
Class precedence list: TOKEN STANDARD-OBJECT T
Class init args: KIND TEXT COLUMN LINE
(token-column x)
generic-function
Returns the column of the first character of the token.
(token-kind x)
generic-function
Returns the kind of the token.
(token-line x)
generic-function
Returns the line where the token was found.
(token-text x)
generic-function
Returns the literal text the token.
(ungetchar peek-stream ch)
generic-function
DO: Unread the character CH from the PEEK-STREAM.

WORD-EQUAL