Your IP : 216.73.216.213
Z��Z�/ � @ s� d d l m Z d d l Z e j d e d e d e d e � d d l m Z d d l m Z m Z m
Z
e � Z Gd
d � d e � Z d S) � )�absolute_importN�BOL�EOL�EOF� NOT_FOUND� )�Errors)r r r c @ s� e Z d Z d Z d d d d � Z d d � Z d d � Z d
d � Z d d
� Z d d � Z d d � Z
d d � Z d d d � Z d d � Z
d S)�Scannera�
A Scanner is used to read tokens from a stream of characters
using the token set specified by a Plex.Lexicon.
Constructor:
Scanner(lexicon, stream, name = '')
See the docstring of the __init__ method for details.
Methods:
See the docstrings of the individual methods for more
information.
read() --> (value, text)
Reads the next lexical token from the stream.
position() --> (name, line, col)
Returns the position of the last token read using the
read() method.
begin(state_name)
Causes scanner to change state.
produce(value [, text])
Causes return of a token value to the caller of the
Scanner.
� Nc C s� d | _ d | _ d | _ d | _ d | _ d | _ d | _ d | _ d | _ d | _ d | _
| | _ | | _ | | _
g | _ d | _ | j d � d | _ d | _ d | _ t | _ d | _ | d k r� | d | d | _ | _ d S)a�
Scanner(lexicon, stream, name = '')
|lexicon| is a Plex.Lexicon instance specifying the lexical tokens
to be recognised.
|stream| can be a file object or anything which implements a
compatible read() method.
|name| is optional, and may be the name of the file being
scanned or any other identifying string.
r r
r N� )�trace�buffer�
buf_start_pos�next_pos�cur_pos�cur_line� start_pos�
start_line� start_col�text�
state_name�lexicon�stream�name�queue�
initial_state�begin�cur_line_startr �cur_char� input_state)�selfr r r Z initial_pos� r! �G/opt/alt/python35/lib64/python3.5/site-packages/Cython/Plex/Scanners.py�__init__H s0
zScanner.__init__c C s� | j } xs | s~ | j � \ | _ } | d k rM | j d � | j � q | j | | j � } | d k r | j | � q W| d } | d =| S)a"
Read the next lexical token from the stream and return a
tuple (value, text), where |value| is the value associated with
the token as specified by the Lexicon, and |text| is the actual
string read from the stream. Returns (None, '') on end of file.
Nr )r � scan_a_tokenr �produce�eofZperform)r r �action�value�resultr! r! r" �readp s
z Scanner.readc C s | j | _ | j | _ | j | j | _ | j � } | d k r� | j ri t d | | j | j f � | j | j | j
| j | j
� } | | f S| j | j k r� | j t k r� | j
� | j d k s� | j t k r� d St j | | j � � d S)z�
Read the next input sequence recognised by the machine
and return (text, action). Returns ('', None) on end of
file.
Nz"Scanner: read: Performing %s %d:%dr
)r
N)r r r r r r �run_machine_inlinedr �printr
r r r � next_charr r ZUnrecognizedInputr )r r' r r! r! r" r$ � s$
zScanner.scan_a_tokenc C sv | j } | j } | j } | j } | j } | j } | j } | j } | j } t | � }
d \ } } }
} } } } | j
} x�| r� t d | d | | t | � f � | d } | d k r� | | | | | | | f \ } } }
} } } } | } | j
| t � } | t k r-| o*| j
d � } | r�| rKt d | d � | } | d k rT| } | | } | |
k r�| | } | d 7} n� | j | } | j j d
� } | j | d � | } | | _ | | 7} | | _ t | � }
| | 8} | r| | } | d 7} n d } | d k r6t } d } q�| sKt } d
} q�| } q| d k rod } d } q| d k r�| d 7} | } } t } d } q| d
k r�t } d } qd } q� | r�t d � | d k r| | |
| | | | f \ } } } } } } } n d } Pq� W| | _ | | _ | | _ | | _ | | _ | | _ | rr| d k rrt d | � | S)z;
Inlined version of run_machine for speed.
Nr r
zState %d, %d/%d:%s -->�numberr' �elsezState %dr i �
r � � � ZblockedzDoing %s)Nr r r r
r r )r r r r r r r r
r �lenr r, �repr�getr r r r* r r r )r �stater r r r r r r
r Zbuf_lenZb_actionZ b_cur_posZ
b_cur_lineZb_cur_line_startZ
b_cur_charZ
b_input_stateZ
b_next_posr r' �cZ new_stateZ buf_index�discard�datar! r! r" r+ � s�
0
' zScanner.run_machine_inlinedc C sV | j } | j r, t d d | | j f � | d k r� | j | _ | j � } | d k rq t | _ d | _ q&| s� t | _ d | _ q&| | _ n� | d k r� d | _ d | _ nm | d k r� | j d 7_ | j | _ | _ t
| _ d | _ n* | d k r t | _ d | _ n d
| _ | j rRt d | | j t | j � f � d S)
NzScanner: next: %s [%d] %d� � r r0 r r1 r2 r3 r
z--> [%d] %d %sz )
r r r, r r Z read_charr r r r r r r5 )r r r8 r! r! r" r- s6 zScanner.next_charc C s | j | j | j f S)am
Return a tuple (name, line, col) representing the location of
the last token read using the read() method. |name| is the
name that was provided to the Scanner constructor; |line|
is the line number in the stream (1-based); |col| is the
position within the line of the first character of the token
(0-based).
)r r r )r r! r! r" �position( s zScanner.positionc C s
| j � S)zOPython accessible wrapper around position(), only for error reporting.
)r= )r r! r! r" � get_position3 s zScanner.get_positionc C s" | j j | � | _ | | _ d S)z8Set the current state of the scanner to the named state.N)r Zget_initial_stater r )r r r! r! r" r 8 s z
Scanner.beginc C s/ | d k r | j } | j j | | f � d S)a�
Called from an action procedure, causes |value| to be returned
as the token value from read(). If |text| is supplied, it is
returned in place of the scanned text.
produce() can be called more than once during a single call to an action
procedure, in which case the tokens are queued up and returned one
at a time by subsequent calls to read(), until the queue is empty,
whereupon scanning resumes.
N)r r �append)r r( r r! r! r" r% > s zScanner.producec C s d S)z_
Override this method if you want something to be done at
end of file.
Nr! )r r! r! r" r& M s z Scanner.eof)�__name__�
__module__� __qualname__�__doc__r# r* r$ r+ r- r= r> r r% r& r! r! r! r" r s ( j r )
�
__future__r �cython�declare�objectr
r ZRegexpsr r r r r r! r! r! r" �<module>
s "