3 rWЧ@s"dZdZddlZddlZddlZddlZddlZddlZyejej fZ Wne k rde e fZ YnXejdZGdddeZGdddeZGd d d eZGd d d eZGd ddZddZddZddZddZddZddZGdddeZd%dd Zd&d!d"Zd#d$Z e Z!dS)'z3.9z3.8Nz^[a-zA-Z0-9_]+$c@seZdZddZdS)LexErrorcCs|f|_||_dS)N)argstext)selfmessagesr/usr/lib/python3.6/lex.py__init__:szLexError.__init__N)__name__ __module__ __qualname__r rrrr r9src@seZdZddZddZdS)LexTokencCsd|j|j|j|jfS)NzLexToken(%s,%r,%d,%d))typevaluelinenolexpos)rrrr __str__AszLexToken.__str__cCst|S)N)str)rrrr __repr__DszLexToken.__repr__N)r r r rrrrrr r@src@s4eZdZddZddZddZddZeZeZd S) PlyLoggercCs ||_dS)N)f)rrrrr r LszPlyLogger.__init__cOs|jj||ddS)N )rwrite)rmsgrkwargsrrr criticalOszPlyLogger.criticalcOs|jjd||ddS)Nz WARNING: r)rr)rrrrrrr warningRszPlyLogger.warningcOs|jjd||ddS)NzERROR: r)rr)rrrrrrr errorUszPlyLogger.errorN) r r r r rrrinfodebugrrrr rKs rc@seZdZddZddZdS) NullLoggercCs|S)Nr)rnamerrr __getattribute__^szNullLogger.__getattribute__cOs|S)Nr)rrrrrr __call__aszNullLogger.__call__N)r r r r#r$rrrr r!]sr!c@s|eZdZddZdddZdddZd d Zd d Zd dZddZ ddZ ddZ ddZ ddZ ddZddZeZdS)LexercCsd|_d|_i|_i|_i|_d|_g|_d|_i|_i|_ i|_ d|_ d|_ d|_ d|_d|_d|_d|_d|_d|_d|_d|_d|_dS)NINITIALrF)lexre lexretext lexstaterelexstateretextlexstaterenameslexstate lexstatestack lexstateinfolexstateignorelexstateerrorf lexstateeoff lexreflagslexdatarlexlen lexerrorflexeoff lextokens lexignore lexliterals lexmoduler lexoptimize)rrrr r ts.zLexer.__init__Nc Cstj|}|ri}x|jjD]\}}g}x\|D]T\}}g} xF|D]>} | sV| d rb| j| qB| jt|| dj| dfqBWq0W|j|| f|||<qW||_i|_x(|jjD]\}} t|| j|j|<qW||_|S)Nrr()copyr+itemsappendgetattrr r2r<) robjectcZnewtabkeyZritemZnewreZcreZfindexZ newfindexrefrrr clones(   & z Lexer.cloner'cCst|tjrtd|jdd}tjj||d}t|d}|j d|t f|j dt t |j dt t |j|j d t |j|j d t |j|j d t |ji}xb|jjD]T\}}g} x>t||j||j|D]"\\} } } } | j| t| | fqW| ||<qW|j d t ||j d t |ji}x,|jjD]\}}|rl|jnd||<qXW|j dt |i}x,|jjD]\}}|r|jnd||<qW|j dt |WdQRXdS)Nz&Won't overwrite existing lextab module.r(z.pywzJ# %s.py. This file automatically created by PLY (version %s). Don't edit! z_tabversion = %s z_lextokens = set(%s) z_lexreflags = %s z_lexliterals = %s z_lexstateinfo = %s z_lexstatere = %s z_lexstateignore = %s z_lexstateerrorf = %s z_lexstateeoff = %s ) isinstancetypes ModuleTypeIOErrorsplitospathjoinopenr __version__repr__tabversion__tupler9r4r;r0r+r?zipr,r-r@_funcs_to_namesr1r2r r3)rlextab outputdirZ basetabmodulefilenameZtfZtabre statenamelretitempatfuncZretextrenamesZtaberrrEZtabeofrrr writetabs6 ( zLexer.writetabc CsZt|tjr|}ntd|tj|}t|ddtkr@td|j |_ |j |_ |j |_|j t|jB|_|j|_|j|_i|_i|_xh|jjD]Z\}}g}g}x4|D],\}} |jtj||j tjBt| |fqW||j|<||j|<qWi|_x&|jjD]\}} || |j|<qWi|_ x&|j!jD]\}} || |j |<q0W|j"ddS)Nz import %sZ _tabversionz0.0zInconsistent PLY versionr&)#rJrKrLexecsysmodulesrArU ImportErrorZ _lextokensr9Z _lexreflagsr4Z _lexliteralsr;set lextokens_allZ _lexstateinfor0Z_lexstateignorer1r+r,Z _lexstaterer?r@recompileVERBOSE_names_to_funcsr2Z_lexstateerrorfr3Z _lexstateeoffbegin) rZtabfilefdictrYr\r]r^Ztxtitemr_Z func_namerErrr readtabs8   ( z Lexer.readtabcCs8|dd}t|tstd||_d|_t||_dS)Nr(zExpected a stringr)rJ StringTypes ValueErrorr5rlenr6)rrrCrrr inputs   z Lexer.inputcCsd||jkrtd|j||_|j||_|jj|d|_|jj|d|_ |j j|d|_ ||_ dS)NzUndefined stater') r+rqr)r,r*r1getr:r2r7r3r8r.)rstaterrr rms   z Lexer.begincCs|jj|j|j|dS)N)r/r@r.rm)rrurrr push_stateszLexer.push_statecCs|j|jjdS)N)rmr/pop)rrrr pop_stateszLexer.pop_statecCs|jS)N)r.)rrrr current_state!szLexer.current_statecCs|j|7_dS)N)r)rnrrr skip'sz Lexer.skipc Cs~|j}|j}|j}|j}x||kr|||kr<|d7}qx|jD]\}}|j||}|s`qFt}|j|_|j |_ ||_|j } || \} |_ | s|j r|j |_|S|j }P|j }||_ ||_||_| |} | s|j}|j}P|js(| j |jkr(td| jj| jj| j| j f||d| SW|||jkrrt}|||_|j |_ |j|_ ||_|d|_|S|jrt}|j|d|_|j |_ d|_ ||_ ||_||_|j|} ||jkrtd||||d|j}| sq| S||_td|||f||dqW|jr\t}d|_ d|_|j |_ ||_||_ ||_|j|} | S|d|_|jdkrztddS) Nr(z4%s:%d: Rule '%s' returned an unknown token type '%s'rz&Scanning error. Illegal character '%s'z"Illegal character '%s' at index %deofr'z"No input string given with input())rr6r:r5r)matchrgrouprr lastindexrendlexerZlexmatchr=rhr__code__ co_filenameco_firstlinenor r;r7r8 RuntimeError) rrr6r:r5r) lexindexfuncmtokir`Znewtokrrr token1s         "   z Lexer.tokencCs|S)Nr)rrrr __iter__szLexer.__iter__cCs|j}|dkrt|S)N)r StopIteration)rtrrr nextsz Lexer.next)N)r')r r r r rFrbrorsrmrvrxryr{rrr__next__rrrr r%ss  %(   nr%cCst|d|jS)Nregex)rA__doc__)r`rrr _get_regexsrcCs0tj|}|jj}|j|jkr,|j|j|S)N)rd _getframe f_globalsr>f_localsupdate)Zlevelsrldictrrr get_caller_module_dicts     rcCsJg}x@t||D]2\}}|r8|dr8|j||dfq|j|qW|S)Nrr()rWr@)Zfunclistnamelistresultrr"rrr rXs  rXcCsHg}x>|D]6}|r6|dr6|j||d|dfq |j|q W|S)Nrr()r@)rrnrrzrrr rls   rlc Csj|sgSdj|}ytj|tj|B}dgt|jjd}|dd}x|jjD]z\}} |j|d} t | t j t j fkr| ||f|| <||| <qV| dk rV||| <|j ddkrd|| <qVd||f|| <qVW||fg|g|gfStk rdtt|d} | dkrd} t|d| |||\} } }t|| d|||\}}}| || |||fSXdS)N|r(ignore_r)NN)rQrirjrkmax groupindexvaluesr?rtrrK FunctionType MethodTypefind Exceptionintrr_form_master_re)Zrelistreflagsrtoknamesrr)rZ lexindexnamesrrZhandlerZllistr]ZlnamesZrlistZrreZrnamesrrr rs2      rcCsd}|jd}x0t|dddD]\}}||kr"|dkr"Pq"W|dkrZt|d|}nd}d|krnt|}dj||d}||fS)Nr(_ANYr&)r&)rN enumeraterVrQ)rnamesZnonstatepartsrpartstatesZ tokennamerrr _statetokens rc@sfeZdZdddZddZddZd d Zd d Zd dZddZ ddZ ddZ ddZ ddZ dS) LexerReflectNrcCsL||_d|_g|_||_ddi|_t|_d|_|dkrBtt j n||_ dS)Nr& inclusiveF) rZ error_functokensr stateinforgrerrrdstderrlog)rrrrrrr r /s zLexerReflect.__init__cCs$|j|j|j|jdS)N) get_tokens get_literals get_states get_rules)rrrr get_all:szLexerReflect.get_allcCs|j|j|j|jS)N)validate_tokensvalidate_literalsvalidate_rulesr)rrrr validate_allAszLexerReflect.validate_allcCsp|jjdd}|s(|jjdd|_dSt|ttfsL|jjdd|_dS|sf|jjdd|_dS||_dS)NrzNo token list is definedTztokens must be a list or tupleztokens is empty)rrtrrrJlistrVr)rrrrr rHs   zLexerReflect.get_tokenscCsTi}xJ|jD]@}tj|s.|jjd|d|_||krD|jjd|d||<q WdS)NzBad token name '%s'TzToken '%s' multiply definedr()r_is_identifierr}rrr)rZ terminalsrzrrr r\s  zLexerReflect.validate_tokenscCs |jjdd|_|jsd|_dS)Nliteralsr')rrtr)rrrr rgszLexerReflect.get_literalsc CspyDx>|jD]4}t|t s&t|dkr |jjdt|d|_q WWn&tk rj|jjdd|_YnXdS)Nr(z.Invalid literal %s. Must be a single characterTzIInvalid literals specification. literals must be a sequence of characters)rrJrprrrrrT TypeError)rrCrrr rms  zLexerReflect.validate_literalscCs|jjdd|_|jrt|jttfs:|jjdd|_nx|jD]}t|t s^t|dkrx|jjdt |d|_qB|\}}t|t s|jjdt |d|_qB|dkp|dks|jjd |d|_qB||j kr|jjd |d|_qB||j |<qBWdS) Nrz)states must be defined as a tuple or listTrzMInvalid state specifier %s. Must be a tuple (statename,'exclusive|inclusive')zState name %s must be a stringr exclusivez:State type for state %s must be 'inclusive' or 'exclusive'zState '%s' already defined) rrtrrJrVrrrrrrTrpr)rrr"Z statetyperrr rxs0    zLexerReflect.get_statesc CsRdd|jD}i|_i|_i|_i|_i|_i|_x"|jD]}g|j|<g|j|<qsz*LexerReflect.get_rules..rz+No rules of the form t_rulename are definedTr$rr|ignorez,%s:%d: Rule '%s' must be defined as a string\z#%s contains a literal backslash '\'z'Rule '%s' must be defined as a functionz&%s not defined as a function or stringcSs |djjS)Nr()rr)xrrr sz(LexerReflect.get_rules..)rDcSs t|dS)Nr()rr)rrrr rs)rDreverse)rrfuncsymstrsymrerrorfeoffrrrrrrhasattrrrrr r@rJrprrsort) rZtsymbolsrrrrtoknamelinefilerrr rsb                  zLexerReflect.get_rulescCsx~|jD]r}x|j|D]r\}}|jj}|jj}tj|}|jj||j |}t |t j rjd}nd}|jj } | |kr|jjd|||jd|_q| |kr|jjd|||jd|_qt|s|jjd|||jd|_qyJtjd|t|ftj|jB} | jdr*|jjd |||jd|_Wqtjk r} zD|jjd |||j| d t|krz|jjd |||jd|_WYdd} ~ XqXqWx|j|D]\} } |j | }|d kr|jjd| d|_q||jkr|jddkr|jjd| |d|_qy@tjd| | ftj|jB} | jdrN|jjd| d|_WnTtjk r} z4|jjd| | d | kr|jjd| d|_WYdd} ~ XnXqW|j| r|j| r|jjd|d|_|jj|d}|r |}|jj}|jj}tj|}|jj|t |t j r,d}nd}|jj } | |kr\|jjd|||jd|_| |kr |jjd|||jd|_q Wx|jD]}|j|qWdS)Nrr(z'%s:%d: Rule '%s' has too many argumentsTz%%s:%d: Rule '%s' requires an argumentz2%s:%d: No regular expression defined for rule '%s'z (?P<%s>%s)r'z<%s:%d: Regular expression for rule '%s' matches empty stringz3%s:%d: Invalid regular expression for rule '%s'. %s#z6%s:%d. Make sure '#' in rule '%s' is escaped with '\#'rz'Rule '%s' must be defined as a functionrrz-Rule '%s' defined for an unspecified token %sz5Regular expression for rule '%s' matches empty stringz,Invalid regular expression for rule '%s'. %sz/Make sure '#' in rule '%s' is escaped with '\#'zNo rules defined for state '%s')rrrrrinspectZ getmodulereaddrrJrKr co_argcountrrr rrirjrkrr}rrrrrtvalidate_module)rrufnamerrrmodulerZreqargsnargsrCer"rZefuncrrr rs                 zLexerReflect.validate_rulesc Csytj|\}}Wntk r&dSXtjd}tjd}i}|d7}xv|D]n}|j|}|sj|j|}|r|jd} |j| } | s||| <n$tj|} |j j d| || | d|_ |d7}qNWdS)Nz\s*def\s+(t_[a-zA-Z_0-9]*)\(z\s*(t_[a-zA-Z_0-9]*)\s*=r(z7%s:%d: Rule %s redefined. Previously defined on line %dT) rZgetsourcelinesrMrirjr}r~rtZ getsourcefilerr) rrlinesZlinenZfreZsreZ counthashrrr"prevr[rrr r@s*         zLexerReflect.validate_module)Nr)r r r r rrrrrrrrrrrrrr r.s   BgrFrYc %s|dkr d}d} ddi} t} || _| dkr6ttj} |rL|dkrLttj}|rT|rfddtD} t| } d| krtj| dj| d<nt d} | j d }|rt |t rd |kr|d |}t | | |d }|j|s|jrtd |o|r4y | j|| | ja| ja| a| Stk r2YnX|rd|jd |j|jd|j|jd|jt| _x|jD]}| jj|qtWt |jttfrt|jdj |j| _!n|j| _!| jt| j!B| _"|j} i}x| D]}g}xX|j#|D]J\}}|j$j%}|j$j&}|j'd|t(|f|r|jd|t(||qWx@|j)|D]2\}}|j'd||f|rP|jd|||qPW|||<qW|r|jdxt|D]l}t*|||| |j+\}}}|| j,|<|| j-|<|| j.|<|rx&t/|D]\}}|jd|||qWqWxl| j0D]`\}}|dkr$|dkr$| j,|j1| j,d| j-|j1| j-d| j.|j1| j.dq$W| | _2| j,d| _3| j-d| _4|| _5|j6| _7| j7j dd| _8|j9| _:|j9j dd| _;| j;s| j|j=j dd| _?x| j0D]\} }|dkr\| |j9kr:| j.__file__r r __package__rG)rrzCan't build lexerzlex: tokens = %rzlex: literals = %rzlex: states = %rrz (?P<%s>%s)z(lex: Adding rule %s -> '%s' (state '%s')z#lex: ==== MASTER REGEXS FOLLOW ====z"lex: state '%s' : regex[%d] = '%s'r'zNo t_error rule is definedrz1No error rule is defined for exclusive state '%s'z2No ignore rule is defined for exclusive state '%s'r(z import %sz#Couldn't write lextab module %r. %srI)Jr%r=rrdrdirdictrerrrtrJrrrr SyntaxErrorrorrsrrfrrrrrgr9rrrVrrQr;rhrrrrr@rrrrr+r,r-rr?extendr0r)r*r4rr1r:rr2r7rrr3r8rKrLrNrcrArOrPdirnamerbrM)%rrBr optimizerYrZnowarnrZZdebuglogZerrorlogrrZlexobjZ_itemsZpkgZlinforzZregexsruZ regex_listrrrrr"rr)Zre_textZre_namesrrstyperZsrcfilerZpkgnamerr)rr lex_s                             $rcCs|sVy&tjd}t|}|j}|jWn*tk rTtjjdtjj}YnX|rb|j }nt }|||rz|j }nt }x0|}|sPtjjd|j |j |j |jfqWdS)Nr(z/Reading from standard input (type EOF to end): z(%s,%r,%d,%d) )rdargvrRreadclose IndexErrorstdoutrstdinrsrrrrr)rdatar[rZ_inputZ_tokenrrrr runmains*   rcsfdd}|S)Ncs tdrt|_n|_|S)Nr$)rrr)r)rrr set_regexBs  zTOKEN..set_regexr)rrr)rr TOKENAs r) NNFFrYrFNNN)NN)"rSrUrirdrKr>rOrZ StringTypeZ UnicodeTyperpAttributeErrorrbytesrjrrrrBrrr!r%rrrXrlrrrrrrTokenrrrr "sD  F  (3 @ "