£Á°èZ¨Ä…–K§‚«“ô4“ÒÙ´dîfUÙÃÅ WKbyʦ•êŽ…È®FÒ¿ÊÎóCozá¬S@6{Í:›œêZÌ:Š•_%:¢¾¾~;‘Ã~芩ÊǍí`ÔÑ©ú뙵'5I¿fš×WO%ø9¾«¾DK|€ùÍD”Ýs]nHÕ¶êםӼ㞪éUWŸÈË%DÒÕ¬ï‘]/Åcx ‰ï2ß]ä6G[]S£Ôϯrs{úëóµmÒï#UQxo·õÞCe]"±/aÙ&Eã4ú9Jé_ÞåëdãöKë)AÞ ¯¹ægƒÛowЍø^d™ý½ßB7áyMä9ÜÖUã !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! a XC?hsk@sdZdZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z mZddlmZddlmZmZdd lmZd!d d ZGd ddeZddZddZddZddZddZGdddeZGdddeZ GdddeZ!Gdd d e Z"dS)"zRefactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. z#Guido van Rossum N)chain)drivertokenizetoken) find_root)pytreepygram) btm_matcherTcCsTt|ggdg}g}t|jD].\}}}|dr |rD|dd}||q |S)zEReturn a sorted list of all available fix names in the given package.*fix_N) __import__pkgutilZ iter_modules__path__ startswithappend)Z fixer_pkgZ remove_prefixZpkgZ fix_namesfindernameZispkgr5/opt/alt/python39/lib64/python3.9/lib2to3/refactor.pyget_all_fix_namess   rc@s eZdZdS) _EveryNodeN__name__ __module__ __qualname__rrrrr+srcCst|tjtjfr(|jdur t|jhSt|tjrH|jrDt|jStt|tj rt }|jD]}|D]}| t|qhq`|St d|dS)zf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s) isinstancerZ NodePatternZ LeafPatterntyperZNegatedPatternZcontent_get_head_typesZWildcardPatternsetupdate Exception)Zpatrpxrrrr/s     rc Cstt}g}|D]v}|jrbzt|j}WntyF||Yq0|D]}|||qLq|jdur~||j|q||qtt j j t j j D]}|||qt|S)z^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N) collections defaultdictlistpatternrrrZ _accept_typerr python_grammarZ symbol2numbervaluestokensextenddict)Z fixer_listZ head_nodesZeveryfixerZheadsZ node_typerrr_get_headnode_dictKs$     r0csfddtdDS)zN Return the fully qualified names for fixers in the package pkg_name. csg|]}d|qS.r).0fix_nameZpkg_namerr hsz+get_fixers_from_package..F)rr5rr5rget_fixers_from_packageds r7cCs|SNr)objrrr _identityksr:csVd}tt|jfdd}ttjtjtj h}t }z|\}}||vrTq>q>|tj krl|rfq6d}q>|tj kr6|dkr6|\}}|tj ks|dkrq6|\}}|tj ks|dkrq6|\}}|tj kr|dkr|\}}|tj kr4|||\}}|tj ks.|d kr"q4|\}}qq>q6q>WntyLYn0t|S) NFcst}|d|dfS)Nrr)next)tokgenrradvancersz(_detect_future_features..advanceTfromZ __future__import(,)rgenerate_tokensioStringIOreadline frozensetrNEWLINENLCOMMENTr STRINGNAMEOPadd StopIteration)sourceZhave_docstringr?ignorefeaturestpvaluerr=r_detect_future_featuresosB           rVc@seZdZdZdS) FixerErrorzA fixer could not be loaded.N)rrr__doc__rrrrrWsrWc@seZdZddddZdZdZd4ddZdd Zd d Zd d Z ddZ ddZ d5ddZ d6ddZ ddZd7ddZddZd8ddZddZd d!Zd9d"d#Zd:d$d%Zd&Zd'Zd(d)Zd*d+Zd,d-Zd.d/Zd0d1Zd2d3ZdS);RefactoringToolF)print_function exec_functionwrite_unchanged_filesZFixr NcCsJ||_|p g|_|j|_|dur0|j|tj|_|jdrR|jj d=n|jdrf|jj d=|j d|_ g|_ t d|_g|_d|_tj|jtj|jd |_|\|_|_g|_t|_g|_g|_t|j|jD]H}|j r|j!|q||jvr|j"|q||jvr|j"|qt#|j|_$t#|j|_%dS) zInitializer. Args: fixer_names: a list of fixers to import options: a dict with configuration. explicit: a list of fixers to run even if they are explicit. NrZprintr[execr\rYF)convertlogger)&fixersexplicit_default_optionscopyoptionsr!r r*grammarkeywordsgetr\errorsloggingZ getLoggerr` fixer_logwroterZDriverrr_ get_fixers pre_order post_orderfilesbmZ BottomMatcherBMZ bmi_pre_orderZbmi_post_orderrZ BM_compatibleZ add_fixerrr0bmi_pre_order_headsbmi_post_order_heads)selfZ fixer_namesrerbr/rrr__init__sB             zRefactoringTool.__init__c CsVg}g}|jD]}t|iidg}|ddd}||jrR|t|jd}|d}|jddd |D}zt ||}Wn$t yt d ||fdYn0||j |j } | jr|jd ur||jvr|d |q|d || jdkr|| q| jdkr|| qt d| jqtd} |j| d|j| d||fS)aInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. r r2rN_cSsg|] }|qSr)title)r3r$rrrr6z.RefactoringTool.get_fixers..zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %sZpreZpostzIllegal fixer order: %rZ run_orderkey)rarrsplitr FILE_PREFIXlensplit CLASS_PREFIXjoingetattrAttributeErrorrWrerkrb log_message log_debugorderroperator attrgettersort) ruZpre_order_fixersZpost_order_fixersZ fix_mod_pathmodr4parts class_nameZ fix_classr/Zkey_funcrrrrms:             zRefactoringTool.get_fixerscOsdS)zCalled when an error occurs.Nr)rumsgargskwdsrrr log_errorszRefactoringTool.log_errorcGs|r ||}|j|dS)zHook to log a message.N)r`inforurrrrrrszRefactoringTool.log_messagecGs|r ||}|j|dSr8)r`debugrrrrr szRefactoringTool.log_debugcCsdS)zTCalled with the old version, new version, and filename of a refactored file.Nr)ruold_textnew_textfilenameequalrrr print_outputszRefactoringTool.print_outputcCs8|D].}tj|r$||||q||||qdS)z)Refactor a list of files and directories.N)ospathisdir refactor_dir refactor_file)ruitemswrite doctests_onlyZ dir_or_filerrrrefactors zRefactoringTool.refactorc Cstjd}t|D]\}}}|d||||D]>}|ds>tj|d|kr>tj||} | | ||q>dd|D|dd<qdS)zDescends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. pyzDescending into %sr2rcSsg|]}|ds|qSr1)r)r3Zdnrrrr62r{z0RefactoringTool.refactor_dir..N) rextsepwalkrrrrsplitextrr) ruZdir_namerrZpy_extdirpathZdirnames filenamesrfullnamerrrr s   zRefactoringTool.refactor_dirc Cszt|d}Wn4tyB}z|d||WYd}~dSd}~00zt|jd}W|n |0tj|d|dd}||fWdS1s0YdS) zG Do our best to decode a Python source file correctly. rbzCan't open %s: %sN)NNrr#ryencodingnewline) openOSErrorrrdetect_encodingrGcloserEread)rurferrrrrr_read_python_source4sz#RefactoringTool._read_python_sourcecCs||\}}|durdS|d7}|rn|d||||}|jsL||kr`||||||q|d|nH|||}|js|r|jr|jt|dd|||dn |d|dS)zRefactors a file.N zRefactoring doctests in %szNo doctest changes in %srw)rrzNo changes in %s)rrrefactor_docstringr\processed_filerefactor_string was_changedstr)rurrrinputroutputtreerrrrDs    zRefactoringTool.refactor_filec Cst|}d|vrtj|j_zfz|j|}WnFtyr}z.|d||jj |WYd}~W|j|j_dSd}~00W|j|j_n |j|j_0||_ | d|| |||S)aFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. rZzCan't parse %s: %s: %sNzRefactoring %s) rVr Z!python_grammar_no_print_statementrrfZ parse_stringr"r __class__rfuture_featuresr refactor_tree)rudatarrSrrrrrr[s"       zRefactoringTool.refactor_stringcCstj}|rN|d||d}|js2||krB||d|q|dn:||d}|jsj|r~|jr~|t |d|n |ddS)NzRefactoring doctests in stdinzzNo doctest changes in stdinzNo changes in stdin) sysstdinrrrr\rrrr)rurrrrrrrrefactor_stdinvs     zRefactoringTool.refactor_stdinc Cst|j|jD]}|||q||j|||j||j| }t | r|jj D]B}||vrj||rj||j tjjdd|jr||j tjjdt||D]}|||vr|||z t|WntyYqYn0|jr||jvrq||}|r|||}|dur|||D] }|js\g|_|j|qJ|j| }|D]*} | |vrg|| <|| || qqqjqTt|j|jD]}|||q|jS)aRefactors a parse tree (modifying the tree in place). For compatible patterns the bottom matcher module is used. Otherwise the tree is traversed node-to-node for matches. Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. T)r}reverser|N)rrnroZ start_tree traverse_byrsrtrrrunZleavesanyr+rarrZBaseZdepthZkeep_line_orderZ get_linenor(remover ValueErrorZfixers_appliedmatch transformreplacerr-Z finish_treer) rurrr/Z match_setnoderesultsnewZ new_matchesZfxrrrrrsJ        zRefactoringTool.refactor_treecCsV|sdS|D]D}||jD]4}||}|r|||}|dur|||}qq dS)aTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N)rrrr)ruraZ traversalrr/rrrrrrs    zRefactoringTool.traverse_bycCs|j||dur.||d}|dur.dS||k}||||||r`|d||js`dS|rv|||||n |d|dS)zR Called when a file has been refactored and there may be changes. NrzNo changes to %szNot writing changes to %s)rprrrrr\ write_file)rurrrrrrrrrrs  zRefactoringTool.processed_filec Csztj|d|dd}Wn4tyJ}z|d||WYd}~dSd}~00|Rz||Wn2ty}z|d||WYd}~n d}~00Wdn1s0Y|d|d|_dS) zWrites a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. wryrzCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)rErrrrrrl)rurrrrfprrrrrsB zRefactoringTool.write_filez>>> z... c Csg}d}d}d}d}|jddD]}|d7}||jr~|durZ|||||||}|g}||j} |d| }q |dur|||js|||jdkr| |q |dur||||||d}d}| |q |dur ||||||d |S)aRefactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) NrTkeependsrrry) splitlineslstriprPS1r-refactor_doctestfindPS2rstriprr) rurrresultblockZ block_linenoindentlinenolineirrrrsD        z"RefactoringTool.refactor_docstringc sz||}Wnhtyz}zPjtjrL|D]}d|dq4d|||j j ||WYd}~Sd}~00 ||rt |j dd}|d|d||dd} }|dds|dd7<j|d g}|r|fd d |D7}|S) zRefactors one doctest. A doctest is given as a block of lines, the first of which starts with ">>>" (possibly indented), while the remaining lines start with "..." (identically indented). z Source: %srz+Can't parse docstring in %s line %s: %s: %sNTrrrwrcsg|]}j|qSr)r)r3rrrurrr6^r{z4RefactoringTool.refactor_doctest..) parse_blockr"r`Z isEnabledForrjDEBUGrrrrrrrrendswithrpop) rurrrrrrrrZclippedrrrrDs& "z RefactoringTool.refactor_doctestcCs|jr d}nd}|js$|d|n"|d||jD]}||q6|jrl|d|jD]}||q\|jrt|jdkr|dn|dt|j|jD]"\}}}|j|g|Ri|qdS) Nwerez need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:rzThere was 1 error:zThere were %d errors:)rlrprrkrir)rurfilemessagerrrrrr summarizeas$       zRefactoringTool.summarizecCs"|j||||}t|_|S)zParses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. )rZ parse_tokens wrap_toksrHr)rurrrrrrrrxszRefactoringTool.parse_blockc csdt|||j}|D]F\}}\}}\} } } ||d7}| |d7} ||||f| | f| fVqdS)z;Wraps a tokenize stream to systematically modify start/end.rN)rrD gen_lines__next__) rurrrr,rrUZline0Zcol0Zline1Zcol1Z line_textrrrrs   zRefactoringTool.wrap_toksccsx||j}||j}|}|D]N}||r>|t|dVn(||dkrVdVntd||f|}qdVqldS)zGenerates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. Nrzline=%r, prefix=%rry)rrrrrAssertionError)rurrprefix1Zprefix2prefixrrrrrs   zRefactoringTool.gen_lines)NN)FF)FF)FF)F)NFN)N)rrrrcrrrvrmrrrrrrrrrrrrrrrrrrrrrrrrrrrYs@ 7(   O  + rYc@s eZdZdS)MultiprocessingUnsupportedNrrrrrrsrcsBeZdZfddZd fdd ZfddZfd d ZZS) MultiprocessRefactoringToolcs&tt|j|i|d|_d|_dSr8)superrrvqueue output_lockrurkwargsrrrrvsz$MultiprocessRefactoringTool.__init__Frc s>|dkrtt|||Sz ddlWnty>tYn0jdurRtd_ _ fddt |D}zn|D] }| qtt|||Wj t |D]}jdq|D]}|r| qd_nLj t |D]}jdq|D]}|r| qd_0dS)Nrrz already doing multiple processescsg|]}jjdqS))target)ZProcess_child)r3rmultiprocessingrurrr6sz8MultiprocessRefactoringTool.refactor..)rrrr ImportErrorrr RuntimeErrorZ JoinableQueueZLockrrangestartrputZis_alive)rurrrZ num_processesZ processesr$rrrrrsF                z$MultiprocessRefactoringTool.refactorcs\|j}|durX|\}}z$tt|j|i|W|jn |j0|j}q dSr8)rrhrrrZ task_done)ruZtaskrrrrrrs  z"MultiprocessRefactoringTool._childcs6|jdur|j||fntt|j|i|SdSr8)rrrrrrrrrrs  z)MultiprocessRefactoringTool.refactor_file)FFr)rrrrvrrr __classcell__rrrrrs   r)T)#rX __author__rErrrrjrr& itertoolsrZpgen2rrrZ fixer_utilrryrr r rqrr"rrr0r7r:rVrWobjectrYrrrrrrs8    (