{ fc @ s d Z d d l m Z d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
Z
d d l m Z d d l
m Z m Z m Z d d l m Z d d l m Z m Z d d
l m Z d d l m Z e d Z d
e f d YZ d Z d Z d Z d Z! e j" d d f k rgd d l# Z# e# j$ Z% d Z& d Z' n e$ Z% e! Z& e! Z' d Z( d e f d YZ) d e* f d YZ+ d e f d YZ, d e+ f d YZ- d S( s Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
i( t with_statements# Guido van Rossum N( t chaini ( t drivert tokenizet token( t find_root( t pytreet pygram( t btm_utils( t btm_matcherc C sz t | g g d g } g } xU t j | j D]A \ } } } | j d r1 | rb | d } n | j | q1 q1 W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_i ( t
__import__t pkgutilt iter_modulest __path__t
startswitht append( t fixer_pkgt
remove_prefixt pkgt fix_namest findert namet ispkg( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt get_all_fix_names" s "
t
_EveryNodec B s e Z RS( ( t __name__t
__module__( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR . s c C s t | t j t j f rC | j d k r3 t n t | j g St | t j rt | j rk t
| j St n t | t j r t } x5 | j D]* } x! | D] } | j t
| q Wq W| St
d | d S( sf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t
isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR% 2 s
c C s t j t } g } x | D] } | j r y t | j } Wn t k
r^ | j | q XxU | D] } | | j | qf Wq | j d k r | | j j | q | j | q Wx: t t
j j j
t
j j D] } | | j | q Wt | S( s^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N( t collectionst defaultdictt listt patternR% R R t _accept_typeR! R R t python_grammart
symbol2numbert
itervaluest tokenst extendt dict( t
fixer_listt
head_nodest everyt fixert headst node_type( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _get_headnode_dictN s"
c C s( g t | t D] } | d | ^ q S( sN
Return the fully qualified names for fixers in the package pkg_name.
t .( R t False( t pkg_namet fix_name( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt get_fixers_from_packageg s c C s | S( N( ( t obj( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _identityn s i i c C s | j d d S( Nu
u
( t replace( t input( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _from_system_newlinesu s c C s* t j d k r" | j d t j S| Sd S( Ns
u
( t ost linesepRF ( RG ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _to_system_newlinesw s c s t } t j t j | j f d } t t j t j t j f } t
} ykxdt r| \ } } | | k r q] q] | t j k r | r Pn t } q] | t j
k r| d k r| \ } } | t j
k s | d k r Pn | \ } } | t j
k s| d k r Pn | \ } } | t j k rY| d k rY| \ } } n xa | t j
k r| j | | \ } } | t j k s| d k rPn | \ } } q\Wq] Pq] WWn t k
rn Xt | S( Nc s j } | d | d f S( Ni i ( t next( t tok( t gen( s( /usr/lib64/python2.7/lib2to3/refactor.pyt advance s u fromu
__future__u importu (u ,( R@ R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR" t Truet STRINGt NAMEt OPt addt
StopIteration( t sourcet have_docstringRO t ignoret featurest tpt value( ( RN s( /usr/lib64/python2.7/lib2to3/refactor.pyt _detect_future_features sD
t
FixerErrorc B s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyRd s t RefactoringToolc B s! e Z i e d 6e d 6Z d Z d Z d d d Z d Z d Z d Z
d Z d Z e e d
Z
e e d Z d Z e e d
Z d Z e d Z d Z d Z d e d d Z d d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( t print_functiont write_unchanged_filest FixR c C s | | _ | p g | _ | j j | _ | d k rI | j j | n | j d re t j | _ n t j
| _ | j j d | _ g | _
t j d | _ g | _ t | _ t j | j d t j d | j | _ | j \ | _ | _ g | _ t j | _ g | _ g | _ x} t! | j | j D]f } | j" rT| j j# | q2| | j k rv| j j$ | q2| | j k r2| j j$ | q2q2Wt% | j | _& t% | j | _' d S( s Initializer.
Args:
fixer_names: a list of fixers to import
options: a dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Rg Rh Rf t convertt loggerN(( t fixerst explicitt _default_optionst copyt optionsR! R' R t! python_grammar_no_print_statementt grammarR2 t getRh t errorst loggingt getLoggerRk t fixer_logR@ t wroteR t DriverR Rj t
get_fixerst pre_ordert
post_ordert filest bmt
BottomMatchert BMt
bmi_pre_ordert bmi_post_orderR t
BM_compatiblet add_fixerR R> t bmi_pre_order_headst bmi_post_order_heads( t selft fixer_namesRp Rm R; ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt __init__ s<
c C s g } g } x| j D]} t | i i d g } | j d d d } | j | j rr | t | j } n | j d } | j d j g | D] } | j ^ q } y t
| | } Wn' t k
r t d | | f n X| | j
| j }
|
j r?| j t k r?| | j k r?| j d | q n | j d | |
j d
k rn| j |
q |
j d k r| j |
q t d |
j q Wt j d
} | j d | | j d | | | f S( s Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
R
R? i it _t s Can't find %s.%ss Skipping optional fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rl R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRd Rp Rw Rm RW t log_messaget log_debugt orderR t operatort
attrgettert sort( R t pre_order_fixerst post_order_fixerst fix_mod_patht modRB t partsR+ t
class_namet fix_classR; t key_func( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyRz s8 /
c O s d S( s Called when an error occurs.N( ( R t msgt argst kwds( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt log_error s c G s' | r | | } n | j j | d S( s Hook to log a message.N( Rk t info( R R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s
c G s' | r | | } n | j j | d S( N( Rk t debug( R R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s
c C s d S( sT Called with the old version, new version, and filename of a
refactored file.N( ( R t old_textt new_textt filenamet equal( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt print_output! s c C sP xI | D]A } t j j | r5 | j | | | q | j | | | q Wd S( s) Refactor a list of files and directories.N( RI t patht isdirt refactor_dirt
refactor_file( R t itemst writet
doctests_onlyt dir_or_file( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt refactor& s
c C s t j d } x t j | D] \ } } } | j d | | j | j xe | D]] } | j d rW t j j | d | k rW t j j | | } | j | | | qW qW Wg | D] }
|
j d s |
^ q | (q Wd S( s Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
t pys Descending into %sR? i N(
RI t extsept walkR R R R t splitextR R ( R t dir_nameR R t py_extt dirpatht dirnamest filenamesR t fullnamet dn( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR / s
c C s y t | d } Wn' t k
r< } | j d | | d SXz t j | j d } Wd | j Xt | d d | } t | j
| f SWd QXd S( sG
Do our best to decode a Python source file correctly.
t rbs Can't open %s: %si NR* t encoding( NN( t opent IOErrorR R! R t detect_encodingRR t closet _open_with_encodingRH t read( R R t ft errR ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt _read_python_sourceC s c C s | j | \ } } | d k r% d S| d 7} | r | j d | | j | | } | j sl | | k r | j | | | | | q | j d | nc | j | | } | j s | r | j r | j t | d | d | d | n | j d | d S( s Refactors a file.Nu
s Refactoring doctests in %ss No doctest changes in %siR R s No changes in %s( R R! R t refactor_docstringRh t processed_filet refactor_stringt was_changedt unicode( R R R R RG R t outputt tree( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR S s
c C s t | } d | k r* t j | j _ n zM y | j j | } Wn0 t k
ru } | j d | | j j | d SXWd | j | j _ X| | _
| j d | | j | | | S( sF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
Rg s Can't parse %s: %s: %sNs Refactoring %s(
Rc R Rq R Rr t parse_stringR( R t __class__R t future_featuresR t
refactor_tree( R t dataR R` R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR j s c C s t j j } | ro | j d | j | d } | j sI | | k r_ | j | d | q | j d nS | j | d } | j s | r | j r | j t | d | n
| j d d S( Ns Refactoring doctests in stdins s No doctest changes in stdins No changes in stdin(
t syst stdinR R R Rh R R R R ( R R RG R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt refactor_stdin s
c
C s x- t | j | j D] } | j | | q W| j | j | j | j | j | j | j j | j } xt
| j rcx| j j D]} | | k r | | r | | j
d t j j d t | j r | | j
d t j j n x[t | | D]F} | | | k r9| | j | n y t | Wn t k
r]qn X| j r|| | j k r|qn | j | } | r| j | | } | d k rU| j | x9 | j D]+ } | j sg | _ n | j j | qW| j j | j } x? | D]4 } | | k r6g | | >> s ... c
C s g } d } d } d } d } x+| j t D]} | d 7} | j j | j r | d k r | j | j | | | | n | } | g } | j | j } | | } q. | d k r| j | | j s | | | j j
d k r| j | q. | d k r/| j | j | | | | n d } d } | j | q. W| d k rz| j | j | | | | n d j | S( s Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
i i u
u N(
R! t
splitlinesRW t lstripR t PS1R6 t refactor_doctestt findt PS2t rstripR R (
R RG R t resultt blockt block_linenot indentt linenot linet i( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR ( s:
c
C ss y | j | | | } Wnu t k
r } | j j t j rm x* | D] } | j d | j d qG Wn | j d | | | j j
| | SX| j | | rot | j
t } | | d | | d } } | d g | d k s t | | d j d s | d c d 7>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
s
Source: %su
s+ Can't parse docstring in %s line %s: %s: %si ii ( t parse_blockR( Rk t isEnabledForRu t DEBUGR R R R R R R R RW t AssertionErrort endswithR t popR (
R R R R R R R R R t clipped( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR S s&
# .c C s | j r d } n d } | j s4 | j d | n1 | j d | x | j D] } | j | qN W| j r | j d x! | j D] } | j | q Wn | j rt | j d k r | j d n | j d t | j x0 | j D]" \ } } } | j | | | q Wn d S( Nt weres
need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rx R} R Rw Rt R ( R R t filet messageR R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt summarizep s$
c C s1 | j j | j | | | } t | _ | S( s Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
( R t parse_tokenst wrap_toksRS R ( R R R R R ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s ! c c s t j | j | | j } xe | D]] \ } } \ } } \ } }
} | | d 7} | | d 7} | | | | f | |
f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RP t gen_linesRL ( R R R R R5 R Rb t line0t col0t line1t col1t line_text( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s
(c c s | | j } | | j } | } xi | D]a } | j | rN | t | Vn4 | | j d k rl d Vn t d | | f | } q' Wx t r d Vq Wd S( s Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
u
s line=%r, prefix=%rR N( R R R R R R RW ( R R R t prefix1t prefix2t prefixR ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s
N( R R R@ Rn R R R! R Rz R R R R R R R R R R R R R R R R R R R R R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyRf s:
4 ( O +
t MultiprocessingUnsupportedc B s e Z RS( ( R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s t MultiprocessRefactoringToolc B s5 e Z d Z e e d d Z d Z d Z RS( c O s/ t t | j | | d | _ d | _ d S( N( t superR R R! t queuet output_lock( R R t kwargs( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s i c C sf | d k r( t t | j | | | Sy d d l } Wn t k
rQ t n X| j d k rp t d n | j | _ | j
| _ g t | D] } | j
d | j ^ q } z; x | D] } | j q Wt t | j | | | Wd | j j x$ t | D] } | j j d qWx' | D] } | j r5| j q5q5Wd | _ Xd S( Ni is already doing multiple processest target( R! R R t multiprocessingt ImportErrorR R" R! t RuntimeErrort
JoinableQueuet LockR# t xranget Processt _childt startR t putt is_alive( R R R R t
num_processesR& R t processesR+ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s2
+
c C so | j j } xY | d k rj | \ } } z t t | j | | Wd | j j X| j j } q Wd S( N( R" Rs R! R! R R t task_done( R t taskR R$ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR- s c O sE | j d k r( | j j | | f n t t | j | | Sd S( N( R" R! R/ R! R R ( R R R$ ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s ( R R R R@ R R- R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyR s
(. Re t
__future__R t
__author__RI R
R Ru R R- RQ t itertoolsR t pgen2R R R t
fixer_utilR R R R R t buR R~ RW R R( R R% R> RC RE t version_infot codecsR R RH RK Rc Rd t objectRf R R ( ( ( s( /usr/lib64/python2.7/lib2to3/refactor.pyt sH (