ó ^ bc@sTdgZddlZddlZddlZddlZddlZddlmZmZm Z ddl m Z m Z dZ dZdZejƒZd„Zd efd „ƒYZefed „Zdefd „ƒYZd efd„ƒYZeZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdS(tPooliÿÿÿÿN(tProcesst cpu_countt TimeoutError(tFinalizetdebugiiicCs t|ŒS(N(tmap(targs((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pytmapstar@stMaybeEncodingErrorcBs)eZdZd„Zd„Zd„ZRS(sVWraps possible unpickleable errors, so they can be safely sent through the socket.cCsAt|ƒ|_t|ƒ|_tt|ƒj|j|jƒdS(N(treprtexctvaluetsuperR t__init__(tselfR R ((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRKscCsd|j|jfS(Ns(Error sending result: '%s'. Reason: '%s'(R R (R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt__str__Ps cCsdt|ƒS(Ns(tstr(R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt__repr__Ts(t__name__t __module__t__doc__RRR(((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyR Gs  cCs»|j}|j}t|dƒr>|jjƒ|jjƒn|dk rW||Œnd}xI|dks~|r¨||kr¨y |ƒ}Wn"ttfk r¯t dƒPnX|dkrÊt dƒPn|\} } } } } yt | | | Žf}Wnt k r}t |f}nXy|| | |fƒWnMt k r€}t ||dƒ}t d|ƒ|| | t |ffƒnXd}} }} } } |d7}q`Wt d|ƒdS(Nt_writeris)worker got EOFError or IOError -- exitingsworker got sentinel -- exitingis0Possible encoding error while sending result: %ssworker exiting after %d tasks(tputtgetthasattrRtcloset_readertNonetEOFErrortIOErrorRtTruet ExceptiontFalseR (tinqueuetoutqueuet initializertinitargstmaxtasksRRt completedttasktjobtitfuncRtkwdstresulttetwrapped((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pytworkerXs@     !    cBseZdZeZddddd„Zd„Zd„Zd„Zd„Z did„Z dd„Z dd „Z dd „Z didd „Zddd „Zed „ƒZed„ƒZed„ƒZed„ƒZd„Zd„Zd„Zd„Zed„ƒZed„ƒZRS(sH Class which supports an async version of the `apply()` builtin c Cs/|jƒtjƒ|_i|_t|_||_||_||_|dkr|y t ƒ}Wq|t k rxd}q|Xn|dkr—t dƒ‚n|dk rÂt |dƒ rÂtdƒ‚n||_g|_|jƒtjdtjd|fƒ|_t|j_t|j_|jjƒtjdtjd|j|j|j|j|jfƒ|_t|j_t|j_|jjƒtjdtjd|j|j|jfƒ|_ t|j _t|j _|j jƒt!||j"d|j|j#|j|j|j|j|j |jfddƒ|_$dS( Nis&Number of processes must be at least 1t__call__sinitializer must be a callablettargetRt exitpriorityi(%t _setup_queuestQueuet _taskqueuet_cachetRUNt_statet_maxtasksperchildt _initializert _initargsRRtNotImplementedErrort ValueErrorRt TypeErrort _processest_poolt_repopulate_poolt threadingtThreadRt_handle_workerst_worker_handlerRtdaemontstartt _handle_taskst _quick_putt _outqueuet _task_handlert_handle_resultst _quick_gett_result_handlerRt_terminate_poolt_inqueuet _terminate(Rt processesR$R%tmaxtasksperchild((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRŠsX                               cCswt}xjttt|jƒƒƒD]M}|j|}|jdk r"td|ƒ|jƒt }|j|=q"q"W|S(sCleanup after any worker processes which have exited due to reaching their specified lifetime. Returns True if any workers were cleaned up. scleaning up worker %dN( R!treversedtrangetlenRAtexitcodeRRtjoinR(RtcleanedR*R0((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt_join_exited_workersÅs"  c Cs¦xŸt|jt|jƒƒD]}|jdtd|j|j|j|j |j fƒ}|jj |ƒ|j j ddƒ|_ t|_|jƒtdƒqWdS(s€Bring the number of pool processes up to the specified number, for use after reaping workers which have exited. R2RRt PoolWorkers added workerN(RVR@RWRARR0RQRKR;R<R:tappendtnametreplaceRRGRHR(RR*tw((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRBÔs#   cCs|jƒr|jƒndS(sEClean up any exited workers and start replacements for them. N(R[RB(R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt_maintain_pooläs cCsPddlm}|ƒ|_|ƒ|_|jjj|_|jjj|_ dS(Ni(t SimpleQueue( tqueuesRbRQRKRtsendRJRtrecvRN(RRb((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyR4ês   cCs|j|||ƒjƒS(s1 Equivalent of `apply()` builtin (t apply_asyncR(RR+RR,((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pytapplyñscCs|j|||ƒjƒS(s/ Equivalent of `map()` builtin (t map_asyncR(RR+titerablet chunksize((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRøsics¶|dkrQt|jƒ‰|jj‡‡fd†t|ƒDƒˆjfƒˆStjˆ||ƒ}t|jƒ‰|jj‡fd†t|ƒDƒˆjfƒd„ˆDƒSdS(sZ Equivalent of `itertools.imap()` -- can be MUCH slower than `Pool.map()` ic3s0|]&\}}ˆj|ˆ|fifVqdS(N(t_job(t.0R*tx(R+R-(s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys sc3s0|]&\}}ˆj|t|fifVqdS(N(RkR(RlR*Rm(R-(s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys scss"|]}|D] }|Vq qdS(N((Rltchunktitem((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys sN(t IMapIteratorR7R6Rt enumeratet _set_lengthRt _get_tasks(RR+RiRjt task_batches((R+R-s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pytimapÿs cs¶|dkrQt|jƒ‰|jj‡‡fd†t|ƒDƒˆjfƒˆStjˆ||ƒ}t|jƒ‰|jj‡fd†t|ƒDƒˆjfƒd„ˆDƒSdS(sK Like `imap()` method but ordering of results is arbitrary ic3s0|]&\}}ˆj|ˆ|fifVqdS(N(Rk(RlR*Rm(R+R-(s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys sc3s0|]&\}}ˆj|t|fifVqdS(N(RkR(RlR*Rm(R-(s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys scss"|]}|D] }|Vq qdS(N((RlRnRo((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys !sN(tIMapUnorderedIteratorR7R6RRqRrRRs(RR+RiRjRt((R+R-s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pytimap_unordereds cCsAt|j|ƒ}|jj|jd|||fgdfƒ|S(s> Asynchronous equivalent of `apply()` builtin N(t ApplyResultR7R6RRkR(RR+RR,tcallbackR-((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRf#s+csæt|dƒst|ƒ}n|dkrhtt|ƒt|jƒdƒ\}}|rh|d7}qhnt|ƒdkrƒd}ntj|||ƒ}t|j |t|ƒ|ƒ‰|j j ‡fd†t |ƒDƒdfƒˆS(s< Asynchronous equivalent of `map()` builtin t__len__iiic3s0|]&\}}ˆj|t|fifVqdS(N(RkR(RlR*Rm(R-(s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pys =sN( RtlistRtdivmodRWRARRst MapResultR7R6RRq(RR+RiRjRytextraRt((R-s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRh,s ( cCsotjƒ}xB|jtks6|jrP|jtkrP|jƒtjdƒqW|j j dƒt dƒdS(Ngš™™™™™¹?sworker handler exiting( RCtcurrent_threadR9R8R7t TERMINATERattimetsleepR6RRR(tpooltthread((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyREAs  * cCs tjƒ}xt|jdƒD]q\}}d}d} zGyÃx»t|ƒD]‰\} }|jrmtdƒPny||ƒWqJtk rÒ} |d \} } y|| j | t | fƒWqÓt k rÎqÓXqJXqJW|røtdƒ|| dƒnwPWn}tk r|} |r|d nd \} } | |krX|| j | dt | fƒn|r}tdƒ|| dƒq}nXWdd}}} XqWtdƒy@tdƒ|j dƒtd ƒx|D]}|dƒqÉWWnt k rûtd ƒnXtd ƒdS( Niÿÿÿÿs'task handler found thread._state != RUNisdoing set_length()iistask handler got sentinels/task handler sending sentinel to result handlers(task handler sending sentinel to workerss/task handler got IOError when sending sentinelsstask handler exiting(ii(RCRtiterRRRqR9RR t_setR!tKeyErrorRR(t taskqueueRR#RƒtcacheR„ttaskseqt set_lengthR(R*R.R)tindtextp((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRINsR       !       cCsútjƒ}x°y |ƒ}Wn"ttfk r@tdƒdSX|jrXtdƒPn|dkrrtdƒPn|\}}}y||j||ƒWntk r¬nXd}}}qWx°|rq|jt krqy |ƒ}Wn"ttfk rtdƒdSX|dkr%tdƒqÂn|\}}}y||j||ƒWntk r_nXd}}}qÂWt |dƒrÝtdƒy5x.t dƒD] }|j j ƒs´Pn|ƒq›WWqÝttfk rÙqÝXntdt|ƒ|jƒdS( Ns.result handler got EOFError/IOError -- exitings,result handler found thread._state=TERMINATEsresult handler got sentinels&result handler ignoring extra sentinelRs"ensuring that outqueue is not fulli s7result handler exiting: len(cache)=%s, thread._state=%s(RCRRRRR9RR†R‡R€RRVRtpollRW(R#RR‰R„R(R)R*tobj((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRM‚sZ              ccsDt|ƒ}x1ttj||ƒƒ}|s1dS||fVqWdS(N(R…ttuplet itertoolstislice(R+tittsizeRm((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRs¾s  cCstdƒ‚dS(Ns:pool objects cannot be passed between processes or pickled(R=(R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt __reduce__ÇscCs5tdƒ|jtkr1t|_t|j_ndS(Ns closing pool(RR9R8tCLOSERF(R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRÌs  cCs-tdƒt|_t|j_|jƒdS(Nsterminating pool(RR€R9RFRR(R((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt terminateÒs   cCsStdƒ|jjƒ|jjƒ|jjƒx|jD]}|jƒq;WdS(Ns joining pool(RRFRYRLRORA(RRŽ((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRYØs     cCsWtdƒ|jjƒx9|jƒrR|jjƒrR|jjƒtjdƒqWdS(Ns7removing tasks from inqueue until task handler finishedi( Rt_rlocktacquiretis_aliveRRReRR‚(R"t task_handlerR•((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyt_help_stuff_finishás    c Cs‘tdƒt|_t|_tdƒ|j||t|ƒƒt|_|jdƒtdƒtjƒ|k r|j dƒn|rÔt |ddƒrÔtdƒx-|D]"} | j dkr«| j ƒq«q«Wntdƒtjƒ|k r|j dƒntd ƒtjƒ|k r,|j dƒn|rt |ddƒrtd ƒx;|D]0} | j ƒrVtd | jƒ| j ƒqVqVWndS( Nsfinalizing pools&helping task handler/workers to finishsjoining worker handlerg}Ô%­I²TiR˜sterminating workerssjoining task handlersjoining result handlersjoining pool workersscleaning up worker %d(RR€R9RRWRRRCRRYRRXR˜R›tpid( tclsRˆR"R#Rƒtworker_handlerRœtresult_handlerR‰RŽ((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRPês6              N((((RRRRRRR[RBRaR4RgRRuRwRfRht staticmethodRERIRMRsR–RR˜RYRt classmethodRP(((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyR„s0  :         4<     RxcBsDeZd„Zd„Zd„Zdd„Zdd„Zd„ZRS(cCsStjtjƒƒ|_tjƒ|_||_t|_ ||_ |||js  N( RRRR«R­RR®RR†(((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRxs      R}cBseZd„Zd„ZRS(cCstj|||ƒt|_dg||_||_|dkr`d|_t|_||j =n||t ||ƒ|_dS(Ni( RxRRR¬RR±t _chunksizet _number_leftR©Rktbool(RR‰RjtlengthRy((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyRRs      cCs|\}}|r¹||j||j|d|j+|jd8_|jdkr|jrn|j|jƒn|j|j=|jjƒzt|_ |jj ƒWd|jj ƒXqnWt |_ ||_|j|j=|jjƒzt|_ |jj ƒWd|jj ƒXdS(Nii(R±R³R´RªR7RkR¦RšRR©R²R¯R!R¬(RR*tsuccess_resulttsuccessR-((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyR†^s* "         (RRRR†(((s9/opt/alt/python27/lib64/python2.7/multiprocessing/pool.pyR}Ps RpcBs>eZd„Zd„Zdd„ZeZd„Zd„ZRS(cCsktjtjƒƒ|_tjƒ|_||_tj ƒ|_ d|_ d|_ i|_|||j#s,        ,ÿ™.-I