£Á°èZ¨Ä…–K§‚«“ô4“ÒÙ´dîfUÙÃÅ WKbyʦ•êŽ…È®FÒ¿ÊÎóCozá¬S@6{Í:›œêZÌ:Š•_%:¢¾¾~;‘Ã~芩ÊǍí`ÔÑ©ú뙵'5I¿fš×WO%ø9¾«¾DK|€ùÍD”Ýs]nHÕ¶êםӼ㞪éUWŸÈË%DÒÕ¬ï‘]/Åcx ‰ï2ß]ä6G[]S£Ôϯrs{úëóµmÒï#UQxo·õÞCe]"±/aÙ&Eã4ú9Jé_ÞåëdãöKë)AÞ ¯¹ægƒÛowЍø^d™ý½ßB7áyMä9ÜÖUã !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! B YI~dh@s0ddgZddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm Z m Z dZ dZ dZeZdd Zd d ZGd d d eZGdddZddZGdddeZd$ddZddZGdddeZGdddeZeZGdddeZGdd d eZGd!d"d"eZ Gd#ddeZ!dS)%Pool ThreadPoolN)util) get_context TimeoutErrorcCs tt|S)N)listmap)argsr 9/opt/alt/python37/lib64/python3.7/multiprocessing/pool.pymapstar+srcCstt|d|dS)Nrr)r itertoolsstarmap)r r r r starmapstar.src@seZdZddZddZdS)RemoteTracebackcCs ||_dS)N)tb)selfrr r r __init__6szRemoteTraceback.__init__cCs|jS)N)r)rr r r __str__8szRemoteTraceback.__str__N)__name__ __module__ __qualname__rrr r r r r5src@seZdZddZddZdS)ExceptionWithTracebackcCs0tt|||}d|}||_d||_dS)Nz """ %s""") tracebackformat_exceptiontypejoinexcr)rr rr r r r<s zExceptionWithTraceback.__init__cCst|j|jffS)N) rebuild_excr r)rr r r __reduce__Asz!ExceptionWithTraceback.__reduce__N)rrrrr"r r r r r;srcCst||_|S)N)r __cause__)r rr r r r!Ds r!cs,eZdZfddZddZddZZS)MaybeEncodingErrorcs.t||_t||_tt||j|jdS)N)reprr valuesuperr$r)rr r&) __class__r r rPs  zMaybeEncodingError.__init__cCsd|j|jfS)Nz(Error sending result: '%s'. Reason: '%s')r&r )rr r r rUszMaybeEncodingError.__str__cCsd|jj|fS)Nz<%s: %s>)r(r)rr r r __repr__YszMaybeEncodingError.__repr__)rrrrrr) __classcell__r r )r(r r$Ls r$r Fc Cs|dk r(t|tr|dks(td||j}|j}t|drR|j|j |dk rb||d}xP|dks|r||kry |} Wn$t t fk rt dPYnX| dkrt dP| \} } } } }yd| | |f}WnHtk r.}z(|r| tk rt||j}d|f}Wdd}~XYnXy|| | |fWnRtk r}z2t||d}t d ||| | d|ffWdd}~XYnXd} } }} } }|d7}qjWt d |dS) NrzMaxtasks {!r} is not valid_writerrz)worker got EOFError or OSError -- exitingzworker got sentinel -- exitingTFz0Possible encoding error while sending result: %szworker exiting after %d tasks) isinstanceintAssertionErrorformatputgethasattrr+close_readerEOFErrorOSErrorrdebug Exception_helper_reraises_exceptionr __traceback__r$)inqueueoutqueue initializerinitargsZmaxtasksZwrap_exceptionr0r1Z completedtaskjobifuncr kwdsresultewrappedr r r worker]sJ       $ rGcCs|dS)Nr )Zexr r r r9sr9c@s(eZdZdZddZd=ddZdd Zd d Zd d ZddZ difddZ d>ddZ d?ddZ d@ddZ ddZdAddZdBddZdiddfdd ZdCd!d"ZdDd#d$Zed%d&Zed'd(Zed)d*Zed+d,Zd-d.Zd/d0Zd1d2Zd3d4Zed5d6Zed7d8Zd9d:Z d;d<Z!dS)ErTcOs|jj||S)N)_ctxProcess)rr rCr r r rIsz Pool.ProcessNr c Csl|pt|_|t|_i|_t|_||_ ||_ ||_ |dkrPt pNd}|dkr`td|dk rxt|sxtd||_g|_|tjtj|fd|_d|j_t|j_|jtjtj|j|j|j|j|jfd|_d|j_t|j_|jtjtj|j|j |jfd|_!d|j!_t|j!_|j!t"j#||j$|j|j%|j|j|j|j|j!|jfdd|_&dS)Nrz&Number of processes must be at least 1zinitializer must be a callable)targetr T)r Z exitpriority)'rrH _setup_queuesqueue SimpleQueue _taskqueue_cacheRUN_state_maxtasksperchild _initializer _initargsos cpu_count ValueErrorcallable TypeError _processes_pool_repopulate_pool threadingZThreadr_handle_workers_worker_handlerdaemonstart _handle_tasks _quick_put _outqueue _task_handler_handle_results _quick_get_result_handlerrZFinalize_terminate_pool_inqueue _terminate)r processesr=r>Zmaxtasksperchildcontextr r r rsT         z Pool.__init__cCsZd}xPttt|jD]:}|j|}|jdk rtd||d}|j|=qW|S)NFzcleaning up worker %dT)reversedrangelenr\exitcoderr7r)rZcleanedrArGr r r _join_exited_workerss   zPool._join_exited_workersc Cs|xvt|jt|jD]^}|jt|j|j|j|j |j |j fd}|j ||j dd|_ d|_|tdqWdS)N)rJr rIZ PoolWorkerTz added worker)rpr[rqr\rIrGrkrerTrUrS_wrap_exceptionappendnamereplacerarbrr7)rrAwr r r r]s  zPool._repopulate_poolcCs|r|dS)N)rsr])rr r r _maintain_poolszPool._maintain_poolcCs4|j|_|j|_|jjj|_|jjj|_ dS)N) rHrNrkrer+sendrdr4recvrh)rr r r rLs   zPool._setup_queuescCs||||S)N) apply_asyncr1)rrBr rCr r r applysz Pool.applycCs|||t|S)N) _map_asyncrr1)rrBiterable chunksizer r r r szPool.mapcCs|||t|S)N)r~rr1)rrBrrr r r rsz Pool.starmapcCs|||t|||S)N)r~r)rrBrrcallbackerror_callbackr r r starmap_asyncs zPool.starmap_asyncc csny0d}x&t|D]\}}||||fifVqWWn8tk rh}z||dt|fifVWdd}~XYnXdS)Nr) enumerater8r9)rZ result_jobrBrrAxrEr r r _guarded_task_generations zPool._guarded_task_generationrcCs|jtkrtd|dkrFt|j}|j||j|||j f|S|dkr\td |t |||}t|j}|j||jt ||j fdd|DSdS)NzPool not runningrzChunksize must be 1+, not {0:n}css|]}|D] }|Vq qdS)Nr ).0chunkitemr r r EszPool.imap..)rRrQrX IMapIteratorrPrOr0r_job _set_lengthr/r _get_tasksr)rrBrrrD task_batchesr r r imap)s(     z Pool.imapcCs|jtkrtd|dkrFt|j}|j||j|||j f|S|dkr\td |t |||}t|j}|j||jt ||j fdd|DSdS)NzPool not runningrzChunksize must be 1+, not {0!r}css|]}|D] }|Vq qdS)Nr )rrrr r r rbsz&Pool.imap_unordered..)rRrQrXIMapUnorderedIteratorrPrOr0rrrr/rrr)rrBrrrDrr r r imap_unorderedGs&      zPool.imap_unorderedcCsB|jtkrtdt|j||}|j|jd|||fgdf|S)NzPool not runningr)rRrQrX ApplyResultrPrOr0r)rrBr rCrrrDr r r r|ds  zPool.apply_asynccCs|||t|||S)N)r~r)rrBrrrrr r r map_asyncoszPool.map_asyncc Cs|jtkrtdt|ds$t|}|dkrTtt|t|jd\}}|rT|d7}t|dkrdd}t |||}t |j |t|||d} |j || j||df| S)NzPool not running__len__rr)r)rRrQrXr2r divmodrqr\rr MapResultrPrOr0rr) rrBrZmapperrrrZextrarrDr r r r~ws&   zPool._map_asynccCsTt}x0|jtks$|jr8|jtkr8|tdq W|j dt ddS)Ng?zworker handler exiting) r^current_threadrRrQrP TERMINATErytimesleeprOr0rr7)poolthreadr r r r_s  zPool._handle_workersc Csjt}xt|jdD]\}}d}zx|D]}|jr@tdPy ||Wq*tk r} zB|dd\} } y|| | d| fWnt k rYnXWdd} ~ XYq*Xq*W|rtd|r|dnd} || dwPWdd}}} XqWtdy:td| dtd x|D]} |dq&WWn t k rZtd YnXtd dS) Nz'task handler found thread._state != RUNrFzdoing set_length()rrztask handler got sentinelz/task handler sending sentinel to result handlerz(task handler sending sentinel to workersz/task handler got OSError when sending sentinelsztask handler exiting) r^riterr1rRrr7r8_setKeyErrorr0r6) taskqueuer0r<rcacherZtaskseqZ set_lengthr?rEr@idxpr r r rcsB          zPool._handle_tasksc Cst}xy |}Wn"ttfk r6tddSX|jrJtdP|dkr^tdP|\}}}y||||Wntk rYnXd}}}q Wx|r>|jt kr>y |}Wn"ttfk rtddSX|dkrtdq|\}}}y||||Wntk r.YnXd}}}qWt |drtdy,x&t dD]}|j stP|qbWWnttfk rYnXtdt||jdS) Nz.result handler got EOFError/OSError -- exitingz,result handler found thread._state=TERMINATEzresult handler got sentinelz&result handler ignoring extra sentinelr4z"ensuring that outqueue is not full z7result handler exiting: len(cache)=%s, thread._state=%s)r^rr6r5rr7rRrrrr2rpr4pollrq)r<r1rrr?r@rAobjr r r rgsZ             zPool._handle_resultsccs4t|}x&tt||}|s"dS||fVq WdS)N)rtuplerislice)rBitsizerr r r r s zPool._get_taskscCs tddS)Nz:pool objects cannot be passed between processes or pickled)NotImplementedError)rr r r r"szPool.__reduce__cCs&td|jtkr"t|_t|j_dS)Nz closing pool)rr7rRrQCLOSEr`)rr r r r3s  z Pool.closecCs$tdt|_t|j_|dS)Nzterminating pool)rr7rrRr`rl)rr r r terminate s zPool.terminatecCsntd|jtkrtdn|jttfkr4td|j|j |j x|j D] }|qZWdS)Nz joining poolzPool is still runningzIn unknown state) rr7rRrQrXrrr`rrfrir\)rrr r r r&s       z Pool.joincCsDtd|jx*|r>|jr>|jt dqWdS)Nz7removing tasks from inqueue until task handler finishedr) rr7Z_rlockacquireis_aliver4rr{rr)r; task_handlerrr r r _help_stuff_finish2s    zPool._help_stuff_finishc CsJtdt|_t|_td|||t||sNt|dkrNtdt|_|dtdt |k r|| |rt |ddrtdx|D]} | j dkr| qWtdt |k r| td t |k r| |rFt |ddrFtd x0|D](} | rtd | j| qWdS) Nzfinalizing poolz&helping task handler/workers to finishrz.Cannot have cache with result_hander not alivezjoining worker handlerrzterminating workerszjoining task handlerzjoining result handlerzjoining pool workerszcleaning up worker %d)rr7rrRrrqrr.r0r^rrr2rrrpid) clsrr;r<rZworker_handlerrZresult_handlerrrr r r rj;s<                zPool._terminate_poolcCs|S)Nr )rr r r __enter__kszPool.__enter__cCs |dS)N)r)rexc_typeZexc_valZexc_tbr r r __exit__nsz Pool.__exit__)NNr NN)N)N)NNN)r)r)NNN)NNN)"rrrrtrIrrsr]ryrLr}r rrrrrr|rr~ staticmethodr_rcrgrr"r3rrr classmethodrjrrr r r r rsD 8         . <  0c@s@eZdZddZddZddZddd Zdd d Zd d ZdS)rcCs4t|_tt|_||_||_||_|||j<dS)N) r^ZEvent_eventnext job_counterrrP _callback_error_callback)rrrrr r r rws   zApplyResult.__init__cCs |jS)N)rZis_set)rr r r readyszApplyResult.readycCs|std||jS)Nz{0!r} not ready)rrXr/_success)rr r r successfulszApplyResult.successfulNcCs|j|dS)N)rwait)rtimeoutr r r rszApplyResult.waitcCs,|||st|jr"|jS|jdS)N)rrrr_value)rrr r r r1s  zApplyResult.getcCsT|\|_|_|jr$|jr$||j|jr<|js<||j|j|j|j=dS)N)rrrrrsetrPr)rrArr r r rs      zApplyResult._set)N)N) rrrrrrrr1rr r r r rus   rc@seZdZddZddZdS)rcCsftj||||dd|_dg||_||_|dkrLd|_|j||j=n||t |||_dS)N)rTr) rrrr _chunksize _number_leftrrrbool)rrrlengthrrr r r rs    zMapResult.__init__cCs|jd8_|\}}|rp|jrp||j||j|d|j<|jdkr|jrZ||j|j|j=|jnF|s|jrd|_||_|jdkr|j r| |j|j|j=|jdS)NrrF) rrrrrrPrrrr)rrAZsuccess_resultsuccessrDr r r rs"         zMapResult._setN)rrrrrr r r r rs rc@s:eZdZddZddZd ddZeZdd Zd d ZdS) rcCsJtt|_tt|_||_t |_ d|_ d|_ i|_ |||j<dS)Nr)r^Z ConditionZLock_condrrrrP collectionsdeque_items_index_length _unsorted)rrr r r rs  zIMapIterator.__init__cCs|S)Nr )rr r r __iter__szIMapIterator.__iter__Nc Cs|jy|j}Wnntk r|j|jkr8td|j|y|j}Wn,tk r~|j|jkrttdtdYnXYnXWdQRX|\}}|r|S|dS)N) rrpopleft IndexErrorrr StopIterationrr)rrrrr&r r r rs"   zIMapIterator.nextc Cs|j|j|krr|j||jd7_x8|j|jkrd|j|j}|j||jd7_q.W|jn ||j|<|j|jkr|j|j =WdQRXdS)Nr) rrrrurpopnotifyrrPr)rrArr r r rs      zIMapIterator._setc Cs<|j,||_|j|jkr.|j|j|j=WdQRXdS)N)rrrrrPr)rrr r r rs   zIMapIterator._set_length)N) rrrrrr__next__rrr r r r rs   rc@seZdZddZdS)rc CsP|j@|j||jd7_|j|j|jkrB|j|j=WdQRXdS)Nr)rrrurrrrPr)rrArr r r r s    zIMapUnorderedIterator._setN)rrrrr r r r r src@s:eZdZdZeddZd ddZdd Zed d ZdS) rFcOsddlm}|||S)Nr)rI)ZdummyrI)r rCrIr r r rIs zThreadPool.ProcessNr cCst||||dS)N)rr)rrmr=r>r r r r!szThreadPool.__init__cCs,t|_t|_|jj|_|jj|_dS)N)rMrNrkrer0rdr1rh)rr r r rL$s   zThreadPool._setup_queuescCsNyx|jddqWWntjk r,YnXxt|D]}|dq8WdS)NF)block)r1rMZEmptyrpr0)r;rrrAr r r r*szThreadPool._help_stuff_finish)NNr ) rrrrtrrIrrLrr r r r rs   )Nr NF)"__all__r^rMrrrVrrrrrrrQrrcountrrrr8rrr!r$rGr9objectrrZ AsyncResultrrrrr r r r  s@   ,f')@