a ƹDf)@s0dZddgZGdddeZGdddeZdS)a} This module contains the base classes for pathos pool and pipe objects, and describes the map and pipe interfaces. A pipe is defined as a connection between two 'nodes', where a node is something that does work. A pipe may be a one-way or two-way connection. A map is defined as a one-to-many connection between nodes. In both map and pipe connections, results from the connected nodes can be returned to the calling node. There are several variants of pipe and map, such as whether the connection is blocking, or ordered, or asynchronous. For pipes, derived methods must overwrite the 'pipe' method, while maps must overwrite the 'map' method. Pipes and maps are available from worker pool objects, where the work is done by any of the workers in the pool. For more specific point-to-point connections (such as a pipe between two specific compute nodes), use the pipe object directly. Usage ===== A typical call to a pathos map will roughly follow this example: >>> # instantiate and configure the worker pool >>> from pathos.pools import ProcessPool >>> pool = ProcessPool(nodes=4) >>> >>> # do a blocking map on the chosen function >>> results = pool.map(pow, [1,2,3,4], [5,6,7,8]) >>> >>> # do a non-blocking map, then extract the results from the iterator >>> results = pool.imap(pow, [1,2,3,4], [5,6,7,8]) >>> print("...") >>> results = list(results) >>> >>> # do an asynchronous map, then get the results >>> results = pool.amap(pow, [1,2,3,4], [5,6,7,8]) >>> while not results.ready(): ... time.sleep(5); print(".", end=' ') ... >>> results = results.get() Notes ===== Each of the pathos worker pools rely on a different transport protocol (e.g. threads, multiprocessing, etc), where the use of each pool comes with a few caveats. See the usage documentation and examples for each worker pool for more information. AbstractPipeConnectionAbstractWorkerPoolc@s eZdZdZddZddZdS)rz9 AbstractPipeConnection base class for pathos pipes. cOst|dS)z} Required input: ??? Additional inputs: ??? Important class members: ??? Other class members: ??? N)object__init__selfargskwdsr e/nfs/NAS7/SABIOD/METHODE/ermites/ermites_venv/lib/python3.9/site-packages/pathos/abstract_launcher.pyrAs zAbstractPipeConnection.__init__cCs d|jjS)Nz  __class____name__rr r r __repr__QszAbstractPipeConnection.__repr__N)r __module__ __qualname____doc__rrr r r r r=sc@seZdZdZdZddZddZddZd d Zd d Z d dZ ddZ ddZ ddZ ddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'S)(rz5 AbstractWorkerPool base class for pathos pools. cOs,t||j|i||dd|_dS)a Important class members: nodes - number (and potentially description) of workers ncpus - number of worker processors servers - list of worker servers scheduler - the associated scheduler workdir - associated $WORKDIR for scratch calculations/files Other class members: scatter - True, if uses 'scatter-gather' (instead of 'worker-pool') source - False, if minimal use of TemporaryFiles is desired timeout - number of seconds to wait for return value from scheduler idN)rr_AbstractWorkerPool__initgetZ_idrr r r r\s zAbstractWorkerPool.__init__cCs|SNr rr r r __enter__nszAbstractWorkerPool.__enter__cGsdSrr )rrr r r __exit__pszAbstractWorkerPool.__exit__cOsnt|r>z|d}d}t|WqLty:|d}YqL0n|d|j}z ||_WntyhYn0dS)z+default filter for __init__ inputs nodesz0got multiple values for keyword argument 'nodes'N)len TypeErrorKeyErrorr_AbstractWorkerPool__nodesr)rrrrmsgr r r Z__initss   zAbstractWorkerPool.__initcOs2|rz|dg}Wnty,tdYn0dS)z&default filter for map inputs rz map() requires at least two argsN IndexErrorrrfrrZargzr r r Z__maps  zAbstractWorkerPool.__mapcOs2|rz|dg}Wnty,tdYn0dS)z'default filter for imap inputs rz'imap() must have at least two argumentsNr!r#r r r Z__imaps  zAbstractWorkerPool.__imapcOs|rz|jj}t|j}t|}||}||krX||krXtd|t|t|fnV||krtd|t|t|fn.|||krtd|t||t|fWn Yn0dS)z'default filter for pipe inputs z-%s() takes at exactly %s arguments (%s given)z*%s() takes at most %s arguments (%s given)z+%s() takes at least %s arguments (%s given)N)__code__ co_argcountr __defaults__rr str)rr$rrvarsZdefsZarglenZminlenr r r Z__pipes    &zAbstractWorkerPool.__pipecOstdS)z4Create a new server if one isn't already initializedNNotImplementedErrorrr r r _serveszAbstractWorkerPool._servecCstdS)z!Remove server with matching stateNr*rr r r clearszAbstractWorkerPool.clearcOstdS)arun a batch of jobs with a blocking and ordered map Returns a list of results of applying the function f to the items of the argument sequence(s). If more than one sequence is given, the function is called with an argument list consisting of the corresponding item of each sequence. Some maps accept the `chunksize` keyword, which causes the sequence to be split into tasks of approximately the given size. Nr*rr$rrr r r maps zAbstractWorkerPool.mapcOstdS)arun a batch of jobs with a non-blocking and ordered map Returns a list iterator of results of applying the function f to the items of the argument sequence(s). If more than one sequence is given, the function is called with an argument list consisting of the corresponding item of each sequence. Some maps accept the `chunksize` keyword, which causes the sequence to be split into tasks of approximately the given size. Nr*r.r r r imaps zAbstractWorkerPool.imapcOstdS)arun a batch of jobs with a non-blocking and unordered map Returns a list iterator of results of applying the function f to the items of the argument sequence(s). If more than one sequence is given, the function is called with an argument list consisting of the corresponding item of each sequence. The order of the resulting sequence is not guaranteed. Some maps accept the `chunksize` keyword, which causes the sequence to be split into tasks of approximately the given size. Nr*r.r r r uimaps zAbstractWorkerPool.uimapcOstdS)arun a batch of jobs with an asynchronous map Returns a results object which containts the results of applying the function f to the items of the argument sequence(s). If more than one sequence is given, the function is called with an argument list consisting of the corresponding item of each sequence. To retrieve the results, call the get() method on the returned results object. The call to get() is blocking, until all results are retrieved. Use the ready() method on the result object to check if all results are ready. Some maps accept the `chunksize` keyword, which causes the sequence to be split into tasks of approximately the given size. Nr*r.r r r amapszAbstractWorkerPool.amapcOstdS)zsubmit a job and block until results are available Returns result of calling the function f on a selected worker. This function will block until results are available. Nr*r.r r r pipeszAbstractWorkerPool.pipecOstdS)aosubmit a job asynchronously to a queue Returns a results object which containts the result of calling the function f on a selected worker. To retrieve the results, call the get() method on the returned results object. The call to get() is blocking, until the result is available. Use the ready() method on the results object to check if the result is ready. Nr*r.r r r apipes zAbstractWorkerPool.apipecCs d|jjS)Nz r rr r r rszAbstractWorkerPool.__repr__cCs|jS)z#get the number of nodes in the pool)rrr r r Z __get_nodesszAbstractWorkerPool.__get_nodescCs tddS)z#set the number of nodes in the poolznodes is a read-only attributeN)r)rrr r r Z __set_nodes szAbstractWorkerPool.__set_nodesN)r rrrrrrrrZ_AbstractWorkerPool__mapZ_AbstractWorkerPool__imapZ_AbstractWorkerPool__piper,r-r/r0r1r2r3r4rZ_AbstractWorkerPool__get_nodesZ_AbstractWorkerPool__set_nodesr r r r rWs*    N)r__all__rrrr r r r s3