a tDf0@sdZddlmZddlZddlmZmZmZddlm Z ddl m Z ddl Z ddlmZddlmZd d Ze jZe jZe jZe jZee jZee jZee jZee jZe jZe jZe jZe jZe jZe j Z e j!Z!e j"Z"e j#Z#e$e%e j&Z&Wdn1s 0Ye j'Z'e$e%e j(Z(Wdn1sB0Ye j)Z)e$e%e j*Z*Wdn1sz0Yd*d d Z+d+ddZ,ddZ-ddZ.ddZ/ddZ0ddZ1d,ddZ2d-ddZ3d.dd Z4d/d"d#Z5d$d%Z6d&d'Z7d(d)Z8dS)0z- A set of NumPy functions to apply per chunk ) annotationsN) ContainerIterableSequencewraps)Integral)concat)flattencstdfdd }|S)zU A wrapper for functions that don't provide keepdims to ensure that they do. Nc s|g|Rd|i|}|s"|S|}|dur8t|j}t|tttfsN|g}t}t|jD]$}||vrt|d7}q^|tdf7}q^||}|S)NaxisN)rangendim isinstancerrrtupleslice) xr keepdimsargskwargsraxesZr_sliceZ each_axis a_callable]/nfs/NAS7/SABIOD/METHODE/ermites/ermites_venv/lib/python3.9/site-packages/dask/array/chunk.pykeepdims_wrapped_callables  z3keepdims_wrapper..keepdims_wrapped_callable)NNr)rrrrrkeepdims_wrappersrFc stjD]}|vr d|<q |rHtfddtjD}|ttfddtjD}||fdttdjddi|S)aCoarsen array by applying reduction to fixed size neighborhoods Parameters ---------- reduction: function Function like np.sum, np.mean, etc... x: np.ndarray Array to be coarsened axes: dict Mapping of axis to coarsening factor Examples -------- >>> x = np.array([1, 2, 3, 4, 5, 6]) >>> coarsen(np.sum, x, {0: 2}) array([ 3, 7, 11]) >>> coarsen(np.max, x, {0: 3}) array([3, 6]) Provide dictionary of scale per dimension >>> x = np.arange(24).reshape((4, 6)) >>> x array([[ 0, 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19, 20, 21, 22, 23]]) >>> coarsen(np.min, x, {0: 2, 1: 3}) array([[ 0, 3], [12, 15]]) You must avoid excess elements explicitly >>> x = np.array([1, 2, 3, 4, 5, 6, 7, 8]) >>> coarsen(np.min, x, {0: 3}, trim_excess=True) array([1, 4]) c3s<|]4\}}||r*td|| ntddVqdS)rNr).0idrrr szcoarsen..cs&g|]}j|||fqSr)shaper r!rrrr zcoarsen..r )r rr enumerater%r Zreshape)Z reductionrrZ trim_excessrr!indZnewshaperr'rcoarsenTs(  "r-csNttrg|jttr8fddt|jD|tddDS)aCTrim boundaries off of array >>> x = np.arange(24).reshape((4, 6)) >>> trim(x, axes={0: 0, 1: 1}) array([[ 1, 2, 3, 4], [ 7, 8, 9, 10], [13, 14, 15, 16], [19, 20, 21, 22]]) >>> trim(x, axes={0: 1, 1: 1}) array([[ 7, 8, 9, 10], [13, 14, 15, 16]]) csg|]}|dqS)r)getr&r#rrr(r)ztrim..css"|]}t||r| ndVqdSr r)r axrrrr$r)ztrim..)rrrdictr r)rrrr#rtrims    r1cs||dus Jdt||jkr*|Stj|| d}|dkrPt| dnt| |tfddt|jDS)aChunk and combine function of topk Extract the k largest elements from a on the given axis. If k is negative, extract the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. Trr Nc3s"|]}|krntdVqdSr rr&r Zk_slicerrr$r)ztopk..)absr%np partitionrrr rakr rrr3rtopks r:cs\|dus Jt|||}dtj|d}|dkr<|S|tfddt|jDS)zmFinal aggregation function of topk Invoke topk one final time and then sort the results internally. Trr2c3s*|]"}|krtdddntdVqdSNrr&r2rrr$sz!topk_aggregate..)r:r5sortrr rr7rr2rtopk_aggregates  r>cCs||fS)z^Preparatory step for argtopk Put data together with its original indices in a tuple. r)r8idxrrrargtopk_preprocesssr@cs|dus Jdt|trXtt|}tdd|D}tdd|D}n|\}}t||jkrv|Stj|| d}|dkrt| dnt| |t fdd t |j D}t ||t ||fS) a)Chunk and combine function of argtopk Extract the indices of the k largest elements from a on the given axis. If k is negative, extract the indices of the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. TrcSsg|] \}}|qSrr)r ai_rrrr(r)zargtopk..cSsg|]\}}t||jqSr)r5Z broadcast_tor%)r rAZidxirrrr(r)r2Nc3s"|]}|krntdVqdSr rr&r3rrr$r)zargtopk..) rlistr r5Z concatenater4r%Z argpartitionrrr rtake_along_axisZ a_plus_idxr9r rr8r?Zidx2rr3rargtopks   "rFcs|dus Jt|dkr|n|d}t|||\}}dtj|d}t||}|dkrf|S|tfddt|jDS)zFinal aggregation function of argtopk Invoke argtopk one final time, sort the results internally, drop the data and return the index only. Trrr2c3s*|]"}|krtdddntdVqdSr;rr&r2rrr$sz$argtopk_aggregate..)lenrFr5ZargsortrDrr rrErr2rargtopk_aggregates  rHcCs:ddlm}||||||d}t||kr6|ddS|S)Nr) arange_safeliker<)dask.array.utilsrIrG)startstopsteplengthdtyperKrIresrrraranges rSTcCsDddlm}t||r|}t||r0|}tj|||||dS)Nr)Array)endpointrQ)Zdask.array.corerTrZcomputer5linspace)rMrNnumrUrQrTrrrrV s    rVcKs|j|fi|Sr )astype)rZ astype_dtyperrrrrXsrXCcCs|dkrBztj||d}Wnty6t|}Yn0||Sztj||d}Wntypt|}Yn0|j|jSdS)NrYrJ)r5Zascontiguousarray TypeErrorviewZasfortranarrayT)rrQorderrrrr[s   r[csddlm}m}|||dtjtdk||dk|jk@}||tfddt |j DS)aChunk function of `slice_with_int_dask_array_on_axis`. Slice one chunk of x by one chunk of idx. Parameters ---------- x: ndarray, any dtype, any shape i-th chunk of x idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx (cartesian product with the chunks of x) offset: ndarray, shape=(1, ), dtype=int64 Index of the first element along axis of the current chunk of x x_size: int Total size of the x da.Array along axis axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- x sliced along axis, using only the elements of idx that fall inside the current chunk. r) asarray_safemeta_from_arrayrJc3s"|]}|krntdVqdSr rr&r r?rrr$Sr)z,slice_with_int_dask_array..) rLr^r_rXr5int64wherer%rr r)rr?offsetZx_sizer r^r_ idx_filterrr`rslice_with_int_dask_array(s rec s|tj}t|dk|t||}d}d}t||D]Z}||k|||k@}t|}t||d|d7||7}|jdkr<||d7}q<|tfddt |j DS)aSFinal aggregation function of `slice_with_int_dask_array_on_axis`. Aggregate all chunks of x by one chunk of idx, reordering the output of `slice_with_int_dask_array`. Note that there is no combine function, as a recursive aggregation (e.g. with split_every) would not give any benefit. Parameters ---------- idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx chunk_outputs: ndarray concatenation along axis of the outputs of `slice_with_int_dask_array` for all chunks of x and the j-th chunk of idx x_chunks: tuple dask chunks of the x da.Array along axis, e.g. ``(3, 3, 2)`` axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- Selection from all chunks of x for the j-th chunk of idx, in the correct order rrr<c3s"|]}|krntdVqdSr rr&r Z idx_finalrrr$sz6slice_with_int_dask_array_aggregate..) rXr5rarbsumZ zeros_likeZcumsumsizerr r) r?Z chunk_outputsZx_chunksr Zx_chunk_offsetZchunk_output_offsetZx_chunkrdZidx_cumrrfr#slice_with_int_dask_array_aggregateVs"    ric Csxz ||}Wn.ty:}ztd|WYd}~n d}~00z$|jjs^|jd|jkr^|}WntyrYn0|S)afGetitem function This function creates a copy of the desired selection for array-like inputs when the selection is smaller than half of the original array. This avoids excess memory usage when extracting a small portion from a large array. For more information, see https://numpy.org/doc/stable/reference/arrays.indexing.html#basic-slicing-and-indexing. Parameters ---------- obj: ndarray, string, tuple, list Object to get item from. index: int, list[int], slice() Desired selection to extract from obj. Returns ------- Selection obj[index] zTArray chunk size or shape is unknown. Possible solution with x.compute_chunk_sizes()Nr*) IndexError ValueErrorflagsZowndatarhcopyAttributeError)objindexresulterrrgetitems   rs)F)N)N)TN)N)rY)9__doc__ __future__r contextlibcollections.abcrrr functoolsrZnumbersrnumpyr5Ztlzr Z dask.corer rrgprodminmaxZargminZ nanargminZargmaxZ nanargmaxanyallZnansumZnanprodZ nancumprodZ nancumsumZnanminZnanmaxmeansuppressrnZnanmeanvarZnanvarZstdZnanstdr-r1r:r>r@rFrHrSrVrXr[rerirsrrrrs`     #     & & & 9    .8