a d?Z>Gd@dAdAeeZ?dS)Iz\ HDBSCAN: Hierarchical Density-Based Spatial Clustering of Applications with Noise N) BaseEstimator ClusterMixin)pairwise_distances)issparse)KDTreeBallTree)Memory)warn) check_array) cpu_count)csgraph)single_linkagemst_linkage_coremst_linkage_core_vectorlabel) condense_treecompute_stability get_clustersoutlier_scores)mutual_reachabilitysparse_mutual_reachability)KDTreeBoruvkaAlgorithmBallTreeBoruvkaAlgorithm)DistanceMetric) CondensedTreeSingleLinkageTreeMinimumSpanningTree)PredictionData) euclideanl2 minkowskipZ manhattanZ cityblockl1Z chebyshevZinfinity) Z braycurtisZcanberraZdiceZhammingZ haversineZjaccardZ mahalanobisZrogerstanimotoZ russellraoZ seuclideanZ sokalmichenerZ sokalsneathcosinearccos)isclose eomFc Cs:t||}t|} t|| |||||\} } } | | | ||fS)z\Converts a pretrained tree and cluster size into a set of labels and probabilities. )rrr) Xsingle_linkage_treemin_cluster_sizecluster_selection_methodallow_single_clustermatch_reference_implementationcluster_selection_epsilonmax_cluster_sizeZcondensed_treeZstability_dictlabelsZ probabilitiesZ stabilitiesr3]/nfs/NAS7/SABIOD/METHODE/ermites/ermites_venv/lib/python3.9/site-packages/hdbscan/hdbscan_.py_tree_to_labelsAs  r5?r!cKs|dkrt|||d}nD|dkr6t|fddi|}n&|dkrH|}nt|fd|i|}t|rt|||||||fi|St|||} t| } t| jd rt dt |rR| } t | d dd D]|\} } t t| t| d | dd }t|| d| ddft}||| d k}t|d ksBJ|d | d <qnd} | t| jdddf} t| }|| fS) Nr!)metricr"r%r9r$ precomputedr8zThe minimum spanning tree contains edge weights with value infinity. Potentially, you are missing too many distances in the initial distance matrix for the given neighborhood size.r r)rcopyr_hdbscan_sparse_distance_matrixrrnpisinfTanyr UserWarning enumeratewherer&intZ intersect1dastypelenargsortr)r* min_samplesalphar9r" leaf_sizegen_min_span_treekwargsZdistance_matrixmutual_reachability_min_spanning_treeresult_min_span_treeindexrow candidatesr+r3r3r4_hdbscan_generic]sP    $rS(cKst|s Jtj|ddddkr(td|}|dd} t||| |d} tj| ddddkrptd||ft| } | } | | } t | | fj }|t |j d ddfd }t |}|r||fS|dfSdS) NF)ZdirectedZ return_labelsr zSparse distance matrix has multiple connected components! That is, there exist groups of points that are completely disjoint -- there are no distance relations connecting them Run hdbscan on each component.max_distr))Z min_pointsrUrIzThere exists points with less than %s neighbors. Ensure your distance matrix has non zeros values for at least `min_sample`=%s neighbors for each points (i.e. K-nn graph), or specify a `max_dist` to use when distances are missing.r8r)rr Zconnected_components ValueErrortolilgetrZminimum_spanning_treeZnonzeror=vstackr?rGr)r*rHrIr9r"rJrKrLZ lil_matrixrUrMZsparse_min_spanning_treeZnonzerosZ nonzero_valsrNr+r3r3r4r<s<    r<c Ks|jtjkr|tj}|jds4tj|tjdd}t|f||d|}tj |fi|} |j ||dddddddd fj dd } t || | |} | t | jd ddf} t| } |r| | fS| dfSdS NZ C_CONTIGUOUSC)dtypeorderr9rJr T)kZdualtreeZ breadth_firstr)r]r8)r\r=float64rEflagsarraydoublerr get_metricqueryr;rrGr?r r*rHrIr9r"rJrKrLtreeZ dist_metricZcore_distancesrNr+r3r3r4_hdbscan_prims_kdtrees*     ric Ks|jtjkr|tj}|jds4tj|tjdd}t|f||d|}tj |fi|} |j ||dddddddd fj dd } t || | |} | t | jd ddf} t| } |r| | fS| dfSdSrZ)r\r=rarErbrcrdrrrerfr;rrGr?rrgr3r3r4_hdbscan_prims_balltrees*     rjTc Ks|dkr d}|dkr(ttd|d}|jtjkr@|tj}t|f||d| } t| |f||d||d| } | } t | j d} | | ddf} t | }|r|| fS|dfSdSNr r^)r9rJapprox_min_span_treeZn_jobsr8) maxr r\r=rarErr spanning_treerGr?r)r*rHrIr9r"rJrnrKcore_dist_n_jobsrLrhalgrNZ row_orderr+r3r3r4_hdbscan_boruvka_kdtree>s2    rsc Ks|dkr d}|dkr(ttd|d}|jtjkr@|tj}t|f||d| } t| |f||d||d| } | } | t | j dddf} t | } |r| | fS| dfSdSrl) ror r\r=rarErrrprGr?r)r*rHrIr9r"rJrnrKrqrLrhrrrNr+r3r3r4_hdbscan_boruvka_balltreejs0    rtcCs"|}d|t|<t|dS)zaPerform check_array(X) after removing infinite values (numpy.inf) from the given distance matrix.r N)r;r=r>r )r*tmpr3r3r4!check_precomputed_distance_matrixsrvcCst|}t|}t|D]>\}\}}}} ||kr:||}n||}||||| f||<qg} |dd} |D]} | | | ddfqltj| dtjfdtjfdtfdtjfgd} t| |}|S)a Takes an internal condensed_tree structure and adds back in a set of points that were initially detected as non-finite and returns that new tree. These points will all be split off from the maximal node at lambda zero and considered noise points. Parameters ---------- tree: condensed_tree internal_to_raw: dict a mapping from internal integer index to the raw integer index finite_index: ndarray Boolean array of which entries in the raw data were finite rr parentchild lambda_val child_size)r\)rFrBappendr=rcZintpfloat)rhinternal_to_rawoutliers finite_count outlier_countirwrxryrzZ outlier_listrootoutlier outlier_treer3r3r4remap_condensed_trees*   rcCst|}t|}t|D]d\}\}}}} ||krB||||df<n||||df<||krl||||df<q||||df<qtt|df} ||jdddd} ||jddd} t|D]2\}} | | dtj| df| |<| d7} | d7} qt|| g}|S)a  Takes an internal single_linkage_tree structure and adds back in a set of points that were initially detected as non-finite and returns that new tree. These points will all be merged into the final node at np.inf distance and considered noise points. Parameters ---------- tree: single_linkage_tree internal_to_raw: dict a mapping from internal integer index to the raw integer index finite_index: ndarray Boolean array of which entries in the raw data were finite rr rkr8rm)rFrBr=zerosshaperoinfrY)rhr}r~rrrleftrightdistancesizerZlast_cluster_idZlast_cluster_sizerr3r3r4remap_single_linkage_trees$ rcCs2t|rtt|jStt|SdS)zLReturns true only if all the values of a ndarray or sparse matrix are finiteN)rr=ZalltrueisfiniteZtocoodata)matrixr3r3r4 is_finitesrcCsRt|r(tddt|jD}n&tt|jdd|j dkd}|S)zQReturns the indices of the purely finite rows of a sparse matrix or dense ndarraycSs$g|]\}}tt|r|qSr3)r=allr).0rrQr3r3r4 z*get_finite_row_indices..r axisr) rr=rcrBrWrrCrsumr)rZ row_indicesr3r3r4get_finite_row_indicess &rbestverbosec KsN|dur |}tt|tjr0tt|tjs8td|dksH|dkrPtd|dkr`tdtt|tjrzt|}t|tus|dkrtdt|tr|dkrtd |dkrtd |d kr|durtd |dkrtd |r|d}|d}d} |dvrtd|dks(t|r8t |ddd}nt |t| t rXt | dd} |j d}t|d|}|dkr~d}| dkr&|dkrt|r| dkrtd| dkr| t||||||| fi|\}}q2| dkr&|tvrtd| t||||||| fi|\}}n| dkrj|tvrBtd| t||||||| fi|\}}n| dkr|tvrtd| t||||||| | | f i|\}}np| dkr|tvrtd|j d|d krtd!| t||||||| | | f i|\}}n td"| n t|s:|tvrb| t||||||| fi|\}}n|tvr|j dd#kr| t||||||| fi|\}}n*| t||||||| | | f i|\}}nb|j dd#kr| t||||||| fi|\}}n*| t||||||| | | f i|\}}t|||||||||fS)$aPerform HDBSCAN clustering from a vector array or distance matrix. Parameters ---------- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples) A feature array, or array of distances between samples if ``metric='precomputed'``. min_cluster_size : int, optional (default=5) The minimum number of samples in a group for that group to be considered a cluster; groupings smaller than this size will be left as noise. min_samples : int, optional (default=None) The number of samples in a neighborhood for a point to be considered as a core point. This includes the point itself. defaults to the min_cluster_size. cluster_selection_epsilon: float, optional (default=0.0) A distance threshold. Clusters below this value will be merged. See [3]_ for more information. Note that this should not be used if we want to predict the cluster labels for new points in future (e.g. using approximate_predict), as the approximate_predict function is not aware of this argument. alpha : float, optional (default=1.0) A distance scaling parameter as used in robust single linkage. See [2]_ for more information. max_cluster_size : int, optional (default=0) A limit to the size of clusters returned by the eom algorithm. Has no effect when using leaf clustering (where clusters are usually small regardless) and can also be overridden in rare cases by a high value for cluster_selection_epsilon. Note that this should not be used if we want to predict the cluster labels for new points in future (e.g. using approximate_predict), as the approximate_predict function is not aware of this argument. metric : string or callable, optional (default='minkowski') The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by metrics.pairwise.pairwise_distances for its metric parameter. If metric is "precomputed", X is assumed to be a distance matrix and must be square. p : int, optional (default=2) p value to use if using the minkowski metric. leaf_size : int, optional (default=40) Leaf size for trees responsible for fast nearest neighbour queries. algorithm : string, optional (default='best') Exactly which algorithm to use; hdbscan has variants specialised for different characteristics of the data. By default this is set to ``best`` which chooses the "best" algorithm given the nature of the data. You can force other options if you believe you know better. Options are: * ``best`` * ``generic`` * ``prims_kdtree`` * ``prims_balltree`` * ``boruvka_kdtree`` * ``boruvka_balltree`` memory : instance of joblib.Memory or string, optional Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory. approx_min_span_tree : bool, optional (default=True) Whether to accept an only approximate minimum spanning tree. For some algorithms this can provide a significant speedup, but the resulting clustering may be of marginally lower quality. If you are willing to sacrifice speed for correctness you may want to explore this; in general this should be left at the default True. gen_min_span_tree : bool, optional (default=False) Whether to generate the minimum spanning tree for later analysis. core_dist_n_jobs : int, optional (default=4) Number of parallel jobs to run in core distance computations (if supported by the specific algorithm). For ``core_dist_n_jobs`` below -1, (n_cpus + 1 + core_dist_n_jobs) are used. cluster_selection_method : string, optional (default='eom') The method used to select clusters from the condensed tree. The standard approach for HDBSCAN* is to use an Excess of Mass algorithm to find the most persistent clusters. Alternatively you can instead select the clusters at the leaves of the tree -- this provides the most fine grained and homogeneous clusters. Options are: * ``eom`` * ``leaf`` allow_single_cluster : bool, optional (default=False) By default HDBSCAN* will not produce a single cluster, setting this to t=True will override this and allow single cluster results in the case that you feel this is a valid result for your dataset. (default False) match_reference_implementation : bool, optional (default=False) There exist some interpretational differences between this HDBSCAN* implementation and the original authors reference implementation in Java. This can result in very minor differences in clustering results. Setting this flag to True will, at a some performance cost, ensure that the clustering results match the reference implementation. **kwargs : optional Arguments passed to the distance metric Returns ------- labels : ndarray, shape (n_samples, ) Cluster labels for each point. Noisy samples are given the label -1. probabilities : ndarray, shape (n_samples, ) Cluster membership strengths for each point. Noisy samples are assigned 0. cluster_persistence : array, shape (n_clusters, ) A score of how persistent each cluster is. A score of 1.0 represents a perfectly stable cluster that persists over all distance scales, while a score of 0.0 represents a perfectly ephemeral cluster. These scores can be guage the relative coherence of the clusters output by the algorithm. condensed_tree : record array The condensed cluster hierarchy used to generate clusters. single_linkage_tree : ndarray, shape (n_samples - 1, 4) The single linkage tree produced during clustering in scipy hierarchical clustering format (see http://docs.scipy.org/doc/scipy/reference/cluster.hierarchy.html). min_spanning_tree : ndarray, shape (n_samples - 1, 3) The minimum spanning as an edgelist. If gen_min_span_tree was False this will be None. References ---------- .. [1] Campello, R. J., Moulavi, D., & Sander, J. (2013, April). Density-based clustering based on hierarchical density estimates. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 160-172). Springer Berlin Heidelberg. .. [2] Chaudhuri, K., & Dasgupta, S. (2010). Rates of convergence for the cluster tree. In Advances in Neural Information Processing Systems (pp. 343-351). .. [3] Malzer, C., & Baum, M. (2019). A Hybrid Approach To Hierarchical Density-based Cluster Selection. arxiv preprint 1911.02282. Nz2Min samples and min cluster size must be integers!rz:Min samples and Min cluster size must be positive integersr z)Min cluster size must be greater than oner)z9Epsilon must be a float value greater than or equal to 0!z4Alpha must be a positive float value greater than 0!z!Leaf size must be greater than 0!r!z/Minkowski metric given but no p value supplied!z6Minkowski metric with negative p value is not defined!F)r(ZleafzEInvalid Cluster Selection Method: %s Should be one of: "eom", "leaf" r:csr accept_sparseZforce_all_finiterrZgenericz6Sparse data matrices only support algorithm 'generic'.Z prims_kdtreez.Cannot use Prim's with KDTree for this metric!Zprims_balltreez0Cannot use Prim's with BallTree for this metric!Zboruvka_kdtreez/Cannot use Boruvka with KDTree for this metric!Zboruvka_balltreez1Cannot use Boruvka with BallTree for this metric!i>zA large dataset size and small leaf_size may induce excessive memory usage. If you are running out of memory consider increasing the ``leaf_size`` parameter.z#Unknown algorithm type %s specified<)r=Z issubdtypetypeintegerrVr| isinstance TypeErrorrr rvstrrrmincacherSKDTREE_VALID_METRICSriBALLTREE_VALID_METRICSrjrsr rt FAST_METRICSr5)r*r,rHrIr0r1r9r"rJ algorithmmemoryrnrKrqr-r.r/rLrr+rOr3r3r4hdbscans1                                rc@seZdZdZddddddddd eddd d d d dd d d fddZd+ddZd,ddZddZddZ ddZ d-ddZ e ddZ e dd Ze d!d"Ze d#d$Ze d%d&Ze d'd(Ze d)d*ZdS).HDBSCANaf%Perform HDBSCAN clustering from vector array or distance matrix. HDBSCAN - Hierarchical Density-Based Spatial Clustering of Applications with Noise. Performs DBSCAN over varying epsilon values and integrates the result to find a clustering that gives the best stability over epsilon. This allows HDBSCAN to find clusters of varying densities (unlike DBSCAN), and be more robust to parameter selection. Parameters ---------- min_cluster_size : int, optional (default=5) The minimum size of clusters; single linkage splits that contain fewer points than this will be considered points "falling out" of a cluster rather than a cluster splitting into two new clusters. min_samples : int, optional (default=None) The number of samples in a neighbourhood for a point to be considered a core point. metric : string, or callable, optional (default='euclidean') The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by metrics.pairwise.pairwise_distances for its metric parameter. If metric is "precomputed", X is assumed to be a distance matrix and must be square. p : int, optional (default=None) p value to use if using the minkowski metric. alpha : float, optional (default=1.0) A distance scaling parameter as used in robust single linkage. See [3]_ for more information. cluster_selection_epsilon: float, optional (default=0.0) A distance threshold. Clusters below this value will be merged. See [5]_ for more information. algorithm : string, optional (default='best') Exactly which algorithm to use; hdbscan has variants specialised for different characteristics of the data. By default this is set to ``best`` which chooses the "best" algorithm given the nature of the data. You can force other options if you believe you know better. Options are: * ``best`` * ``generic`` * ``prims_kdtree`` * ``prims_balltree`` * ``boruvka_kdtree`` * ``boruvka_balltree`` leaf_size: int, optional (default=40) If using a space tree algorithm (kdtree, or balltree) the number of points ina leaf node of the tree. This does not alter the resulting clustering, but may have an effect on the runtime of the algorithm. memory : Instance of joblib.Memory or string (optional) Used to cache the output of the computation of the tree. By default, no caching is done. If a string is given, it is the path to the caching directory. approx_min_span_tree : bool, optional (default=True) Whether to accept an only approximate minimum spanning tree. For some algorithms this can provide a significant speedup, but the resulting clustering may be of marginally lower quality. If you are willing to sacrifice speed for correctness you may want to explore this; in general this should be left at the default True. gen_min_span_tree: bool, optional (default=False) Whether to generate the minimum spanning tree with regard to mutual reachability distance for later analysis. core_dist_n_jobs : int, optional (default=4) Number of parallel jobs to run in core distance computations (if supported by the specific algorithm). For ``core_dist_n_jobs`` below -1, (n_cpus + 1 + core_dist_n_jobs) are used. cluster_selection_method : string, optional (default='eom') The method used to select clusters from the condensed tree. The standard approach for HDBSCAN* is to use an Excess of Mass algorithm to find the most persistent clusters. Alternatively you can instead select the clusters at the leaves of the tree -- this provides the most fine grained and homogeneous clusters. Options are: * ``eom`` * ``leaf`` allow_single_cluster : bool, optional (default=False) By default HDBSCAN* will not produce a single cluster, setting this to True will override this and allow single cluster results in the case that you feel this is a valid result for your dataset. prediction_data : boolean, optional Whether to generate extra cached data for predicting labels or membership vectors for new unseen points later. If you wish to persist the clustering object for later re-use you probably want to set this to True. (default False) match_reference_implementation : bool, optional (default=False) There exist some interpretational differences between this HDBSCAN* implementation and the original authors reference implementation in Java. This can result in very minor differences in clustering results. Setting this flag to True will, at a some performance cost, ensure that the clustering results match the reference implementation. **kwargs : optional Arguments passed to the distance metric Attributes ---------- labels_ : ndarray, shape (n_samples, ) Cluster labels for each point in the dataset given to fit(). Noisy samples are given the label -1. probabilities_ : ndarray, shape (n_samples, ) The strength with which each sample is a member of its assigned cluster. Noise points have probability zero; points in clusters have values assigned proportional to the degree that they persist as part of the cluster. cluster_persistence_ : ndarray, shape (n_clusters, ) A score of how persistent each cluster is. A score of 1.0 represents a perfectly stable cluster that persists over all distance scales, while a score of 0.0 represents a perfectly ephemeral cluster. These scores can be guage the relative coherence of the clusters output by the algorithm. condensed_tree_ : CondensedTree object The condensed tree produced by HDBSCAN. The object has methods for converting to pandas, networkx, and plotting. single_linkage_tree_ : SingleLinkageTree object The single linkage tree produced by HDBSCAN. The object has methods for converting to pandas, networkx, and plotting. minimum_spanning_tree_ : MinimumSpanningTree object The minimum spanning tree of the mutual reachability graph generated by HDBSCAN. Note that this is not generated by default and will only be available if `gen_min_span_tree` was set to True on object creation. Even then in some optimized cases a tre may not be generated. outlier_scores_ : ndarray, shape (n_samples, ) Outlier scores for clustered points; the larger the score the more outlier-like the point. Useful as an outlier detection technique. Based on the GLOSH algorithm by Campello, Moulavi, Zimek and Sander. prediction_data_ : PredictionData object Cached data used for predicting the cluster labels of new or unseen points. Necessary only if you are using functions from ``hdbscan.prediction`` (see :func:`~hdbscan.prediction.approximate_predict`, :func:`~hdbscan.prediction.membership_vector`, and :func:`~hdbscan.prediction.all_points_membership_vectors`). exemplars_ : list A list of exemplar points for clusters. Since HDBSCAN supports arbitrary shapes for clusters we cannot provide a single cluster exemplar per cluster. Instead a list is returned with each element of the list being a numpy array of exemplar points for a cluster -- these points are the "most representative" points of the cluster. relative_validity_ : float A fast approximation of the Density Based Cluster Validity (DBCV) score [4]. The only differece, and the speed, comes from the fact that this relative_validity_ is computed using the mutual- reachability minimum spanning tree, i.e. minimum_spanning_tree_, instead of the all-points minimum spanning tree used in the reference. This score might not be an objective measure of the goodness of clusterering. It may only be used to compare results across different choices of hyper-parameters, therefore is only a relative score. References ---------- .. [1] Campello, R. J., Moulavi, D., & Sander, J. (2013, April). Density-based clustering based on hierarchical density estimates. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 160-172). Springer Berlin Heidelberg. .. [2] Campello, R. J., Moulavi, D., Zimek, A., & Sander, J. (2015). Hierarchical density estimates for data clustering, visualization, and outlier detection. ACM Transactions on Knowledge Discovery from Data (TKDD), 10(1), 5. .. [3] Chaudhuri, K., & Dasgupta, S. (2010). Rates of convergence for the cluster tree. In Advances in Neural Information Processing Systems (pp. 343-351). .. [4] Moulavi, D., Jaskowiak, P.A., Campello, R.J., Zimek, A. and Sander, J., 2014. Density-Based Clustering Validation. In SDM (pp. 839-847). .. [5] Malzer, C., & Baum, M. (2019). A Hybrid Approach To Hierarchical Density-based Cluster Selection. arxiv preprint 1911.02282. r6Nr)rrr7rrTrTFrkr(cKs||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||_ ||_||_||_||_d|_d|_d|_d|_d|_d|_d|_dS)N)r,rHrIr1r0r9r"rrJrrnrKrqr-r.r/prediction_data_metric_kwargs_condensed_tree_single_linkage_tree_min_spanning_tree _raw_data_outlier_scores_prediction_data_relative_validity)selfr,rHr0r1r9rIr"rrJrrnrKrqr-r.rr/rLr3r3r4__init__Ls2zHDBSCAN.__init__c Csz|jdkrt|ddd}||_t||_|jr|t|}||}ddttt||D}t t t|j dt |}q|}n&t |rt|dd}|}n t ||}|}|d d ||jt|fi|\|_|_|_|_|_|_|jdkrf|jsft|j|||_t|j|||_t|j dd }|j||<||_t|j d} |j| |<| |_|jrv||S) aPerform HDBSCAN clustering from features or distance matrix. Parameters ---------- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples) A feature array, or array of distances between samples if ``metric='precomputed'``. Returns ------- self : object Returns self r:rFrcSsi|]\}}||qSr3r3)rxyr3r3r4 szHDBSCAN.fit..r)rrNr`) r9r rrZ _all_finiterziprangerFlistsetrrrv get_paramspopupdaterrlabels_probabilities_Zcluster_persistence_rrrrrr=fullrrgenerate_prediction_data) rr*rZ finite_indexZ clean_datar}r~rLZ new_labelsZnew_probabilitiesr3r3r4fit}sX        z HDBSCAN.fitcCs|||jS)aPerforms clustering on X and returns cluster labels. Parameters ---------- X : array or sparse (CSR) matrix of shape (n_samples, n_features), or array of shape (n_samples, n_samples) A feature array, or array of distances between samples if ``metric='precomputed'``. Returns ------- y : ndarray, shape (n_samples, ) cluster labels )rr)rr*rr3r3r4 fit_predicts zHDBSCAN.fit_predictcCs||jtvrp|jp|j}|jtvr&d}n$|jtvr6d}ntd|jdSt|j |j |f||jd|j |_ ntddS)z Create data that caches intermediate results used for predicting the label of new/unseen points. This data is only useful if you are intending to use functions from ``hdbscan.prediction``. ZkdtreeZballtreez,Metric {} not supported for prediction data!N) tree_typer9z~Cannot generate prediction data for non-vectorspace inputs -- access to the source data ratherthan mere distances is required!) r9rrHr,rrr formatrrcondensed_tree_rr)rrHrr3r3r4rs*     z HDBSCAN.generate_prediction_datacCsPt|dstd|dkr"td|j|k}|j|}|j|}tj||ddS)aOProvide an approximate representative point for a given cluster. Note that this technique assumes a euclidean metric for speed of computation. For more general metrics use the ``weighted_cluster_medoid`` method which is slower, but can work with the metric the model trained with. Parameters ---------- cluster_id: int The id of the cluster to compute a centroid for. Returns ------- centroid: array of shape (n_features,) A representative centroid for cluster ``cluster_id``. rModel has not been fit to datar`MCannot calculate weighted centroid for -1 cluster since it is a noise clusterr)weightsr)hasattrAttributeErrorrVrrrr=Zaverage)r cluster_idmask cluster_datacluster_membership_strengthsr3r3r4weighted_cluster_centroids    z!HDBSCAN.weighted_cluster_centroidcCszt|dstd|dkr"td|j|k}|j|}|j|}t|fd|ji|j}||}t |j dd}||S)a/Provide an approximate representative point for a given cluster. Note that this technique can be very slow and memory intensive for large clusters. For faster results use the ``weighted_cluster_centroid`` method which is faster, but assumes a euclidean metric. Parameters ---------- cluster_id: int The id of the cluster to compute a medoid for. Returns ------- centroid: array of shape (n_features,) A representative medoid for cluster ``cluster_id``. rrr`rr9r r) rrrVrrrrr9rr=Zargminr)rrrrrZdist_matZ medoid_indexr3r3r4weighted_cluster_medoids&    zHDBSCAN.weighted_cluster_medoidcCs|jj||dS)aReturn clustering that would be equivalent to running DBSCAN* for a particular cut_distance (or epsilon) DBSCAN* can be thought of as DBSCAN without the border points. As such these results may differ slightly from sklearns implementation of dbscan in the non-core points. This can also be thought of as a flat clustering derived from constant height cut through the single linkage tree. This represents the result of selecting a cut value for robust single linkage clustering. The `min_cluster_size` allows the flat clustering to declare noise points (and cluster smaller than `min_cluster_size`). Parameters ---------- cut_distance : float The mutual reachability distance cut value to use to generate a flat clustering. min_cluster_size : int, optional Clusters smaller than this value with be called 'noise' and remain unclustered in the resulting flat clustering. Returns ------- labels : array [n_samples] An array of cluster labels, one per datapoint. Unclustered points are assigned the label -1. ) cut_distancer,)single_linkage_tree_r)rrr,r3r3r4dbscan_clusteringCszHDBSCAN.dbscan_clusteringcCs|jdurtdn|jSdS)Nz No prediction data was generated)rrrr3r3r4prediction_data_es  zHDBSCAN.prediction_data_cCs8|jdur|jS|jdur,t|j|_|jStddSNz7No condensed tree was generated; try running fit first.)rrrrrr3r3r4outlier_scores_ls   zHDBSCAN.outlier_scores_cCs(|jdurt|j|j|jStddSr)rrr-r.rrr3r3r4rys zHDBSCAN.condensed_tree_cCs |jdurt|jStddS)Nz.cs g|]}||qSr3r3r)V_index cluster_sizetotalr3r4rr)rrKrrr=ZbincountrrFrrZonesrZ to_pandasZiterrowsrDrorrCr)rr2sizesZ noise_sizeZ num_clustersZmin_outlier_sepZcorrection_constZ max_distanceZmst_dfZedgelabel1label2lengthZ correctionZscorer3)rrrrrr4relative_validity_s\           zHDBSCAN.relative_validity_)N)N)r6)__name__ __module__ __qualname____doc__rrrrrrrrpropertyrrrrrrrr3r3r3r4rsPK  1 O   % "      r)r'r(FFr)r)r6r7r!r8NF)r6r7r!r8rTF)r6r7r!r8rTF)r6r7r!r8rTF)r6r7r!r8rTTFrk)r6r7r!r8rTTFrk)@rnumpyr=Z sklearn.baserrZsklearn.metricsrZ scipy.sparserZsklearn.neighborsrrZjoblibrwarningsr Z sklearn.utilsr Zjoblib.parallelr r Z_hdbscan_linkagerrrrZ _hdbscan_treerrrrZ_hdbscan_reachabilityrrZ_hdbscan_boruvkarrZ dist_metricsrZplotsrrrZ predictionrrrrr&r5rSr<rirjrsrtrvrrrrrrr3r3r3r4s             N F , ) . ++'