EFFECTIVE METHOD FOR OPTIMAL CLUSTERING IN UNSUPERVISED LEARNING

Abstract

K-Means is one of the most common clustering methods, but it has a serious limitation. It needs the number of clusters (k) to be set before running, and the wrong choice may lead to weak results. To deal with this problem, the so-called Dual Mean Approach is introduced within the paper. Unlike standard techniques, it does not fix k in advance but seeks the best k by considering together two measures: the average distance between points inside a cluster and the average distance between points from different clusters. The use of both at the same time allows seeing more clearly into the structure of the data. The main idea is to balance compactness within clusters and separation between clusters. Tests on a few benchmark datasets show that this approach manages to estimate k values that are close to the actual labels. It also results in more consistent results when compared to the Elbow Method.