Data Mining Hub
Advanced data mining and clustering analytics platform built with Next.js frontend and Flask backend. Features multiple clustering algorithms, real-time data preprocessing, statistical analysis, and comprehensive visualization tools for complex dataset analysis.
Overview
Data Mining Hub is a sophisticated analytics platform designed for advanced data exploration and clustering analysis. The application provides a seamless integration between a modern Next.js frontend and a powerful Flask backend, enabling researchers and data scientists to upload datasets, perform real-time statistical analysis, apply multiple clustering algorithms, and visualize results with interactive charts and dendrograms. The platform supports K-Means, K-Medoids, DBSCAN, and Hierarchical clustering with comprehensive performance metrics including Silhouette Score, Davies-Bouldin Index, and Calinski-Harabasz Score.
Outcomes
- 4 clustering algorithms with 3 validity metrics behind a 20+ endpoint API
- Real-time preprocessing, PCA and visualization for uploaded datasets
Key Features
- CSV and ARFF file upload with automatic parsing
- Real-time data preview with column selection
- Comprehensive statistical analysis (mean, median, mode, missing values)
- Data preprocessing with multiple fill strategies
- Data normalization and standardization
- K-Means clustering with elbow method visualization
- K-Medoids robust clustering with representative points
- DBSCAN density-based spatial clustering
- Hierarchical clustering with dendrogram visualization
- PCA dimensionality reduction for high-dimensional data
- Multi-algorithm performance comparison
- Silhouette, Davies-Bouldin, and Calinski-Harabasz metrics
- Real-time plot generation (matplotlib backend)
- Export results in CSV format
- CORS-enabled REST API with 20+ endpoints
Role
Full-Stack Developer & Data Science Engineer
Challenge
Building a complex full-stack application that integrates a responsive Next.js frontend with a robust Flask backend capable of handling large datasets, implementing multiple clustering algorithms with real-time visualization, and ensuring smooth data flow with proper error handling and performance optimization.
Solution
Implemented a modular component architecture with TypeScript, created a Flask API with 20+ endpoints for data processing and clustering, integrated scikit-learn for the machine learning algorithms, used pandas and NumPy for efficient data manipulation, and rendered real-time visualizations with matplotlib.