2. Unsupervised learning

• 2.1. Gaussian mixture models
  • 2.1.1. Gaussian Mixture
    • 2.1.1.1. Pros and cons of class GaussianMixture
      • 2.1.1.1.1. Pros
      • 2.1.1.1.2. Cons
    • 2.1.1.2. Selecting the number of components in a classical Gaussian Mixture Model
    • 2.1.1.3. Estimation algorithm Expectation-maximization
  • 2.1.2. Variational Bayesian Gaussian Mixture
    • 2.1.2.1. Estimation algorithm: variational inference
    • 2.1.2.2. Pros and cons of variational inference with BayesianGaussianMixture
      • 2.1.2.2.1. Pros
      • 2.1.2.2.2. Cons
    • 2.1.2.3. The Dirichlet Process
• 2.2. Manifold learning
  • 2.2.1. Introduction
  • 2.2.2. Isomap
    • 2.2.2.1. Complexity
  • 2.2.3. Locally Linear Embedding
    • 2.2.3.1. Complexity
  • 2.2.4. Modified Locally Linear Embedding
    • 2.2.4.1. Complexity
  • 2.2.5. Hessian Eigenmapping
    • 2.2.5.1. Complexity
  • 2.2.6. Spectral Embedding
    • 2.2.6.1. Complexity
  • 2.2.7. Local Tangent Space Alignment
    • 2.2.7.1. Complexity
  • 2.2.8. Multi-dimensional Scaling (MDS)
    • 2.2.8.1. Metric MDS
    • 2.2.8.2. Nonmetric MDS
  • 2.2.9. t-distributed Stochastic Neighbor Embedding (t-SNE)
    • 2.2.9.1. Optimizing t-SNE
    • 2.2.9.2. Barnes-Hut t-SNE
  • 2.2.10. Tips on practical use
• 2.3. Clustering
  • 2.3.1. Overview of clustering methods
  • 2.3.2. K-means
    • 2.3.2.1. Mini Batch K-Means
  • 2.3.3. Affinity Propagation
  • 2.3.4. Mean Shift
  • 2.3.5. Spectral clustering
    • 2.3.5.1. Different label assignment strategies
    • 2.3.5.2. Spectral Clustering Graphs
  • 2.3.6. Hierarchical clustering
    • 2.3.6.1. Different linkage type: Ward, complete, average, and single linkage
    • 2.3.6.2. Adding connectivity constraints
    • 2.3.6.3. Varying the metric
  • 2.3.7. DBSCAN
  • 2.3.8. OPTICS
  • 2.3.9. Birch
  • 2.3.10. Clustering performance evaluation
    • 2.3.10.1. Adjusted Rand index
      • 2.3.10.1.1. Advantages
      • 2.3.10.1.2. Drawbacks
      • 2.3.10.1.3. Mathematical formulation
    • 2.3.10.2. Mutual Information based scores
      • 2.3.10.2.1. Advantages
      • 2.3.10.2.2. Drawbacks
      • 2.3.10.2.3. Mathematical formulation
    • 2.3.10.3. Homogeneity, completeness and V-measure
      • 2.3.10.3.1. Advantages
      • 2.3.10.3.2. Drawbacks
      • 2.3.10.3.3. Mathematical formulation
    • 2.3.10.4. Fowlkes-Mallows scores
      • 2.3.10.4.1. Advantages
      • 2.3.10.4.2. Drawbacks
    • 2.3.10.5. Silhouette Coefficient
      • 2.3.10.5.1. Advantages
      • 2.3.10.5.2. Drawbacks
    • 2.3.10.6. Calinski-Harabasz Index
      • 2.3.10.6.1. Advantages
      • 2.3.10.6.2. Drawbacks
    • 2.3.10.7. Davies-Bouldin Index
      • 2.3.10.7.1. Advantages
      • 2.3.10.7.2. Drawbacks
    • 2.3.10.8. Contingency Matrix
      • 2.3.10.8.1. Advantages
      • 2.3.10.8.2. Drawbacks
• 2.4. Biclustering
  • 2.4.1. Spectral Co-Clustering
    • 2.4.1.1. Mathematical formulation
  • 2.4.2. Spectral Biclustering
    • 2.4.2.1. Mathematical formulation
  • 2.4.3. Biclustering evaluation
• 2.5. Decomposing signals in components (matrix factorization problems)
  • 2.5.1. Principal component analysis (PCA)
    • 2.5.1.1. Exact PCA and probabilistic interpretation
    • 2.5.1.2. Incremental PCA
    • 2.5.1.3. PCA using randomized SVD
    • 2.5.1.4. Kernel PCA
    • 2.5.1.5. Sparse principal components analysis (SparsePCA and MiniBatchSparsePCA)
  • 2.5.2. Truncated singular value decomposition and latent semantic analysis
  • 2.5.3. Dictionary Learning
    • 2.5.3.1. Sparse coding with a precomputed dictionary
    • 2.5.3.2. Generic dictionary learning
    • 2.5.3.3. Mini-batch dictionary learning
  • 2.5.4. Factor Analysis
  • 2.5.5. Independent component analysis (ICA)
  • 2.5.6. Non-negative matrix factorization (NMF or NNMF)
    • 2.5.6.1. NMF with the Frobenius norm
    • 2.5.6.2. NMF with a beta-divergence
  • 2.5.7. Latent Dirichlet Allocation (LDA)
• 2.6. Covariance estimation
  • 2.6.1. Empirical covariance
  • 2.6.2. Shrunk Covariance
    • 2.6.2.1. Basic shrinkage
    • 2.6.2.2. Ledoit-Wolf shrinkage
    • 2.6.2.3. Oracle Approximating Shrinkage
  • 2.6.3. Sparse inverse covariance
  • 2.6.4. Robust Covariance Estimation
    • 2.6.4.1. Minimum Covariance Determinant
• 2.7. Novelty and Outlier Detection
  • 2.7.1. Overview of outlier detection methods
  • 2.7.2. Novelty Detection
  • 2.7.3. Outlier Detection
    • 2.7.3.1. Fitting an elliptic envelope
    • 2.7.3.2. Isolation Forest
    • 2.7.3.3. Local Outlier Factor
  • 2.7.4. Novelty detection with Local Outlier Factor
• 2.8. Density Estimation
  • 2.8.1. Density Estimation: Histograms
  • 2.8.2. Kernel Density Estimation
• 2.9. Neural network models (unsupervised)
  • 2.9.1. Restricted Boltzmann machines
    • 2.9.1.1. Graphical model and parametrization
    • 2.9.1.2. Bernoulli Restricted Boltzmann machines
    • 2.9.1.3. Stochastic Maximum Likelihood learning