.. _example_applications_plot_stock_market.py: ======================================= Visualizing the stock market structure ======================================= This example employs several unsupervised learning techniques to extract the stock market structure from variations in historical quotes. The quantity that we use is the daily variation in quote price: quotes that are linked tend to cofluctuate during a day. .. _stock_market: Learning a graph structure -------------------------- We use sparse inverse covariance estimation to find which quotes are correlated conditionally on the others. Specifically, sparse inverse covariance gives us a graph, that is a list of connection. For each symbol, the symbols that it is connected too are those useful to explain its fluctuations. Clustering ---------- We use clustering to group together quotes that behave similarly. Here, amongst the :ref:`various clustering techniques ` available in the scikit-learn, we use :ref:`affinity_propagation` as it does not enforce equal-size clusters, and it can choose automatically the number of clusters from the data. Note that this gives us a different indication than the graph, as the graph reflects conditional relations between variables, while the clustering reflects marginal properties: variables clustered together can be considered as having a similar impact at the level of the full stock market. Embedding in 2D space --------------------- For visualization purposes, we need to lay out the different symbols on a 2D canvas. For this we use :ref:`manifold` techniques to retrieve 2D embedding. Visualization ------------- The output of the 3 models are combined in a 2D graph where nodes represents the stocks and edges the: - cluster labels are used to define the color of the nodes - the sparse covariance model is used to display the strength of the edges - the 2D embedding is used to position the nodes in the plan This example has a fair amount of visualization-related code, as visualization is crucial here to display the graph. One of the challenge is to position the labels minimizing overlap. For this we use an heuristic based on the direction of the nearest neighbor along each axis. .. image:: images/plot_stock_market_001.png :align: center **Script output**:: Cluster 1: Time Warner, Comcast, Cablevision Cluster 2: Wells Fargo, American express, JPMorgan Chase, General Electrics, Pfizer, Ryder, AIG, Mc Donalds, Bank of America, DuPont de Nemours, Home Depot, Goldman Sachs, Wal-Mart, Ford Cluster 3: Microsoft, Cisco, 3M, Texas instruments, HP, Dell, IBM, SAP Cluster 4: Honda, Caterpillar, Mitsubishi, Canon, Toyota, Unilever, Navistar, Marriott, Xerox, Sony Cluster 5: Lookheed Martin, General Dynamics, Raytheon, Northrop Grumman, Boeing Cluster 6: ConocoPhillips, Valero Energy, Exxon, Chevron, Total Cluster 7: Kraft Foods Cluster 8: CVS, Walgreen Cluster 9: Sanofi-Aventis, GlaxoSmithKline, Novartis Cluster 10: Coca Cola, Kellogg, Pepsi Cluster 11: Apple, Yahoo, Amazon Cluster 12: Colgate-Palmolive, Kimberly-Clark, Procter Gamble **Python source code:** :download:`plot_stock_market.py ` .. literalinclude:: plot_stock_market.py :lines: 62- **Total running time of the example:** 11.27 seconds ( 0 minutes 11.27 seconds)