Next-Generation (Clustered) Heat Maps V2
|Description||Next-Generation (Clustered) Heat Maps are interactive heat maps that enable the user to zoom and pan across the heat map, alter its color scheme, generate production quality PDFs, and link out from rows, columns, and individual heatmap entries to related statistics, databases and other information.|
Bradley M. Broom, Michael C. Ryan, Robert E. Brown, Futa Ikeda, Mark Stucky, David W. Kane, James Melott, Chris Wakefield, Tod D. Casasent, Rehan Akbani and John N. Weinstein, A Galaxy Implementation of Next-Generation Clustered Heatmaps for Interactive Exploration of Molecular Profiling Data. Cancer Research 77(21): e23-e26 (2017).
Michael C. Ryan, Mark Stucky, Chris Wakefield, James M. Melott, Rehan Akbani, John N. Weinstein, and Bradley M. Broom, Interactive Clustered Heat Map Builder: An easy web-based tool for creating sophisticated clustered heat maps. F1000Research 2019, 8 (ISCB Comm J):1750.
|Help and Support|
|Contact||Bradley M. Broom|
The NG-CHM Heat Map Viewer is a dynamic, graphical environment for exploration of clustered or non-clustered heat map data in a web browser. It supports zooming, panning, searching, covariate bars, and link-outs that enable deep exploration of patterns and associations in heat maps.
We provide a variety of tools for building your own NG-CHMs. These tools are described in section Building NG-CHMs.
We post updates and news to our website and social media accounts. Follow us for the latest updates:
Select the sample heat map link below to use the NG-CHM viewer to dynamically explore a sample heat map. Check out the Video Tutorials section below for an introduction to the NG-CHM features and functions.
The NG-CHM Heat Map Viewer contains two panels. The Summary Panel on the left provides a full view of your heat map. The Detail Panel on the right provides a zoomed in view of a portion of the heat map. Click anywhere on the summary picture to see the detail of that portion of the map. The green box on the Summary Panel indicates the portion of the heat map currently displayed in the Detail Panel. You may zoom in or out with the mouse scroll wheel or zoom in/out buttons to see smaller or larger portions of the map in the Detail View. Double clicking on a cell in the Detail Panel will also zoom in and center on that cell and shift-double click will zoom out. When the zoom level permits, row and column labels will be displayed in the Detail Panel. You may also use arrow keys to move up/down/left/right one row in the Detail Panel. The divider bar may be moved by clicking on it and dragging left or right to change the relative size of the Summary and Detail Panels.
We have a video series of short tutorial videos to introduce the various capabilities and controls of the NG-CHM Viewer.
There are a variety of buttons on the header bar to manipulate the heat map view. Each has a tooltip that can be displayed by hovering over the button for a view seconds with the mouse pointer.
|Zoom in on the Detail Panel making rows and columns larger. Mouse scroll forward will also zoom in.|
|Zoom out on the Detail Panel to show more rows and columns. Mouse scroll backward will also zoom out.|
|Set the Detail Panel in normal mode where the data points are square and the number of rows/columns shown will be controlled only by the zoom level. Normal mode contrasts with ribbon view modes. When selected, the background of the button will be darker.|
|Set the Detail Panel in ribbon horizontal mode. It is often useful to be able to see the full width of the heat map but also see the row labels. Ribbon horizontal mode shows a horizontal slice of the heat map containing all the columns and a few of the rows. The zoom level will then control how many rows are shown. When selected the button background will be darker. Select the normal mode button to exit ribbon view.|
|Set the Detail Panel in ribbon vertical mode. It is sometimes useful to be able to see the full height of the heat map but also see the column labels. Ribbon vertical mode shows a vertical slice of the heat map containing all the rows and a few of the columns. The zoom level will then control how many columns are shown. When selected the button background will be darker. Select the normal mode button to exit ribbon view.|
|Search:||Enter a search term in the search box and hit enter or click the Go button to look for the search term in the heat map labels. Search will select matching labels and the -> and <- buttons can then be used to jump to selected rows/columns in the Detail Panel. See | Search ] below for more details.|
|This button will split the Summary Panel and Detail Panel into two coordinated browser instances. This is helpful if you are working on a computer with multiple monitors and would like to see the Summary Panel on one screen and Detail Panel on another.|
|If you have split the Summary and Detail Panel into two browsers, this will join them back together into a single browser.|
|This button opens up a PDF generation window that allows you to generate a PDF document of the heat map. The PDF window allows selection of various options for the generation of the PDF. The detail window in the generated PDF will be in the same position and zoom level as is currently shown in the viewer.|
|The save button allows users to save a local copy of a heat map. Changes made through the configuration panel are saved to the local copy of the heat map. Heat maps are stored as a collection of files compressed into a zip file. Local heat map files can be viewed in several ways including a local stand-alone version of NG-CHM Heat Map Viewer. Please see [[NG-CHM-V2:Overview#Save and Run Local Copy|
|The gear button opens the heat map display properties panel. Many items of the heat map can be configured including heat map colors, label sizes, which co-variate bars to display, and data layer breakpoints. Please see [[NG-CHM-V2:Overview#Configuration Settings|
|For advanced maps with multiple data layers, this control allows you to toggle between the different data layers. Please see [[NG-CHM-V2:Overview#Data Layers|
The search function on the toolbar is used to find specific rows or columns in the heat map by searching the labels. Enter a search term in the search box and hit enter or click the Go button. By default search will perform partial matches. If you want to find only exact matches place quotes characters around each search term. For example:
Search: Gene_2 will find rows Gene_2, Gene_21, Gene_283, etc.
Search: “Gene_2” will find only the row Gene_2
Multiple search terms can be entered separated by spaces.
The search box will turn yellow if some search terms are not found and will turn red if no search terms are found. Use the -> and <- buttons next to the search box to have the detail view jump to the next or previous row / column that match the search terms.
The labels of rows and columns that match the search term will be selected (green background) and a green highlight will be displayed in the margin of Summary Panel at the position of matching rows/columns. To clear selections from a search, use the red X button next to the search box.
Detail Panel Selecting rows and/or columns in the heat map is useful because operations like copy to clipboard or link out functions may be performed on selections.
Rows or columns can be selected through a search or may be selected by clicking on a row or column label. To select multiple labels use shift-click to select a block of labels or control-click to individually select multiple labels. When labels are selected, you can right click on one of them to access a pop-up menu of actions that can be performed on the selected labels. For advanced maps, a variety of context specific link out options may be available.
Drag selection is also possible. On the Detail Panel hold down shift and the left mouse button while dragging the mouse pointer across the rows and columns you wish to select. Matrix data specific link out functions can be accessed by right clicking on the resulting green selection box in the Detail Panel.
For clustered heat maps, It is also possible to select rows / columns by clicking on the dendrogram branch in the Detail Panel of the rows/columns you wish to select.
Selection operations performed on the Summary Panel will not select rows and columns but will select the view shown in the Detail Panel. Use shift and hold down the left mouse button while dragging the mouse pointer across the region that you would like to view in the Summary Panel. The Detail Panel will select the closest matching zoom level for the region you wish to view.
For clustered heat maps, a selection option in the Summary Panel can be used to view a section of the map associated with a dendrogram branch. This feature is particularly useful for investigation of sub-sections of the heat map related class of samples (e.g. subgroup of cancer patients identified by a dendrogram branch). Selecting a dendrogram branch in the Summary Panel will lock the Detail Panel to only portion of the heat map associated with the selected branch of the dendrogram. The selected portion of the map is visible as it normally is in the Summary Panel but the non-selected part of the heat map is grayed out. You can scroll up/down or left/right through dendrogram branch data in the Detail Panel. This view is essentially a restricted ribbon view. To get out of the sub-dendrogram view, click on the normal mode button.
In the Detail Panel, some heat maps have a link out capability that supports deeper exploration of heat map data. Pre-constructed link out functions are available for certain data types, for example Hugo gene symbols. If at the time a heat map is constructed, the label data type for rows and or columns is associated with available link out functions, then a link out menu can be accessed.
To access link out functions, right click on a label for link outs for just that label or select multiple labels use the links in the “Linkouts for entire selection” section for the selected group of labels.
Link outs will take you to different websites to get detailed information about a row or column. Some link outs are available for single row/column selections. Other link outs in the “Linkouts for entire selection” section support multiple rows / columns. For example, if multiple gene symbols are selected the cBioPortal link will send the list of gene symbols to cBioPortal.
Some link outs are also available for matrix selections (green boxes on the Detail Panel). If a section of the matrix is selected via a drag selection or shift selection of rows and columns, then you can right click on the matrix selection area surrounded by the green box to access the matrix link out menu.
Even if a heat map does not have specific link out functions for the data types in the heat map, a row, column, and matrix link out menu is available to copy selected labels to the clipboard.
Many aspects of the presentation of heat maps in the NG-CHM viewer can be customized. The control panel is accessed by clicking on the gear icon on the far right side of the tools icons. The control panel has three tabs for different aspects of heat map customization. If a heat map is ‘read only’, then changes made in the control panel will be displayed for the duration of a browser session and can be saved to a local copy of the heat map on your local computer. See [[NG-CHM-V2:Overview#Save and Run Local Copy | Save]] below. If the heat map is not ‘read-only’, then you can save configuration changes to the server and everyone who subsequently views the heat map will see it with your changes.
Configure Rows and Columns
Indicates the version of NG-CHM Software used to create the heat map.
Indicates if a map is locked such that configuration changes can not be saved to the server. If so, changes can be saved to a local copy.
Total Rows Count of the number of rows in the heat map.
Labels Type Describes the type of data represented by the row and identified by the row label (e.g. Gene Symbol). In advanced maps there can be additional row level metadata which is not displayed. If so these will be shown in this field separated by commas.
Ordering Method The method used to order the rows in the heat map. Currently can be: hierarchical clustering, original order, or random.
Agglomeration Method This is applicable only when rows are ordered by hierarchical clustering and describes the algorithm used for clustering. The currently supported methods are: Average Linkage, Complete Linkage, Single Linkage, Ward, Mcquitty, Median, or Centroid.
Distance Metric This is applicable only when rows are ordered by hierarchical clustering and describes the metric used to calculate the distance between any two rows. The currently supported metrics are: euclidean, binary, manhattan, maximum, canberra, minkowski, or correlation.
Show Dendrogram: For maps with hierarchical clustering, may be configured to hide/show the dendrogram for rows in the summary panel and/or the detail panel.
Dendrogram Height: Use the dropdown to increase or decrease the amount of space used in the Summary and Detail view for the dendrogram. Increasing the size of the dendrogram will decrease the space available for heat map values and vice-versa.
Same as above but applied to the columns.
Maximum Length: Row / Column labels may need to be trimmed in the Detail Panel display. This values allows you to increase/decrease the truncation point for labels. Longer labels will leave less room for display of the other heat map elements.
Trim Text From: If labels need to be trimmed, this option sets the method to use to shorten the strings: trim the beginning, remove characters from the middle, or truncate.
The Data Layers tab is used to configure the matrix (center) portion of the heat map.
For advanced maps, there may be multiple data layers loaded. If this is the casee you need to select the layer that you wish to configure from the Data Layer drop down menu. For regular heat maps constructed from a single data matrix, this selection does not need to be used.
Breakpoint/Color A set of breakpoints/colors determines how the heat map data is displayed. If the value of a cell is exactly the same as a breakpoint then it will be the color assigned to the breakpoint. If the value falls between two breakpoints, the color will be an interpolation of the two breakpoint colors based on where the value falls between the breakpoints. The color will be a blend of the color of the two breakpoints but may more strongly favor one breakpoint color if it is closer to one breakpoint than the other. If a value is below the first breakpoint or above the last breakpoint, it gets the color of the first/last breakpoint as appropriate.
To change the display of heat map data, you may click on and change the color associated with a breakpoint, change the value of a breakpoint, or add/remove breakpoints. Breakpoints should be entered in ascending numerical order.
Missing Color: If there are missing data values in the matrix, this color will be used for them in the matrix display.
Choose a pre-defined color palette If you would like to change the colors used for the data in your heat map but don’t want to set them individually, just click on a pre-defined color palette in the list below this header.
Grid Lines: In the Detail Panel, lines are drawn to separate the data points when the zoom level permits. If you would prefer that the grid lines not be displayed, uncheck the Show option. If you do want grid lines but the current grid line color does not contrast well with the colors you selected for your heat map data, click on the grid line color box to change the grid line color.
Selection Color: Selected rows or columns are indicated by the selection color. This is the color of the square drawn in the heat map as well as the highlight color for the labels of selected rows/columns. Click on the color box to change the selection color.
Covariate bars display information associated with the rows and/or columns in your heat map, for example the smoking status of patients. If your heat map has covariate bars, this panel can be used to configure them.
This dropdown is used to select the covariate bar that you wish to configure.
Note there is a special item in this menu, ALL, that will be selected by default. When ALL is selected, a list of all of the covariate bars in the heat map is displayed and you can select which covariates you would like to be displayed and adjust the height of the covariate bars. The checkbox to the left of the Show column header can be used to hide or show all covariates. If the heat map has lots of covariate bars, the covariate filter can be used to select a subset of the covariates. For example ‘mutation’ could be applied as a filter to get a list of just the mutation covariates and then the checkbox next to the Show column could be used to hide all of the mutation covariate bars and the remaining covariates would be displayed.
If a specific covariate is selected, the panel will change to look like this. The coloring scheme of the selected covariate can then be modified.
Covariate Bar:The selected covariate bar.
Bar Position Indicates whether the classification bar is for rows or columns.
Bar Type There are two types of covariate bars: continuous (range of values like age), and discrete (specific list of items like smoker status). The type will affect the breakpoint definition below.
For discrete covariate bars, there are not breakpoints per se but instead there is a list of categories and colors:
Category One entry per distinct category in the covariate bar. Click on the color next to a given category to change the color for that category.
For continuous covariate bars, the color schemes work like those on the heat map data where there are breakpoints and the color of the covariate for a given row/column is interpolated based on the breakpoints.
Category Provide the breakpoint values and colors. Covariate breakpoints should be in ascending numerical order. Covariates that are less than the first breakpoint or greater than the last breakpoint will get the color of the first or last breakpoint as appropriate. Covariates that fall between breakpoints will get a color that is a blend of the two breakpoint colors based on where the value falls between the two breakpoints.
Missing: The color assigned to rows or columns that were not in the covariate data. Click on the color to change the missing covariate color.
Choose a pre-defined color palette If you don’t want to set classification colors individually, you can pick one of the color palettes to change the colors of all of the classifications.
Most heat maps have a single matrix of data but in some advanced heat maps multiple matrices are loaded so that users can switch rapidly back and forth to see multiple measurements in the context of the clustered heat map. For example, raw values and normalized values could be in different data layers of a single heat map. Only the first data layer is used for clustering but other layers can provide rich detail when investigating sections of the heat map.
To control the display of data layers, click on the data layers icon. This icon will only be present when multiple data layers are included in a heat map. Clicking on the icon will open the data layers control. There may be many data layers in a heat map and this control is designed to allow for quick toggling between two of the layers. Select one layer that you wish to use for toggling in the first drop-down and the alternate toggle layer in the second dropdown. You can then toggle between the two layers by clicking on the radio button next to the data layers or by hitting the F2 key on your keyboard. The currently selected data layer is highlighted with a yellow background in the data layer control. Click on the data layer icon again to close the control.
The NG-CHM Viewer is generally run by opening a heat map on a website or by visualizing an NG-CHM heat map in Galaxy. It is possible, however, to save a local copy of a heat map and view it with a locally installed copy of the NG-CHM viewer. This is useful if you want to customize and save a copy of a read-only heat map or if you want to view heatmaps without installing the heat map on a web server. To save a local copy of a heat map, just click on the save icon. If you have made changes to the heat map like color changes or adjusting label lengths, when you click on the save icon, you will get a dialog like this one.
If you have full rights to the heat map, you can either save the changes to the server (Save Original) so that all users will see your updated version of the heat map or you can just save a local copy of the changes to your computer (Save to .ngchm). If the heat map is read-only, you will only have the option of saving a local copy. Depending on your browser settings, you will either be prompted for the location where you would like to save the heat map or it will go to the browser Downloads folder.
After downloading the heat map, a second dialog will allow you to install a local copy of the NG-CHM Viewer if you do not yet have it on your computer.
Select the Download Viewer button on the next dialog to download a local copy of the viewer. Again you will be prompted for where you would like to save the ngChmApp.html file or it will go to your browser’s Downloads folder. Just double-click on the ngChmApp.html file to run the viewer in your browser.
Finally, to open any map that you have stored on your machine, click the Browse button in the viewer to select the .ngchm file. This will open the heat map in the local NG-CHM viewer.
First, download and unzip the Viewer software into the directory of your choice:
Then pull down a sample NG-CHM heat map and store it locally (use the save button on a map in the TCGA portal or download this one):
Finally, run the heatmap viewer by clicking on the chm.html file in the folder where you installed the viewer. When it opens in your browser, click on the Browse… button and select the sample heat map file tcga_rnaseqv2_kirc_v2.0_gene_sample.ngchm that you downloaded.
For building your own NG-CHMs, we provide a variety of tools catering to a wide variety of different use cases:
The NG-CHM Builder lets you build NG-CHMs using a browser-based graphical user interface. This is the most convenient option for building NG-CHMs and requires no special informatics expertise. For building test NG-CHMs, a small test data set can be loaded easily. Although the builder supports many commonly-used options (such as common clustering methods), uncommon or specialized methods are not supported by this builder.
Galaxy is a popular open source platform for performing bioinformatics analysis in a documented, repeatable process. NG-CHM tools are available for creating and viewing heat maps on the Galaxy platform.
Installing the NG-CHM Galaxy Tools: The NG-CHM Galaxy tools are available in the Galaxy tool shed. Two components need to be installed to fully utilize the NG-CHM heat map tools in Galaxy: the heat map generator tool for creating heat maps and the NG-CHM Viewer plug-in for visualizing and dynamically exploring heat maps. To install the NG-CHM tools in an existing Galaxy server, please follow the instructions on our Galaxy tool shed page. Search for ‘ng-chm’ in the toolshed or use this link
Creating an NG-CHM using the NG-CHM Galaxy Tools: The NG-CHM Generator tool is used to create an NG-CHM heat map. The options for heat map generation are described below.
Input Matrix - Select a data matrix that is already loaded in your current history using a Get Data tool. The data matrix needs to be a tab delimited text file with column identifiers in the first row and row identifiers in the first column. For example, column identifiers could be patient IDs and row identifiers could be gene symbols. The other cells of the matrix should be numeric values. It is often advisable to perform normalization on data prior to clustering and several other Galaxy tools are available to perform data matrix normalization.
Heat Map Name - A short user-defined name for the heat map that will be displayed as its title.
Heat Map Description - A longer user-defined description for the heat map that will be available in the viewer via mouse over of the name.
Data Summarization Method - For large data matrices, it is necessary to summarize multiple values into a single pixel in the summary NG-CHM heat map panel. The possible summarization methods include: average, sample, and mode. Average will compute an average value to represent several data values. Sample will just draw every Nth value. Mode will use the most prevalent value as the summary value. In general, average works best to summarize the data in an intuitive fashion. In the case of categorical data, Mode will be a better summary value.
Row / Column Ordering Method - Generally user are interested in clustering both rows and columns but for some analysis it may be useful to cluster just rows or columns. Use the Ordering Method selections to indicate which axes you wish to cluster and for those not being clustered whether you would like original data order or random order.
Row / Column Distance Metric - If hierarchical clustering is selected for ordering method, a technique for measuring distance between rows/columns must be selected. The available distance measures are provided by R and include: Euclidean, Binary, Manhattan, Maximum, Canberra, Minkowski, and Correlation.
Row / Column Clustering Method - If hierarchical clustering is selected for ordering method, an algorithm for clustering must be selected. The algorithms are provided by the R hclust function and include: Complete Linkage, Single Linkage, Ward, Mcquitty, Median, and Centroid. If you are not familiar with these algorithms, Ward is often a good initial choice.
Row / Column Tree Cut Covariate - NG-CHMs can show annotation data associated with rows / columns as covariate bars. The tool can automatically generate a category covariate bar using the clustering information by ‘cutting’ the data using the N largest clusters in the dendrogram. Set this option to something other than 0 to get a category classification bar with the number of categories (between 2 and 10)
Row / Column Linkout Data Type - NG-CHM supports linkout functions for the rows or columns of some heat maps. If your data identifiers (row / column headers) are of a type for which NG-CHM provides linkouts, select the data type to activate linkouts in the viewer. For example, if your rows are gene symbols select the ‘Gene HUGO symbol’ to activate many link out options in the Heat Map Viewer.
Covariate Bars - As mentioned above, NG-CHMs can show annotation data associated with rows / columns as covariate bars. For example, if your columns are patients and you want to display smoker status in the heat map, the smoking status of the patients can be loaded as a covariate bar. Any number of covariate bars can be added. Select the + Insert Covariate Bars to add one.
Axix Covariate Name - User-defined name for the covariate that will be displayed in the heat map.
Covariate File - A tab-delimited text file with covariate data. The first column of the file must contain row or column identifiers that match the row / column identifiers in the data matrix file. The second column must contain the covariate data. Covariate data can be categories (e.g. smoker, non-smoker) or continuous data (21, 34, 22, 41).
Axis Covariate Type - Select whether the covariate bar is a row or column bar and whether it is categorical data or continuous data.
After the NG-CHM Generator tool completes running, you can view and explore the resulting heat map by clicking on the heat map item in the Galaxy history. Once the history item is open, there is a visualization button on the bottom of the history item (circled in red above). Select the NG-CHM Heat Map Viewer after clicking the visualize icon. Note: you must be logged on in Galaxy to see the visualization icon.
Heat maps can be explored in galaxy using zoom, pan, selection, link outs, and customization as described in the sections above. Also, you can click on the download icon in the Galaxy history (disk icon) to get a local copy of the heat map that can be shared and viewed with the stand-alone NG-CHM viewer outside of Galaxy.
Docker is a container system that runs on Windows, Apple, and Unix systems. Docker makes it fast and easy to run software packages because a Docker image includes all of the software and system dependencies needed to run the application. We have created a Docker image that has a full implementation of Galaxy with the NG-CHM tools pre-installed. The NG-CHM Galaxy Docker image is available on Docker Hub.
Please see our Docker Hub page:
The Docker Hub page provides instructions on installing and running the NG-CHM Galaxy image.
Also, see our tutorial video for an overview of running a Docker NG-CHM Galaxy container and creating/viewing heat maps.
We provide an R package for creating NG-CHMs within the R statistical computing environment. The R package provides the greatest range of options for creating an NG-CHM. For instance, you can use any clustering method available within R or its package library to order the rows and/or columns in the NG-CHM. Using the R package requires programming expertise.