Computer Vision and Pattern Discovery

Figure 1.

2. Single-Cell In Situ Spatial Omics at Subcellular Resolution

We work on spatial omics research in collaborative with Genome Institute of Singapore, Singapore Immunology Network, National Cancer Centre Singapore, Singapore General Hospital and Duke-NUS Medical School. By utilizing spatial omics techniques, such as MERFISH, we can detect the expression and physical location of thousands of DNA and RNA molecules. This approach allows us to study the spatial organization, cell types, and intercellular communication. Specifically, we proposed a new method called BANKSY1, which achieves more accurate cell type clustering by augmenting the transcriptomic profile of each cell with a weighted sum of its spatial neighbors' transcriptomes. Additionally, we investigated DNA copy number variation (CNV) in colorectal cancer and have identified distinct CNV profiles of colorectal subtypes, which can aid in cancer state prediction and prognostic treatment.

Related Publication

Singhal, V., Chou, N., Lee, J., Yue, Y., Liu, J., Chock, W. K., ... & Prabhakar, S. (2024). BANKSY unifies cell typing and tissue domain segmentation for scalable spatial omics data analysis. Nature Genetics, 1-11.

3. A bioinspired nanocoating technology for improved germination and plant growth

As part of a collaborative effort with the Institute of Materials Research and Engineering ASTAR (IMRE) we developed a research support application to monitor the growth of plants during the germination stage. The application uses computer vision to detect the plants from mobile phone images and can compute useful metrics regarding their growth. The application also includes an easy-to-use graphical interface that allows the user to set up the experiments, import and analyze the images.

Figure 2. processing pipeline of plant monitoring software

4. Automated analysis of coronary angiography

Coronary angiography is the gold standard imaging technique in the diagnosis of coronary artery disease. This method involves the acquisition of moving X-ray images from various angles around the patient's torso following the injection of iodine-based contrast. Traditionally, the analysis and interpretation of coronary angiogram is performed by trained cardiologists. However, the process can be time-consuming and subject to inter-observer variability. In collaboration with the National Heart Centre Singapore, our project explores AI approaches to automate the analysis of angiogram video sequences, aiming to develop a quantitative assessment tool that provides repeatable and objective measurements.

Figure 3. In the left panel, the original coronary angiography frame is displayed. The middle panel illustrates the annotated coronary artery (highlighted in green) and stenosis (highlighted in red) on the coronary angiography frame. The right panel shows the model's prediction of the coronary artery (highlighted in green) and stenosis (highlighted in red).

5. Intra-tumor Heterogeneity Through the Lens of Image Analysis

Cancer is a leading global cause of death, responsible for 9.7 million deaths in 2022. Cancer begins with cells acquiring mutations, growing uncontrollably, and forming tumors. With latest advances in precision medicine, clinicians are using molecular level information to design treatment. For example, tumor purity (fraction of cancer cells out of all cells in given tissue area), FGA (Fraction Genome Altered), EGFR mutation status, etc, are needed for better cancer treatment. These molecular level information needs additional resources in the current clinical workflow.

Currently pathologists review the whole slide images as the first assessment of cancer. The (expensive) molecular tests are carried out as and when needed and are not in routine workflow. Our team is developing deep learning models to quantify tumor by analyzing histopathology images. One such model, based on Multiple Instance Learning (MIL), predicts tumor purity from digital slides. Accurate tumor purity estimation is crucial for selecting samples for genomic sequencing. Our tumor purity model has shown high consistency with genomic tumor purity values across eight TCGA cohorts and a local Singapore cohort. Our model also provides tumor purity maps, revealing spatial variation within sections and aiding in understanding the tumor microenvironment.

Additionally, we are developing models for tumor staging, mutation analysis, and survival analysis. Our aim is to support medical professionals in diagnosis, treatment planning, medication management, and precision medicine, addressing future healthcare demands more effectively.

Figure 4. Spatial Tumor Purity Maps [1] facilitates:

• Region selection for macro-dissection in low-purity samples
• Better understanding of tumor micro-environment

Related Publication

Oner, M. U., Chen, J., Revkov, E., James, A., Heng, S. Y., Kaya, A. N., ... & Lee, H. K. (2022). Obtaining spatially resolved tumor purity maps using deep multiple instance learning in a pan-cancer study. Patterns, 3(2)

6. A Trimodal AI Framework for RNA Target Discovery

Identifying disease targets remains a major challenge due to the lack of access to high-quality molecular data at the "scene of the crime", i.e., within the context of the diseased tissue. Spatial omics (SO) technology has emerged as a powerful tool for high-throughput profiling of RNAs within tissues. However, SO data is expensive and difficult to collect, and has not yet been proven in the clinic. Conversely, histopathological whole-slide imaging (WSI) is routinely used in histopathology for diagnosis and prognosis, yet it lacks the molecular insights crucial for biomarker discovery.

This project pioneers a tri-modal AI framework that integrates molecular (SO), histopathological (WSI), and clinical data to identify novel disease targets. By leveraging national-scale data repositories, including an extensive Asian disease cohort, our integrative model unlocks the latent potential of existing WSI and clinical data resources through multi-modal learning. A central aim of this effort is the development of an SO data generator that can impute spatial omics profiles from WSI data. This generator will address the roadblock of SO data scarcity and pave the way for the discovery of novel targets.

Figure 5. Trimodal AI Framework for RNA Target Discovery