Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics

Matteo Togninalli, Xu Wang, Tim Kucera, Sandesh Shrestha, Philomin Juliana, Suchismita Mondal, Francisco Pinto, Velu Govindan, Leonardo Crespo-Herrera, Julio Huerta-Espino, Ravi P Singh, Karsten Borgwardt, Jesse Poland

Research output: Contribution to journalArticlepeer-review


Motivation: Developing new crop varieties with superior performance is highly important to ensure robust and sustainable global food security. The speed of variety development is limited by long field cycles and advanced generation selections in plant breeding programs. While methods to predict yield from genotype or phenotype data have been proposed, improved performance and integrated models are needed. Results: We propose a machine learning model that leverages both genotype and phenotype measurements by fusing genetic variants with multiple data sources collected by unmanned aerial systems. We use a deep multiple instance learning framework with an attention mechanism that sheds light on the importance given to each input during prediction, enhancing interpretability. Our model reaches 0.754 ± 0.024 Pearson correlation coefficient when predicting yield in similar environmental conditions; a 34.8% improvement over the genotype-only linear baseline (0.559 ± 0.050). We further predict yield on new lines in an unseen environment using only genotypes, obtaining a prediction accuracy of 0.386 ± 0.010, a 13.5% improvement over the linear baseline. Our multi-modal deep learning architecture efficiently accounts for plant health and environment, distilling the genetic contribution and providing excellent predictions. Yield prediction algorithms leveraging phenotypic observations during training therefore promise to improve breeding programs, ultimately speeding up delivery of improved varieties.
Original languageEnglish (US)
StatePublished - May 23 2023

ASJC Scopus subject areas

  • Biochemistry
  • Computational Theory and Mathematics
  • Computational Mathematics
  • Molecular Biology
  • Statistics and Probability
  • Computer Science Applications


Dive into the research topics of 'Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics'. Together they form a unique fingerprint.

Cite this