Machine learning helps plant science turn over a new leaf

Date:10-09-2019

Father of genetics Gregor Mendel spent years tediously observing and measuring pea plant traits by hand in the 1800s to uncover the basics of genetic inheritance. Today, botanists can track the traits, or phenotypes, of hundreds or thousands of plants much more quickly, with automated camera systems. Now, Salk researchers have helped speed up plant phenotyping even more, with machine-learning algorithms that teach a computer system to analyze three-dimensional shapes of the branches and leaves of a plant. The study, published in Plant Physiology on October 7, 2019, may help scientists better quantify how plants respond to climate change, genetic mutations or other factors.

“What we’ve done is develop a suite of tools that helps address some common phenotyping challenges,” says Saket Navlakha, an associate professor in Salk’s Integrative Biology Laboratory and Pioneer Fund Developmental Chair.


A plant’s environment helps dictate its structure, which is related to its health. Scientists trying to understand plant growth, engineer more resilient plants or boost crop production often want to measure detailed characteristics of a plant’s leaves and shoots. To do this phenotyping in a high-throughput way, many researchers use camera systems that take images of each plant from various angles and assemble a three-dimensional model. However, some measurements are hard to take with these stitched-together images.

Recently, some have turned to a new method, called 3D laser scanning, to capture the structure of plant architectures. Researchers shine a laser at each plant to “paint” its surface with the beam. The resulting data—called a 3D point cloud—portrays the fine detail of the plant’s surface. But quantitatively analyzing the point clouds can be challenging since the technology is so new and the datasets so large.

“The resolution and accuracy of this data is much higher,” says Navlakha. “But the methods that have been developed for analyzing leaves and branches in 2D images don’t work as well for these 3D point clouds.”

Navlakha, along with UC San Diego graduate student Illia Ziamtsov, used a 3D laser scanner to scan 54 tomato and tobacco plants grown in a variety of conditions. Then, they inputted the resulting 3D point clouds into machine-learning algorithms that let them teach the program how to phenotype the plants. The technique involved the researchers first indicating manually where leaves and shoots on the plants were. Then, the software began to automatically recognize these features.

“It’s like teaching things to a baby,” says Navlakha. “You give them examples of what a leaf looks like and what a branch looks like, and eventually they can identify a plant they’ve never seen before and pick out the leaves and branches.”

The researchers focused on teaching the program to make three phenotype measurements that scientists often use—separating stems from leaves, counting leaves and their size, and outlining the branching patterns of a plant. They found they were successful: for example, the method had a 97.8 percent accuracy at identifying stems and leaves.

“This kind of object detection has been used in self-driving cars and for identifying construction and furniture items,” says Ziamtsov. “But applying it to plants is totally novel.”

Navlakha and Ziamtsov want to continue fine-tuning the approach; differentiating two close-together leaves can still be challenging, for instance. And the current version of the software may not work on all types of plants. They hope to generalize the software to work on plants from vines to trees, and also to analyze roots.

“There are a lot of challenges in agriculture right now to try and increase crop production and sequester carbon better,” says Navlakha. “We hope our tool can help biologists address some of these broader challenges.”

Navlakha and Ziamtsov will release their software as open-source for other researchers to use. They hope the software will speed up plant research, since it makes high-throughput phenotyping faster and easier.

“Doing this kind of analysis by hand is very laborious,” says Ziamtsov. “Our tool does it quickly and pretty accurately.”

The work was supported by grants from the Pew Charitable Trusts, the National Science Foundation and the National Institutes of Health.

More information:
Illia Ziamtsov et al, Machine learning approaches to improve three basic plant phenotyping tasks using 3D point clouds, Plant Physiology (2019). DOI: 10.1104/pp.19.00524