Digital phenotyping technologies set to take off across CGIAR

New and more accessible digital phenotyping technologies and methods can help breeding programs collect data faster and better. EiB is partnering across CGIAR to expand, accelerate and streamline adoption.

In the last few years, there has been a tool, technique and technology revolution in phenotyping. To ensure these are better streamlined across CGIAR and national crop breeding programs, CGIAR Excellence in Breeding (EiB) and partners deployed the Unmanned Aerial Vehicle (UAV) initiative. The project led to the development of key operating procedures and a model for a more centralized approach to rolling out the technology.

The project involved the International Maize and Wheat Improvement Center (CIMMYT), International Crops Research Institute for the Semi-Arid Tropics (ICRISAT) and CGIAR Big Data, and aimed to kick off a small revolution in the way operations teams measure plant traits.

CGIAR drone team in a maize field in India (Source: ICRISAT)

Benefits of technology

In the plant breeding world, measuring plant traits can be time-consuming and challenging. However, with digital phenotyping, programs can expect to see an improvement in speed, accuracy and efficiency, and potentially reduced costs.

“Technicians face practical problems on the ground to generate data,” says Sobhan Sajja, head of ICRISAT’s crop improvement team. “Large fields take lot of time in capturing data manually resulting in improper data sometimes. Drones cover this gap”.

For example, instead of sending teams of data gatherers out to physically examine plots, drones fitted with various cameras and sensors can collect data just by flying over the plot. Satellite images and portable devices/smartphones can also be used to speed up data collection and ensure better consistency. This means measuring traits such as plant size, ripening time or even disease can be done much faster, and with sufficient accuracy. It also makes it easier to include important traits that are often ignored due to time-consuming manual assessment processes, such as staygreen traits.

Drone captures images of fields as part of UAV data analysis project (Source: ICRISAT)

The Digital Phenotyping skills gap

However, the reach of the technologies is still limited. Part of the reason lies in the skillsets needed to use such techniques, and a gap in understanding between the breeding programs and the potential technology providers.

The initiative team identified two main limitations to using UAV-based imaging technology. The first is generating quality images. Standard operating procedures (SOP’s) are needed for each flight, (including guidance on technical issues such as ground control points, flight height and speed, minimum image overlap). Unless these SOPs are rigorously followed, the quality of the images would not allow the generation of reliable data.

The second limitation was the processing of image data. A data analysis pipeline — which contains several essential, skills-based, intermediary steps to check data quality — needed to be developed.

To overcome these gaps and hasten the adoption of digital phenotyping across CGIAR and NARES breeding programs, the initiative teams set out to streamline efforts across CGIAR centers, in the scope of a regional hub. To do so, the project worked with Hiphen — a crop image analysis company working with commercial breeding companies.

Launch of the UAV initiative

To serve as a hub for the project, the team selected ICRISAT’s Patancheru's location in India, where an array of CGIAR crops could be represented and developed for validation experiments. Then came the challenge of selecting crops that not only represented CGIAR’s crop diversity but that could be cultivated in one location. The crops selected for the experiments were: maize, sorghum, chickpea and pigeon pea.

But the real challenges came as the program was about to take off — new regulations had been put in place by the Indian government, making the operators not authorized to fly drones until they could fulfill these. It took several months to complete this process.

“The drone regulation in India is a work in progress as regulating texts evolve very quickly,” explains project coordinator Vincent Vadez. “The openness of drone regulation has to be considered very closely to expand this initiative to other countries.”

While awaiting approval, the team held rounds of meetings with crop breeders and others to understand the trait requirements of each crop (relevant to product profiles) and to streamline different actors from legislative to drone piloting to image analysis.

Then, Hiphen set about developing SOPs using ground logistics tables developed for each crop. This provided drone operators with a document to follow over the course of the season. This included timetables for flights, and the type of flights needed (e.g. flying heights, type of camera).

The first flights took place in early February after drone permits were finally granted. Although it was too late in the growing season to measure a number of traits, the team chose to fly — no matter what — to ensure the project could generate some quality images and run through the process involving all the various actors and steps.

Operators reached high flight quality levels while fixing issues along the way. They acquired additional small equipment — an iPad to monitor the flight, a GPS for drone positioning. Improvements and adjustments were also needed on the logistic front, to ensure that each actor fulfilled their activities in time for others down in the chain. For instance, GPS location of a field and a field map were necessary before the ground control points could be placed across the field. This allowed the drone pilot to perform the initial flight at the time of sowing. Failing one of these essential steps would have caused problems along the full chain of steps.

Other improvements needed to be made along the way. For instance, it was found that the image overlap on the post-rainy sorghum crop was insufficient and of low quality, even though this image overlap was sufficient for other crops. There, Hiphen guided the operators with a revised SOP, testing the different percentages of overlap.

In all, the drones undertook 58 flights over the four crops during rainy and post-rainy seasons. Data gathered included plant count, canopy height, plant height, leaf senescence, and plot heterogeneity.

Drone image and analysis showing mean height of maize crops (Source: Hiphen)

Image data was then input into Hiphen’s Coverfield platform. Technicians evaluated the quality of these images to construct corrected “ortho-images.” A number of image datasets were considered unfit for analysis – particularly the insufficient image overlap of sorghum. But where possible, ortho-images were constructed and plots were identified and geo-localized from the images. The resulting data was then loaded on the Cloverfield platform for each flight, and breeders were trained on accessing the data.

Defining the engineering process

The efforts underscore that the work is a complete engineering process where roles, responsibilities, and timelines need to be tightly defined. The project has given a jump start to CGIAR/NARES efforts to expand.

“Even with the challenges of often working remotely, COVID and permit delays, we are pleased with what we’ve learned,” says Gustavo Teixeira, EiB operations and phenotyping module lead. “We have a good process in place, SOPs that can be adapted, and a lot of learning on how to create a data analysis pipeline.”

Large-scale adoption of drone-based phenotyping has real potential to help agricultural research and seed industries overcome human error, inefficiencies and hardware costs. To spur scaling across CGIAR, EiB and partners plan to create a business proposal for replacing current phenotyping methods. It needs to examine cost, training, deployment plans, and the required infrastructure per station. Then, the next step is to deploy digital phenotyping tools in other regions, particularly in Africa.

The project’s results are exciting, but even beyond the progress on technology and adoption issues, there are other challenges. As the UAV initiative expands to new crop varieties, defining the most desirable traits to analyze, and defining plot quality will be vital steps. But it’s all central to helping propel forward EiB’s goals of crop breeding modernization and genetic gain increases.