Take the time to evaluate new techniques properly

Monday, January 2, 2012

It takes more time and effort, but you get a better payback when you implement a change on your farm that is based on solid data rather than on single comparisons

by KEITH REID

Every year, a range of new techniques are developed for you to try on your farm. Some come with lots of data to support them, and others with almost none. The ultimate test, though, is whether they work on your own farm. 

A fair, unbiased assessment of any new technique will be able to show clearly the differences between the old and the new way of doing things. It takes some pre-planning for on-farm trials to meet this standard, so that the results don't slow down or reverse progress towards higher yields and more environmentally responsible farming. You can deal with this situation by avoiding a few of the more common errors in assessing new techniques.

Split-field trials. Comparing two halves of a field that have different treatments might seem to be a good way to evaluate whether those treatments are actually different.  However, even in the most consistent fields, one side differs from the other and there is no way of knowing whether the difference comes from the treatments or from the underlying variability. 

The way to get around this is to use alternate strips of land for each treatment. If the difference between treatments is consistent across the field, you can be confident that the treatment is actually causing the difference. Statisticians call this replication.

By the way, if you can see that one of the strips has been damaged by something other than the treatment (a flooded area or wildlife damage, for example), you should exclude it from the assessment. Replication accounts for the variability we can't see, but that doesn't mean we shouldn't account for the variability we can see.

Unfair application of treatments. When comparing a new practice on your farm to "business as usual," it can be a challenge to account for all the variables that would allow for a fair comparison, while still getting the work done on time. Although a complex change is more likely to include some factor that would favour one practice over another, this would not come into play if either practice were used on the entire field. This is not unique to on-farm trials, and you should always look at the results of research trials to see if this type of scenario has influenced the results.

An example of this type of complexity would be comparing tillage systems. I have seen many trials that attempt to be "fair" by planting both a no-till and tilled treatment on the same date.

In an ideal spring this might be reasonable. Very few springs, however, provide ideal conditions. Persistent rainy weather can lead to a situation where delaying the no-till planting until the tillage has been done in the other treatment means a delay of several weeks.

Alternatively, the no-till treatment might take a few days longer to dry to planting condition, and the tilled treatment might get excessively dry during that time. In either case, the yield differences at the end of the year wouldn't reflect what either system was capable of producing with proper management.

Compound this with changes in planter settings that might or might not have been made, differences in weed pressure and changes in pest populations, and it becomes clear that keeping good field notes during the season is important for interpreting what caused the yield results at the end of the season.

Single-year or single-location comparisons. Capt. Edward A. Murphy, a U.S. Air Force engineer, is famous for the saying, "If anything can go wrong, it will." While we have seen this statement proven many times, the chances of things going wrong will be much less if the comparisons are repeated over a number of different fields and a number of different years.

It takes more time and effort, but there is a much better payback when you implement a change on your farm that is based on solid data rather than on single comparisons. BF

Keith Reid is Manager (Eastern Canada), Soil Nutrient and GHG Management,
Agriculture and Agri-Food Canada, Guelph.