Do Agricultural Market Advisory Services Beat the Market? Evidence
from the Corn and Soybean Markets Over 1995-1998
Scott H. Irwin, Darrel
L. Good, Joao Martines-Filho
and Thomas E. Jackson [*]
Copyright 2000 by Scott H. Irwin, Darrel L. Good,
Joao Martines-Filho and Thomas E. Jackson. All rights reserved. Readers may make
verbatim copies of this document for non-commercial purposes by any means, provided
that this copyright notice appears on all such copies.
The advisory service marketing recommendations used
in this research represent the best efforts of the AgMAS Project staff to accurately
and fairly interpret the information made available by each advisory program.
In cases where a recommendation is vague or unclear, some judgment is exercised
as to whether or not to include that particular recommendation or how to implement
the recommendation. Given that some recommendations are subject to interpretation,
the possibility is acknowledged that the AgMAS track record of recommendations
for a given program may differ from that stated by the advisory service, or from
that recorded by another subscriber. In addition, the net advisory prices presented
in this report may differ substantially from those computed by an advisory service
or another subscriber due to differences in simulation assumptions, particularly
with respect to the geographic location of production, cash and forward contract
prices, expected and actual yields, carrying charges and government programs
The purpose of this paper is to address two basic performance questions for
market advisory services: 1) Do market advisory services, on average, outperform
an appropriate market benchmark? and 2) Do market advisory services exhibit persistence
in their performance from year-to-year? Data on corn and soybean net price received
for advisory services, as reported by the AgMAS Project, are available for the
1995, 1996, 1997 and 1998 crop years. Performance test results suggest that,
on average, market advisory services exhibit a small ability to "beat the
market" for the 1995 through 1998 corn and soybean crops. It is debatable
whether the performance of advisory services also is economically significant.
The predictability results provide little evidence that future advisory service
pricing performance can be predicted from past performance.
Do Agricultural Market Advisory Services Beat the Market? Evidence from the
Corn and Soybean Markets Over 1995-1998
Farmers in the US continue to identify price and income risk as
one of their greatest management challenges. Using a survey of midwestern grain
farmers, Patrick and Ullerich (1996) report that price variability is the highest
rated source of risk by crop farmers. Coble, Patrick, Knight and Baquet (1999)
survey farmers in Indiana, Mississippi, Nebraska and Texas and find that crop
price variability, by a wide margin, is rated as having the most potential to
affect farm income. Norvell and Lattz (1999) survey a random sample of Illinois
farmers and show that price and income risk management rank second (following
computer education and training) among ten business categories in which farmers
identify needs for additional consulting services. The desire for greater assistance
with price and income risk management is not limited to large farms, as the proportion
of farmers expressing this preference actually is highest for those operating
medium-sized Illinois farms (500-999 acres).
Farmers view market advisory services as a significant source of
market information and advice in their quest to manage price risks associated
with grain marketing. In a rating of seventeen risk management information sources,
Patrick and Ullerich (1996) report that the rank of market advisors and computerized
information services is surpassed only by farm records. Schroeder, Parcell, Kastens
and Dhuyvetter (1998) find that a sample of Kansas farmers rank market advisory
services as the number one source of information for developing price expectations.
Norvell and Lattz (1999) find that twenty-one percent of Illinois respondents
currently use marketing consultants, and that such consultants tie for first (with
accountants), in a list of seven, as likely to be most important to their business
in the future.
Given the high value that farmers place upon market advisory
services, it is somewhat surprising that only two academic studies investigate
the pricing performance of advisory services.
The dearth of studies seems even more anomalous in light of the large number of
studies on grain marketing strategies. The lack of studies on market advisory services
is most likely due to the difficulty in obtaining data on the stream of recommendations
provided by services.
Gehrt and Good (1993) analyze the performance of five advisory
services for corn and soybeans over the 1985 through 1989 crop years. Assuming a representative farmer follows the
hedging and cash market recommendations for each advisory service, a net price
received for each year is computed and compared to a benchmark price. They generally
find that corn and soybean farmers obtained a higher price by following the marketing
recommendations of advisory services. Martines-Filho (1996) examines the pre-harvest
corn and soybean marketing recommendations of six market advisory services over
1991 through 1994. He computes the harvest time revenue that results from a representative
farmer following the pre-harvest futures and options hedging recommendations and
selling 100 percent of production at harvest. Average advisory service revenue
over the four years is larger than benchmark revenue for both corn and soybeans.
While a useful starting point, the two previous studies have important
limitations. First, the sample of advisory services is quite small, with the
largest sample including only six advisory services. Second, the results may
be biased due to the nature of the sample selection process. The literature on
the performance of mutual funds and investment newsletters highlights the sample
selection biases that plague many performance results (e.g., Brown, Goetzmann,
Ibbotson, and Ross, 1992; Jaffe and Mahoney, 1999; Metrick, 1999). The most relevant
bias for previous studies of market advisory services is survivorship bias, which
results from tracking only advisory services that remain in business at the end
of a sample period.
The previous discussion suggests the academic literature provides
farmers with little basis for evaluating and selecting advisory services. In
1994, the Agricultural Market Advisory Service (AgMAS) Project was initiated,
with the goal of providing unbiased and rigorous evaluation of market advisory
services for farmers. The AgMAS Project has collected marketing recommendations
for about 25 market advisory services each crop year. The AgMAS Project subscribes
to all of the services that are followed, and as a result, "real-time"
recommendations are obtained. This prevents the data from being subject to survivorship
After the stream of recommendations is collected by AgMAS staff
for a given commodity in a particular crop year, the net price that would have
been received by a farmer that precisely follows the set of marketing recommendations
is computed. This net price is the weighted average of the cash sale price plus
or minus gains/losses associated with futures and options transactions. Brokerage
costs are accounted for, as are the costs of storing any portion of the crop beyond
harvest. So far, the AgMAS Project has reported corn and soybean results for
the 1995, 1996, 1997 and 1998 crop years. (Good, Irwin, Jackson, and Price, 1997;
Jackson, Irwin, and Good, 1998; Jackson, Irwin, and Good, 1999; Good, Irwin, Jackson,
Jirik and Martines-Filho, 2000).
The annual AgMAS comparison of net price received for advisory
services provides important information that farmers can use in selecting a service.
However, the comparisons to date are descriptive only and do not rigorously address
the central questions regarding pricing performance. Following the literature
on mutual fund and investment newsletter performance (e.g., Jaffe and Mahoney,
1999), two basic questions need to be answered: 1) Do market advisory services,
on average, outperform an appropriate market benchmark? and 2) Do market advisory
services exhibit persistence in their performance from year-to-year?
The purpose of this report is to address the previous two questions
for corn and soybeans using the net advisory prices reported by the AgMAS Project
for the 1995, 1996, 1997 and 1998 crop years. The results update those found
in Irwin, Jackson and Good (1999) by adding data for the 1998 crop year. At least
21 advisory services are included in the evaluations for each commodity and crop
year. While the sample of advisory services is non-random, it is constructed
to be generally representative of the majority of advisory services offered to
farmers. The availability of only four crop years is a limitation of the analysis,
but the time period considered does include years of rapidly increasing and decreasing
corn and soybean prices.
The tests used to determine average performance of market advisory
services and predictability of performance through time have been widely applied
in the financial literature (e.g., Elton, Gruber, and Rentzler, 1987; Lakonishok,
Shleifer and Vishny, 1992; Irwin, Zulauf, and Ward, 1994; Jaffe and Mahoney, 1999;
Metrick, 1999; Carpenter and Lynch, 1999). Two tests of performance relative
to a benchmark are used: i) the proportion of services exceeding the benchmark
price and ii) the average percentage difference between the net price of services
and the benchmark price. Three tests of predictability are used: i) the correlation
of advisory service pricing performance measures from year-to-year, ii) the predictability
of “winner” and “loser” categories from year-to-year and iii) the differences
between pricing performance measures for “top” and “bottom” performing advisory
Data on Advisory Service Recommendations
The market advisory services included in this evaluation do not
comprise the population of market advisory services available to farmers. The
included services also are not a random sample of the population of market advisory
services. Neither approach is feasible because no public agency or trade group
assembles a list of advisory services that could be considered the "population."
Furthermore, there is not a generally agreed upon definition of an agricultural
market advisory service. To assemble a sample of services for the AgMAS Project,
criteria were developed to define an agricultural market advisory service and
a list of services assembled.
The first criterion used to identify services is that a service
has to provide marketing advice to farmers. Some of the services tracked by the
AgMAS Project do provide speculative trading advice, but that advice must be clearly
differentiated from marketing advice to farmers for the service to be included.
The terms "speculative" trading of futures and options versus the use
of futures and options for "hedging" purposes are used for identification
purposes only. A discussion of what types of futures and options trading activities
constitute hedging, as opposed to speculating, is not considered.
The second criterion is that specific advice must be given for
making cash sales of the commodity, in addition to any futures or options hedging
activities. In fact, some marketing programs evaluated by the AgMAS Project do
not make any futures and options recommendations. However, marketing programs
that make futures and options hedging recommendations, but fail to clearly state
when cash sales should be made, or the amount to be sold, are not considered.
The original sample of market advisory services that met the two
criteria were drawn from the list of "Premium Services" available from
the two major agricultural satellite networks, Data Transmission Network (DTN)
and FarmDayta in the summer of 1994.,
 While the
list of advisory services available from these networks was by no means exhaustive,
it did have the considerable merit of meeting a market test. Presumably, the
services offered by the networks were those most in demand by farm subscribers
to the networks. In addition, the list of available services was cross-checked
with other farm publications to confirm that widely-followed advisory firms were
included in the sample. It seems reasonable to argue that the resulting sample
of services was (and remains) generally representative of the majority of advisory
services available to farmers.
The original sample for 1995 includes 25 market advisory services
for both corn and soybeans. For a variety of reasons, deletions and additions
to the original sample occur over time.
In 1996, the total number of advisory services is 26 for corn and 24 for soybeans,
while in 1997 the total is 23 for corn and 21 for soybeans. In 1998, the total
is again 23 for corn, but the total number of services for soybeans increases
to 22. A directory of the advisory services included in the study can
be found at the AgMAS Project website (http://web.aces.uiuc.edu/farmdoc/agmas/).
As mentioned earlier, sample selection biases may plague advisory
service databases. The first form is survival bias, which occurs if only advisory
services that remain in business at the end of a given period are included
in the sample. Survival bias significantly biases measures of performance upwards
since "survivors" typically have higher performance than "non-survivors"
(Brown, Goetzmann, Ibbotson, and Ross, 1992). This form of bias should not be
present in the AgMAS database of advisory services because all services ever tracked
are included in the sample. The second and more subtle form of bias is hindsight
bias, which occurs if data from prior periods are "back-filled" at the
point in time when an advisory service is added to the database. Statistically,
this has the same effect as survivorship bias because data from surviving advisory
services are back-filled. This form of bias should not be present in the AgMAS
database because recommendations are not back-filled when an advisory service
is added. Instead, recommendations are collected only for the crop year after
a decision has been made to add an advisory service to the database.
The actual daily process of collecting recommendations for the
sample of advisory services begins with the purchase of subscriptions to each
of the services. Staff members of the AgMAS Project read the information provided
by each advisory service on a daily basis. The information is received electronically,
via DTN, websites or e-mail. For the services that provide two daily updates,
typically in the morning and at noon, information is read in the morning and afternoon.
In this way, the actions of a farmer-subscriber are simulated in “real-time.”
The recommendations of each advisory service are recorded separately.
Some advisory services offer two or more distinct marketing programs. This typically
takes the form of one set of advice for marketers who are willing to use futures
and options (although futures and options are not always used), and a separate
set of advice for farmers who only wish to make cash sales. In this situation, both strategies are
recorded and treated as distinct strategies to be evaluated.
Several procedures are used to check the recorded recommendations
for accuracy and completeness. Whenever possible, recorded recommendations are
cross-checked against later status reports provided by the relevant advisory service.
Also, at the completion of the crop year, it is confirmed whether cash sales total
exactly 100%, all futures positions are offset, and all options positions are
offset or expire worthless.
The final set of recommendations attributed to each advisory service
represents the best efforts of the AgMAS Project staff to accurately and fairly
interpret the information made available by each advisory service. In cases where
a recommendation is considered vague or unclear, some judgment is exercised as
to whether or not to include that particular recommendation. This occurs most
often when a service suggests “a farmer might consider” a position, or when minimal
guidance is given as to the quantity to be bought or sold. Given that some recommendations
are subject to interpretation, the possibility is acknowledged that the AgMAS
track record of recommendations for a given service may differ from that stated
by the advisory service, or from that recorded by another subscriber.
Calculation of Net Advisory Service Prices
At the end of a crop year, all of the (filled) recommendations
are aligned in chronological order. The advice for a given crop year is considered
to be complete for each advisory service when cumulative cash sales of the commodity
reach 100%, all open futures positions covering the crop are offset, all open
option positions covering the crop are either offset or expired, and the advisory
service discontinues giving advice for that crop year. The returns to each recommendation
are then calculated in order to arrive at a weighted-average net price that would
be received by a farmer who precisely follows the marketing advice (as recorded
by the AgMAS Project).
In order to simulate a consistent and comparable set of results
across the different advisory services, certain explicit assumptions are made.
These assumptions are intended to accurately depict marketing conditions for a
representative, central Illinois farm. An overview of the simulation assumptions
is presented below. Complete details of the simulation assumptions can be found
in Good, Irwin, Jackson, Jirik and Martines-Filho (2000).
The simulation is designed to reflect conditions facing a representative
central Illinois corn and soybean farmer. Whenever possible, data are collected
for the Central Crop Reporting District in Illinois as defined by the National
Agricultural Statistics Service (NASS) of the US Department of Agriculture (USDA).
Eleven counties (DeWitt, Logan, McLean, Marshall, Macon, Mason, Menard, Peoria,
Stark, Tazewell, and Woodford) make up this District.
A two-year marketing window, spanning September of the year before
harvest through August of the year after harvest, is used in the analysis. For
example, the 1997 marketing window is September 1, 1996 through August 31, 1998.
The beginning date is selected because services in the sample generally begin
to make recommendations around this date. The ending date is selected to be consistent
with the ending date for corn and soybean crop years as defined by the US Department
of Agriculture (USDA). There are a few exceptions to the marketing window definition.
Some advisory services have relatively small amounts (15% or less) of cash corn
or soybeans unsold as of the end of a window. Several advisory services also
begin pre-harvest hedges prior to the September 1 start of the window. In these
cases, the actual sales recommendations on the indicated dates are recorded. Finally,
note that throughout the remainder of this report, the term "crop year"
is used to represent the two-year marketing window.
The price assigned to each cash sale recommendation is the central
Illinois closing, or overnight, bid. The data are collected and reported by the
Illinois Department of Ag Market News. The central Illinois price is the mid-point
of the range of bids by elevators in the North Central and South Central Price
Reporting Districts, as defined by the Illinois Department of Ag Market News.
Prices in this 25-county area best reflect prices for the assumed geographic location
of the representative central Illinois farmer (Central Illinois Crop Reporting
District). The central Illinois market also is used for cash-forward contract
transactions. Futures prices and options premia are Chicago Board of Trade (CBOT)
Since most of the advisory service recommendations are given in
terms of the proportion of total production (e.g.,, “sell 5% of 1997 crop today”),
some assumption must be made about the amount of production to be marketed. For
the purposes of this study, if the per-acre yield is assumed to be 100 bushels,
then a recommendation to sell 5% of the corn crop translates into selling 5 bushels.
When all of the advice for the crop year has been carried out, the final per-bushel
selling price is the average price for each transaction weighted by the amount
marketed in each transaction.
When making hedging or forward contracting decisions prior to harvest,
the actual yield is unknown. Hence, an assumption regarding the amount of expected
production per acre is necessary to accurately reflect the returns to marketing
advice. Prior to harvest, the best estimate of the current year’s expected yield
is assumed to be a function of yield in previous years. In this study, the assumed
yield prior to harvest is based on a linear regression trend yield, while the
actual reported yield is used from the harvest period forward.
Brokerage costs are incurred when farmers open or lift positions
in futures and options markets. For the purposes of this study, it is assumed
that brokerage costs are $50 per contract for a round-turn for futures transactions,
and $30 per contract to enter or exit an options position. Further, it is assumed
that CBOT corn and soybean futures are used, and the contract size for each commodity
is 5,000 bushels. Therefore, per-bushel brokerage costs are 1 cent per bushel
for a round-turn futures transaction and 0.6 cents per bushel for each options
An important element in assessing returns to an advisory service
is the economic cost associated with storing grain instead of selling grain immediately
at harvest. The cost of storing grain after harvest (carrying costs) consists
of two components: physical storage charges and the opportunity cost incurred
by foregoing sales when the crop is harvested. Physical storage charges can apply
to off-farm (commercial) storage, on-farm storage, or some combination of the
two. Opportunity cost is the same regardless of the type of physical storage.
For the purposes of this study, it is assumed that all storage
occurs off-farm at commercial sites. Carrying costs are assumed to begin after
the last day of harvest. Physical storage charges are assumed to be a flat 13
cents per bushel from the end of harvest through December 31. After January 1,
physical storage charges are assumed to be 2 cents per month (per bushel), with
this charge pro-rated to the day when the cash sale is made. The storage costs
represent the typical storage charges quoted in a non-random telephone survey
of central Illinois elevators.
The interest charge for storing grain is the interest rate compounded
daily from the end of harvest to the date of sale. The interest rate used is
the average rate for all commercial agricultural loans for the fourth quarter
of the harvest year and the first three quarters of the next calendar year as
reported in the Agricultural Finance Databook published by the Board of
Governors of the Federal Reserve Board. This interest rate has been around 9%
per year for the four years of this study.
In addition to the storage and interest costs, another charge is
assigned to corn (but not soybeans). This charge, referred to as a “shrink charge”,
is commonly deducted by commercial elevators on “dry” corn that is delivered to
the elevator to be stored, and reflects a charge for drying and volume reduction
(shrinkage) which occurs in drying the corn from (typically) 15% to 14% moisture.
The charge for drying is a flat 2 cents per bushel, while the charge for volume
reduction is 1.3% per bushel. The charge for this volume reduction is calculated
as 1.3% times the average harvest-time cash price for each crop year. For example,
for the 1998 corn crop the harvest-time cash price was $1.91 per bushel, so the
charge for volume reduction was 2.5 cents per bushel ($1.91*0.013).
LDP and Marketing Assistance Loan Payments
The price of both corn and soybeans is below the loan rate during
significant periods of time in the 1998-1999 crop year, so that use of the marketing
loan program is an important part of marketing strategies during this period.
Most of the advisory services tracked by the AgMAS Project for the 1998 crop make
specific recommendations regarding the timing and method of implementing the loan
program for the entire corn and soybean crops. These recommendations are implemented
as given wherever feasible. Several decision rules have to be developed even
in this case, in particular, for pre-harvest forward contracts. For a few services,
loan recommendations are incomplete or not made at all. For these cases, it is
necessary to develop a more complete set of decision rules for implementing the
loan program in the marketing of corn and soybeans. All loan-related decision
rules are based on the assumption of a “prudent” or “rational” farmer, within
the context of the intent of the loan program. More specifically, it is assumed
that a farmer will take advantage of the price protection offered by the loan
program, even in the absence of specific advice from an advisory service. Further
information on the decision rules used to implement marketing loan recommendations
can be found in Good, Irwin, Jackson, Jirik and Martines-Filho (2000).
Simply comparing the net price received across advisory services
will not answer the question of whether advisory services as a group enhance the
income of farm subscribers. Instead, a comparison to a benchmark price (or prices)
is needed to evaluate the performance of advisory services relative to pricing
opportunities offered by the market. In the stock market, mutual funds are evaluated
with respect to market benchmark performance criteria (e.g., Bodie, Kane, and
Marcus, 1989). These benchmarks typically are indexes of stock market returns
over the period of evaluation, such as the Dow Jones Industrial Average and Standard
and Poor’s 500.
The selection of appropriate benchmarks for advisory service performance
evaluations is treated thoroughly in a report by Good, Irwin and Jackson (1998).
They argue that, conceptually, a useful benchmark should: 1) be simple
to understand and to calculate; 2) represent the returns to a marketing strategy
that could be implemented by farmers; 3) be directly comparable
to the net advisory price received from following the recommendations of a market
advisory service; 4) not be a function of the actual recommendations of the advisory
services or of the actual marketing behavior of farmers, but rather should be
external to their marketing activities; and 5) be stable, so that
it represents the range of prices made available by the market throughout the
crop year instead of representing the price during a small segment of the crop
year. The market benchmark price that Good, Irwin and Jackson argue is the most
consistent with the above criteria is the average cash price for corn and soybeans
over the entire marketing horizon. The marketing window used in the AgMAS project
for a given crop spans two calendar years, beginning on the first business day
of September in the year prior to harvest, and extends through the last business
day of August in the year after harvest. As its name suggests, the benchmark
is calculated as the average of the daily central Illinois cash grain bids available
for the two-year marketing window. Pre-harvest cash prices represent cash-forward
bids for harvest delivery in central Illinois, while daily spot prices for central
Illinois are used for the post-harvest period.
Three adjustments are made to the daily cash prices to make the
average cash price benchmark consistent with the calculated net advisory prices
for each marketing program. First, instead of taking the simple average of the
daily prices, a weighted average price is calculated to account for changing yield
expectations. The daily weighting factors for pre-harvest prices are based on
the calculated trend yield, while the weighting of the post-harvest prices is
based on the actual reported yield for central Illinois. The second adjustment
to the daily cash prices is to adjust the post-harvest cash prices to a harvest
equivalent by subtracting carrying charges. The daily carrying charges are calculated
in the same manner as those for the net advisory price. Complete details of the
construction of this benchmark price can be found in Good, Irwin and Jackson (1998).
A third adjustment to the average cash price benchmark is made
only for 1998. This adjustment is based on the logic that a “prudent” or “rational”
farmer will take advantage of the price protection offered by the marketing loan
program when following the benchmark average price strategy. Based on this argument,
the average cash price benchmark is adjusted by the addition of marketing loan
benefits. Bushels marketed in the pre-harvest period according to the benchmark
strategy (approximately 53 percent) are treated as forward contracts with the
benefits assigned at harvest. Bushels marketed each day in the post-harvest period
(approximately 47 percent) are awarded marketing loan benefits in existence for
that particular day.
In order to test the sensitivity of performance results to the
choice of market benchmark, two alternative versions of the previous average
cash price benchmark also are considered in the analysis. The first alternative
benchmark averages prices for the 20-month period starting in January of the year
of harvest and ending in August of the year after harvest. The only difference
between this alternative and the 24-month benchmark is the exclusion of the pre-harvest
period previous to January. Hence, this alternative benchmark places more weight
on post-harvest prices than pre-harvest prices. The second alternative benchmark
averages prices only for the 16-month period starting in May of the year of harvest
and ending in August of the year after harvest.
Net Price Received Results for 1995 - 1998
Net price received for the sample of market advisory services for
the 1995, 1996, 1997 and 1998 crop years is reported in Tables
1 and 2. Note that some of the market advisory services
included in the table are not evaluated for all four years. The four-year averages
and standard deviations are calculated only for the 19 services that are evaluated
for all four years.
As shown in Table 1, the average
net advisory price for corn ranges from $2.17 per bushel in 1998 to $3.04 per
bushel in 1995. The four-year average for the 19 services is $2.53 per bushel.
The range of four-year average net advisory prices is large, with a low of $2.36
and a high of $2.83. Not surprisingly, the range within the individual years
is even more substantial. The most dramatic example is 1995, where the minimum
is $2.29 per bushel and the maximum is $3.90 per bushel. Even in years with less
market price volatility, such as 1998, the range in performance is just under
$0.60 per bushel.
The three alternative market benchmark prices for corn are shown
at the bottom of Table 1. Four-year averages of
the market benchmarks differ by two cents per bushel or less. However, this masks
large differences within some of the years, particularly 1995. These data suggest
advisory service performance results for corn may be sensitive to the selected
As reported in Table 2, the average
net advisory price for corn ranges from $5.82 per bushel in 1998 to $7.27 per
bushel in 1996. The four-year average for the 19 services is $6.50 per bushel.
Again, the range of four-year average net advisory prices is large, with a low
of $6.32 and a high of $6.88. As with corn, the range within the individual years
is even more substantial. The most dramatic example is 1995, where the range
in advisory prices exceeds two dollars per bushel.
Since many subscribers to market advisory services produce both
corn and soybeans, it is of interest to examine a combined measure of corn and
soybean pricing performance for each market advisory service. One way to aggregate
the results is to calculate the per-acre revenues implied by the pricing performance
results. The per-acre revenue for each commodity
is found by multiplying the net advisory price for each market advisory service
by the actual central Illinois corn or soybean yield for each year. A simple
average of the two per acre revenues is then taken to reflect a farm that uses
a 50/50 rotation of corn and soybeans.
Table 3 contains the combined corn
and soybeans revenue results. As with Tables 1 and
2, a four-year average is calculated only for services that were included
in the study for all four years. In addition, market advisory services that provide
recommendations for corn but not soybeans are excluded. The four-year average
revenue for all 19 market advisory services is $325 per acre. The four-year average
for individual services ranges from a low of $312 per acre to a high of $349 per
Statistical Tests of Market Advisory Service Pricing Performance
Two statistical tests are used to test the null hypothesis that
average market advisory service pricing performance does not differ from that
of the market benchmark. The first test is based on the proportion of services
exceeding the benchmark price. This test is considered because it is not influenced
by extremely high or low advisory prices. The second test is based on the average
percentage difference between the net price of services and the benchmark price.
This test is useful because it takes into account the average magnitude of differences
from the benchmark.
Independence of Observations
Before considering the statistical tests and results, an important
issue needs to be explored that may have a substantial impact on the results.
The issue is whether the sample observations on net advisory price are independent,
both within and across years. The most likely form of dependence is positive
correlation, which, if ignored, would cause sample standard deviation estimates
across advisory services to be understated. This in turn would cause the statistical
significance of hypothesis test results to be overstated.
There are several potential ways that independence could be violated
in the sample of market advisory service prices. One potential violation is positive
correlation of corn pricing performance for a market advisory service in a given
year with its soybean pricing performance in the same year. In other words, do
services that do well in corn also tend to do well in soybeans in the same year?
If so, statistical tests that pool pricing performance of services for corn and
soybeans may overstate the significance of positive or negative performance because
the standard deviation across the corn and soybean observations would be understated.
Correlation across corn and soybeans in a given year is computed
three ways. First, the correlation of rank across corn and soybeans for a given
year is computed. To do this, the rank of each advisory service with respect to
the other services is calculated separately for corn and soybeans. The services
are ranked in descending order. For example, the service with the highest net
advisory price is ranked number one, and the service with the lowest net advisory
price is assigned a number equal to the total number of observations for that
commodity in the given year. The final step is to compute the correlation of
the corn and soybean ranks. Second, the simple correlation between the net advisory
corn and soybean price levels is computed for a given year. Third, the correlation
of advisory service performance with respect to the 24-month market benchmark
price is calculated. The “return” to market advice is calculated
as the percentage difference between the net advisory price and the 24-month market
benchmark price for the commodity. A graphical view of the rank correlations
is presented in Figure 1.
The correlation results for market advisory corn and soybean pricing
performance within the same crop year are presented in Table
4. The results are similar across the different measures of correlation.
Significant positive correlation between corn and soybean pricing results is found
in 1995 and 1997, but not for 1996 or 1998. This may be due to the fact that
the price patterns for corn and soybeans were somewhat different for the 1996
and 1998 crop years, while corn and soybean prices moved (generally) in the same
direction during the 1995 and 1997 crop years. While market advisory services
do not make exactly the same recommendations for corn and soybeans in any given
year, there often is a significantly positive correlation in their corn and soybean
pricing performance. This suggests it is inappropriate to pool separate corn
and soybean pricing results when conducting statistical tests.
A second potential source of dependence is correlation of net advisory
prices through time for a given service and commodity. This form of correlation
may exist due to persistence in the performance of advisory services through time
(winners continue to win, losers continue to lose). It may also exist due to
the overlapping nature of the crop years; each crop year is two calendar years
long, and each set of contiguous crop years overlaps by one year. If this correlation
through time exists, it would be inappropriate to pool samples of net advisory
prices across crop years for the same reason as discussed above. As will be shown
in the following section, this form of correlation generally is minimal, and therefore,
it is reasonable to pool net advisory prices across crop years.
A third potential source of dependence perhaps is less obvious.
It is possible that net advisory prices for a given commodity and crop year are
correlated because of the existence of similar programs offered by the same market
advisory service. For example, AgriVisor offers four marketing programs, which
may not differ substantially in outcomes due to similar methods of analysis and
similar underlying strategies. The potential impact of this form of correlation
is examined by creating one net advisory price for each of the market advisory
firms that offer multiple programs. A single price is computed by averaging net
advisory prices across programs for a given year and commodity. Pricing performance
results are qualitatively similar to those using the full set of disaggregated
advisory prices, suggesting that net prices of advisory programs for the same
firm are uncorrelated or no more correlated than net prices from different firms.
Hence, use of net advisory prices by program in tests of market performance does
not appear to be a substantive problem.
A formal test of the null hypothesis that the proportion of advisory
services "beating" the market benchmark is insignificant requires the
specification of an appropriate test statistic. First, define the sample estimate
of the proportion for a given year and commodity as,
where k is the number of advisory services that have net prices exceeding
the market benchmark price and n is the total number of advisory services
in the sample. Anderson, Sweeney and Williams (1996) show that the sample estimator
of the proportion, ,
is distributed binomially with an expected value of p and a standard error
of, where p
is the true value of the proportion in the population. They also note that the
sampling distribution of is approximately
normal so long as and . Since both conditions
are met for all of the samples considered here, the normality approximation is
invoked. The form of the test statistic based on the above assumptions is,
where p0 is the assumed value of p under
the null hypothesis. The remaining issue is the expected proportion (p0)
under the null hypothesis. The efficient market hypothesis (Fama, 1970) implies
that the expected probability of “beating the market” is the same as the result
of flipping a coin and showing heads, or 0.5. Setting , the test statistic
A formal test of the null hypothesis that the average percentage
difference between the net price of services and the benchmark price is zero also
requires the specification of an appropriate test statistic. First, define the
percentage difference for the ith advisory service for a given
crop year and commodity as,
where NAPi is the net advisory price for the ith
advisory service and BP is the market benchmark price for the same commodity
and crop year. The sampling distribution of is
well-known and does not need to be described in detail here. The test statistic
for a null hypothesis of zero average percentage difference is,
where is the estimated
standard deviation of the differences across the n advisory services in
the sample. The t-statistic follows a t-distribution with n-1
degrees of freedom.
As noted earlier, ri can be thought of as the
“return” to following the recommendations of a particular market advisory service.
This raises the question of whether the calculated “returns” are risk-adjusted.
One method of adjusting returns for risk that has been used in a number of studies
stock investment strategies (e.g., Friend, Blume and Crocket, 1970; Ritter, 1991)
is to match the average risk of the investments to the risk of the benchmark.
Hence, if the average risk of advisory services is equal to risk of the market
benchmark, then market advisory returns can be considered risk-adjusted returns.
Evidence on the appropriateness of this “risk-matching” assumption for advisory
services can be found in Tables 1, 2 and 3, where
the standard deviations for the advisory services and market benchmarks can be
found in the last column of each table. As shown in Table 1, the average standard
deviation for advisory services in corn is $0.42 per bushel, near the middle of
the range of standard deviations for the three benchmarks. Examining Table
2, the average standard deviation for advisory services in soybeans is $0.67
per bushel, again near the middle of the range of standard deviations for the
three benchmarks. Turning to Table 3, the average
standard deviation for advisory service revenue is $36 per acre, near the top
of the range of standard deviations for the three benchmarks. Overall, the comparisons
suggest the risk of the market benchmarks roughly matches the average risk of
the advisory services, and hence, computed “returns” may be considered risk-adjusted.
However, given the short time-period considered in these comparisons, a risk-adjusted
interpretation of advisory returns should be treated with a good bit of caution.
It is important to emphasize that the tests discussed in this section
address the pricing performance of market advisory services as a group.
In other words, average pricing performance across all services is considered.
This is a different issue than the pricing performance of a particular advisory
service. It is possible that advisory services as a group fail to beat the market,
yet at the same time there exist a small number of services that are exceptions
to this outcome. In the stock market, this argument is often made with respect
to the performance of the Fidelity Magellan Fund. Testing whether an “exceptional”
advisory service beats the market requires more data than is available for this
study and different statistical methods (Marcus, 1990).
Performance Test Results
Table 5 reports results of the proportional
test of corn pricing performance for each year and all four years pooled. Statistical
significance is based on a null hypothesis proportion of 0.5, the same as the
proportion of heads observed in the flips of a fair coin. Individual year results
are quite sensitive to the benchmark considered. For example, the proportion
of services above the 24-month benchmark price in 1995 is 0.72 and statistically
different from 0.5, while the proportion of services above the 16-month benchmark
is only 0.12 and also significantly different from 0.5. A similar contrast in
test results is found in 1998. The overall proportions for the four years are
not as variable across the benchmarks, ranging from 0.46 to 0.59. Pooled four-year
proportions based on the 24-month and 20-month proportions are insignificantly
different from 0.5, while the 16-month benchmark proportion is significant at
the ten-percent level.
Table 6 shows the results of the
proportional test of soybean pricing performance for each year and all four years
pooled. Like corn, individual year results are sensitive to the benchmark considered.
The most dramatic contrast again can be found in 1995, where the proportion of
services above the 24-month benchmark price is 0.84 and significantly different
from 0.5, while the proportion of services above the 16-month benchmark is 0.56
and not significantly different from 0.5. Despite the variation across benchmarks
in the individual years, the pooled proportions for the four years are similar
across benchmarks, ranging only from 0.65 to 0.73. All of the four-year proportions
are significantly greater than 0.5 at the one-percent level.
Table 7 reports proportional test
results for combined corn and soybean revenue. Given the evidence of positive
correlation between the pricing performance of advisory services for corn and
soybeans in the same year, it is inappropriate to simply pool the separate net
price observations for corn and soybeans to test combined performance. Instead,
corn and soybean net prices are aggregated to form a single observation on per-acre
revenue for each advisor and year, and then proportions are computed. The per-acre
combined revenues are those first presented in Table
3, with the per-acre revenue for each commodity found by multiplying the net
advisory price for each market advisory service by the actual central Illinois
corn or soybean yield for each year. A simple average of the two per acre revenues
is then taken to reflect a farm that uses a 50/50 rotation of corn and soybeans.
As would be expected, the proportions for revenue per acre tend to fall between
the proportions for corn and soybean net advisory prices and show a similar pattern
of variation across the alternative benchmarks in a given year. Combined corn
and soybean performance for the entire four-year period varies little across the
benchmarks, with the proportion of services above the benchmark ranging from 0.55
to 0.62. Four-year proportions are significantly different from 0.5 for the 20-month
and 16-month benchmark, but not the 24-month benchmark.
Results for the average return test of pricing performance are
reported in Tables 8, 9 and 10. Individual year
and four-year average test results for corn, shown in Table
8, are qualitatively similar to the proportional test results. Point estimates
of the four-year average return range from –0.26 to 1.54 percent. However, none
of the four-year average returns for the three benchmarks are significantly different
from zero. Individual year and four-year average results for soybeans, reported
in Table 9, also are qualitatively similar to the
proportional test results. Point estimates of the four-year average soybean return
range from 2.17 to 3.00 percent, substantially higher than for corn. All three
of the four-year average soybean returns are significantly different from zero.
Results of the average return test for combined corn and soybean revenue, found
in Table 10, also differ little from the proportional
test results. Point estimates of the four-year average revenue return range from
0.90 to 2.08 percent, which, as expected, is between the ranges for corn and soybeans.
Four-year average revenue returns are significantly different from zero for the
20-month and 16-month benchmarks, but not the 24-month benchmark.
In statistical terms, the pricing performance test results presented
in this section are fairly clear. Minimal evidence is found that market advisory
services consistently and significantly “beat the market” in corn. There is substantial
evidence that market advisory services consistently and significantly “beat the
market” in soybeans. When corn and soybean net advisory prices are combined into
revenue per acre, evidence also is found that market advisory services significantly
outperform the market. Overall, the statistical results suggest that market advisory
services have some ability to outperform broad market benchmarks.
Given the statistical results summarized above, a relevant question
to ask is whether the pricing performance of advisory services also is economically
significant. While "economic significance" is a vague concept, it is
important nonetheless. Perhaps the best perspective on this question is gained
by re-examining returns for corn and soybean revenue per acre. Given the sensitivity
of measured returns to the benchmark considered, the best point estimate of revenue
returns probably is the simple average across the three benchmarks. This “grand
average” revenue return across all four crop years and three benchmarks is 1.4
percent, which translates into about $4 per acre above benchmark revenue. While this level of return is probably best
characterized as “small,” it also appears to be non-trivial, particularly in comparison
to the cost of the services. Good, Irwin, Jackson, Jirik and Martines-Filho (2000)
report that the average cost of the services is $295 for the 1998 crop year. For
a 1,000 acre corn and soybean farm, this translates into an average cost of about
30 cents per acre. Put in different terms, this is roughly equal to the average
50/50 revenue from one acre of corn and soybeans over 1995-1998. There are two
important reasons to be cautious about concluding that advisory returns generate
even a "small" level of economic significance: i) the results are based
on a limited sample of years, and ii) returns tend to be concentrated in one market,
The results of the analysis have implications for the ongoing debate
about market efficiency and marketing strategies in agriculture. One view is
that grain markets (cash, futures and options) are not efficient and, therefore,
provide opportunities for farmers to systematically earn additional profits through
marketing (e.g Wisner, Blue and Baldwin, 1998). The other view is that grain
markets are at least efficient with respect to the type of strategies available
to farmers (e.g.,, Zulauf and Irwin, 1998). Since the return of advisory services
over 1995-1998 significantly exceeds transactions costs in several cases, including
the cost of the services, the results potentially imply a rejection of market
efficiency in the sense of Grossman and Stiglitz (1980). A firm conclusion cannot be reached due to the
uncertainties pointed out with respect to economic significance. In addition,
there is uncertainty about the appropriate adjustment for risk or a complete accounting
for the costs of implementing advisory service recommendations. It may be that
important costs are ignored, such as search costs, monitoring costs and related
management costs. Nevertheless, the performance results suggest market advisory
services, at least to a modest extent, have some access to information not available
to other market participants and/or superior analytical skills.
Finally, it is interesting to compare the pricing performance results
for market advisory services to that of other investment professionals. Malkiel
(1999) reports that only 33 percent of active mutual fund managers beat the returns
to the S&P 500 stock index over 1974-1998. Clements (1999) notes that only
9 percent of active managers beat the S&P 500 in the decade ending in 1998.
By comparison, the performance of agricultural market advisory services is quite
strong, with a little more than half of the services beating the market in corn
and about two-thirds beating the market in soybeans. This divergence may simply
reflect a unique time period in corn and soybean markets, relatively less efficient
commodity markets, the skillfulness of advisory services, or an inappropriate
adjustment for advisory service risk.
Predictability of Advisory Service Performance
Even if, as a group, advisory services generate positive returns,
there is a wide range in performance for any given year. For example, soybean
net advisory prices for 1995 vary from $5.71 per bushel to $7.94 per bushel (see
Table 2). While this example probably is the most
dramatic, the variation across advisors in other cases is substantial. This raises
the important question of the predictability of advisory service performance from
year-to-year. In other words, is past performance indicative of future results?
Three tests of predictability are used: i) the correlation of advisory service
prices, ranks and percentage differences from the benchmark across overlapping
and non-overlapping pairs of adjacent crop years, ii) predictability of “winner”
and “loser” categories across overlapping and non-overlapping pairs of adjacent
crop years and iii) differences between prices, ranks and percentage differences
from the benchmark for “top” and “bottom” performing advisory services across
overlapping and non-overlapping pairs of adjacent crop years. The testing procedures
have been widely applied in studies of financial investment performance (e.g.,
Elton, Gruber, and Rentzler, 1987; Irwin, Zulauf and Ward, 1994; Lakonishok, Shleifer
and Vishny, 1992; Malkiel, 1995).
The distinction between overlapping and non-overlapping market
years is due to the fact that each marketing window is two calendar years in length,
and hence, two adjacent marketing windows overlap by one calendar year. This
overlap may influence predictability results, in that persistence between overlapping
years may be due to “true” persistence in performance or the overlapping nature
of the periods of comparison. Persistence for non-overlapping years presumably
reflects only “true” persistence in pricing performance.
The first test of predictability is based on the correlation between
performance measures of individual market advisory services across overlapping
and non-overlapping pairs of crop years. Brorsen and Townsend (1998) show that
this type of test is reasonably powerful in detecting performance persistence
in managed futures funds if it exists. For a given commodity, the first step in
this testing procedure is to form the sample of all advisory services that are
active in both adjacent years (overlapping or non-overlapping). The second step
is to rank each advisory service in the first year of the pair (e.g., t
= 1997) based on net price received. Then the services are sorted in descending
order. For example, the service with the highest net advisory price is ranked
number one, and the service with the lowest net advisory price is assigned a rank
equal to the total number of services for that commodity in the given year. The
third step is to sort and rank the sample of services in the second year of the
pair (e.g., t + 1 = 1998). The fourth step is to estimate the correlation
coefficient between performance measures for the two adjacent crop years t
and t+1 as follows,
where is the sample
average of net advisory prices for year t, is the sample average
of net advisory ranks for years t and is the sample averages
of net advisory percentage differences from the market benchmark for years t.
Finally, using Bartlett’s approximation for the standard error () of the correlation
coefficient, the following test statistic is used to test the null hypothesis
of no predictability across the adjacent pair of years,
where j = NAPt,t+1, RKt,t+1
and rt,t+1. The sampling distribution of the test statistic
Zj approximately follows a standard, normal distribution.
The second test of predictability is based on placing advisory
services into “winner” and “loser” categories across overlapping and non-overlapping
pairs of adjacent crop years. The resulting 2 x 2 contingency table of winner
and loser counts allows the use of non-parametric statistical testing procedures.
Carpenter and Lynch (1999) indicate this test is well-specified and powerful in
detecting persistence in mutual fund returns. For a given commodity, the first
step in this testing procedure is to form the sample of all advisory services
that are active in both adjacent years (overlapping or non-overlapping). The
second step is to rank each advisory service in the first year of the pair (e.g.,
t = 1997) based on net price received. Then the services are sorted in
descending order. The third step is to form two groups of services in the first
year of the pair: winners are those services in the top half of the rankings and
losers are services in the bottom half. The third step is to rank each advisory
service in the second year of the pair (e.g.,t +1 = 1998) based on net
price received and once again form winner” and loser groups of services. The fourth
step is to compute the following counts for the advisory services in the pair
of years: WW = winner t-winner t+1, WL = winner t-loser
t+1, LW = loser t-winner t+1, LL = loser t-loser
t+1. The fifth step is to compute the following odds ratio,
The sampling distribution of the test statistic Zt,t+1
asymptotically follows a standard, normal distribution
The third test of predictability is based on the differences between
prices, ranks and percentage differences from the benchmark for “top” and “bottom”
performing advisory services across overlapping and non-overlapping pairs of adjacent
crop years. This test is based on the observation that predictability in advisory
service performance may not exist across all advisory services, but it is possible
that sub-groups of advisory services may exhibit predictability. In particular,
predictability may only be found at the extremes of performance. That is, only
top-performing services in one year may tend to perform well in the next year,
or only poor-performing services may perform poorly in the next year. Carpenter
and Lynch (1999) indicate this type of test also is well-specified and powerful
in detecting persistence in mutual fund returns. It is also robust to the presence
of survivorship bias in returns.
For a given commodity, the first step in this testing procedure
is to sort services by pricing performance in the first year of the pair and group
services by quantiles (thirds and fourths). The second step is to compute the
average pricing performance for the quantiles formed in the first year of the
pair in the second year of the pair. For example, the pricing performance of the
top fourth quantile formed in 1995 is computed for 1996. The third step is to
compute the following differences in pricing performance for the top- and bottom-performing
where and are the average net
advisory prices for the top and bottom quantiles (thirds or fourths) formed in
year t and tracked in year t+1, respectively, and are the average net
advisory ranks for the top and bottom quantiles (thirds or fourths) formed in
year t and tracked in year t+1, respectively, and and are the average net
advisory returns for the top and bottom quantiles (thirds or fourths) formed in
year t and tracked in year t+1, respectively. The fourth step is
to estimate the mean and standard deviation of the above differences across all
possible pairs of years. Finally, the following test statistic can be used to
test the null hypothesis of no predictability,
where is the mean estimate
across the possible pairs of years, is the standard deviation
estimate across the possible pairs of years and . In the case of
overlapping crop years, since there are three
pairs of years (1995/1996, 1996/1997, 1997/1998). In the case of non-overlapping
crop years, since there are two
pairs of years (1995/1997, 1996/1998).
Predictability Test Results
Results of the test of predictability based on the correlation
between performance measures of individual market advisory services across overlapping
pairs of crop years are presented in Table 11. Figure 2
presents a graphical illustration of the rank correlation across crop years for
corn. Figure 3 shows the same relationships for
soybeans, and Figure 4 for revenue. Turning to
corn, correlation coefficients for 1995 vs. 1996 and 1997 vs. 1998 generally are
in the range of 0.50 to 0.65 for all three performance measures. Five of the
six correlation coefficients for these pairs of years are significantly different
from zero. In contrast, each of the three correlations estimated for 1996 vs.
1997 is moderately negative and insignificant. The net result is a small average
correlation coefficient across the three pairs of years, about 0.25 to 0.30.
These comparisons suggest some predictability of pricing performance in corn through
All of the estimated correlation coefficients for soybeans are
positive, but only one is significantly different from zero (net price correlation,
1997 vs. 1998). When averaged across the three pairs of crop years, the correlations
are only about 0.20 to 0.25. This evidence suggests, at best, limited predictability
of pricing performance for soybeans.
Revenue correlation coefficients for 1995 vs. 1996 and 1997 vs.1998
range widely, from a low of 0.15 to a high of 0.58. Four of the six correlation
coefficients for these pairs of years are significantly different from zero. Each
of the three correlations estimated for 1996 vs. 1997 is moderately negative and
insignificant. Once again, the net result is a small average correlation coefficient
across the three pairs of years. However, these comparisons do suggest some predictability
of revenue performance over time.
Results of the test of predictability based on the correlation
between performance measures of individual market advisory services across non-overlapping
pairs of crop years are presented in Table 12. The
results for corn differ sharply from those for overlapping years. Five of the
six estimated correlations for corn are negative. Most striking is the large
absolute magnitude and significance of the correlations for 1995 vs. 1997. These
correlations are statistically significant and range between –0.52 and –0.68.
The average correlation for the two pairs of non-overlapping years ranges from
–0.19 to –0.38. Soybean correlation coefficients for non-overlapping years also
tend to be negative, but none are significantly different from zero. Revenue
correlation coefficients for non-overlapping years show a similar pattern to those
for corn. Overall, the non-overlapping results tend to be in the opposite direction
of the correlations observed for overlapping years, and suggests correlation of
performance through time is quite fragile, in the sense of being sensitive to
the nature of the comparisons.
Results of the “winner” and “loser” predictability test for overlapping
crop years are shown in Table 13. It is worth noting
that this test of predictability is not as sensitive to outliers in pricing performance,
either positive or negative, as the previous correlation tests. Hence, it is
possible for the results to differ across the two sets of tests. The winner and
loser counts for corn, soybeans and revenue indicate little difference in the
odds of a winner or loser in one period being a winner or loser in the subsequent
period. As an example, consider the results for corn in 1996 and 1997. Of the
eleven winners in 1996, six are winners (top half) in 1997 and five are losers
(bottom half). The corresponding odds ratio is 1.44, which indicates that the
odds (6/5) of a winning service in 1996 being a winning service in 1997 are only
1.44 times the odds (5/6) of a losing service in 1996 being a winning service
in 1997. The odds ratio for all the cases in Table 13 ranges from 0.44 to 3.06.
None of the odds ratios are significantly different from one. This evidence provides
no indication of predictability of advisory service pricing performance.
Results of the winner and loser predictability test for non-overlapping
crop years are shown in Table 14. The winner and
loser counts for corn are slightly more favorable, given that the odds ratio for
1996 and 1998 is significantly different from one. However, the pooled results
for corn are insignificant. Soybean results are quite similar to the overlapping
case, with no significant odds ratios. Likewise, revenue results for overlapping
comparisons indicate no significant odds ratios.
Results for the test of predictability based on the difference
between pricing performance for “top” and “bottom” performing advisory services
across overlapping pairs of adjacent crop years are shown in Table
15. Nominally there is some evidence that top services outperform bottom
services. In all cases, the average net advisory price for services in the top
quantile (thirds or fourths) exceeds the average net advisory price for services
in the bottom quantile. This is most evident when comparing average prices for
the top fourth and bottom fourth, with net prices for the top group exceeding
those of the bottom group by $0.17 and $0.18 per bushel for corn and soybeans,
respectively. Revenue for the top fourth exceeds the revenue of the bottom fourth
by an average of $8 per acre. However, t-statistics indicate that none
of the positive price premiums for top performers is significantly different from
zero, although some of the lack of significance certainly can be attributed to
the fact that only three observations are used to compute the test statistics.
Results for the test of predictability based on the difference
between pricing performance for “top” and “bottom” performing advisory services
across non-overlapping pairs of adjacent crop years are shown in Table
16. These results tend to be just the opposite of those observed for overlapping
years. In all cases, the average net advisory price for services in the top quantile
(thirds or fourths) is below the average net advisory price for services in the
bottom quantile. For example, net prices for the top fourth of services in corn
and soybeans, on average, are $0.12 and $0.13 per bushel, respectively, less than
the comparable average prices for bottom fourth services. Revenue for the top
fourth is below the revenue of the bottom fourth by an average of $12 per acre.
Once again, t-statistics indicate that none of the negative premiums for
top performers is significantly different from zero. It is worth noting that all
of the top third and top fourth quantiles generate average returns that are substantially
negative, so these “top” services not only trail bottom performers, but also the
The practical implications of the contrary top- and bottom-performer
results (at least nominally) for overlapping versus non-overlapping years are
striking. Consider the case of a farmer who uses 1995 performance results to
select a top-fourth advisory service. As shown in Table
A1 in the Appendix, the 1995 and 1996 comparisons suggest that services in
the top fourth outperform services in the bottom fourth by $0.17 per bushel.
However, since the 1995 marketing window ends on August 31, 1996, halfway through
the 1996 marketing window and one day before the beginning of the 1997 marketing
window, the farmer could fully implement their choice of advisory service only
for the 1997 crop. The comparisons in Table A10 show that top-performing advisory
services in 1995 tend to be the bottom-performing services in 1997, just the opposite
of what the farmer expected. In fact, bottom-performing services outperform top-performing
services in 1997 by $0.29 per bushel. Similar results tend to be found for other
years and for soybeans and revenue.
Overall, the test results presented in this section provide little
evidence that future advisory service performance can be usefully predicted from
past performance. Most test results show no statistically significant predictability.
When predictability is found, it is sensitive the nature of the comparisons (overlapping
versus non-overlapping crop years) and statistical test considered. The previous
conclusion does not mean it is impossible to predict advisory service performance.
There may be other variables associated with performance that can be used for
prediction. For example, Chevalier and Ellison (1999) study whether mutual fund
performance is related to characteristics of fund managers that indicate ability,
knowledge or effort, and find that managers who attended higher-SAT undergraduate
institutions generate systematically higher returns. Barber and Odean (2000)
examine the trading records of individual stock investors and report that frequent
trading substantially depresses investment returns. Similar factors, such as
education of advisors, cash only services versus futures and options services,
frequency of futures and options trading, or storage costs, may be useful in predicting
the performance of agricultural market advisory services.
Farmers view market advisory services as a significant source of
market information and advice in their quest to manage price risks associated
with grain marketing. Given the high value that farmers place upon market advisory
services, it is somewhat surprising that only two academic studies investigate
the pricing performance of advisory services. The lack of studies is most likely
due to the difficulty in obtaining data on the stream of recommendations provided
In 1994, the Agricultural Market Advisory Service (AgMAS) Project
was initiated, with the goal of providing unbiased and rigorous evaluation of
market advisory services for crop farmers. The AgMAS Project has collected marketing
recommendations for about 25 market advisory services each crop year. The Project
subscribes to all of the services that are followed, and as a result, "real-time"
recommendations are obtained. This prevents the data from being subject to survivorship
and hindsight biases.
The purpose of this paper is to address two basic performance questions
for corn and soybeans using the net price received reported by the AgMAS Project
for the 1995, 1996, 1997 and 1998 crop years. The two basic questions are: 1)
Do market advisory services, on average, outperform an appropriate market benchmark?
and 2) Do market advisory services exhibit persistence in their performance from
year-to-year? At least 21 advisory services are included in the evaluations for
each commodity and crop year. While the sample of advisory services is non-random,
it is constructed to be generally representative of the majority of advisory services
available to farmers. The tests used to determine average performance of market
advisory services and predictability of performance through time have been widely
applied in the financial literature.
Tests of pricing performance relative to a market benchmark are
based on the proportion of services exceeding the benchmark price and the average
percentage difference between the net price of services and the benchmark price.
In statistical terms, the pricing performance test results provide little evidence
that market advisory services consistently and significantly “beat the market”
in corn. There is substantial evidence that market advisory services consistently
and significantly “beat the market” in soybeans. When corn and soybean net advisory
prices are combined into revenue per acre, some evidence also is found that market
advisory services significantly outperform the market. Overall, the statistical
results suggest that market advisory services have some ability to outperform
broad market benchmarks.
It is debatable whether the performance of advisory services also
is economically significant. Perhaps the best perspective on this question is
gained by examining returns for corn and soybean revenue per acre. For all three
crop years, returns averaged 1.4 percent above benchmark revenue, which translates
into about $4 per acre. While this level of return is probably best characterized
as “small,” it also appears to be non-trivial, particularly in comparison to the
cost of the services. However, there are two important reasons to be cautious
about concluding that advisory returns generate even a "small" level
of economic significance: i) the results are based on a small sample of years,
and ii) returns are concentrated in only one market, soybeans.
Three tests of predictability are used and, in general, the they
provide little evidence that advisory service pricing performance can be predicted
from year-to-year. The average correlation coefficient relating performance from
one year to the next generally is insignificantly different from zero. Winner
and loser counts for corn, soybeans and revenue indicate little difference in
the odds of a winner or loser in one period being a winner or loser in the subsequent
period. Finally, average performance of top-performing services is insignificantly
different from that of bottom-performing services.
In conclusion, the results of this study suggest that market advisory
services exhibited some ability to "beat the market" for the 1995 through
1998 corn and soybean crops. Possible explanations for this result include: i)
a unique time period in corn and soybean markets, ii) inefficient commodity markets,
iii) the skillfulness of advisory services or iv) a return to risk. Determining
which explanation is correct will be an important subject for future research
as more data on market advisory service performance becomes available.
Anderson, D.R., D.J. Sweeney and T.A. Williams. Statistics for Business
and Economics, Sixth Edition. West Publishing Company: Minneapolis/St. Paul,
Barber, B.M. and T. Odean." "Trading is Hazardous to Your Wealth:
The Common Stock Investment Performance of Individual Investors." Journal
of Finance 55(2000):773-806.
Bodie, Z., A. Kane, and A.J. Marcus. Investments. Irwin: Homewood,
Brorsen, B.W. and J. Townsend. "Performance Persistence for Managed Futures."
Proceedings of the 1998 NCR‑134 Conference on Applied Commodity Price
Analysis, Forecasting, and Market Risk Management. Department of Agricultural
Economics, Kansas State University, pp. 337-354.
Brown, S. J., W. Goetzmann, R.G.Ibbotson,
and S.A.Ross. "Survivorship Bias in Performance Studies." Review
of Financial Studies 5(1992):553-580.
Carpenter, J.N. and A.W. Lynch. “ Survivorship Bias and Attrition Effects in
Measures of Performance Persistence.” Journal of Financial Economics 54(1999):337-374.
Chevalier, J. and G. Ellison. "Are Some Mutual Fund Managers Better Than
Others? Cross-Sectional Patterns in Behavior and Performance." Journal
of Finance 54(1999):875-899.
Christensen, R. Log-Linear Models and Logistic Regression, Second Edition.
New York: Springer-Verlag, 1997.
Clements, J. "Debunking Some Mutual-Fund Myths." The Wall Street
Journal, March 16, 1999.
Coble, K.H., G.F. Patrick, T.O. Knight, and A.E. Baquet.
“Crop Producer Risk Management Survey: A Preliminary Summary of Selected Data.”
Information Report 99-001, Department of Agricultural Economics, Mississippi State
University, September 1999.
E. J. Elton, M. J. Gruber, and J. C. Rentzler. "Professionally Managed,
Publicly Traded Commodity Funds." Journal of Business 60(1987):175-199.
Fama, E. "Efficient Capital Markets: A Review
of Theory and Empirical Work." Journal of Finance 30(1970):1043-1053.
Friend, I., M. Blume and J. Crockett. Mutual
Funds and Other Institutional Investors. New York: McGraw-Hill, 1970.
Gehrt, D. and Good. "Evaluation of Market Advisory Services for Corn
and Soybeans." Journal of the American Society of Farm Managers and Rural
Good, D.L., S.H. Irwin, T.E. Jackson, M.A. Jirik and J. Martines-Filho“1998 Pricing Performance of Market Advisory
Services for Corn and Soybeans.” AgMAS Project Research Report 2000-01, February
Good, D.L., S.H. Irwin, T.E. Jackson, and G.K Price. "1995 Pricing Performance
of Market Advisory Services for Corn and Soybeans." AgMAS Project Research
Report 1997-01, March 1997.
Good, D.L., T.E. Jackson, and S.H. Irwin. "Development of a Market Benchmark
for AgMAS Performance Evaluations." AgMAS Project Research Report 1998-02,
Grossman, S. and J. E. Stiglitz. "On the Impossibility
of Informationally Efficient Markets." The American Economic Review
Irwin, S.H., C.R. Zulauf, and B.L. Ward. "The Predictability of Managed
Futures Returns." Journal of Derivatives 2(1994): 20-27.
Irwin, S.H., T.E. Jackson and D.L.
Good. "Do Agricultural Market Advisory Services Beat the Market? Evidence
from the Corn and Soybean Markets Over 1995-1997." AgMAS Project Research
Report 1999-03, October 1999.
Jackson, T.E., S.H. Irwin, and D.L. Good. "1996
Pricing Performance of Market Advisory Services for Corn and Soybeans."
AgMAS Project Research Report 1998-01, January 1998.
Jackson, T.E., S.H. Irwin, and D.L. Good. "1997 Pricing Performance of
Market Advisory Services for Corn and Soybeans." AgMAS Project Research Report
1999-01, February 1999.
Jaffe, J.F. and J.M. Mahoney. "The Performance of Investment Newsletters."
Journal of Financial Economics 53(1999):289-307.
Kastens, T.L. and T.C. Schroeder. "Efficiency Tests of July Kansas City
Wheat Futures." Journal of Agricultural and Resource Economics 21(1996):187-198.
King, R.P., L.S. Lev and W.E. Nefstad. "A Position Report for Farm-Level
Marketing Management." Review of Agricultural Economics 17(1995):205-212.
Lakonishok, J., A. Shleifer, and R.W. Vishny. "The Structure and Performance
of the Money Management Industry." Brookings Papers: Microeconomics
Malkiel, B.G. “Returns from Investing in Equity Mutual Funds 1971 to 1991.”
Journal of Finance
Malkiel, B.G. A Random Walk Down Wall Street. New York: W.W. Norton
and Company, 1999.
Marcus, A.J. “The Magellan Fund and Market Efficiency.” Journal of Portfolio
Marten, J. "Farmers Want Market News, Not Advice." Farm Journal
Extra, June 1994.
Martines-Filho, J.G. Pre-Harvest Marketing Strategies for Corn and Soybeans:
A Comparison of Optimal Hedging Models and Market Advisory Service Recommendations.
Unpublished Ph.D. dissertation, The Ohio State University, 1996.
Metrick, A. "Performance Evaluation with Transactions Data: The Stock
Selection of Investment Newsletters." Journal of Finance 54(1999):1743-1776.
Norvell, J. M. and D. H. Lattz. “Value-Added Crops,
GPS Technology and Consultant Survey: Summary of a 1998 Survey to Illinois Farmers.”
Working Paper, College of Agricultural, Consumer, and Environmental Sciences,
University of Illinois, July 1999.
Otte, J. "Marketing Matters -- How Well Do Market Advisors Deliver?"
Prairie Farmer, July 19, 1986.
Patrick, G.F. and S. Ullerich. "Information Sources and Risk Attitudes
of Large-Scale Farmers, Farm Managers, and Agricultural Bankers." Agribusiness
Patrick, G.F., W.N. Musser, and D.T. Eckman. "Forward Marketing Practices
and Attitudes of Large-Scale Midwestern Grain Farmers." Review of Agricultural
Powers, L. "How to Measure Your Pro's Performance." Top Farmer,
April 1993, p. 17.
Ritter, J.R. "The Long-Run Performance of Initial Public Offerings."
Journal of Finance 46(1991):3-27.
Schroeder, T.C., J.L. Parcell, T.L. Kastens, and K.C. Dhuyvetter. "Perceptions
of Marketing Strategies; Farmers vs. Extension Economists. Journal of Agricultural
and Resource Economics 23(1998):279-293.
Wisner, R.N., E.N. Blue and E.D. Baldwin. “Preharvest Marketing Strategies
Increase Net Returns for Corn and Soybean Growers.” Review of Agricultural
Zulauf, C.R. and S.H. Irwin. “Market Efficiency and Marketing to Enhance Income
of Crop Farmers.” Review of Agricultural Economics 20(1998):308-331.
[*] Scott H. Irwin and Darrel
L. Good are Professors in the Department of Agricultural and Consumer Economics
at the University of Illinois at Urbana-Champaign. Joao Martines-Filho is the
Manager of the AgMAS and farmdoc Projects in the Department of Agricultural and
Consumer Economics at the University of Illinois at Urbana-Champaign. Thomas
E. Jackson is Manager of the US Agriculture Forecast with WEFA, Inc. and former
Manager of the AgMAS Project. Funding for the AgMAS Project is provided by the
following organizations: American Farm Bureau Foundation for Agriculture; Illinois
Council on Food and Agricultural Research; Cooperative State Research, Education,
and Extension Service, U.S. Department of Agriculture; Economic Research Service,
U.S. Department of Agriculture; and the Risk Management Agency, U.S. Department
of Agriculture. The authors gratefully acknowledge the valuable comments of members
of the AgMAS Project Review Panel and seminar participants at the 1999 NCR-134
Conference, Cornell University, Southern Illinois University and the University
of Illinois at Urbana-Champaign.
King, Lev and Nefstad (1995) examine the corn and soybean recommendations of two
market advisory services for a single year. The focus of their study is not pricing
performance, but a demonstration of the market accounting program Market Tools.
Several analyses have appeared in the popular farm press. Marten (1984) examines
the performance of six advisory services for corn and soybeans over 1981 through
1983. Otte (1986) investigates the performance of three services for corn over
the period 1980 through 1984. Each of these studies indicates the average price
generated by the services exceeds a benchmark price (e.g., selling 100 percent
at harvest). More recent evaluations appear in Top Producer magazine (e.g.,
Powers, 1993). In this case, evaluations of corn, wheat, and soybean recommendations
from advisory services are reported on a regular basis. Kastens and Schroeder
(1996) examine futures trading profits based on the information reported in Top
Producer for the 1998-1996 crop years. They find negative trading profits
for wheat and positive trading profits for corn and soybeans.
 See Zulauf and Irwin (1998) for a classification
and review of marketing strategy studies.
 Throughout this report, the term "crop
year" refers to the marketing window for a particular crop. This is done
to simplify the presentation and discussion of market advisory service performance
results. A “crop year” is more than twelve calendar months in length and includes
pre-harvest and post-harvest marketing periods.
 When the AgMAS study began in 1994,
DTN and FarmDayta were separate companies. The two companies merged in 1996.
 This assumption subsequently is relaxed
to reflect the growing importance of alternative means of electronic delivery
of market advisory services. Beginning in 1997, a service that meets the original
two criteria and is available on a "real-time" basis electronically
may be included in the sample. Two examples are Utterback Marketing Service,
which is carried on a World Wide Web site, and Ag Review, which is available via
e-mail. Both are for-pay subscription services.
 Progressive Ag is included in the study
for the 1996, 1997 and 1998 marketing periods, but is not included in 1995 because
it had not yet come to the project's attention. Utterback Marketing Services
is included in 1997 and 1998, but is not included in 1995 or 1996 because its
marketing programs were not deemed to be clear enough to be followed by the AgMAS
project. Ag Alert for Ontario was included in 1996, but its advice is geared
to Canadian farmers and was not deemed to be generalizable to U.S. farmers, and
subsequently was dropped. Grain Field Report, Harris Weather/Elliott Advisory,
North American Ag, and Prosperous Farmer are included in 1995 and/or 1996, but
are not included in 1997 or 1998 because they no longer provide specific recommendations
regarding cash sales. Agri-Edge is included in previous reports, but the program
was discontinued during the 1997 marketing period. Allendale futures & options
and Ag Line by Doane hedge are programs introduced for the 1996 marketing period
for corn only. The Ag Line by Doane hedge program for soybeans is first tracked
for the 1998 marketing period.
 Some of the programs that are depicted
as “cash-only” do in fact have some futures-related activity, due to the use of
hedge-to-arrive contracts, basis contracts, and some use of options.
 There are a few instances where a service
clearly differentiates strategies based on the availability of on-farm versus
off-farm (commercial) storage. In these instances, recorded recommendations reflect
the off-farm storage strategy. Otherwise, services do not differentiate strategies
according to the availability of on-farm storage.
 These results originally are presented
in Good, Irwin, Jackson, Jirik and Martines-Filho (2000). Complete details regarding
the components of the net prices (futures and options gains and losses, net cash
price, etc.) can be found in this report.
 Note that return in this case refers
to return net of marketing costs but no other production costs.
 Return correlations are invariant
to the particular benchmark chosen to compute returns. Hence, correlations are
presented only for 24-month benchmark returns.
 These results are not presented due
to space constraints, but are available from the authors upon request.
 The calculation of revenue per acre
ignores economies of size that may accrue to larger farms implementing the recommendations.
It also ignores contract "lumpiness" problems that may be significant
for smaller farms.
 Adding the subscription cost of services
to the transactions costs considered in computing net advisory prices does not
alter the performance results. For a 1,000 acre farm, subscription costs amount
to less than one-tenth of one percent of the average corn and soybean revenue
 As noted earlier, return correlations
are invariant to the particular benchmark chosen to compute returns. Hence, correlations
are presented only for 24-month benchmark returns.
Click here for Order Form