Part 2: The Impact of Experiment Design on Trial Results
In part one of this series, we explored the effect of feed formulation on results of enzyme evaluation trials. The formulation considerations we discussed included substrate-enzyme interaction, utilizing the matrix value for each enzyme to maintain nutrient balance and the potential impact of feed ingredient variation.
Another area which can significantly impact confidence in enzyme evaluation is the trial design. Deficiencies in trial design can lead to incorrect decisions.
There are two types of decision errors:
- An example of one error (α) is when we conclude that a certain enzyme releases X amount of energy when in the reality it does not.
- A different error (β) occurs when we conclude that a certain enzyme does NOT release X amount of energy when in reality it does.
When designing an enzyme evaluation trial, the aim is to minimize both types of errors. This can be accomplished by controlling factors of interest and separating sources of extraneous variation in the trial design. The accuracy of the results and the ultimate decision can be improved by using statistically designed experiments and applying accepted techniques. Such techniques include blocking, randomization and replication.
Reduce Experimental Error by Blocking
The experimental unit in a trial is the physical entity to which the treatment is randomly assigned (i.e., the cage in which the chicken is placed). These units must be similar to each other to minimize variation caused by factors other than the experimental diets, which is the factor of interest.
For example, if cages are used in two separate rooms, then we would expect that the environmental conditions such as humidity, light intensity and temperature will vary systematically between rooms. This could create consistent (non-random) variation in responses between the rooms, leading to higher experimental errors. Another example is using the same room but with cages stacked in separate batteries. This creates a location difference of each battery inside the room and could also create differences among the experimental units (cages).
In cases of expected or known variation between cages, it is recommended to apply blocking. Blocking is a technique whereby experimental units (cages) are placed into groups based on their similarity with respect to characteristics that may affect the response. This way experimental units within each block will be more homogenous than those between blocks. Blocking enables us to separate the variability associated with the block factor (room) from the experimental error, improving the chance of detecting a real enzyme effect in a trial.
Apply Non-Bias Randomization
To minimize the possibility of experimental bias which could arise from systematic assignments of treatments to experimental units, it is necessary to randomly assign the treatments. For example, it is convenient to assign the same treatment to cages that are adjacent to each other for ease of handling, feeding and weighing birds (Figure A).
However, if there is a systematic bias between cages from left to right, the observed difference among treatments could be misinterpreted as a treatment effect when it is actually caused by the location. Therefore, when assigning treatments to different cages, it must be done randomly. Figure B below illustrates an example of randomly assigning 6 treatments over 24 cages.
Choose the Appropriate Number of Replicates
Replicates are independent measurements of the same treatment. With a greater number of replicates, the estimation of the treatment values is more precise and increases the probability of detecting a pre-defined difference between treatments (defined as “d”). However, due to the cost of running an extra-large number of replicates (labor, time and availability of experimental units), a minimum number of replicates is often applied to each experiment to achieve the pre-determined level of significance (a=0.05) and statistical power (b=0.10, meaning 90% probability of detecting a treatment difference of d).
Figures A and B (above) show 4 replicates of each treatment. This approach provides several independent observations for any particular treatment and several measurements of each treatment. Whether the number of replicates in this example is adequate can be determined mathematically using a, b, d and expected within-treatment variation. Always keep in mind when interpreting trial results that a “no significant treatment effects” could be simply due to insufficient replication.
In summary, the decisions made based on statistical inference of trial data depends on the methods applied, such as blocking, randomization of treatments and number of replicates. Involving an expert who understands both the actual production environment and the statistical methods is crucial to achieve confidence in trial results.
Ott, R. L., and M. Longnecker. 2001. An introduction to statistical methods and data analysis. 6th edition. Brooks/Cole, Cengage Learning, CA.
Sprinthall, R. C. 2000. Basic statistical analysis. 6th edition. Allyn and Bacon, Boston.