Hero image Hero image


Test Better by Design

Well-designed tests result in reliable and robust answers
to important questions about product quality

Sensory testing is a valuable risk management tool that can prevent companies from suffering product disasters and help inform new product development. There are three “big” things that sensory science can accomplish.

  • We can determine if a product has changed (Discrimination)

  • We can measure the sensory properties of our products (Descriptive Analysis)

  • We can determine consumer responses to our product (Consumer Research)

All of these methods can be used to manage risk. They provide the information that responsible management requires to make decisions that are basic to continued success. Is the contemplated ingredient or process change to reduce cost going to be detected by our consumers? Will sales suffer due to consumer disappointment? What makes our product unique in the market? What sensory characteristics are related to consumer acceptance? Do consumers like the new variety as much, or more than the existing product? In every case, we want a test that gives us confidence in the business decision that we must make.

Consequently, getting the test right is of paramount importance. That means we have to design the test properly to assure a reliable and robust outcome. Otherwise, we may as well not bother testing at all.

In all sensory tests, people are our judges. Every one of us can be influenced by the way in which the test is conducted. The most common source of bias is the effect of order of sample evaluation. The first sample we see or taste is perceived differently because it’s a fresh experience. When we get the second sample we have the first sample’s impact influencing our response. We can’t eliminate this first position effect entirely, but we can compensate for it by design. See this YouTube video from the University of Western Sydney for an explanation on order effects:

The Importance of Experimental Design

The experimental design, that is the order of sample presentation, can be rotated and balanced to ensure that all samples are treated in the same manner and that the test is fair. If we know that the first position effect is really big, for example if testing alcoholic beverages or coffee, then we recommend a “warm-up” sample that is neither evaluated nor measured. If we insert a dummy sample at the start of each sample set, then the effect is equal for all samples in the test. If we have a large number of products to test, experimental designs can be used to ensure that all possible pairs of samples are seen together, similarly we can even ensure that triplets are evaluated in a balanced design.

Complete Block Designs

Finally, if we simply have too many samples to evaluate in one sitting, we can use sessions to manage our study. The large complete block design (CBD) can be sliced into columns and a limited number of columns may be presented in each sitting. This way the whole block can be evaluated over several days, or even weeks if the samples are sufficiently stable to permit doing so.

Complete Block Design (with Sessions).png

Balanced Incomplete Block Designs

We can also opt for balanced incomplete block (BIB) designs that ensure that all samples are seen equally, but no one sees all the samples. The size of the block is based upon what an assessor can manage to evaluate in one session without compromising data quality by fatigue or carry-over. It is possible to balance blocks over sessions to actually have all samples seen by each assessor. This takes quite a bit of planning and sample preparation. 

Sensory Informed Designs

The most powerful BIB designs are Sensory Informed Designs (SID). This research has been conducted at Compusense over the last five years and has resulted in a number of presentations and publications in journals. The technique starts with determining the sensory profiles of the products being studied through descriptive analysis. We ensure that the products chosen cover the sensory space, the range of attributes and their intensities, of the product category. The incomplete blocks are made using products that have a sensory contrast and can be used to cluster our consumers on their liking response. No one has to see every product for us to find out their sensory drivers of liking (attributes that are more critical than others in determining consumer preferences for a product). SID category appraisals have been conducted with consumers only testing 6 out of 16 possible products. This approach cuts the cost of these studies in half and delivers excellent products development guidance.

To learn more about Sensory Informed Design, visit https://www.compusense.com/en/research-and-community/articles/oral-presentations/sensory-informed-incomplete-block-designs/