Assessment of enjoyment value in DLG Quality Tests

Authors:

• Professor Emeritus Dr. Goetz Hildebrandt, Institute of Food Hygiene, Goetz.Hildebrandt@fu-berlin.de
• Jörg Jacob, Leipzig, H.JoergJacob@t-online.de
• Bianca Schneider-Häder, Elisabeth Hamacher, DLG e.V. - Competence Center Food Frankfurt/M., Germany, sensorik@DLG.org
• Britta Loewe-Stanienda, DLG Test Center Food, B.Loewe-Stanienda@DLG.org

Sensory assessment/tasting as neutral guidance for consumers

According to natural science criteria, foods serve both nutritional and enjoyment purposes. Whether a food can provide us with enjoyment depends largely on the attributes of appearance, consistency, odour and taste. Thanks to intensive inhouse and external monitoring, foods generally satisfy the requirements of product composition and product safety, but consumers cannot assume that they will also like the taste, or in other words that the enjoyment value of a food will satisfy their requirements. Official surveillance only steps in here in the event of substantial deviations, while minor failings remain unchallenged. The quality tests conducted by NGOs, magazines, TV broadcasters or consumer organizations admittedly often include a sensory test, but they are limited to just individual products. Accordingly, there is a lack of any instance that provides consumers with the essential objective information they need about the enjoyment value of a large number of food types from as many producers as possible and in this way support their purchasing decision. By combining an easily understandable award system with core competence in sensory testing, DLG as a longstanding, traditional test institute has taken on this public-serving task. At its annual International DLG Quality Tests, DLG tests processed foods from all areas of conventional production, as well as organic products and foods for the elderly.

Objectivity, expert knowledge and inner standard

Despite all efforts with electronic noses and tongues and other analytical apparatus, only humans are able to conduct an authentic and holistic assessment of enjoyment value on the basis of a sensory test. Accordingly the gathering, description and assessment of a sensory profile should be carried out by suitable test persons. In order to ensure the most neutral possible and at the same time proper findings and analysis, DLG uses the services of acknowledged experts. Such experts must not only be experienced specialists familiar with the theory and practice of sensory testing (= sensory assessors), but should also be distinguished by product-related expertise (= technical experts). Such skills are defined as profound knowledge about the production, properties and marketability of the tested products. As a DIN standard formulates, this should result in a representative and up-to-date internalized standard – largely acquired through professional experience, tuition and further training – regarding the prevailing opinion about the sensory quality of the foods to be tested. As a result of this “inner standard”, experts are able to make an objective comparison between standards of quality and their realization. The fact that the examination of foods for edibility by government inspection laboratories derives from the same principle, namely assessment based on a professionally grounded “inner standard”, confirms the scientific respectability of the DLG method.

Does it not run contrary to human, emotion-guided nature to demand absolute neutrality from an expert in the assessment of enjoyment value? The answer is “no”, for the ability to be objective results from the essential feature of our intelligence that enables us to inhibit personal impulses and adopt the perspective of others (known as the theory of mind). And nothing less than an application of human intelligence, namely culinary intelligence, is required for sensory expert opinions. Alongside the perception and consideration of usual consumption habits, neutrality also demands the willingness to leave oneself behind and where necessary to relativism one’s own opinion or even correct one’s personal “inner standard”.

Promoting the diversity of foods

In a free market economy, a certain variability of the sensory attributes within a food type is a feature of competition, for only distinguishing factors enable consumers to make choices geared to their own needs. That is why experts should suppress their own specific preferences when testing. Instead, their “inner standard” must take into account the complete “hedonic bandwidth” as standard. This reference value can be defined as the accurate overall picture of the entirety (cluster) of sound samples of a food type satisfying the prevailing opinion in all sensory properties. As the only true product for all consumers does not exist, a DLG expert for example will also never answer the question which of the two market leaders of caffeine-based soft drinks he would recommend to consumers on sensory grounds. If they do not show any measurable defects, both variants evidently satisfy the general type standard for such foods, and although they display sensory differences in the quality test, they are deemed to be of equal value. The same applies for the features “meltingly smooth” or “firm to the bite” as stand-alone attributes for milk chocolate, while bitter chocolate should not immediately melt in the mouth. Thus the suitability of test persons depends crucially on their ability to distinguish between desired diversity and undesirable deviation from the consumer expectation.

Uniform rules for awarding DLG medals

The standard by which the quality of a food sample is measured at a DLG Quality Test is the overall picture (“inner standard”) of a product with rounded and harmonized appearance, consistency, odour and taste. Deviations in the sector of craft perfection, known as the hedonic bandwidth, are considered to be faults. Such orientation to negative deviations from the optimal quality standard is more concrete and more objective than attempts to assess a product partially or even exclusively via positive attributes (e.g. “attractive presentation”, “type-specific texture” or “tart-tangy aroma”).

Within the framework of a DLG Quality Test, therefore, test persons must be able to identify and name any existing product defects precisely. Because of this concrete fault-specific aspect, the DLG Test falls under the category of “descriptive tests”. If a product displays sensorially clear deviations from the norm, points are subtracted from the maximum score (5 points) awarded for sound products in accordance with binding calculation rules. The fault is weighted depending on its significance on the one hand (deviations in taste, for example, are more serious than deviations in outer appearance) and intensity (scaling”) on the other. The total score achieved then results in the quality classification Gold, Silver, Bronze or “no award”. Thus for instance a slight deviation in taste (e.g. a little “greasy”) or a perceptible shortfall in consistency (e.g. “rubbery”) in a bockwurst sausage would lead to a Silver Medal, while a noticeable fault in appearance (e.g. “greying”) would lead to an award in Bronze. Because such medal awards signify a quality classification, in scientific language the DLG Test belongs to the “descriptive tests with integrated assessment”. 

Test findings as the basis for optimizing products

For each submitter of food samples, a Gold Medal represents confirmation of their professional qualification that can be documented vis-à-vis customers too with the corresponding certificate for a fault-free product. However, if product defects are ascertained, the producer cannot simply continue business as usual, even in the case of a Silver Medal (= slight deviation), but instead should address the test report, which ranks as an expert opinion, intensively. The standardization of the fault terminology helps to understand the findings, i.e. the defects that can occur in a food have been defined and compiled as a list in the DLG test form. In the tests, the experts then only need to select the appropriate expression for a deviation identified. This standard vocabulary (wording) aims to ensure that all participants understand the same thing by the same term. Naturally the jurors can also add supplementary remarks or more specific notes.

If defects are pointed up in the expert opinion, the submitter of the sample must first decide whether they want to remain by their line or accept the complaint. If they decide to eliminate the defect, the clear and distinct description of the fault enables them to intervene selectively in the production process or the formulation. In this correction, the overall concept of the food is under scrutiny because the sensory assessment is not just limited to odour and taste, but also covers the consistency as well as the outer and inner appearance and the composition, and these features often mutually influence each other. Without doubt chemical, physical or microbiological analyses also contain essential information, but no other examination method supplies such complex findings for marketable foods as sensory testing. That is why the enjoyment value can also be described as “the soul of a product”.

The question of that „certain extra“

There have repeatedly been efforts not just to award a Gold Medal for the fault-free product, but also to provide additional recognition for samples with that “certain extra”. Their uniqueness could result from a special selection of raw materials, formulation, production method or presentation. As spectacular and attention-grabbing as awards such as “best of the show” or “top of the tops” may be, in the standardized DLG assessment system a “more than perfect” condition is however encountered too rarely to be able to justify this plausibly for consumers and fellow competitors. It would be easy to overstep the boundary towards individual preference testing here, which is all the more possible as the expert’s customary definition does not and cannot require any objective “inner standard” for what is “perceived as exceptionally positive”. In the case of German wines with their often metaphorical attribute descriptions, special awards are assigned by a separate panel of testers in order to ensure comparability with other quality labels.

One expert alone is not enough

The reliability and credibility of sensory test results stand and fall with the quality of the experts’ “inner standard”. These persons must possess objective criteria for the sensory profile of a food that satisfies general standards and not simply their own respective personal preferences. However, the requirement that concepts of the cluster of fault-free products be absolutely congruent for all experts is totally unrealistic. Instead, the “inner standard” possesses an individual variation that is all the greater, the more complex a food composition is. Such divergences become less important when a number of experts are assigned to describe and assess a sample. Depending on the type of food and the availability of testers, they work in groups of three, five or ten persons. The experts first assess the food on their own, independently of the others. If the evaluation of the test sheets does not reveal consistent findings, the group comes together and discusses points in dispute with a view to reaching a consensus. By comparison with completely separate individual evaluations, this method makes it possible to stabilize the test result. 

Consumer expectations and their significance for the DLG Quality Test

The objectivity of the DLG Quality Tests is ensured by assigning experts as testers. Alongside their sensory capabilities, these persons are characterized by a both representative and up-to-date internalized standard regarding generally customary demands made of the enjoyment value of a food. Such a product- related ideal picture is largely acquired through professional experience, tuition and further training and covers the bandwidth of perfect products. By contrast with experts from the fields of production, trade, surveillance and academia, “normal consumers” do not possess this background knowledge. Instead they follow their own, subjective concepts and can therefore only be deployed for popularity testing where the market prospects of products are to be assessed with the help of large numbers of “testers”. However, even without the specific inclusion of untrained consumers in the DLG test, the focus is on the expectations of this group. For only an expert who is able to assess the preferences of customers in a representative fashion and orients his assessment to these deserves the title of expert.

Measures to secure the quality of actual testing

Voluntary tests such as the DLG Quality Tests can only be successful if both the test customers, in other words the producers submitting the samples, and the addressees, i.e. the consumers, trust the examination and scoring system. In order to achieve credibility, the organizational structures are subject to constant revision, the training and selection of testers is perfected, and the scientific foundations are adapted to new findings. As documentary evidence of these efforts, it has been possible to certify the DLG testing methodology in accordance with ISO and DIN. The training and appointment of suitable persons as certified DLG experts lies in the self-interest of many firms and examination establishments, including official surveillance bodies.

One quite essential organizational measure is the rendering anonymous of samples, i.e. the foods are tested without the testers knowing the producer, the brand or the price. Further key management elements include the use of standardized record sheets with defined fault attributes, instructions for testers and examination of alignment samples at the beginning of the event, the inspection of reserve samples in cases of doubt and supervision by test group leaders and test officers including an evaluation of testers. Furthermore, spot post-checks of products are conducted by contract laboratories in order to track down any possible special production batches. The fact that hardly any indications of tampering/manipulation are found here is plausible, as such deceptions require an incommensurately high outlay and moreover are quite likely to fail due to a lack of corresponding experience.While at the beginning of the DLG Quality Tests the individual professional experience of the experts alone was sufficient to legitimize their evaluations, modern sensory testing calls for back-up in the form of a sound scientific basis. A two-stage procedure consisting of an individual test and subsequent group consensus-finding was developed as an appropriate solution for a descriptive test with integrated assessment such as is required by the comparative quality tests.

Many DLG experts from the fields of science and academia, our own Sensory Committee, the organizing of scientific seminars and congresses and the award of research contracts ensure that this concept is kept under constant scrutiny.

Stand-alone features of the DLG Quality Test

Originally created to improve the quality of German agricultural products, the DLG Quality Tests have not only become increasingly popular among domestic producers, but are also becoming ever more significant outside Germany too. This international recognition is manifested on the one hand in the submission of samples by foreign producers, and on the other hand in the conduct of quality tests in the field, e.g. in the Czech Republic or Japan. Such success motivates imitators, mainly at a regional level and without certification by the organizer. The vast number of product competitions always uses the concept, the organizational principles and as far as possible also the co-participation of DLG experts. Even if isolated modifications are undertaken (assignment of lay testers or cup trophies instead of medals), there is no independent further development of the testing system. Only at DLG does a team of full-time staff engage in constant dialogue with a large number of honorary experts from the fields of business, food surveillance and research to ensure that new findings are incorporated promptly and in line with practice in the sensory quality tests. As a result of this consistent improvement, DLG will remain the standard for quality tests.

Further reading

Hildebrandt, G., Jacob, J., Loewe-Stanienda, B., Oehlenschläger, J. und Schneider-Häder, B. (2012): Descriptive sensory analysis with integrated quality rating as a tool for quality testing of commercial food products. Journal of Food Safety and Food Quality 63 (5), 155-162