- Ass. Prof. Dipl.Ing., Dr. nat.techn. Klaus Dürrschmid, Institute of Food Sciences, University of Natural Resources and Life Sciences (BOKU), Vienna; email@example.com
Further information on the topic:
- Dürrschmid, K. (2009). Gustatorische Wahrnehmungen gezielt abwandeln Hamburg: Behr‘s Verlag
Among other things, the DLG study conducted in 2018 on the “Reduction of sugar, fat and salt in food – between feasibility and consumer expectations” examined to what degree the reduction of sugar, fat and salt can be technologically successfully realised without considerably impairing the taste and texture of the product in the process. As part of an expert survey and comprehensive market research, a compilation was made as to which reduction methods are technologically possible and how great their familiarity or practical use is in food processing. The consideration of the “multi-sensory interaction” is relevant for sensory changes to food and beverages. It was apparent in the results of the expert survey that the familiarity and the practical knowledge in this regard are not yet broadly prevalent. Approximately 75 % of the study participants are unfamiliar with this subject or know only little about how it could be applied in practice. This Expert Knowledge represents Part 3 of the explanations on the main topic of “Reformulation” and is intended to increase the transparency with regard to “multi-sensory interactions”, present current scientific findings and make suggestions for the respective formulation management in practice. For all sensory changes to products offered on the market should be well-prepared in a strategic, conceptional and communicative (claims) sense in order to achieve the smallest possible modifications of the taste profiles and to maintain consumer confidence in the expected product quality.
The sensory analysis of food traditionally has a one-dimensional focus. This means that individual sensory properties regarding the product characteristics appearance, odour or aroma, flavour and texture or haptics, and in some cases also the acoustics, are generally examined. However, during daily consumption, consumers perceive various sensory modalities in combination originating from the product or its consumption or sales environment. These multi-modal perceptions and cross-modal interactions then result in the actual consumption preferences. In the packaging sector, the findings from the multi-sensory interaction have already been implemented in production concepts for some time now. In addition to the shape and choice of colours, here in particular the haptics also play a very important role for the product quality induced at the point of sale . In the food sector, the analysis of the multi-modal influencing factors on product purchasing and consumption – both at the place of consumption (e.g. at a restaurant) and at the point of sale – has also been the object of various research projects for some time now. In the context of reformulation efforts for the reduction of salt, sugar and fat in food as well, the consideration and analysis of cross-modal interactions of ingredients in the product recipes plays a major role.
As is generally known, human beings have several sensory systems which, when stimulated by transmitting stimuli and processing in the brain, result in sensations and perceptions which are referred to as sensory modalities, such as seeing, hearing, smelling and tasting. Within these modalities, there are various sensory qualities, like the colours red, blue and yellow for seeing, or sweet, sour and salty for tasting. Surprisingly, just how many sensory modalities human beings possess is debatable, as for example heat, cold, the trigeminal sense and the sense of balance are sensory modalities of their own.
In everyday life, we assume that each sensory modality functions independently of the other modalities, however this assumption is incorrect. Virtually all of our everyday perceptions are multi-modal, i.e. the corresponding complex perceptions result from the activities of several modalities simultaneously, and these activities mutually affect each other on various levels in the development of perceptions. This not only results in multi-modally triggered perceptions, but also in multi-modal or cross-modal interactions.
Uni-modal perceptions, i.e. perceptions which actually only result from the stimulation of one sensory system, are mainly an exception in the food and nutrition sector. Even if one takes a mouthful of a watery sugar solution, the temperature of the solution is perceived, as well as its viscosity and its colour before sampling it. We taste sweetness and intuitively assume that this flavour would be independent of the colour of the sampled product, its texture and temperature which are also perceived. In fact, the intensity of the taste is, however, very much affected by the viscosity of the solution, its temperature and colour. We perceive pink-coloured solutions as sweeter than colourless solutions and cool solutions appear to be less sweet. As our perceptual system has learned that intensely sweet sugar solutions have a certain viscosity, elimination of our perception of sweetness with a sweetness blocker results in the same, equally viscous but now no longer sweet-tasting sugar solution suddenly seeming watery and no longer as viscous to us.
This means that we must understand our sensory systems as a network with which the activation of each sensory modality node of the network can affect the other sensory modalities. Taste affects viscosity, colour affects odour and taste, etc. Figure 1 shows a schematic representation of which and how the seven most important sensory modalities have been scientifi cally documented to infl uence each other in pairs up until now.
Multi-modal perceptions and cross-modal interactions are not only relevant on the level of the food product itself, but also with regard to the packaging, the point of sale in the retail food trade and the point of consumption in the catering sector.
Multi-modal products are products that address several sensory modalities. They are processed in several parts of the brain, and therefore the triggered perception is also more complex and extensive, which leads to improved brand awareness and greater product loyalty. On the other hand, products which are less complex in a sensory regard can produce boredom or even an aversion when consumed over a longer period, and therefore are characterised by poorer sales in the long term. Multi-modality of products can also be used to differentiate them from the competition and to improve recognisability in order to stimulate sales. The packaging as an element of the product is always multi-modal; it not only addresses the visual sense, but also the haptics, the acoustic and the olfactory sense, and therefore affects the perception of the product itself via expectations, associations and resulting cross-modal interactions.
The place of consumption of food in a restaurant, café, canteen, airplane, movie theatre, etc. is always characterised by a multi-modal context. The atmosphere, music and sounds, lighting, table layout, fl owers, menu design, dishes, room fragrancing, air quality  and many other factors affect the persons eating in their interaction with the food and beverages, however also with the other persons. The scientifi c literature on this subject has greatly developed in the past several years and will become even more extensive due to the increasing importance of eating away from home in developed societies [5, 48, 51, 52, 55, 69, 73, 74].
Multi-modality also plays a major role at the point of sale of food. The atmosphere, lighting, background music, colours, odours and a great deal more were already examined and it has become apparent that both the type and the intensity of the stimuli can affect buying behaviour . Lutsch, Scharf and Zanger (2015)  showed that the motivation of consumers, their needs and their goals considerably determined their behaviour, and that the multi-modal stimuli and their intensity must be exactly coordinated with the needs they are intended to activate.
Overview of modulations of taste perceptions
The following is intended to show the main multi-modalities and to describe examples of neurophysiological cross-modal interactions (see Table 1). Some constellations are already being implemented in practice, while others are currently being discussed and require further research activities.
The flavour is of particular importance for the overall perception of a food. A flavour perception comprises all sensory perceptions which occur in the oral and nasal areas during eating and also, if volatile aromas from the oral area are perceived retronasally via the receptors of the olfactory mucosa of the nose, the brain also localises them as a perception in the oral area. This flavour illusion is multi-modal, because several sensory systems are involved in its occurrence and cross-modal, because a sensory system can be modified with other interactions and the perception. Above all, there are a number of interesting interactions between odour and flavour. The flavouring maltol is, for example, added to chocolate and other confectionery and helps to save 5-15 % of the sucrose there by enabling us to perceive the sweetness more intensively. Maltol also improves the feeling of reduced-fat food in the mouth. Another well-documented interaction is that between strawberry aroma and the sweet taste. Strawberry aroma increases the sweetness intensity. How and why these stable but not very intense effects occur is under discussion, however in particular cultural connections of odours with flavours and those formed through personal experience are assumed. This is referred to as a learned synaesthesia, i.e. a learned, involuntary connection of one sensory modality to another. However, the concept of the double encoding theory could take effect, according to which information can be retained better and can be recognised more reliably when it is encoded double, namely in the codes of two sensory systems.
The modulation of taste perceptions using chemical substances is a research field with results which could intervene in our taste perceptions in the near future.
Colour and flavour
Moir (1936)  showed that the change in colour of a food also changes the perceived flavour. Red colour in white wine convinces many consumers that it is red wine and red colour frequently causes a considerable increase in the perceived sweetness intensity. Many studies show clear effects of colour on the perceived flavour and the taste intensity, however in contrast others do not. Spence (2013)  showed that the manner in which colours affect flavour and taste is very highly dependent on the expectations triggered by the colour, and is therefore not independent of the culinary culture. A blue colour can, for example, trigger the expectation of blueberry flavour for Europeans, while in contrast it leads to the expectation of mint flavour for the Vietnamese (Listerine). If the perceived flavour is then not too far away from the expected flavour (black current expected, blackberry actually used), then the consumer will report the expected flavour. This effect is known as expectation bias. Only if the difference between the expected and the perceived flavour is too great, is the expectation not confirmed and confusion results.
Taste and odour
The interactions between odour and taste perceptions are described as very robust in the related literature. Odours can change the perceived taste intensity, and vice-versa the taste can modulate the aroma perception. The strengthening of the sweetness perception by certain odours is especially well-documented. The “sweet” odour of caramel, vanilla, strawberry, lychee and mint aromas, which have no taste in themselves, is capable of intensifying the perceived sweetness in a product [19, 58, 59, 62].
The results were similar for many other odour-taste interactions: Citrus and strawberry aroma increases the sour sensation of citric acid , while soy sauce and anchovy aroma (umami sources) increase the salty taste . As a result, odours which generally regularly occur together with a sweet, bitter, salty or sour taste in food are capable of strengthening the associated taste, even if they are only added in very small concentrations. These interactions which intensify the test impression are very important for practical applications in the context of reformulation and the reduction of sugar and salt. Through the specific use of defined aroma combinations in the recipe, it is possible among other things to save a considerable amount of sugar or salt, which is to be evaluated as positive from the perspective of health and preventative medicine.
In this context, the intensification of the salty taste with aromas (OISE or odour induced saltiness enhancement) to save salt which – when enjoyed in excessive amounts – is made responsible for high blood pressure [34, 35, 44, 45, 64]. OISE can especially be observed at low salt concentrations; if the salt concentration is already high, then aromas that increase the saltiness are weak . There are also aromas that increase the fat perception [63, 64].
An odour and taste interaction combined with an effect that increases the taste sensation is also attributable to the kokumi. Like “umami”, the term “kokumi” originates from Japan. However, in contrast to umami (salts of the amino acid glutamine), kokumi is not a basic taste, but instead a sensation based on the interaction of several protein molecule groups with food ingredients and affects the taste perception. Research results around the Japanese scientist Kuroda et.al. [30, 41] focus on studies with mixtures of amino acid molecules, so-called γ-glutamyl peptide chains. Kokumi peptides, like γ-glutamyl valyl glycine chains (combination of the amino acids glutamine, valine and glycine), primarily form in dishes cooked long or fermented, like bean-and-meat stews, fermented black beans or matured Gouda cheese. Even small concentrations of kokumi, which is in itself tasteless, are sufficient to achieve a sensory effect. Various studies conducted by Kuroda et.al. prove that with the kokumi effect a more intensive perception of saltiness and of spicy notes and meat aromas (umami enhancement), an enhancement of the feeling in the mouth (full-flavoured) and in some cases also a mouth-filling fat impression can be achieved. Results in the context of salt and fat reduction must be re-evaluated.
Less often a weakening of the taste intensity can be observed, e.g. Angelica oil reduces the sweetness intensity. Interesting applications of such suppressing effects exist both in the food and in the pharmaceuticals sector for reducing a bitter or sour taste. Here among other things so-called bitter-blockers, e.g. AMP adenosine monophosphate and other nucleotides which naturally occur in food, are used for this purpose.
Conversely, flavourings can also intensify the retronasal aroma perception. Davidson, Linforth, Hollowood and Taylor (1999)  have already shown that the decreasing perceptual intensity of menthol in chewing gum is not due to the decreasing menthol concentration, but rather due to the declining sweetness of the chewing gum. Dalton, Doolittle, Nagata and Breslin (2000)  found that saccharine in a concentration already below the perceptive threshold strongly increases the perceptual intensity of benzaldehyde (cherry/almond aroma). And strawberry aroma is also increased by a sweet taste. Another familiar effect is also that sodium chloride and sodium glutamate make the flavour of soups and other meat products appear more intense . Most scientific publications describe the influence of a sweet or sour taste on the flavour perception [16, 21, 23, 28, 38, 50, 67, 77] that generally consists of a sweetener additive increasing the perceived intensity of an aroma, which usually occurs in food together with a sweet taste.
As the increase in the sweetness intensity is only possible with aromas which have often been perceived in combination with sweetness by the test persons, it is obvious that the increased sweetness is a learned effect . This effect is frequently referred to as learned synaesthesia . However, Auvray and Spence (2008)  take the view that this interaction only occurs, because taste and odour are transformed in the act of eating into a singular modality, the flavour.
Neurophysiological studies support these findings by showing that the gustatory and olfactory cortex do not function independently of each other, but instead are closely interlinked. The olfactory cortex is heavily influenced and sometimes even controlled by the gustatory cortex .
Another interesting observation is that especially a sweet taste can also affect the overall intensity of the flavour [23, 24, 50]. The cause for this phenomenon could lie in the fact that retronasal perceptions are localised in the oral cavity, and therefore an intensive gustatory perception also leads to a high flavour perception, even if only a few aromas reach the nose from the oral cavity .
Trigeminal perceptions and taste and aroma
The trigeminal nerve is multi-modal per se, i.e. it reacts to several stimulus groups, such as mechano-textural stimuli, heat and chemical substances (e.g. Capsaicin – heat-spiciness stimulus). The description of the triggered sensations is extremely variable and can be spicy, burning, hot, irritating, painful, etc. From a certain concentration, most odours and flavours primarily also activate the trigeminal nerve and that makes the analysis of the interactions between trigeminal stimuli and odour and taste perceptions difficult .
Trigeminally effective carbon dioxide can increase the intensity of sourness and saltiness, but not of sweetness [12, 78]. Cowart (1998)  found that carbon dioxide weakens the sweet and salty taste, and Saint-Eve et al. (2010)  added that it weakens the sweetness of a beverage and increases the acidity and the perceived aroma intensity.
The effect of the spicy substance Capsaicin on the taste perception was examined most frequently and it becomes apparent that it weakens a sweet, salty and bitter taste [31, 32]. One cause of this effect could be that the burning of Capsaicin takes away attention from the taste perception, resulting in the taste being evaluated as less intensive. In contrast, the burning of Capsaicin is intensified by sodium chloride . The interaction between trigeminal and olfactory stimuli has been examined very little. Kobal and Hummel (1988)  found that when using CO2-vanilin mixtures, the perceived intensity of the vanilla odour was reduced by CO2, however was unable to clarify the neurophysiological relationships.
Texture and aroma, texture and taste
The scientific literature on interactions between texture and aroma or taste are contradictory and the main cause for this is probably that these interactions are highly dependent on the structure and the composition of the product examined .
With regard to the interactions between the texture and the taste, usually a decrease in the perceived taste intensity with an increase in the viscosity due to the addition of thickening agents is reported [13, 26, 37]. The underlying mechanism could be that the flavourings in a more viscous matrix in the mouth are more poorly released, and are therefore perceived as less intensive, however cognitive interactions could not be excluded up until now [6, 33, 36].
Texture and odour/aroma: Some studies were unable to prove any effect of the texture on the aroma perception if the texture modification tended to be minor [37, 67]. However, in addition there are a large number of studies which illustrate that the addition of thickening agents results in a reduction in the perceived aroma intensity [4, 22, 27, 49, 76]. It was often apparent that the type of thickening agent had a major influence on the aroma-reducing effect. The reduced availability of the flavouring is also named here as a cause for the effect found. However, the chemical bonding of the flavourings and the manner in which the examined gel or food is orally processed could be causes for the reduction of the perceived aroma intensity [1, 7]. However, cognitive mechanisms with regard to the integration of texture and aroma perception are also being discussed [3, 13, 20, 75].
Up until now, the reverse effect, i.e. the effect of aroma and taste on the texture perception, were examined relatively seldom [10, 37, 57, 67] and the findings are in some cases very contradictory.
Charles et al. (2017)  showed using the example of an apple that there are not only texture-aroma, but also taste-aroma interactions. Added aroma made the perception of hardness and juiciness less dominant, and crispness and mealiness were weakened independently of the fructose concentration. Increasing concentrations of sugar, citric acid and sodium chloride can affect the orally perceived viscosity as well as the astringency [9, 25]. Bult, de Wijk and Hummel (2007)  confirmed that an increasing viscosity reduces both the orthonasally and the retronasally perceived odour intensity, however showed on the other hand that only retronasally perceived odours can intensify the perceived viscosity and creaminess of milk samples.
Food packaging and multi-sensory interactions
The sensory perception of the packaging can change the perception and evaluation of the packaged product via multi-sensory interactions . In particular the colours and shape of the haptic perception, however also the acoustic properties should be mentioned here. The sensory properties of the packaging material are above all especially effective when the food is consumed directly from the wrapping. Especially colours strongly and quickly influence the expectations and associations of the consumer, which can then influence the perception of the product itself. For the choice of colours, the fact that the triggered expectations and associations match and support the perceptions is of primary importance, as otherwise confusion and dissatisfaction occur.
The shape of the package can also significantly affect the expectations and the cross-modal associations, whereby these interactions appear to be determined by the cultural background of the persons examined . Where the shape of bottles is concerned, it must also be ensured that it is experienced as suitable for the beverage. Especially with extruded snack or crisp products, the crackling and rustling of the packaging can also affect the perceived crunchiness of the products. And associative relationships between shapes and basic tastes were also proven. Square and thin shapes are linked to bitter, salty to square shapes, sour to squareness and asymmetry, however sweet to round, symmetrical shapes with volume [70, 71].
The weight and haptics of a package can also modulate the sensory perception of the contents. Tu, Yang, and Ma (2015)  showed, for example, that the haptics of the packaging material for tea beverages can affect the sweet sensory dimension, but not sour or bitter. A beverage in a glass container was evaluated as sweeter than those in an equally heavy plastic sleeve.
Multi-modal perceptions and cross-modal interactions represent factors with the greatest influence on the product perception, especially also in the food sector. The analysis of the respective interactions of the individual sensory properties is extraordinarily complex. Examinations to date were also always only able to analyse a selection and a fraction of the interactions. In addition to the analytically measurable objective influences and interactions with regard to the product composition (intrinsic) or the point of sale or consumption (extrinsic), the affective-hedonistic influencing factors (emotions, state of health, etc.) must also be considered. In total, extraordinarily complex processes and interactions result which must be analysed and taken into account. Research on the multi-sensory interactions is only just beginning and may experience a further impetus in sensory analysis and consumer research with the use of virtual reality and the possibilities in the simulation of various sales and consumption scenarios. In addition, it is also interesting how this affects practice in the food sector. In any case, it should be noted, as experience from previous projects shows, that with regard to recipe management, especially also while taking the cross-modal interactions into account, a step-by-step modification of the product profiles is more promising with regard to the consumer acceptance than sudden recipe changes.