Consumption as a Rhythm

Through Data Visualisation and Sonification models, we present a study of multimodal representations to characterise the Portuguese consumption patterns, gathered from Portuguese hypermarkets and supermarkets over the course of two years. We focus on the rhythmic nature of this data to create and discuss audio and visual representations that highlight disruptions and sudden changes in the normal consumption patterns. For this study, we present two distinct visual and audio representations.

Data

The dataset for this project is concerned with the consumption values in 729 Portuguese supermarkets and hypermarkets of SONAE’s chains. We analysed all the transactions made on these supermarkets and hypermarkets from May 2012 to April 2014. For this work, we aggregate all the purchases in nine distinct categories: Grocery; Alcohol & Sweets; Healthcare; Beauty; Clothes; Furniture; House Care; Culture & Leisure; Pets & Nature Care. Additionally, we grouped those nine categories into groups of three by their type of consumption: essential (Grocery, Health Care, and Clothes); non-essential (Alcohol & Sweets, Beauty, and Culture & Leisure); and unknown (Furniture, House Care, and Pets & Nature Care).

First Approach

**Figure 1**
Screenshot of the first visualisation layout.

**Figure 2**
Representation of each category (shape) and their types (colour): (a) Clothes, essential; (b) Health Care, es- sential; (c) Grocery, essential; (d) Culture & Leisure, non- essential; (e) Beauty, non-essential; (f) Alcohol & Sweets, non-essential; (g) Pets & Nature Care, unknown; (h) House Care, unknown; (i) Furniture, unknown.

For the first visual representation (Fig.1), our main goal was to represent the different categories and to visually perceive their behaviour over the two years. For the representation of each category, we decided to normalise each consumption value by category independently. This enabled us to represent the peaks of consumption in each category, instead of enabling the comparison between them. Each category is represented by a different circular shape, which is coloured depending on the type of consumption it represents: red, for non-essential, green for essential, and grey for unknown (Fig. 2). As the days go by, each shape grows or decreases in size depending on the consumption value.

**Figure 3**
Basic Taiko drums beat for Representation #1.

The first sonification has a time signature of 7/4 and each pulse represents a day of the week. Figure 3 presents the basic beat defined for this sonification: Taiko Drums Big 1 corresponds to essential goods, Taiko Drums Big 2 corresponds to non-essential goods, whereas the transactions from the remaining categories are associated with an ensemble of four Taiko drums. As Fig. 3 suggests this sonification focuses on the transactions of essential and non-essential goods. To accentuate even more the changes in consumption patterns, we added other instruments to the sonification: the transactions of essential goods are also associated with thunder drums; an orchestral snare drum is used for transactions of non- essential goods. The remaining goods are not associated with any other instrument.

Second Approach

**Figure 4**
Screenshot of the second visualisation layout.

For the second visual representation (Fig. 4), our main goal was to highlight the differences between types of consumption. We summed all the categories in each type of consumption and normalised those values by the maximum value of all three types. To distinguish the different types of consumption, we used three colours: green, red and grey to represent, respectively, the essential, non-essential, and unknown consumptions. Each type is represented by a coloured circle. The size of the circle varies depending on the consumption value of each type of consumption on a given day. To emphasise the different consumption values, we also distance each circle from the centre of the canvas. The lower the summed consumption value, the closer the circle is to the centre of the canvas.

As time passes, the circles change position in an invisible circle. Each lap represents a month, and we visually divided the lap into 5 slices (as can be seen in Fig. 4), representing 5 distinct weeks. At the beginning of each month, the circles start in the upper side of the circle (in the first slice). Each slice is further divided in the seven days of the week.

**Figure 5**
Basic Taiko drums beat for Representation #2.

The second sonification, which was designed to focus on the transactions of non-essential and other goods, has a time signature of 7/8. The drums beat for this sonification is depicted in Fig. 5: Taiko Drums Big 1 corresponds to non-essential goods, Taiko Drums Big 2 corresponds to essential goods, and the transactions from the remaining categories are associated with a group of two Taiko drums. Other percussion instruments were added to the model: essential goods are associated with timpani, non-essential goods are associated with thunder-drums, other goods are associated with chimes, which are played at every beat. Chimes were introduced to represent the consumption patterns in terms of pitch, using the same strategy as the one that we defined for the gain of the instruments.

Discussion

The visual artefacts can arguably be regarded as more informative, as they translate consumption to a finer detail. Nevertheless, the proposed multimodality was created with the intent of becoming a richer experience in terms of understanding. The auditory counterpart contributes to a better comprehension of consumption peaks over time. While the sonification model was designed to not consider all types per pulse, the final representations become sufficiently informative as the result of complementing sound with the visual artefacts.

Publication

C. Maçãs, P. Martins, and P. Machado, “Consumption as a Rhythm: A Multimodal Experiment on the Representation of Time-Series,” in 22nd International Conference Information Visualisation, IV 2018, Fisciano, Italy, July 10-13, 2018, 2018, pp. 504-509.
- Bibtex
- |
- Link
- |
- PDF

Author

Pedro Martins