The project aims to design and build a methodology to characterize food consumption behaviour of users in Instagram, which will generate actionable knowledge. The goals of the project are:
1. Large-scale joint processing of food-related hashtags and visual content using machine learning. The work in this goal will provide systematic insights on what social food in Instagram is about, beyond the sole use of hashtags as done in most previous work. The novelty comes from the use of deep learning methods applied on images as a rich data source to filter, discriminate, disambiguate, and enrich food posts in addition to text, as well as from the study of the interplay between multiple hashtags and visual features.
Outcomes: A pipeline that includes a data-driven vocabulary of food-related categories, and a module to categorize Instagram images according to the vocabulary.
2. Discovery of food-related user contextual patterns. The work in this goal is to understand thematic, temporal, and spatial patterns of how users generate food-related posts in everyday life. The novelty comes from the fact that previous work has not focused on individual-level patterns, but rather on population aggregates at county or state levels. Using longitudinal data and unsupervised machine learning methods, the proposed work first includes a systematic analysis of three contextual cues: time (when posts are created), topic (what specific food-related categories are posted, including dependencies among hashtags and visual food categories), and place (what place types are chosen to create posts). The results of this analysis will then be used to automatically identify data-driven, food-related user categories that correspond to coherent patterns along the three contextual dimensions.
Outcomes: A pipeline for data-driven user categorization based on food-related contextual patterns involving thematic, temporal, and spatial patterns.
3. Cross-country comparative study. The methodology proposed in the first two goals will be applied on Instagram data from three different countries to demonstrate the generality of the approach. The novelty of this work comes from the insights that will be generated through the understanding of country variations, both with respect to the generated content (visual and textual) and to emerging human behaviour (with respect to temporal, thematic, or place variations of food consumption habits.) The countries include a combination of Europe (Switzerland, UK, or France) and North America (US or Mexico) to be decided at the onset of the work.
Outcomes: A comparative study of the differences with respect to food-related categories and user behaviour using Instagram data from three countries and the methods developed in Goals 1 and 2.