This thesis investigates the potential of hypergraphs for capturing higher-order relations between objects in a multimodal dataset. These relations are often sub-optimally represented by pairwise connections used in a graph. Hence, in order to unlock the full potential of relational information within a multimodal dataset, this thesis proposes several geometric deep learning approaches for capturing and learning higher-order relations.