In this thesis, we introduce new approaches for training better machine learning models for computer vision tasks in the absence of large labeled datasets. Our approach is based on equipping neural networks with the notion of symmetry for the sake of better learning real-world constraints without observing all their realizations in the training data. We start with explicit mathematical structures such as the scale group. We introduce Scale-Equivariant Steerable Networks, a class of convolutional neural networks that are equipped with an extra notion of scale variations. We present a theory and an effective implementation for these networks. Then we consider a wider group of non-affine transformations. And finally, we demonstrate that models can learn structures themselves by assuming symmetries in the input data.