Representation learning is fundamental to many machine learning techniques, perhaps even more so in the subfield of deep learning. Disentangled representations introduce a form of interpretability such that both humans and our models can understand (to a degree) how decisions are made – or at least, what is important to such decisions. Symmetry based disentangled representations introduce a stricter form of interpretability which ensures the representations are structured based on how the data is observed, which is described by symmetries acting on it. In this work we explore two aspects of symmetry based representations. First we consider methods to learn such representations with a particular focus on doing so without action labelling. Secondly we consider their structure and potential benefits for downstream tasks. We find that through the use of policy gradients, we can successfully learn linear disentangled representation without knowledge of world states or action labels. Indeed our proposed method achieves similar performance to supervised methods. Subsequently, we find that linear disentangled representations are highly structured with respect to a particular symmetry group. This structure allows for better performance than standard disentangled representations on both the tasks of generative factor prediction and observed action prediction.