Traffic congestion in urban areas is a problem forthe environment and the economy. One solution to minimizecongestion is optimizing traffic lights. Traffic signal control isa challenging problem due to the complex traffic flow patterns.Conventional traffic control use pre-coded cycle pattern plans,which suffer from adapting to the complex flow dynamics.Reinforcement Learning allows for dynamic control but is unableto properly catch temporal-feature due to the Markov property.To solve this, recent papers propose incorporating predictionmodules into Reinforcement Learning control, however, thissuffers from additional loss and generalization.To circumvent these issues, we propose Recurrent Light(ReLight), which treats the environment as a Partially ObservableMarkov Decision Process which depends on the history ofprevious belief states. We utilize this dependency to capturespatial-temporal features and utilize an LSTM in the DQNnetwork to capture important long-short term features throughhidden states. To properly capture cycle phases, we propose twosampling and two training strategies. In our experiments, wedemonstrate that ReLight outperforms state-of-the-art modelson one, multi and city-wide datasets.