Being able to better understand and measure what is happening in the world is of great importance for a range of stakeholders, including policy makers. The recent explosion in the availability of data documenting our collective behaviour offers new opportunities to gain insights into our society.

Here, we focus on a series of case studies to demonstrate how new forms of data may be used to help us better understand human behaviour.

Data coming from financial transactions taking place in the stock market can help us better understand financial crises. We analyse a dataset comprising the stocks forming the Dow Jones Industrial Average at a second by second resolution. We investigate changes in stock market prices and how they arise at different time scales, showing a transition between power law and exponential decay in the tails of the distribution of logarithmic returns.

Accurate and quick estimates of the size of a crowd are crucial for the avoidance of crowd disasters. However, existing approaches rely on human judgement and can be slow and costly. Our findings suggest that data from mobile phone networks and social media platforms may allow us to estimate the size of a crowd. Such data could potentially be accessed in real time, leading to shorter delays than those experienced with previous approaches to crowd size estimation.

We also show how communities on a network constructed from our social interactions through smartphones capture the temporal evolution of our behaviour in everyday life.

The complex datasets presented here also require complex methodologies to analyse them. Complexity science, and more specifically network science, has witnessed increasing attention within the scientific community in the last two decades. Here, we will present a new technique to analyse a common feature of many real world complex networks, namely community structure. We show how our methodology addresses many of the drawbacks of current techniques, and we also introduce an efficient algorithm which outperforms analogous methods on a set of standard benchmark networks.

Our findings suggest that the analysis of large complex social datasets coupled with methodological advances can allow us to gain valuable measurements of human behaviour.