Models

How do we use models to find species by their sound?

Mosquitoes communicate with the sound of their wingbeat. In many species whilst mating, males and females try to match each other's tone and if they are not able to do so (sometimes the female is considerably bigger than the male) they will meet at a common harmonic. We can use this characteristic sound and its components, for example its strongest frequency and its temporal signature, to recognise the presence of a mosquito and determine to which species it belongs.

^{Figure 1: A visual representation of a clearly audible mosquito in flight for 5 seconds. The audio wave file is given in the first row. The second row is a short-time Fourier transform spectrogram, which shows how the magnitude of frequency components vary over time. If we think of the buzz of a mosquito as a musical note, we can show that this particular mosquito keeps a near-constant pitch of F# in the bottom row.}

In order to automate this process, we have developed a set of machine learning algorithms that take a mosquito recording, analyse its features (such as those shown in Fig 1.) and use these to classify the signal, determining the most likely species that may have generated it. Many challenges lie in the detection and classification of these sounds, in particular the fact that mosquitoes are inherently quiet and that the characteristics of their sounds vary considerably with the size of the individual, its age and the temperature of the environment.  For more detailed descriptions of our models and methods used, please see our publications.

Appropriate algorithms can then be deployed in a cheap, portable device that can record sounds and analyse them in real time. Smartphones are ideal platforms because they are equipped with a wide range of sensors, as well as being internet-connected, which enables easier data transfer. To learn more about how we deployed smartphones, including the models and algorithms used, please refer to our paper Mosquito detection with low-cost smartphones: data acquisition for malaria research. An overview of how we record the sound of mosquitoes can also be found on our website (Sound).