To potentially address this issue, we could:

• Have efficient models to quickly assess the language that the user is speaking so that the right language is selected for the speech-to-text model.
• Have a better way to handle unsupported languages.

And that’s exactly what we do in our paper!

We present two sets of models, based on TC-ResNets and ECAPA-TDNNs: TC-ResNet10, TC-ResNet14, and LECAPAT (Light ECAPA-Tdnn). These models are orders of magnitude smaller than baselines! Check out this plot, and note the logarithmic scale on the y axis:

When assessed with the VoxLingua107 dataset, these models can be comparable in performance to models that have orders of magnitude more parameters that only address the task of Spoken Language Recognition. In the following Figure, we plot the error rates against the model parameters (so that we ideally want models in the bottom left corner). Our proposed LECAPAT performs almost equally than ECAPA-TDNN, with two orders of magnitude less parameters!

Moreover, the proposed models are fast! Here’s a plot of their real-time factor, the higher the better:

Finally, we present a multilabel approach during training that, without adding any sort of complexity to the model, it allows to handle unsupported classes better, so that the problem motivated in the video will no longer occur! Here the confusion matrix of a standard model (using multiclass) on the left, and our proposed multilabel strategy on the right:

The final row and columns represent the “other” class. We can see how there are so many false positives on the left matrix (e.g., Norwegian is classified almost 30% of times as “other”, and Portugeuese is always classified as “other”. On the other hand, with our proposed multilabel strategy (right), Portuguese is always correctly classified, and Norwegian is only classified as “other” 10% of times.)

And that’s it! This is, to the best of our knowledge, the first paper to address efficiency in the task of Speaker Language Recognition. We hope this work motivates the usage of such tiny models so that our little assistants will no longer get confused when we speak to them in other languages rather than English, without having to manually configure them! Making the world a better place, one step at a time, titans!