Towards Enabling Multilingual Multitask SpeechLLMs in Data-Scarce Settings

Speech-based Large Language Models (SpeechLLMs) are moving towards supporting multiple languages and tasks. However, prior work often studies multilinguality and multitask learning separately or use large training corpora. We present a study of multilingual multitask SpeechLLMs under low-resource conditions, extending beyond ASR to speech translation and topic classification. Building on evidence that high-resource ASR pretraining enables cross-lingual transfer, we examine whether this generalizes to multitask SpeechLLMs. Using only 3-5 hours of labeled data per task and language, we show that bootstrapping from ASR-pretrained projectors enables effective multitask adaptation. We find that cross-lingual performance is influenced by language similarity, while zero-shot generalization does not extend to unseen tasks. Overall, our results demonstrate that ASR-pretrained bootstrapping and limited target-language finetuning enable multilingual multitask SpeechLLMs in low-resource settings.