Programme summary
Our multilingual speech capabilities cover Indic languages, Arabic and European languages including German, French and Spanish. This study examines Qwen3-ASR adaptation for Hindi–English conversations.
Indriya completed an initial six-month programme of applied speech-model research and prototype development. One track investigated adapting Qwen3-ASR to Hindi–English conversations and specialist terminology. The work included dataset preparation, transcript normalization, baseline evaluation and supervised adaptation experiments using selected model configurations.
Adapted checkpoints were compared with their unchanged foundation-model baselines. Error analysis examined language switches, technical terms, numbers and accents. The programme also investigated whether adaptation affected recognition quality on general-language speech.
The resulting prototype supports repeatable transcription experiments and structured error analysis. Model selection, comparative findings and local deployment characteristics are undergoing internal review before pilot release.
1. The transcription problem
A Hindi–English conversation can change language within a speaker’s turn. It can also contain specialist terms, names and numbers that need to remain recognizable in the transcript. A model’s support for both languages is relevant background, but does not settle how it handles their combination in a particular recording.
Our research focused on these points of difficulty. We examined errors by category so that a change in general transcription quality would not obscure what happened to language switches or terminology. Numbers received separate attention because a small recognition error can change the meaning of a statement.
Accent variation formed another part of the investigation. Broader validation across speakers and recording environments remains ongoing.
2. Dataset preparation and normalization
The programme included preparing datasets and normalizing transcripts for adaptation and evaluation. Transcript conventions matter in mixed-language speech: spelling, punctuation and the treatment of borrowed terms can affect how two outputs are compared. A repeatable workflow needs to make those choices explicit.
AI-assisted engineering supported data inspection and preprocessing. Human review governed dataset acceptance and quality assessment.
A future comparative report should document recording sources, dataset sizes, train–evaluation splits and transcript conventions alongside the checkpoint configurations. These details make the evaluation interpretable and reproducible.
3. Baseline and adaptation experiments
The comparison paired adapted checkpoints with their unchanged foundation-model baselines. This established a reference for examining what changed after supervised adaptation.
Qwen3-ASR is the starting model family, developed by the Qwen team at Alibaba Cloud. Its upstream project documents Hindi and English support and adaptation tooling. Our experiments compare adapted checkpoints with their unchanged foundation-model baselines.
Our work builds on the existing foundation model. The adaptation workflow combines data preparation, supervised experiments, evaluation and prototype integration.
4. Error analysis and general-language checks
We examined language switches, technical terms, numbers and accents rather than treating all recognition errors as interchangeable. This helps connect a change in output to a particular difficulty in the recording. It also makes difficult segments easier to review.
The programme investigated whether adaptation changed recognition quality on general-language speech. A task-specific intervention needs that wider check: the intended setting and ordinary speech can respond differently to an adaptation.
The prototype supports structured analysis and repeatable transcription experiments. The next public results report should identify the evaluation material, normalization rules, exact baseline and adapted configurations, error measures and test conditions. Those details will let readers assess where an adaptation helps and where it does not.
5. Training and local inference preparation
The programme investigated a deployment approach using AWS GPU infrastructure for training and controlled evaluation, with Apple Silicon systems supporting local inference experiments. Preparation included checkpoint conversion, runtime compatibility, memory profiling and assessment of accuracy after compression.
These activities examine whether a research artifact can be used under practical operating conditions. Conversion and compression require their own quality checks; a converted checkpoint should not be assumed to preserve the original output simply because it loads.
Pilot preparation connects quality measurements to the selected hardware, runtime and installation procedure.
6. Assets and pilot preparation
The initial programme produced prototype implementations, data-processing workflows, baseline configurations, evaluation procedures and an internal technical report. AI-assisted and agentic workflows supported experiment execution, evaluation-script development, error categorization and documentation, with human oversight.
Selected prototypes are being hardened for pilot deployment. Preparation focuses on reproducibility, installation, monitoring and performance under representative operating conditions. Customer-specific adaptation and broader validation remain ongoing.
Related vision and speech model work
Alongside the Qwen3-ASR study, ongoing research includes NVIDIA speech models, Nemotron 3 Nano Omni, Google Gemma and Apple AFM 3. Their roles differ: transcription, multimodal understanding and speech synthesis require separate comparisons. A model family’s language coverage is not evidence of accuracy on our mixed-language material.
The cloud Mac work examines available inference paths. We evaluate checkpoint adaptation for open-weight models and application integration through Apple’s framework.
References and publication status
References checked on 14 October 2026.