# AnySpeechRecognizer: Features, Models and How to Use It

By Schnalz.net · 29 October 2026

https://schnalz.net/ai-guides/anyspeechrecognizer-features-models-and-how-to-use-it

> AnySpeechRecognizer is an open Swift package that brings third-party speech recognition models and speaker diarization to Apple platforms using Core ML. It mirrors Apple's Speech framework while letting developers swap in models such as NVIDIA Parakeet and Alibaba Qwen3-ASR.

## In short

- It implements Apple's SpeechAnalyzer API while supporting open local models on Core ML.
- Supported speech engines include NVIDIA Parakeet variants and Alibaba Qwen3-ASR.
- Built-in speaker diarization uses NVIDIA Streaming Sortformer to track up to four speakers.
- Models download directly from Hugging Face Hub into the standard local cache.

## What is AnySpeechRecognizer?

AnySpeechRecognizer is a Swift package developed by Hugging Face that implements Apple's speech-to-text API while allowing developers to choose alternative recognition engines. It mirrors the SpeechAnalyzer framework introduced in macOS 26 and iOS 26, using Apple's default engine or local models running on Core ML.

The package also introduces speaker diarization, which tracks who spoke and when. By running open models on device, it lets developers transcribe audio without sending data to external cloud services.

**AnySpeechRecognizer at a glance**

- **Developer**: Hugging Face
- **Target platforms**: macOS 26+, iOS 26+
- **Speech models**: Apple Speech, NVIDIA Parakeet, Alibaba Qwen3-ASR
- **Diarization model**: NVIDIA Streaming Sortformer (v2, v2.1)
- **Runtime engine**: Core ML and GPU
- **Package language**: Swift

## Supported speech recognition models

AnySpeechRecognizer supports multiple open speech-to-text models converted to Core ML format and hosted on the Hugging Face Hub. Developers can select models based on language requirements, latency needs, and available storage.

NVIDIA Parakeet v3 supports 25 European languages with automatic language detection, volatile live results, word timing, and word confidence scores. Moondream post-trained the Parakeet Ultra variant for higher accuracy, while Parakeet Unified EN targets English only. Parakeet models download around 600 MB plus an optional 590 MB encoder for live streams.

Alibaba Qwen3-ASR comes in 0.6B and 1.7B parameters, requiring 0.9 GB to 2.4 GB of storage. It supports 30 languages and 22 Chinese dialects, including Cantonese, Mandarin, Japanese, and Korean, but it runs on the GPU and processes files in chunks up to 40 seconds without word timings or volatile streaming results.

**Comparison of models supported in AnySpeechRecognizer**

| Model | Sizes | Language coverage | Live streaming features | Download size |
| --- | --- | --- | --- | --- |
| Parakeet TDT v3 | 0.6B | 25 European languages | Word timing, confidence, volatile results | ~600 MB (+590 MB live encoder) |
| Parakeet Ultra | 0.6B | 25 European languages | Word timing, confidence, volatile results | ~600 MB (+590 MB live encoder) |
| Parakeet Unified EN | 0.6B | English only | Word timing, confidence, volatile results | ~600 MB (+590 MB live encoder) |
| Qwen3-ASR | 0.6B, 1.7B | 30 languages and 22 Chinese dialects | Single final result per 40s segment | 0.9 GB to 2.4 GB |
| Streaming Sortformer | Not stated | Not stated | Up to 4 concurrent speakers | ~250 MB |

## Speaker diarization with Sortformer

AnySpeechRecognizer adds speaker diarization to Apple's API, an ability missing from the default Speech framework. It integrates NVIDIA's Streaming Sortformer model to detect who is speaking during a conversation.

The diarizer tracks up to four distinct speakers simultaneously, even when their voices overlap. A component named SpeakerTimeline aligns diarizer output with transcriber word timestamps, allowing apps to print formatted transcripts segmented by speaker turn.

The package uses Sortformer v2 by default and includes Sortformer v2.1 as an option. Hugging Face notes that Sortformer v2.1 was trained on additional meeting audio and delivered higher accuracy in meeting tests.

## How to set up and use AnySpeechRecognizer

AnySpeechRecognizer is designed as a drop-in replacement for Apple's SpeechAnalyzer framework in Swift. Developers can replace Apple's native engine by updating framework imports and initializing custom model configurations.

1. Add the AnySpeechRecognizer package to your Swift project and import both Speech and AnySpeechRecognizer in your source files.
2. Initialize SpeechTranscriber with your chosen model configuration, such as ParakeetSpeechModel(variant: .v3) or QwenSpeechModel.
3. Optionally initialize SpeakerDiarizer using SortformerDiarizationModel() to track speaker turns.
4. Call AssetInventory.assetInstallationRequest with your modules and await downloadAndInstall() to download the Core ML model weights from the Hugging Face Hub.
5. Initialize SpeechAnalyzer with your transcriber and diarizer modules, then pass your audio file or live audio stream buffer to receive transcription sequences.

> **Tip**: Mac apps running outside the App Store sandbox store downloaded weights in the standard Hugging Face cache folder, allowing models to be shared directly with local Python scripts.

## Limits and system requirements

AnySpeechRecognizer requires Apple operating system versions that support SpeechAnalyzer, specifically macOS 26 or iOS 26 and later. The package was benchmarked by Hugging Face on macOS 27.0.1 using an M4 Max Mac Studio.

Hardware demands vary across models. Qwen3-ASR executes on the GPU and is significantly slower than Parakeet, making it less practical for real-time live microphone transcription. It also lacks support for partial volatile results, word-level timing, and confidence scores.

Parakeet offers faster processing but requires an extra 8-second live encoder module to reduce live latency. Hugging Face reports that while this live encoder cuts analysis time by approximately one-third, it results in a slightly higher word error rate.

## Frequently asked questions

### What is AnySpeechRecognizer?

AnySpeechRecognizer is an open Swift library created by Hugging Face. It implements Apple's SpeechAnalyzer API while allowing developers to run open Core ML models like Parakeet and Qwen3-ASR on device.

### Which languages does AnySpeechRecognizer support?

Language support depends on the chosen engine. NVIDIA Parakeet handles 25 European languages, while Alibaba Qwen3-ASR supports 30 languages and 22 Chinese dialects, including Cantonese, Mandarin, Japanese, and Korean.

### Does AnySpeechRecognizer include speaker diarization?

Yes, AnySpeechRecognizer includes a SpeakerDiarizer module powered by NVIDIA Streaming Sortformer. It can differentiate and track up to four speakers simultaneously on the same timeline as the transcript.

### Can I add custom speech models to AnySpeechRecognizer?

Yes, developers can connect custom speech engines to AnySpeechRecognizer. The package allows this by implementing the SpeechRecognitionModel protocol for configuration and the SpeechRecognitionExecutor protocol for session execution.

### Where are AnySpeechRecognizer model files stored?

Model assets for AnySpeechRecognizer are downloaded on demand from Hugging Face Hub repositories. On macOS without application sandboxing, files are stored in the standard Hugging Face cache directory.

## Sources

- [Hugging Face: Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms](https://huggingface.co/blog/anydecisionmodel)