Speech

Groq API is the fastest speech-to-text solution available, offering OpenAI-compatible endpoints that enable real-time transcriptions and translations. With Groq API, you can integrate high-quality audio processing into your applications at speeds that rival human interaction.

API Endpoints

We support two endpoints:

Supported Models

Which Whisper Model Should You Use?

Having more choices is great, but let's try to avoid decision paralysis by breaking down the tradeoffs between models to find the one most suitable for your applications:

• If your application is error-sensitive and requires multilingual support, use whisper-large-v3.
• If your application is less sensitive to errors and requires English only, use distil-whisper-large-v3-en.
• If your application requires multilingual support and you need the best price for performance, use whisper-large-v3-turbo.

The following table breaks down the metrics for each model.

Working with Audio Files

Audio Preprocessing

Our speech-to-text models will downsample audio to 16KHz mono before transcribing, which is optimal for speech recognition. This preprocessing can be performed client-side if your original file is extremely large and you want to make it smaller without a loss in quality (without chunking, Groq API speech endpoints accept up to 25MB). We recommend FLAC for lossless compression.

The following ffmpeg command can be used to reduce file size:

ffmpeg \
  -i <your file> \
  -ar 16000 \
  -ac 1 \
  -map 0:a \
  -c:a flac \
  <output file name>.flac

Working with Larger Audio Files

For audio files that exceed our size limits or require more precise control over transcription, we recommend implementing audio chunking. This process involves:

• Breaking the audio into smaller, overlapping segments
• Processing each segment independently
• Combining the results while handling overlapping

To learn more about this process and get code for your own implementation, see the complete audio chunking tutorial in our Groq API Cookbook.

Using the API

The following are optional request parameters you can use in your transcription and translation requests:

Example Usage of Transcription Endpoint

The transcription endpoint allows you to transcribe spoken words in audio or video files.

pip install groq

1import os
2from groq import Groq
3
4# Initialize the Groq client
5client = Groq()
6
7# Specify the path to the audio file
8filename = os.path.dirname(__file__) + "/sample_audio.m4a" # Replace with your audio file!
9
10# Open the audio file
11with open(filename, "rb") as file:
12    # Create a transcription of the audio file
13    transcription = client.audio.transcriptions.create(
14      file=(filename, file.read()), # Required audio file
15      model="whisper-large-v3-turbo", # Required model to use for transcription
16      prompt="Specify context or spelling",  # Optional
17      response_format="json",  # Optional
18      language="en",  # Optional
19      temperature=0.0  # Optional
20    )
21    # Print the transcription text
22    print(transcription.text)

Example Usage of Translation Endpoint

The translation endpoint allows you to translate spoken words in audio or video files to English.

pip install groq

1import os
2from groq import Groq
3
4# Initialize the Groq client
5client = Groq()
6
7# Specify the path to the audio file
8filename = os.path.dirname(__file__) + "/sample_audio.m4a" # Replace with your audio file!
9
10# Open the audio file
11with open(filename, "rb") as file:
12    # Create a translation of the audio file
13    translation = client.audio.translations.create(
14      file=(filename, file.read()), # Required audio file
15      model="whisper-large-v3", # Required model to use for translation
16      prompt="Specify context or spelling",  # Optional
17      response_format="json",  # Optional
18      temperature=0.0  # Optional
19    )
20    # Print the translation text
21    print(translation.text)

Understanding Metadata Fields

When working with Groq API, setting response_format to verbose_json outputs each segment of transcribed text with valuable metadata that helps us understand the quality and characteristics of our transcription, including avg_logprob, compression_ratio, and no_speech_prob.

This information can help us with debugging any transcription issues. Let's examine what this metadata tells us using a real example:

{
  "id": 8,
  "seek": 3000,
  "start": 43.92,
  "end": 50.16,
  "text": " document that the functional specification that you started to read through that isn't just the",
  "tokens": [51061, 4166, 300, 264, 11745, 31256],
  "temperature": 0,
  "avg_logprob": -0.097569615,
  "compression_ratio": 1.6637554,
  "no_speech_prob": 0.012814695
}

As shown in the above example, we receive timing information as well as quality indicators. Let's gain a better understanding of what each field means:

• id:8: The 9th segment in the transcription (counting begins at 0)
• seek: Indicates where in the audio file this segment begins (3000 in this case)
• start and end timestamps: Tell us exactly when this segment occurs in the audio (43.92 to 50.16 seconds in our example)
• avg_logprob (Average Log Probability): -0.097569615 in our example indicates very high confidence. Values closer to 0 suggest better confidence, while more negative values (like -0.5 or lower) might indicate transcription issues.
• no_speech_prob (No Speech Probability): 0.0.012814695 is very low, suggesting this is definitely speech. Higher values (closer to 1) would indicate potential silence or non-speech audio.
• compression_ratio: 1.6637554 is a healthy value, indicating normal speech patterns. Unusual values (very high or low) might suggest issues with speech clarity or word boundaries.

Using Metadata for Debugging

When troubleshooting transcription issues, look for these patterns:

• Low Confidence Sections: If avg_logprob drops significantly (becomes more negative), check for background noise, multiple speakers talking simultaneously, unclear pronunciation, and strong accents. Consider cleaning up the audio in these sections or adjusting chunk sizes around problematic chunk boundaries.
• Non-Speech Detection: High no_speech_prob values might indicate silence periods that could be trimmed, background music or noise, or non-verbal sounds being misinterpreted as speech. Consider noise reduction when preprocessing.
• Unusual Speech Patterns: Unexpected compression_ratio values can reveal stuttering or word repetition, speaker talking unusually fast or slow, or audio quality issues affecting word separation.

Quality Thresholds and Regular Monitoring

We recommend setting acceptable ranges for each metadata value we reviewed above and flagging segments that fall outside these ranges to be able to identify and adjust preprocessing or chunking strategies for flagged sections.

By understanding and monitoring these metadata values, you can significantly improve your transcription quality and quickly identify potential issues in your audio processing pipeline.

We can't wait to see what you build! 🚀