Groq
Login

Structured Outputs

Guarantee model responses strictly conform to your JSON schema for reliable, type-safe data structures.

Introduction

Structured Outputs is a feature that ensures your model responses conform to your provided JSON Schema. The feature offers two modes with different guarantees and requirements:

Best-effort Mode (strict: false)

With strict: false (the default behavior), the model attempts to match your schema but without hard constraints:

  • Valid JSON, but schema adherence not guaranteed - May produce valid JSON that does not match your schema (for example, wrong field types or missing/extra fields)
  • Possible errors and malformed output - Can sometimes produce malformed JSON syntax or trigger 400 errors due to schema validation failures
  • Fewer requirements - More flexible schema constraints, such as optional fields
  • Broader model support - Available on all models that support Structured Outputs

This mode is suitable when you need structured outputs but can handle occasional validation errors with retry logic.

Example usage:

JSON
{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "schema_name",
      "strict": false,  // or omit this field (defaults to false)
      "schema": { ... }
    }
  }
}

Key benefits of Structured Outputs:

  1. Type-safe responses: Reduce validation and retry logic for malformed outputs
  2. Programmatic refusal detection: Detect safety-based model refusals programmatically
  3. Simplified prompting: Less complex prompts needed for consistent formatting

In addition to supporting Structured Outputs in our API, our SDKs also enable you to easily define your schemas with Pydantic and Zod to ensure further type safety. The examples below show how to extract structured information from unstructured text.

Supported models

Structured Outputs is available in two modes: strict: true (with constrained decoding) and strict: false (default, best-effort validation).

Models with Best-effort Mode (strict: false)

The following models support Structured Outputs with strict: false (default), which attempts schema compliance but may occasionally error:

Model IDModel
openai/gpt-oss-20b
GPT-OSS 20B
openai/gpt-oss-120b
GPT-OSS 120B
openai/gpt-oss-safeguard-20b
Safety GPT OSS 20B
moonshotai/kimi-k2-instruct-0905
Kimi K2 Instruct
meta-llama/llama-4-maverick-17b-128e-instruct
Llama 4 Maverick
meta-llama/llama-4-scout-17b-16e-instruct
Llama 4 Scout

For all other models, you can use JSON Object Mode to get a valid JSON object, though it may not match your schema.

Getting a structured response from unstructured text

from groq import Groq
import json

groq = Groq()

response = groq.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[
        {"role": "system", "content": "Extract product review information from the text."},
        {
            "role": "user",
            "content": "I bought the UltraSound Headphones last week and I'm really impressed! The noise cancellation is amazing and the battery lasts all day. Sound quality is crisp and clear. I'd give it 4.5 out of 5 stars.",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "product_review",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "product_name": {"type": "string"},
                    "rating": {"type": "number"},
                    "sentiment": {
                        "type": "string",
                        "enum": ["positive", "negative", "neutral"]
                    },
                    "key_features": {
                        "type": "array",
                        "items": {"type": "string"}
                    }
                },
                "required": ["product_name", "rating", "sentiment", "key_features"],
                "additionalProperties": False
            }
        }
    }
)

result = json.loads(response.choices[0].message.content or "{}")
print(json.dumps(result, indent=2))
Example Output
JSON
{
  product_name: 'UltraSound Headphones',
  rating: 4.5,
  sentiment: 'positive',
  key_features: [
    'amazing noise cancellation',
    'all-day battery life',
    'crisp and clear sound quality'
  ]
}

Choosing Between Strict and Best-effort Mode

Strict Mode (strict: true)Best-effort Mode (strict: false)
Schema adherenceGuaranteed - uses constrained decodingBest-effort - generally compliant
Error handlingNever produces invalid JSONMay occasionally 400 errors or produce syntactically valid but schema-invalid JSON
RequirementsAll fields must be required
additionalProperties: false required		More flexible constraints allowed
Model supportLimited (GPT-OSS 20B, 120B)All Structured Outputs models
When to useProduction apps requiring 100% reliabilityDevelopment, prototyping, or when using unsupported models

Recommendation: Use Strict Mode (strict: true) when available for production applications. Fall back to Best-effort Mode (strict: false) for broader model support or during development.

Examples

SQL Query Generation

You can generate structured SQL queries from natural language descriptions, helping ensure proper syntax and including metadata about the query structure.


from groq import Groq
from pydantic import BaseModel
import json

client = Groq()

class ValidationStatus(BaseModel):
    is_valid: bool
    syntax_errors: list[str]

class SQLQueryGeneration(BaseModel):
    query: str
    query_type: str
    tables_used: list[str]
    estimated_complexity: str
    execution_notes: list[str]
    validation_status: ValidationStatus

response = client.chat.completions.create(
    model="moonshotai/kimi-k2-instruct-0905",
    messages=[
        {
            "role": "system",
            "content": "You are a SQL expert. Generate structured SQL queries from natural language descriptions with proper syntax validation and metadata.",
        },
        {"role": "user", "content": "Find all customers who made orders over $500 in the last 30 days, show their name, email, and total order amount"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "sql_query_generation",
            "schema": SQLQueryGeneration.model_json_schema()
        }
    }
)

sql_query_generation = SQLQueryGeneration.model_validate(json.loads(response.choices[0].message.content))
print(json.dumps(sql_query_generation.model_dump(), indent=2))
Example Output
JSON
{
  "query": "SELECT c.name, c.email, SUM(o.total_amount) as total_order_amount FROM customers c JOIN orders o ON c.customer_id = o.customer_id WHERE o.order_date >= DATE_SUB(NOW(), INTERVAL 30 DAY) AND o.total_amount > 500 GROUP BY c.customer_id, c.name, c.email ORDER BY total_order_amount DESC",
  "query_type": "SELECT",
  "tables_used": ["customers", "orders"],
  "estimated_complexity": "medium",
  "execution_notes": [
    "Query uses JOIN to connect customers and orders tables",
    "DATE_SUB function calculates 30 days ago from current date",
    "GROUP BY aggregates orders per customer",
    "Results ordered by total order amount descending"
  ],
  "validation_status": {
    "is_valid": true,
    "syntax_errors": []
  }
}

Schema Validation Libraries

When working with Structured Outputs, you can use popular schema validation libraries like Zod for TypeScript and Pydantic for Python. These libraries provide type safety, runtime validation, and seamless integration with JSON Schema generation.

Support Ticket Classification

This example demonstrates how to classify customer support tickets using structured schemas with both Zod and Pydantic, ensuring consistent categorization and routing.

TypeScript
import Groq from "groq-sdk";
import { z } from "zod";

const groq = new Groq();

const supportTicketSchema = z.object({
  category: z.enum(["api", "billing", "account", "bug", "feature_request", "integration", "security", "performance"]),
  priority: z.enum(["low", "medium", "high", "critical"]),
  urgency_score: z.number(),
  customer_info: z.object({
    name: z.string(),
    company: z.string().optional(),
    tier: z.enum(["free", "paid", "enterprise", "trial"])
  }),
  technical_details: z.array(z.object({
    component: z.string(),
    error_code: z.string().optional(),
    description: z.string()
  })),
  keywords: z.array(z.string()),
  requires_escalation: z.boolean(),
  estimated_resolution_hours: z.number(),
  follow_up_date: z.string().datetime().optional(),
  summary: z.string()
});

type SupportTicket = z.infer<typeof supportTicketSchema>;

const response = await groq.chat.completions.create({
  model: "moonshotai/kimi-k2-instruct-0905",
  messages: [
    {
      role: "system",
      content: `You are a customer support ticket classifier for SaaS companies. 
                Analyze support tickets and categorize them for efficient routing and resolution.
                Output JSON only using the schema provided.`,
    },
    { 
      role: "user", 
      content: `Hello! I love your product and have been using it for 6 months. 
                I was wondering if you could add a dark mode feature to the dashboard? 
                Many of our team members work late hours and would really appreciate this. 
                Also, it would be great to have keyboard shortcuts for common actions. 
                Not urgent, but would be a nice enhancement! 
                Best, Mike from StartupXYZ`
    },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "support_ticket_classification",
      schema: z.toJSONSchema(supportTicketSchema)
    }
  }
});

const rawResult = JSON.parse(response.choices[0].message.content || "{}");
const result = supportTicketSchema.parse(rawResult);
console.log(result);
Example Output
JSON
{
  "category": "feature_request",
  "priority": "low",
  "urgency_score": 2.5,
  "customer_info": {
      "name": "Mike",
      "company": "StartupXYZ",
      "tier": "paid"
  },
  "technical_details": [
      {
          "component": "dashboard",
          "description": "Request for dark mode feature"
      },
      {
          "component": "user_interface",
          "description": "Request for keyboard shortcuts"
      }
  ],
  "keywords": ["dark mode", "dashboard", "keyboard shortcuts", "enhancement"],
  "requires_escalation": false,
  "estimated_resolution_hours": 40,
  "summary": "Feature request for dark mode and keyboard shortcuts from paying customer"
}

Implementation Guide

Schema Definition

Design your JSON Schema to constrain model responses. Reference the examples above and see supported schema features for technical limitations.

API Integration

Include the schema in your API request using the response_format parameter. Choose between strict: true for guaranteed schema compliance or strict: false for best-effort validation:

Using Strict Mode (strict: true)

Set strict: true for guaranteed schema compliance on supported models:

JSON
response_format: { type: "json_schema", json_schema: { name: "schema_name", strict: true, schema: … } }

Complete implementation example:

from groq import Groq
import json

client = Groq()

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[
        {"role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step."},
        {"role": "user", "content": "how can I solve 8x + 7 = -23"}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "math_response",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "steps": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "explanation": {"type": "string"},
                                "output": {"type": "string"}
                            },
                            "required": ["explanation", "output"],
                            "additionalProperties": False
                        }
                    },
                    "final_answer": {"type": "string"}
                },
                "required": ["steps", "final_answer"],
                "additionalProperties": False
            }
        }
    }
)

result = json.loads(response.choices[0].message.content)
print(json.dumps(result, indent=2))

Error Handling

Error handling differs based on which mode you're using:

With Strict Mode (strict: true)

Constrained decoding guarantees schema-compliant output, so you won't encounter schema validation errors. The model's output will always match your JSON Schema perfectly.

No error handling needed:

Python
# Simple and reliable - no try/catch needed for validation
response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[...],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "schema_name",
            "strict": True,
            "schema": {...}
        }
    }
)

# Output is guaranteed to match schema
data = json.loads(response.choices[0].message.content)

Best Practices

  • User input handling: Include explicit instructions for invalid or incompatible inputs. Models attempt schema adherence even with unrelated data, potentially causing hallucinations. Specify fallback responses (empty fields, error messages) for incompatible inputs.

  • Output quality: Structured outputs are designed to output schema compliance but not semantic accuracy. For persistent errors, refine instructions, add system message examples, or decompose complex tasks. See the prompt engineering guide for optimization techniques.

Migration Guide: Upgrading to Strict Mode

If you're currently using Structured Outputs with strict: false (or without specifying the strict parameter), you can upgrade to strict: true for guaranteed schema compliance. Follow these steps:

Step 1: Verify Model Support

Ensure you're using a model that supports strict: true. See the Supported Models section for more information.

Step 2: Update Your Schema

Make your schema compliant with strict: true requirements:

Mark all fields as required:

JSON
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "number" }
  },
  "required": ["name", "age"]  // ← Ensure all properties are in required array
}

Add additionalProperties: false to all objects:

JSON
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "email": { "type": "string" }
  },
  "required": ["name", "email"],
  "additionalProperties": false  // ← Add this to all objects
}

Handle optional fields with union types:

If you need optional fields, use union types with null:

JSON
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "nickname": { 
      "type": ["string", "null"]  // ← Use union type for optional fields
    }
  },
  "required": ["name", "nickname"],  // ← Field must still be in required array
  "additionalProperties": false
}

Step 3: Update API Calls

Add strict: true to your response_format:

JSON
{
  "model": "openai/gpt-oss-20b",
  "messages": [...],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "schema_name",
      "strict": true,  // ← Add this line
      "schema": {...}
    }
  }
}

Schema Requirements

Structured Outputs supports a JSON Schema subset with specific constraints for performance and reliability.

Supported Data Types

  • Primitives: String, Number, Boolean, Integer
  • Complex: Object, Array, Enum
  • Composition: anyOf (union types)

Schema Constraints by Mode

When using strict: false (default), your schema has more flexibility:

  • Optional fields allowed: Not all properties need to be in required
  • additionalProperties: Can be true or omitted (though false is recommended)
  • More forgiving validation: Best-effort schema matching, but may occasionally produce errors or invalid JSON

Example with optional fields:

JSON
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "nickname": { "type": "string" }
  },
  "required": ["name"]
}

Note: While strict: false is more flexible, following the strict: true requirements will improve output quality and reduce validation errors.


Union types: Each schema within anyOf must comply with all subset restrictions:

JSON
{
  "type": "object",
  "properties": {
    "payment_method": {
      "anyOf": [
        {
          "type": "object",
          "description": "Credit card payment information",
          "properties": {
            "card_number": {
              "type": "string",
              "description": "The credit card number"
            },
            "expiry_date": {
              "type": "string",
              "description": "Card expiration date in MM/YY format"
            },
            "cvv": {
              "type": "string",
              "description": "Card security code"
            }
          },
          "additionalProperties": false,
          "required": ["card_number", "expiry_date", "cvv"]
        },
        {
          "type": "object",
          "description": "Bank transfer payment information",
          "properties": {
            "account_number": {
              "type": "string",
              "description": "Bank account number"
            },
            "routing_number": {
              "type": "string",
              "description": "Bank routing number"
            },
            "bank_name": {
              "type": "string",
              "description": "Name of the bank"
            }
          },
          "additionalProperties": false,
          "required": ["account_number", "routing_number", "bank_name"]
        }
      ]
    }
  },
  "additionalProperties": false,
  "required": ["payment_method"]
}

Reusable subschemas: Define reusable components with $defs and reference them using $ref:

JSON
{
  "type": "object",
  "properties": {
    "milestones": {
      "type": "array",
      "items": {
        "$ref": "#/$defs/milestone"
      }
    },
    "project_status": {
      "type": "string",
      "enum": ["planning", "in_progress", "completed", "on_hold"]
    }
  },
  "$defs": {
    "milestone": {
      "type": "object",
      "properties": {
        "title": {
          "type": "string",
          "description": "Milestone name"
        },
        "deadline": {
          "type": "string",
          "description": "Due date in ISO format"
        },
        "completed": {
          "type": "boolean"
        }
      },
      "required": ["title", "deadline", "completed"],
      "additionalProperties": false
    }
  },
  "required": ["milestones", "project_status"],
  "additionalProperties": false
}

Root recursion: Use # to reference the root schema:

JSON
{
  "name": "organization_chart",
  "description": "Company organizational structure",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "employee_id": {
        "type": "string",
        "description": "Unique employee identifier"
      },
      "name": {
        "type": "string",
        "description": "Employee full name"
      },
      "position": {
        "type": "string",
        "description": "Job title or position",
        "enum": ["CEO", "Manager", "Developer", "Designer", "Analyst", "Intern"]
      },
      "direct_reports": {
        "type": "array",
        "description": "Employees reporting to this person",
        "items": {
          "$ref": "#"
        }
      },
      "contact_info": {
        "type": "array",
        "description": "Contact information for the employee",
        "items": {
          "type": "object",
          "properties": {
            "type": {
              "type": "string",
              "description": "Type of contact info",
              "enum": ["email", "phone", "slack"]
            },
            "value": {
              "type": "string",
              "description": "The contact value"
            }
          },
          "additionalProperties": false,
          "required": ["type", "value"]
        }
      }
    },
    "required": [
      "employee_id",
      "name",
      "position",
      "direct_reports",
      "contact_info"
    ],
    "additionalProperties": false
  }
}

Explicit recursion through definition references:

JSON
{
  "type": "object",
  "properties": {
    "file_system": {
      "$ref": "#/$defs/file_node"
    }
  },
  "$defs": {
    "file_node": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string",
          "description": "File or directory name"
        },
        "type": {
          "type": "string",
          "enum": ["file", "directory"]
        },
        "size": {
          "type": "number",
          "description": "Size in bytes (0 for directories)"
        },
        "children": {
          "anyOf": [
            {
              "type": "array",
              "items": {
                "$ref": "#/$defs/file_node"
              }
            },
            {
              "type": "null"
            }
          ]
        }
      },
      "additionalProperties": false,
      "required": ["name", "type", "size", "children"]
    }
  },
  "additionalProperties": false,
  "required": ["file_system"]
}

JSON Object Mode

JSON Object Mode provides basic JSON output validation without schema enforcement. Unlike Structured Outputs with json_schema mode, it is designed to output valid JSON syntax but not schema compliance. The endpoint will either return valid JSON or throw an error if the model cannot produce valid JSON syntax. Use Structured Outputs when available for your use case.

Strict ModeBest-effort ModeJSON Object Mode
Valid JSONAlways ✓Usually ✓Usually ✓
Schema adherenceGuaranteed ✓Best-effortNo
Can errorNoOccasionallyOccasionally
Requires schemaYesYesNo
Model supportMultiple modelsMultiple modelsAll models
Use caseProduction appsDevelopment, broader compatibilitySimple JSON without schema

Enable JSON Object Mode by setting response_format to { "type": "json_object" }.


Requirements and limitations:

  • Include explicit JSON instructions in your prompt (system message or user input)
  • Outputs are syntactically valid JSON but may not match your intended schema
  • Combine with validation libraries and retry logic for schema compliance

Sentiment Analysis Example

This example shows prompt-guided JSON generation for sentiment analysis, adaptable to classification, extraction, or summarization tasks:

from groq import Groq
import json

client = Groq()

def main():
    response = client.chat.completions.create(
        model="llama-3.3-70b-versatile",
        messages=[
            {
                "role": "system",
                "content": """You are a data analysis API that performs sentiment analysis on text.
                Respond only with JSON using this format:
                {
                    "sentiment_analysis": {
                    "sentiment": "positive|negative|neutral",
                    "confidence_score": 0.95,
                    "key_phrases": [
                        {
                        "phrase": "detected key phrase",
                        "sentiment": "positive|negative|neutral"
                        }
                    ],
                    "summary": "One sentence summary of the overall sentiment"
                    }
                }"""
            },
            {
                "role": "user", 
                "content": "Analyze the sentiment of this customer review: 'I absolutely love this product! The quality exceeded my expectations, though shipping took longer than expected.'"
            }
        ],
        response_format={"type": "json_object"}
    )

    result = json.loads(response.choices[0].message.content)
    print(json.dumps(result, indent=2))

if __name__ == "__main__":
    main()

System prompts structure the output format while maintaining JSON validity. However, keep in mind that the JSON object output may not match your schema.


Example Output
JSON
{
  "sentiment_analysis": {
    "sentiment": "positive",
    "confidence_score": 0.84,
    "key_phrases": [
        {
            "phrase": "absolutely love this product",
            "sentiment": "positive"
        },
        {
            "phrase": "quality exceeded my expectations",
            "sentiment": "positive"
        }
    ],
    "summary": "The reviewer loves the product's quality, but was slightly disappointed with the shipping time."
  }
}

Response structure:

  • sentiment: Classification (positive/negative/neutral)
  • confidence_score: Confidence level (0-1 scale)
  • key_phrases: Extracted phrases with individual sentiment scores
  • summary: Analysis overview and main findings

Was this page helpful?