Skip to main content
Learn to generate synthetic data in Microsoft Foundry for fine-tuning. Synthetic data helps you:
  • Create large, diverse datasets when real data is scarce
  • Preserve privacy while retaining useful structure
  • Generate business‑specific data from your documents or code
  • Cut cost vs manual collection
Create domain-specific data that refines foundation models for your needs—enabling more accurate Q&A in regulated fields and more dependable tool-use by agents—without relying on scarce proprietary datasets. This article covers:
  • Generate synthetic data in the Foundry portal.
  • Prepare and upload a reference file.
  • Configure generation parameters.
  • Review and download results.
  • Integrate generated data into fine-tuning.
  • Apply best practices.
Preview: Functionality, formats, and limits may change. Always validate outputs before production use.

Prerequisites

The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.
  • Use one of the supported regions for synthetic data generation: eastus2, eastus, westus, northcentralus, southcentralus, swedencentral, germanywestcentral, francecentral, uksouth, uaenorth, japaneast, australiaeast

Generate synthetic data for fine-tuning

Foundry provides generators that turn a reference file into task‑ready training data aligned to your fine‑tuning goal.

Overview of synthetic data generators

The preview includes two generator types:

Prepare your reference file

Our generators require a single reference file as the basis for generating new, synthetic examples derived from its content. Supported formats:
  • Simple Q&A: A PDF, Markdown, or plain text document less than 20MB containing the subject knowledge you want the model to learn from.
  • Tool use: A valid 3.0.x or 3.1.x OpenAPI Specification (Swagger) file in JSON less than 20MB that describes the APIs you want the model to learn to call as tools.
Keep formatting clean, avoid noisy boilerplate (for example, navigation menus), and ensure the file directly expresses the target domain or API functionality. Do not include personally identifiable information (PII) or other sensitive data.
In this preview, only one reference file is supported. If you have multiple documents, please combine them into a single file before uploading.

Generate synthetic data in the portal

  1. In the Foundry portal, select Data > Synthetic Data Generation from the left menu.
A screenshot showing an example of the generate data button in Foundry.
  1. Select the Generate data button.
  2. Use the synthetic data generation wizard to create a new data generation job:
A screenshot showing an example of the upload data button in Foundry.
  1. Select a task type: Simple Q&A or Tool use.
  2. Define the generation parameters, if applicable.
  3. Upload your reference file.
  4. Select the number of samples to generate for your dataset (between 50 and 1,000).
  5. Select the model to use for data generation. If the model you select isn’t deployed, a new deployment is created automatically.
  6. Select whether to perform an 80/20 train-validation split on the generated dataset automatically.
  7. (Optional) Provide a custom name to append to the data generation job ID.
  8. Select Generate to submit the job.
  9. Review generated data by selecting the completed job. A preview panel opens on the right side of the screen showing sample outputs and cost metrics.
If you don’t have an available generator model, ask your admin to deploy one, or you need to be assigned the Foundry Owner role to deploy models.

Use your generated synthetic data for fine-tuning

Once the data generation job completes, the resulting dataset is ready for fine-tuning. The output is formatted as JSONL files compatible with Foundry fine-tuning workflows. You can download the dataset for post-processing or use it directly in fine-tuning by selecting the Use this dataset button in the right preview panel. An example of JSONL training data generated by the Simple Q&A generator from a tax statute:
An example of JSONL training data generated by the Tool use generator from an OpenAPI specification (Swagger) for an airline travel booking agent:

Best practices for fine-tuning with synthetic data

Use high-quality reference files

The quality of your reference file directly impacts the quality of the generated synthetic data. Use relevant, well-structured documents or API specifications. Avoid documents with excessive noise, irrelevant information, or poor formatting. For tool use, provide a clear and concise description for each API path and parameter in your API specification. The path description should explain its purpose, and include business policies in the description if needed to help the LLM understand when to use it correctly. Example (note the inclusion of description with business policies for the path and required parameters in the schema):

Start with generating a sample size and iterate

When generating synthetic data for the first time, start with a smaller generation sample size to evaluate the quality of the generated data. Review the outputs and make adjustments to your reference file or generation parameters as needed before scaling up to larger sample sizes. This can help you avoid unnecessary costs and ensure that the generated data meets your requirements.

Experiment with hyperparameters when fine-tuning on synthetic data

When fine-tuning your model with the generated synthetic data, experiment with different hyperparameters such as learning rate, batch size, and number of epochs to find the optimal settings for your specific use case. You may want to use a smaller learning rate when fine-tuning on synthetic data compared to real-world data to curb overfitting. You may also want to experiment with earlier checkpoints if the resulting model underperforms or shows signs of regression on specific evaluation heuristics. This can help you achieve better performance and generalization on your target tasks when training on synthetic data.

Combine synthetic data with real-world data

In some cases, combining synthetic data with real-world data can lead to better model performance. Consider mixing a portion of real-world data with the generated synthetic data during fine-tuning to provide the model with a more diverse and representative training set. This can help improve the model’s ability to generalize to real-world scenarios.

Monitor and audit model performance

After fine-tuning your model with synthetic data, it’s important to monitor and audit its performance on real-world tasks. Evaluate the model using relevant benchmarks and metrics to ensure that it meets your performance requirements. Regularly assess the model’s behavior to identify any potential issues or biases that may arise from training on synthetic data.