Microsoft AI Foundry: Getting Started Guide

What is Microsoft AI Foundry?

Organizations looking to strengthen this area can work with Microsoft AI Foundry development.

Microsoft AI Foundry, formerly Azure AI Foundry, is Microsoft’s platform for building, testing, deploying, and monitoring enterprise AI solutions. It brings together models, agent orchestration, prompt engineering tools, evaluations, and responsible AI guardrails in a single unified experience.

Key Capabilities

Model Catalog

Access a curated catalog of AI models including:

• Azure OpenAI models (GPT-4o, GPT-4, GPT-3.5)
• Open-source models (Llama, Mistral, Phi)
• Custom fine-tuned models
• Embedding models for search and retrieval

Foundry Agent Service

Build and run agentic applications with:

• Hosted agents
• Multi-agent workflows
• Tool-connected actions
• Session memory and runtime controls
• Microsoft 365 deployment options for supported scenarios

Foundry IQ

Ground AI systems with:

• Enterprise and web knowledge retrieval
• Cross-source grounding patterns
• Simplified RAG orchestration
• Better context quality for enterprise agents

Foundry Local

Support edge and privacy-sensitive scenarios with:

• On-device model execution for supported runtimes
• Lower-latency AI experiences
• Better resilience when connectivity is limited
• Flexible patterns for edge and mobile use cases

Prompt Engineering

Build and test prompts with:

• Interactive prompt playground
• System message configuration
• Few-shot example management
• Parameter tuning (temperature, top-p, etc.)
• A/B testing different prompt strategies

RAG (Retrieval-Augmented Generation)

Connect your AI to your own data:

• Index documents from Azure Blob Storage
• Connect to Azure AI Search
• Use SharePoint as a data source
• Build knowledge bases from your existing content

Evaluation & Testing

Measure AI quality systematically:

• Built-in evaluation metrics (groundedness, relevance, coherence)
• Custom evaluation criteria
• Bulk testing with test datasets
• Comparison across model versions
• Agent tracing and workflow-level diagnostics

Responsible AI

Built-in safety features:

• Content filtering for harmful outputs
• Jailbreak detection
• PII detection and redaction
• Custom safety policies

Observability and Governance

Operate AI systems with:

• Tracing and runtime visibility
• Quality, cost, and safety dashboards
• Policy-aligned controls
• Enterprise identity and RBAC integration

Getting Started: Build a RAG Application

Here’s how to build a basic RAG (Retrieval-Augmented Generation) application:

Step 1: Create a Project

1. Go to ai.azure.com
2. Create a new project
3. Select your Azure subscription and resource group
4. Choose your AI hub or create a new one

Step 2: Add Your Data

1. Upload documents to Azure Blob Storage
2. Create an Azure AI Search index
3. Connect the index to your project
4. Configure chunking and embedding settings

Step 3: Configure Your Model

1. Deploy a model (e.g., GPT-4o) from the model catalog
2. Configure the system prompt with your use case context
3. Connect the model to your search index
4. Set RAG parameters (top-k results, search type)

Step 4: Test and Evaluate

1. Use the playground to test queries
2. Create an evaluation dataset
3. Run automated evaluations
4. Iterate on prompts and parameters

Step 5: Deploy

1. Deploy as a managed endpoint
2. Get API keys and endpoint URL
3. Integrate into your application
4. Monitor usage and performance

Best Practices for Enterprise AI

1. Start with clear use cases — don’t build AI for AI’s sake
2. Ground your AI in your data — RAG dramatically reduces hallucination
3. Implement guardrails from day one — content filtering and safety policies
4. Monitor continuously — track groundedness, user satisfaction, and cost
5. Iterate on prompts — prompt engineering is an ongoing process
6. Plan for scale — design for production throughput from the start

Enterprise Considerations

• Data residency — ensure your data stays in your required region
• Authentication — use Azure AD for user-level access control
• Cost management — monitor token usage and set spending limits
• Compliance — AI Foundry supports SOC 2, HIPAA, and GDPR requirements
• Integration — use the REST API or SDK to embed AI into existing applications

Ready to build enterprise AI solutions? Contact Al Rafay Consulting — we specialize in Microsoft AI Foundry implementations for production-grade enterprise applications.