Azure LLM Deployment: Azure OpenAI, AI Foundry, and Self-Managed

Azure's LLM deployment landscape confuses even Azure-committed teams, because two of its surfaces sound like competitors and are not: Azure OpenAI is the specialized service for OpenAI models, and Azure AI Foundry is the superset platform that contains it plus a vendor-spanning catalog, agent tooling, and evaluation. Choosing well on Azure means understanding the layering first, then picking your entry point by need — with self-managed serving on Azure VMs and AKS as the floor beneath when control demands it.

How Azure's AI Services Actually Layer

No — Azure AI Foundry is a superset: it contains Azure OpenAI's capabilities plus the broader platform (model catalog spanning vendors, agent building, evaluation tooling), so the real choice is between consuming OpenAI models through the specialized service or working through the full platform surface that also wraps them.

The layering is confirmed by vendor guidance — Foundry as a superset of Azure OpenAI, not an alternative to it — and the confusion it resolves is visible in Microsoft's own Q&A, where teams ask which to use without realizing the question is really "narrow surface or full platform." One caution attaches: Azure rebrands and reshapes these surfaces periodically, so verify the current names and scopes at deployment time rather than relying on any article's snapshot — including this one.

Choosing the Entry Point by Need

Start with Azure OpenAI when OpenAI models alone meet the need — the narrower surface is simpler to govern; move up to Foundry when catalog breadth (open-weight and third-party models), agent building, or integrated evaluation become requirements — the platform surface earns its complexity when you use its breadth, not before.

• The narrow-first principle: the specialized service gives you everything OpenAI provides with the least surface to govern — access policy, spend, and vendor terms all simplify when the platform scope is narrow.
• The breadth triggers: needing models beyond OpenAI's (the catalog spans vendors), building agents rather than just calling models, or wanting integrated evaluation — each trigger moves you up to Foundry, where those capabilities are native.
• The transition cost is low: because Foundry contains the narrower surface's capabilities, moving up is additive — start narrow, and the entry-point decision is reversible without rework.

Azure's model catalog makes the breadth trigger concrete: browsable by performance, cost, and use case across vendors, it is the capability the narrow surface cannot offer — and the reason multi-model estates end up on the platform surface. The governance note cuts the same way: a narrower surface is easier to govern and a broader one governs more, so the entry point is also a governance-scope decision your security team should see made, not discover.

The Self-Managed Floor: VMs and AKS

Beneath the managed surfaces sits the self-managed floor — Azure VMs and AKS running your choice of serving stack for open-weight models — entered when control over weights, engine, or data path exceeds what any managed surface provides, at the familiar price of operating the stack yourself.

The boundary moves: as managed surfaces add model choice, some workloads climb from the floor to the catalog, and estates should re-verify before each new commitment that self-management still earns its operating price. For teams whose control requirements point to dedicated capacity beyond Azure's shared surfaces — or beyond any single cloud — environments such as OneSource Cloud's private AI infrastructure are the boundary-controlled alternative, evaluated with the same control-versus-operations logic this page applies inside Azure.

FAQ

Should we use Azure OpenAI directly or through AI Foundry?

Directly when OpenAI models alone serve you — the narrower surface is simpler to govern and everything transfers upward. Through Foundry when you need its breadth: third-party and open-weight models, agent tooling, or integrated evaluation — moving up later costs little; starting wide without needing it costs complexity.

Does AI Foundry replace Azure Machine Learning?

No — they coexist with different centers of gravity: Foundry is the GenAI-application surface (models, agents, evaluation) while Azure ML carries the broader classic-ML lifecycle — estates running both should assign workloads by which lifecycle they actually live in, and verify current positioning as Azure evolves the boundaries.

Can we serve open-weight models like Llama on Azure?

Yes, two ways: through the managed catalog where Azure hosts the model, or on the self-managed floor (VMs or AKS with your serving stack) when control over weights and engine matters — the catalog trades control for simplicity, the floor trades it back.