Most buyers evaluating an on-prem LLM focus on GPUs, VRAM and models. Identity gets pushed to the week before go-live, when a security review surfaces questions nobody wrote down during the pilot. By then the choices are expensive to reverse.
Three protocols cover almost every authentication and authorization decision you will make on an on-prem AI system: SAML for user login, OpenID Connect for app-to-app identity, and OAuth 2.0 for agent and API access. They are not interchangeable, and the right split affects how your chat UI, your retrieval service, your agents and your admin tools all talk to each other.
What should sit where?
Why Identity Is the Decision You Keep Putting Off
On-prem AI buyers often assume identity is "whatever Okta or Entra does." That is true until you add a document AI service, a retrieval index, three agents that call internal APIs on behalf of a user, and an admin console used by two teams. Suddenly you have five services that each need to know who the caller is, what they can see, and what they are allowed to do with it.
The stakes are not theoretical. Verizon's 2025 DBIR found that compromised credentials were the initial access vector in 22% of breaches, and that credential stuffing accounted for 19% of all authentication attempts on a median day, rising to 25% in enterprise-sized companies. IBM's 2025 Cost of a Data Breach Report put the average cost of a breach initiated via compromised credentials at $4.67 million per incident. The protocols below exist to take the password out of the flow entirely and to narrow what a stolen token can do.
If you are still deciding what hardware will run all of this, the AI hardware estimator is a reasonable place to start before the identity conversation gets serious.
SAML for User Login at the Front Door
SAML 2.0 was ratified as an OASIS Standard in March 2005, merging SAML 1.1, Liberty Alliance ID-FF 1.2 and Shibboleth into a single specification. Twenty-one years later it is still the lingua franca of enterprise IdPs. If your organization runs Entra ID, Okta, PingFederate or ADFS, your identity team almost certainly has a SAML pattern they already trust.
Use SAML for the human-facing front door of your on-prem LLM: the chat UI, the admin console, the Studio where power users build prompts and agents. The IdP authenticates the user, signs an XML assertion that carries their identity and group membership, and the AI platform validates the signature locally. No password ever touches the AI system.
Two reasons to prefer SAML at this layer:
- Group and role attributes travel in the assertion. Your IdP already knows which users are in Legal, which are in Clinical Research, which are external contractors. Mapping those groups to AI permissions is a one-time configuration rather than a parallel user directory.
- Single logout actually works. When HR disables an account, the SAML session on the AI platform ends on the next assertion check. You do not need to invalidate a sprawl of long-lived tokens.
SAML is verbose, XML-heavy and painful to debug. For user login against an existing enterprise IdP, those are acceptable trade-offs for a protocol your security team already understands.
OIDC for App-to-App Identity Inside the Stack
Once a user is in, the services that make up your AI platform need to talk to each other. The chat UI calls a retrieval service. The retrieval service calls a reranker. An agent calls three internal tools. Each hop should carry an assertion of who the original user is, not a shared service account that erases the audit trail.
This is where OpenID Connect fits. OIDC is a simple identity layer on top of OAuth 2.0 that lets clients verify the end-user's identity based on authentication performed by an authorization server. In practice that means a signed JWT that any service can validate locally, with claims for user ID, groups and session context. The OIDC Final specification launched on February 26, 2014, and nine OpenID Connect specifications are now published as ISO/IEC standards, which matters when a procurement questionnaire asks whether your identity layer sits on a recognized standard.
In the cstmAI stack, OIDC typically brokers between the SAML login at the edge and the OAuth-protected APIs inside. A thin broker converts the SAML assertion into an OIDC ID token, and every internal service validates that token the same way. The retrieval service does not need to understand SAML. The reranker does not need to understand SAML. They understand JWTs, which is a much smaller surface area to get wrong.
This matters more than it sounds. If you are also planning integrations with ERP, CRM and data systems, those systems tend to speak OIDC natively. Standardizing on OIDC between the AI platform and the rest of your estate avoids a second translation layer.
OAuth 2.0 for Agents, Tools and API Access
OAuth 2.0 was published as RFC 6749 by the IETF in October 2012, with D. Hardt as editor, and obsoletes the earlier OAuth 1.0 protocol. It is an authorization framework, not an authentication protocol: it is designed to let a client act on behalf of a resource owner with a scoped, time-limited token.
That is exactly what agents need. An agent that reads a case file, calls a vector search, hits an internal pricing API and writes a draft back to SharePoint is doing four delegated actions on behalf of a user. Each one should carry an access token with the narrowest scope that works and the shortest TTL you can tolerate. The right default is minutes, not hours.
A workable pattern for custom AI agents on an on-prem system looks like this:
- User signs in via SAML, broker issues an OIDC ID token for session identity.
- Agent requests an OAuth access token with scopes like docs.read, vector.search, pricing.read.
- Each downstream API validates the token, checks the scope, logs the call with the original user ID from the ID token.
- Tokens expire quickly. Refresh is explicit and auditable.
Two guardrails worth building in from day one. First, never let an agent hold a long-lived refresh token on behalf of a user without an explicit consent record. Second, treat tool-calling as a privileged operation: an agent calling an API that mutates data should require a tighter scope than one that reads.
Where NIST SP 800-63C Fits
NIST SP 800-63C defines Federation Assurance Level (FAL), which evaluates the security of federated identity assertions, as one of three dimensions of assurance alongside Identity Assurance Level (IAL) and Authenticator Assurance Level (AAL). If you work in healthcare, financial services, government or any sector where identity assurance is written into policy, FAL is the dial that governs how your SAML and OIDC assertions must be signed, encrypted and bound to the user's session.
NIST SP 800-63C-4, published in 2025, supersedes the earlier NIST SP 800-63C and provides technical requirements for identity providers and relying parties implementing federated identity systems and assertions. The practical read: FAL2 is a reasonable floor for internal AI used by employees; FAL3, which requires holder-of-key assertions, is where you land if the AI system handles regulated data and your compliance program says so.
What to Decide Before the Discovery Sprint
A two-week discovery sprint moves faster when identity decisions are already roughed in. You do not need a finished architecture, but you need answers to the questions that will otherwise block week three.
- Which IdP will be the source of truth? Entra ID, Okta, PingFederate, Keycloak, something else. Confirm it supports both SAML and OIDC flows out of the box.
- Which groups map to which AI permissions? Draft a table of three to five roles (reader, author, admin, agent operator, auditor) and the IdP groups that populate them.
- Where does the audit log live? On-box, SIEM, both. Decide whether user ID from the OIDC token is enough, or whether you need the original SAML session ID preserved end-to-end.
- What is the token TTL budget? Short TTLs mean more refresh traffic and more IdP load. Pick a target for access tokens (often 15 minutes), ID tokens (often an hour) and refresh tokens (often a day for agents, longer for humans).
- What assurance level does the workload require? If the answer is FAL3, holder-of-key support has to be in the IdP contract, not retrofitted later.
If some of these answers are "we are not sure yet," that is normal. The first 90 days of on-premises AI are where most of these decisions get stress-tested anyway, and it is cheaper to get them close now than to redo wiring later.
Putting the Three Together
SAML, OIDC and OAuth are not a stack to pick from. They are a stack to use together. SAML gets the user in. OIDC carries their identity between services. OAuth scopes what agents and APIs can do on their behalf. The failure mode on-prem AI buyers see most often is picking one protocol for everything and bending it into roles it was not designed for.
The right time to work through this is before hardware shows up, not after. If you are mapping what your deployment will actually need, get a quote and bring the identity questions above into the discovery conversation. The two weeks you spend there save months of rework when the first real workload hits the system.