Online deployment

Security impact is direct because an online deployment runs model code and often reaches sensitive data, registries, secrets, and downstream services. Risks include over-permissive managed identities, private container image access failures, exposed environment variables, vulnerable base images, and deployments that bypass approved network isolation. Secure designs separate endpoint access from deployment identity, grant least privilege to storage, Container Registry, Key Vault, and data sources, and review custom containers before release. Microsoft regularly patches base images, but teams using custom images remain responsible for updates. Logs must avoid leaking prompts, inputs, keys, customer identifiers, or regulated inference data. Reviewers should treat model-serving code as production software, not only data science output. Review deployment identities before rollout.

Cost impact is direct because managed online deployments consume compute while instances are running. Instance type, GPU use, instance count, always-on production capacity, and extra quota reservation for some SKUs determine the main spend. Multiple blue, green, test, and shadow deployments can quietly double or triple costs if old deployments remain active. Private networking, logging, registries, and storage for model assets also contribute. FinOps reviews should map each deployment to owner, endpoint, traffic percentage, SKU, instance count, idle time, and business purpose. Cost control is not just deletion; it also includes right-sizing, autoscale planning, and retiring unused model versions safely. Scheduled cleanup prevents canary capacity from becoming permanent waste.

Reliability impact is strong because a deployment failure can make an endpoint unavailable or route users to the wrong model behavior. Reliable teams create deployments with enough instances for availability, test locally when possible, review container logs, and shift traffic gradually. They also preserve model and environment assets so reimaging or recovery does not fail later. Blue-green and mirrored deployments reduce blast radius when evaluating new code or model versions. Runbooks should distinguish provisioning failure, image pull failure, scoring-script error, dependency outage, quota shortage, and traffic-routing mistake. Operators should know how to set traffic back to a known healthy deployment quickly. Each release should define evidence needed before deleting rollback capacity.

Performance impact is direct because the deployment controls compute size, instance count, container startup, model loading, request concurrency, and scoring code. A well-tuned endpoint can still perform poorly if the deployment uses an undersized VM, slow model initialization, inefficient preprocessing, or restrictive max concurrent requests. GPU deployments require careful matching between model size, batch behavior, and request latency objectives. Operators should test realistic payloads, watch latency percentiles, failures, CPU or GPU utilization, memory pressure, queueing, and cold-start behavior. Scaling the deployment can improve throughput, but code optimization and model packaging often matter just as much as adding instances safely. Load tests should match realistic payload size, concurrency, and dependency latency before rollout.

Operators manage online deployments through YAML definitions, CLI commands, SDK calls, studio pages, deployment logs, metrics, and endpoint traffic settings. Daily work includes creating deployments, checking provisioning_state, reviewing logs, tuning instance_count, updating environments, allocating traffic, and deleting retired versions. Incident response often starts with az ml online-deployment get-logs and endpoint metrics, then moves to model asset availability, container startup, request payloads, and dependency access. Good operations keep deployment YAML in source control, separate scaling changes from model changes, document identity and network requirements, and capture which deployment served production traffic during each release window and approval. Operators should keep deployment records close to model cards, release notes, and incident timelines. Review them quarterly with owners.