AI and Machine Learning Azure Machine Learning premium

ML datastore

ML datastore is a workspace-level Azure Machine Learning connection to underlying storage used by jobs, data assets, pipelines, and experiments. In everyday Azure work, it appears when ML workloads need to read training data, write outputs, or mount cloud storage without hard-coding every storage detail. The useful mental model is the storage connection definition, not the dataset or the storage account itself. Treat it as an operating decision, not a loose label: identify the owner, scope, dependent workload, monitoring signal, and rollback path before changing it in production.

Back to glossary browser Open Microsoft Learn source
Aliases
AML datastore, Azure ML datastore, Machine Learning datastore, workspace datastore
Difficulty
intermediate
CLI mappings
4
Last verified
2026-05-16T06:31:43Z

Microsoft Learn

Microsoft Learn describes ML datastore as an Azure Machine Learning reference that connects a workspace to external storage such as Blob Storage or Data Lake Storage. Teams use it to let jobs and assets access storage through a governed connection.

Microsoft Learn: Create datastores2026-05-16T06:31:43Z

Technical context

Technically, ML datastore sits in the Azure Machine Learning data-access plane across datastores, storage accounts, credentials or identities, data assets, jobs, and pipelines. Azure represents it through datastore name, storage type, account or container path, authentication method, secret reference, identity settings, and default flags. It usually depends on workspace configuration, storage account access, managed identity or credentials, network reachability, RBAC, and private endpoint design. The important boundary is that a datastore connects to storage; it does not automatically approve every data asset or guarantee data quality.

Why it matters

ML datastore matters because it gives ML teams a reusable and governable storage connection that supports repeatable jobs and safer data access. A weak definition causes teams to change the wrong setting, misread symptoms, or accept defaults that do not fit the workload. The value is not just the feature itself; it is the evidence around it. A strong page explains who owns it, which resource or workflow depends on it, how operators verify health, and what must happen before a production change. That shared understanding makes audits, migrations, scale events, and incidents less chaotic. This keeps owners, operators, and reviewers aligned on the same production evidence.

Where you see it

Signals, screens, and Azure surfaces where this term usually becomes operational.

Signal 01

In the Azure portal, ML datastore appears on ML Studio datastore pages, data asset creation forms, job input selectors, storage account references, and access settings, where operators confirm state, ownership, and release evidence.

Signal 02

In CLI, SDK, REST, or diagnostic output, ML datastore appears as datastore show output, account names, container paths, credential mode, default datastore flags, and network evidence, helping teams compare live state with design.

Signal 03

In architecture, audit, or incident reviews, ML datastore appears when teams discuss data access governance, credential rotation, private storage access, job reproducibility, and workspace ownership, then decide which evidence proves health.

When this becomes relevant

Specific situations where this term helps solve real Azure design, operations, migration, security, reliability, cost, or governance problems.

  • Connect an ML workspace to approved cloud storage.
  • Reuse storage locations across jobs and data assets.
  • Move from secret-based access toward managed identity.
  • Troubleshoot failed data mounts or downloads.

Real-world case studies

Different enterprise-style examples that show the term being used to hit measurable objectives.

Case study 01

Insurance data connection governance.

Scenario, objectives, solution, measured impact, and takeaway.

Scenario

HarborShield Insurance had training scripts that embedded storage account names and SAS tokens, creating security and rotation problems.

Business/Technical Objectives
  • Remove hard-coded storage credentials from ML scripts.
  • Register governed datastore connections.
  • Support credential rotation without rewriting jobs.
  • Improve audit evidence for data access.
Solution Using ML datastore

The ML platform team created datastores for approved claim, policy, and fraud-data containers. Authentication moved to managed identity where possible, with credential-based exceptions stored through the workspace Key Vault process. Data assets referenced the datastores, and training jobs used asset names instead of raw storage paths. CLI output documented datastore names, storage targets, and approval status for each release. The implementation team captured before-and-after evidence, named the support owner, and added a rollback checkpoint so the change could be repeated safely during later releases. They also documented validation commands, expected healthy signals, escalation contacts, and the operational decision that would trigger a rollback during production support. The design review treated configuration, identity, monitoring, cost ownership, and incident response as one operating pattern instead of separate portal tasks.

Results & Business Impact
  • Hard-coded SAS tokens were removed from production scripts.
  • Credential rotation no longer required model-code changes.
  • Audit reviewers received datastore and identity evidence.
  • Failed data-access incidents dropped 57%.
Key Takeaway for Glossary Readers

Datastores make data access an owned platform configuration instead of scattered script knowledge.

Case study 02

Manufacturing lake access.

Scenario, objectives, solution, measured impact, and takeaway.

Scenario

ForgeLine Manufacturing wanted predictive-maintenance models to read plant telemetry from ADLS Gen2 without giving every scientist direct storage keys.

Business/Technical Objectives
  • Expose approved telemetry locations to ML jobs.
  • Keep plant storage accounts behind network controls.
  • Reduce access exceptions for data scientists.
Solution Using ML datastore

Platform engineers registered datastores for each telemetry lake file system and tied access to workspace identity and storage RBAC. Data scientists created data assets from those datastores, while private endpoint and firewall rules stayed under the storage team. CLI checks showed datastore type, target, and workspace scope during release review. The runbook described how to test access before submitting expensive training jobs. The implementation team captured before-and-after evidence, named the support owner, and added a rollback checkpoint so the change could be repeated safely during later releases. They also documented validation commands, expected healthy signals, escalation contacts, and the operational decision that would trigger a rollback during production support. The design review treated configuration, identity, monitoring, cost ownership, and incident response as one operating pattern instead of separate portal tasks.

Results & Business Impact
  • Direct storage-key requests dropped 89%.
  • Predictive maintenance jobs accessed data through governed datastore names.
  • Private storage controls remained owned by the platform team.
  • Training failures from bad paths fell below the monthly target.
Key Takeaway for Glossary Readers

A datastore is the bridge between ML usability and storage security ownership.

Case study 03

Media archive storage abstraction.

Scenario, objectives, solution, measured impact, and takeaway.

Scenario

SilverFrame Media stored video features across multiple containers, and model teams struggled to remember paths during batch scoring.

Business/Technical Objectives
  • Create friendly datastore references for feature storage.
  • Reduce path mistakes in batch jobs.
  • Keep output storage separate from input storage.
  • Improve support handoff during reruns.
Solution Using ML datastore

The team created separate ML datastores for raw features, curated features, and scoring outputs. Batch endpoint deployments and pipeline jobs referenced data assets built on those datastores. Operators used CLI to verify target containers before rerunning failed jobs. Storage permissions, lifecycle policies, and retention were still managed at the storage account level, keeping ML configuration and storage governance aligned. The implementation team captured before-and-after evidence, named the support owner, and added a rollback checkpoint so the change could be repeated safely during later releases. They also documented validation commands, expected healthy signals, escalation contacts, and the operational decision that would trigger a rollback during production support. The design review treated configuration, identity, monitoring, cost ownership, and incident response as one operating pattern instead of separate portal tasks.

Results & Business Impact
  • Path mistakes in batch jobs dropped 81%.
  • Support reruns used approved input and output datastore names.
  • Storage lifecycle policies stayed consistent with media retention rules.
  • Batch scoring handoffs became repeatable across teams.
Key Takeaway for Glossary Readers

Datastores give ML workflows readable storage names while leaving storage controls enforceable.

Why use Azure CLI for this?

Azure CLI is useful for ML datastore because it turns portal state into repeatable evidence. Operators can inspect scope, identity, configuration, metrics, dependencies, and related resources before approving a change. CLI output also supports automation, audit packages, rollback reviews, and incident handoffs.

CLI use cases

  • Inventory ML datastore across the relevant resource, workspace, account, group, endpoint, or scope before a production review.
  • Inspect live ML datastore state during troubleshooting, migration planning, access review, release validation, or rollback confirmation.
  • Export JSON output so reviewers can compare actual configuration with architecture diagrams, source-controlled definitions, and approved runbooks.
  • Run read-only commands first; use create, update, or delete commands only through an approved change path.

Before you run CLI

  • Confirm tenant, subscription, resource group, workspace, account, namespace, server, endpoint, or policy scope before running commands.
  • Verify your role assignment allows the read, write, monitoring, data, or governance action you plan to perform.
  • Choose JSON, table, or TSV output intentionally so the result can be reviewed, scripted, or attached as evidence.
  • For production changes, confirm owner approval, maintenance window, rollback path, cost impact, and dependent workloads first.

What output tells you

  • Names, IDs, scopes, and regions confirm whether you are looking at the intended ML datastore boundary, not a similarly named test asset.
  • State, SKU, version, identity, network, metric, and configuration fields show whether live behavior matches the approved design.
  • Errors, timestamps, and provisioning states help separate service configuration issues from application, data, identity, or caller problems.
  • Saved output gives release, audit, and incident teams a shared record for comparison after the next change.

Mapped Azure CLI commands

Command bundle

az ml datastore list --workspace-name <workspace> --resource-group <group>
az ml datastorediscoverAI and Machine Learning
read-only Microsoft docs
az ml datastore show --name <datastore> --workspace-name <workspace> --resource-group <group>
az ml datastorediscoverAI and Machine Learning
read-only Microsoft docs
az ml datastore create --file datastore.yml --workspace-name <workspace> --resource-group <group>
az ml datastoreprovisionAI and Machine Learning
mutating Microsoft docs
az ml data create --file data.yml --workspace-name <workspace> --resource-group <group>
az ml dataprovisionAI and Machine Learning
mutating Microsoft docs

Architecture context

Architecturally, ML datastore belongs to the Azure Machine Learning data-access plane across datastores, storage accounts, credentials or identities, data assets, jobs, and pipelines. It connects to workspace configuration, storage account access, managed identity or credentials, network reachability, RBAC, and private endpoint design. Treat it as a production boundary with explicit ownership, dependencies, monitoring, and rollback evidence. A diagram or runbook should show who can change it, what resources rely on it, and which outputs prove the intended configuration.

Security

Security for ML datastore focuses on stored credentials, managed identity permissions, private networking, storage ACLs, secret rotation, and who can use the datastore. The main risk is treating it as harmless configuration while it may affect access, exposure, data handling, or automated response. Review who can read, create, update, delete, invoke, or bypass the related resource, and whether that permission is direct, inherited, or granted through a deployment pipeline. Prefer managed identity, least privilege, private access, encryption, monitored changes, and clear exception ownership wherever the Azure service supports those controls. Keep evidence in the change record. This keeps owners, operators, and reviewers aligned on the same production evidence.

Cost

Cost for ML datastore is driven by storage transactions, data movement, failed jobs, duplicated connections, and support time from broken credentials or network paths. Some costs are direct, such as compute, storage, ingestion, action execution, capacity, or retained data. Other costs are indirect: failed retries, duplicated work, noisy alerts, unused resources, delayed migrations, or engineering time spent troubleshooting unclear ownership. FinOps reviews should identify who pays, which metric or SKU drives the bill, and whether a cheaper setting still meets security, reliability, compliance, and performance requirements. Do not cut cost by removing evidence or weakening controls silently. This keeps owners, operators, and reviewers aligned on the same production evidence.

Reliability

Reliability for ML datastore depends on whether jobs can still access data after credential rotation, network changes, account moves, or default datastore updates. The concern is not only that the setting exists; it is whether the workload behaves predictably during deployment, scale, maintenance, dependency loss, retry, recovery, and operator error. Production teams should know which metric, log, activity record, or CLI output proves healthy behavior. They should also document what failure looks like, how to roll back, and which dependent services must be checked before the incident is closed. Good reliability practice makes the term operational, not decorative. This keeps owners, operators, and reviewers aligned on the same production evidence.

Performance

Performance for ML datastore depends on storage latency, mount mode, download speed, data locality, private endpoint routing, file count, and job startup time. The right signal may be request latency, queue depth, startup time, query duration, chart responsiveness, job runtime, throughput, alert delay, or operator time to isolate a bottleneck. Measure before and after important changes rather than assuming the setting improves speed. Keep enough metrics, logs, and command output to explain whether Azure configuration helped the workload, hid the problem, or simply moved the bottleneck to another component. This keeps owners, operators, and reviewers aligned on the same production evidence.

Operations

Operationally, ML datastore requires creating datastores, testing access, rotating credentials, checking default settings, and documenting which workloads rely on each connection. Operators should know which portal blade, CLI command, SDK property, metric, activity log, deployment output, or runbook step shows the live state. Avoid undocumented portal-only edits in production. Use scripts, tags, source-controlled definitions, diagnostics, and change records so support staff can compare actual configuration with the approved design during releases, audits, and incidents. After any change, capture evidence, confirm dependent workloads still behave correctly, and record the owner responsible for follow-up. This keeps owners, operators, and reviewers aligned on the same production evidence.

Common mistakes

  • Changing ML datastore without checking dependent resources, owner approval, monitoring signals, and rollback steps first.
  • Assuming a portal label tells the whole story instead of validating live state through CLI, logs, diagnostics, or activity history.
  • Granting broad permissions for convenience when a narrower role, managed identity, group assignment, or read-only path would work.
  • Optimizing cost or speed while ignoring security, reliability, data exposure, recovery behavior, or user-facing impact.