az monitor app-insights component show --app <app-insights-name> --resource-group <resource-group>
Monitoring and Observability
Application telemetry
Distributed tracing
Distributed tracing follows a request across services, dependencies, and process boundaries so teams can understand latency, failures, and call relationships in a distributed application.
Source: Microsoft Learn - Application Insights OpenTelemetry observability overview Reviewed 2026-05-13
- Exam trap
- Changing production before checking the exact Azure scope, owner, identity, dependency, and rollback or recovery procedure.
- Production check
- Can you name the Azure scope, owner, resource ID, dependency, and rollback or recovery path without guessing?
Article details and learning context
- Aliases
- end-to-end tracing, OpenTelemetry tracing, application distributed trace, transaction trace
- Difficulty
- intermediate
- CLI mappings
- 4
- Last verified
- 2026-05-13
Learning paths
Learn
Observability and Incident Response
Distributed tracing
Related context
Learning path: Observability and Incident Response
Understand the concept
In plain English
Distributed tracing is the observability practice that follows one request across services, dependencies, queues, and processes using correlated trace context. In Azure, it helps teams diagnose latency, identify failing dependencies, explain user-impacting incidents, and connect application telemetry across service boundaries. Plainly, it is a named part of the architecture that operators can point to when they need evidence, ownership, and a safe change path. A useful glossary entry should explain where it appears, what it controls, what depends on it, and which signal proves it is healthy.
Why it matters
Distributed tracing matters because a small Azure design choice can shape customer experience, security posture, operational visibility, and incident recovery. When it is shallowly documented, teams may troubleshoot the wrong tenant, policy, storage account, migration project, disk, telemetry path, or SQL table while the real dependency remains hidden. In enterprise Azure work, the value is shared language: application, platform, security, data, finance, and operations teams can discuss the same object without guessing. That reduces incident time, improves audit quality, clarifies ownership, and makes production changes safer because failure modes and graph relationships are visible before change. Treat Distributed tracing as production owned when customer traffic, regulated data, migration planning, shared infrastructure, or release automation depends on it.
Technical context
Technically, Distributed tracing appears in Application Insights requests, dependencies, traces, OpenTelemetry spans, correlation IDs, W3C trace context, Azure Monitor workbooks, and transaction search and interacts with Application Insights, Azure Monitor, Azure Functions, and Azure App Service. Configuration is reviewed through OpenTelemetry instrumentation, sampling settings, correlation headers, and Application Insights connection string, while operators validate live state through request trace, dependency call, operation ID, and span duration. Scope determines which permissions, logs, commands, and dependencies matter.
Exam context
Compare with
Where it is used
Where you see it
- In Application Insights transaction search, distributed tracing appears as a request tree with child dependency calls and correlated telemetry during production review when operators collect repeatable evidence.
- In OpenTelemetry instrumentation, it appears when services emit spans that share trace context across HTTP, messaging, or function calls during production review when operators collect repeatable evidence.
- In incidents, it appears when operators follow one slow transaction across services to find the failing dependency or bottleneck during production review when operators collect repeatable evidence.
Common situations
- Find which dependency caused a slow customer request.
- Correlate Azure Function, API, and queue processing telemetry.
- Validate OpenTelemetry instrumentation after a release.
Illustrative Azure scenarios
These examples show how the concept can affect design and operations. They are illustrative scenarios, not customer claims.
Scenario 01 Distributed tracing in action for parcel delivery Scenario, objectives, solution, measured impact, and takeaway.
Scenario
ParcelPoint Express, a parcel delivery organization, needed to address customers saw slow tracking updates, but each microservice dashboard looked healthy alone. The architecture team used Distributed tracing as the control point for a measurable production improvement.
Goals
- Trace a request across API, queue, and worker services
- Reduce mean time to identify latency source by 60 percent
- Keep telemetry ingestion cost controlled
Approach using Distributed tracing
Engineers enabled OpenTelemetry instrumentation and routed traces to Application Insights. APIs, Service Bus handlers, and worker services propagated trace context, while sampling rules preserved high-value failed and slow transactions. Workbooks showed request duration, dependency calls, and queue processing together. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership. Runbooks were updated so support engineers could identify the correct scope, identity, dependency, telemetry signal, and approval record without asking the original implementer. The final design connected governance with day-to-day engineering work, which made the change understandable to security, operations, finance, and application stakeholders. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership.
Potential outcomes
- Latency root-cause time dropped from 90 minutes to 22 minutes
- A slow warehouse API dependency was isolated and fixed
- Sampling kept ingestion within the observability budget
What to learn
Distributed tracing shows the whole transaction path when separate dashboards hide the actual bottleneck.
Scenario 02 Distributed tracing in action for digital lending Scenario, objectives, solution, measured impact, and takeaway.
Scenario
GreenField Loans, a digital lending organization, needed to address loan applications occasionally failed after a release, but logs did not connect API requests to downstream scoring calls. The architecture team used Distributed tracing as the control point for a measurable production improvement.
Goals
- Correlate failures across services
- Expose dependency errors in one transaction view
- Reduce rollback decisions based on incomplete evidence
Approach using Distributed tracing
The application team added Azure Monitor OpenTelemetry instrumentation to API and scoring services, verified operation IDs in Application Insights, and queried requests and dependencies during release validation. Deployment gates checked for trace coverage before production rollout continued. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership. Runbooks were updated so support engineers could identify the correct scope, identity, dependency, telemetry signal, and approval record without asking the original implementer. The final design connected governance with day-to-day engineering work, which made the change understandable to security, operations, finance, and application stakeholders. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership. Runbooks were updated so support engineers could identify the correct scope, identity, dependency, telemetry signal, and approval record without asking the original implementer.
Potential outcomes
- Failed scoring dependencies were visible within minutes
- False rollback decisions dropped by 40 percent
- Release validation included trace-continuity evidence
What to learn
Distributed tracing is release evidence, not just an incident-response tool.
Scenario 03 Distributed tracing in action for healthcare integration Scenario, objectives, solution, measured impact, and takeaway.
Scenario
AsterHealth Connect, a healthcare integration organization, needed to address FHIR integration workflows crossed Functions, APIs, and partner endpoints, making patient-message delays hard to diagnose. The architecture team used Distributed tracing as the control point for a measurable production improvement.
Goals
- Trace patient-message flow end to end
- Avoid capturing sensitive payloads in telemetry
- Prove partner latency separately from internal delays
Approach using Distributed tracing
Architects enabled distributed tracing for Azure Functions and API services, configured telemetry processors to avoid sensitive payload capture, and stored traces in an approved Application Insights resource. Transaction search separated internal processing from partner dependency duration. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership. Runbooks were updated so support engineers could identify the correct scope, identity, dependency, telemetry signal, and approval record without asking the original implementer. The final design connected governance with day-to-day engineering work, which made the change understandable to security, operations, finance, and application stakeholders. The team validated Distributed tracing in a lower environment, captured before-and-after evidence, and promoted the change through a controlled release with rollback ownership.
Potential outcomes
- Patient-message delay investigations fell from hours to minutes
- Sensitive payloads were excluded from trace attributes
- Partner SLA conversations used objective dependency evidence
What to learn
Good distributed tracing balances diagnostic depth with privacy and data-retention discipline.
Azure CLI
CLI checks for Distributed tracing are useful because they turn portal assumptions into repeatable evidence. Start with read-only commands that show scope, state, owner, permissions, destinations, configuration, metrics, or discovered inventory. Run mutating, security-impacting, or cost-impacting commands only after approval, because the wrong scope can affect production availability, spend, access, or telemetry.
Useful for
- Find which dependency caused a slow customer request.
- Correlate Azure Function, API, and queue processing telemetry.
- Validate OpenTelemetry instrumentation after a release.
Before you run a command
- Run az account show, confirm tenant and subscription, and verify the operator identity has approved read access for the exact Azure scope.
- Confirm resource group, service name, resource ID, environment, owner, and change record before collecting evidence or modifying production configuration.
- Prefer read-only commands first; review any command that changes access, policy evaluation, disk state, migration discovery, telemetry, or data distribution before running it.
What the output tells you
- Whether the target tenant, policy, storage account, migration project, disk, trace resource, or SQL pool exists at the expected Azure scope.
- Which state, assignment, property, identity, key reference, attachment, metric, trace, table design, or discovered inventory value is visible to the operator.
- Whether the issue is wrong scope, stale configuration, missing permissions, weak evidence, failed discovery, disk pressure, trace sampling, or table distribution skew.
Mapped commands
Distributed tracing operational checks
directaz monitor app-insights query --app <app-id> --analytics-query "requests | take 5"az monitor app-insights query --app <app-id> --analytics-query "dependencies | take 5"az monitor diagnostic-settings list --resource <resource-id> --output tableArchitecture context
Distributed tracing belongs to Monitoring and Observability architecture decisions where identity, monitoring, cost ownership, reliability, and production support need shared evidence.
- Security
- Security for Distributed tracing starts with least privilege, trusted configuration, and evidence that access matches workload risk. Review sensitive data in traces, telemetry ingestion access, prompt or payload capture, data retention, and workspace permissions before approving production use. A common failure is assuming that a working feature, successful deployment, visible resource, or populated dashboard proves the configuration is safe. Use Microsoft Entra groups, managed identities, RBAC, private connectivity, diagnostic logging, source-controlled definitions, and approval records where applicable. Keep exceptions ticketed, time-bounded, and owned. For regulated workloads, align the term with classification, retention, break-glass, and incident-response procedures. Remove broad access, stale keys, unreviewed contributors, and undocumented exception paths before Distributed tracing becomes an incident path.
- Cost
- Cost for Distributed tracing appears through licensing impact, compute capacity, transaction volume, diagnostic retention, policy remediation, storage consumption, migration assessment effort, disk performance choices, and the human effort required to recover from mistakes. Review telemetry ingestion volume, retention period, sampling strategy, workspace queries, and incident triage time before expanding production use. Some costs are direct, such as retained logs, provisioned disks, storage transactions, or SQL pool capacity; others are indirect, such as failed releases, duplicated troubleshooting, emergency restores, and support escalation. Tag related resources, monitor usage, and separate exploratory work from production. A cost review should connect spend to a real owner and measurable value.
- Reliability
- Reliability for Distributed tracing depends on repeatable configuration, tested dependencies, and clear failure signals. Watch trace continuity, sampling loss, dependency instrumentation, clock skew, and telemetry ingestion health because drift often appears later as failed releases, blocked sign-ins, missing telemetry, slow migration assessments, VM disk pressure, or poor query behavior. Use lower environments, source-controlled definitions where possible, deployment validation, monitoring, and recovery notes before changing production. Operators should know which tenant, endpoint, policy, appliance, VM, dependency, or downstream application fails first and which metric or log proves the failure. The goal is predictable recovery: detect Distributed tracing drift, preserve service, restore safely, and explain the incident without guessing.
- Performance
- Performance for Distributed tracing depends on workload shape, service limits, data volume, network path, diagnostic destination, policy evaluation, disk throughput, trace sampling, SQL distribution, and the monitoring path used to confirm success. Review end-to-end latency, dependency duration, instrumentation overhead, sampling choices, and hot path analysis before increasing capacity or retrying blindly. The better fix might be correcting access scope, reducing log noise, improving discovery cadence, choosing a different disk SKU, tuning trace collection, or changing table distribution. Measure under representative production conditions. Operators should connect symptoms to evidence: latency, throttling, backlog, failed operations, dropped logs, skew, or stale state.
- Operations
- Operations for Distributed tracing should focus on ownership, observability, and safe repeatability. Standardize names, tags, owner groups, environment labels, diagnostic destinations, runbook links, approval records, and change windows so support teams do not reverse-engineer the platform during incidents. Use read-only CLI, API, policy, diagnostic, or portal checks first, then compare live state with intended configuration. For production, connect alerts, audit events, cost records, graph links, and release notes to the same term. The support question should be simple: who owns it, what changed, and what proves the current state?. Capture owner, scope, evidence, and recovery procedure before changing Distributed tracing in a production environment.
Common mistakes
- Changing production before checking the exact Azure scope, owner, identity, dependency, and rollback or recovery procedure.
- Treating a portal screenshot as sufficient evidence when CLI output, Activity Logs, diagnostics, and source-controlled configuration are repeatable.
- Assuming a name match proves the correct resource when tenants, subscriptions, disks, storage accounts, workspaces, and SQL pools can look similar.
Copied command