# Tracing with Tempo
DuploCloud's Advanced Observability Suite (AOS) leverages [Tempo](https://grafana.com/docs/tempo/latest/) for tracing, with [Alloy](https://grafana.com/docs/alloy/latest/) and [Beyla](https://grafana.com/docs/beyla/latest/) as the data collectors. Beyla, powered by eBPF (extended Berkeley Packet Filter), enables seamless collection of observability data directly from the system kernel without requiring application-level instrumentation. By attaching to system events like network requests and function calls, Beyla efficiently gathers metrics and traces. For more detailed trace information, users can [fine-tune their applications using the OTEL SDK](/docs/automation-platform/diagnostics-overview/advanced-observability-suite/application-instrumentation.md), integrating traces into logs for enhanced observability.
## What Tracing Can Tell You
Tracing with OpenTelemetry is useful in scenarios where you want to identify, analyze, and resolve performance issues or understand the flow of requests through a distributed system. Tracing is helpful for:
* Debugging latency issues
* End-to-end visibility of requests across services
* Analyzing errors and failures
* Capacity planning and optimization
* Root Cause Analysis (RCA)
* Performance tuning new features
* Validating SLAs
* Exposing misconfigurations
* Understanding user behavior
Using OpenTelemetry with Grafana, you can collect and visualize traces alongside other observability data like metrics and logs, providing a unified view for effective troubleshooting and optimization.
## Exploring Traces in the DuploCloud Advanced Observability Suite
1. In the DuploCloud Portal, navigate to **Administrator** -> **Observability** -> **Advanced** -> **Dashboard** (Administrator AOS Dashboard) **or Observability** -> **Advanced** -> **Dashboard** (Tenant AOS Dashboard). The AOS Dashboard displays.
2. Select your Infrastructure from the **Infrastructure** list box (Administrator AOS Dashboard) or **Tenant** from the Tenant list box (Tenant AOS Dashboard).
3. Click the **Traces** card button. The Grafana **Tracing** dashboard displays.
Grafana Tracing dashboard
4. Use the following functions on the Grafana **Tracing** dashboard to find and view relevant trace data. See the [Grafana documentation](https://grafana.com/docs/grafana/latest/datasources/tempo/query-editor/) for detailed instructions.
* **Search Query Builder:** Use the search query builder to filter and find traces based on parameters like service name, duration, status codes, or custom tags. This lets you quickly narrow down the traces you're interested in and dive into specific details.
* **TraceQL:** TraceQL is a powerful query language designed to provide advanced filtering and exploration of your trace data. It lets you run more complex queries for pinpointing specific traces or span data based on conditions like service dependencies, error rates, or trace attributes.
* **Service Graph:** The Service Graph visualizes the relationships between services in your application. It shows how services communicate with one another and provides a high-level view of trace flow across your system. This view helps you identify bottlenecks or failures in service-to-service interactions.
## Inspecting Traces
When you have located a trace of interest using the steps above, you can inspect specific metrics or individual spans to understand the operations that occurred during the trace.
1. From the Grafana **Tracing** dashboard, click on the trace of interest. The **Trace Timeline Viewer** displays on the right side of the screen. See the [Grafana Tracing documentation](https://grafana.com/docs/grafana/latest/datasources/tempo/tracing-best-practices/) for more details.
The Trace Timeline Viewer in Grafana
### Mapping Traces to Logs, Metrics, and Profiles for Debugging
The Trace Timeline Viewer can be used to map traces with logs, metrics, and profiles. These details offer insights into timing, status, and metadata, which can help identify performance bottlenecks or failures.
* **Trace-to-Logs**: Navigate from a trace directly to relevant logs, allowing you to correlate trace details with log data.
The Grafana trace timeline showing details and a button to open logs for the span
* **Trace-to-Metrics**: Jump to metrics related to a particular exception/span to analyze rates, durations, and other key performance indicators.
Close-up of the Grafana trace timeline with
* **Trace-to-Profiles**: Link trace spans directly to profiling data for deeper analysis of resource usage and performance at the code level. This feature allows you to correlate traces with profiles, such as CPU or memory usage, providing fine-grained insights into your system’s behavior.
For detailed instructions on inspecting traces, spans, and associated logs, see this\
[Grafana documentation page](https://grafana.com/docs/grafana/latest/explore/trace-integration/). For more information about logging with DuploCloud's AOS, see the\
[DuploCloud Logging page](/docs/automation-platform/diagnostics-overview/advanced-observability-suite/logging-with-loki.md).
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.duplocloud.com/docs/automation-platform/diagnostics-overview/advanced-observability-suite/tracing-with-tempo.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.