arrow-left

All pages
gitbookPowered by GitBook
1 of 9

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Setup

Setting up the Standard Observability Suite in DuploCloud

circle-exclamation

Observability components are not turned on by default.

You need to enable the standard observability components before using them. To do this, navigate to Administrator -> Observability -> Standard -> Settings and follow the instructions in the rest of this section.

Metrics Setup

Enabling Metrics collection for centralized monitoring and select Tenants

The Metrics control plane uses Grafana, Prometheus, and Yace. They are only deployable in Default Tenant.

Navigate to Administrator -> Observability -> Standard -> Settings. Select the Monitoring tab to enable Metrics, and click the Enable Monitoring link.

Metrics Enable Monitoring link

In the Monitoring tab, select Enable Centralized Monitoring.

Below, the Monitoring view is shown after the metrics have been enabled. cAdvisor and Node Exporter collect metrics. Selecting a Tenant deploys the containers on all the Hosts in that Tenant.

Monitoring tab after metrics are enabled

Standard Observability Suite

The suite included in the DuploCloud platform

Access the Standard suite by navigating to Observability -> Standard -> Dashboard.

Using DuploCloud's Standard Observability Suite, you have access to:

  • Comprehensive logging

  • Metrics

To use the Standard version, first proceed to .

Auditing

Setting up Auditing in the DuploCloud Portal Standard Observability Suite

The DuploCloud Standard Observability Suite generates audit trails using OpenSearch. Enabling logging is a prerequisite for audit functionality.

In the DuploCloud Portal, navigate to Administrator -> Observability -> Standard -> Settings -> Audit and click the Enable Audit link.

Once enabled, the Audit view is accessible at Administrator -> Observability ->Audit. See the Auditing topic for further details.

The Enable Audit link in the Audit tab
Auditing
Setup

Logging Setup

Setting up Logging in the DuploCloud Portal

Navigate to Administrator -> Observability -> Standard -> Settings, and select the Logging tab to enable Logging. Click on Enable Logging.

Logging is based on OpenSearch and Kibana, deployed in the Tenant of your choice, and configurable, as shown below.

After enabling logging, choose which Tenants to collect logs from. The platform deploys collectors for each Tenant that you enable. Filebeat is the collector for Logs.

Logging tab for adding and enabling Tenant log collection
circle-exclamation

If you have a multi-region setup, create a separate logging infrastructure setup for each region to avoid the cost of cross-region data transfer.

Custom Kibana Logging URL

Configure a custom Kibana URL for viewing Logging data from DuploCloud

DuploCloud provides a default Kibana URL for viewing Logging data, but if you want to update filters or customize further, you can override the default with these settings.

hashtag
Configuring a Custom Kibana Logging URL

  1. Log in to the DuploCloud Portal.

  2. Navigate to Administrator -> System Settings -> System Config.

  3. Click Add. The Add Config pane displays.

  4. In the Config Type list box, select AppConfig.

  5. In the Key list box, choose one of the following keys:

    • Kibana Base URL: This URL points to the proxy or direct access Kibana URL.

    • Kibana Logs URL Template: This URL template links to Kibana’s Logs.

  6. In the Value field, enter the custom URL for the selected key. For example:

    • Kibana Base URL: /proxy/kibana

    • Kibana Logs URL Template: https://kibana.example.com/app/discover#/?_a=(query:(language:kuery,query:'log_type:"application"'))&_g=(time:(from:now-1h,to:now))

circle-info

You can customize the example URLs, replacing placeholders with your own values and modifying the query string to adjust the filters:

  • log_type: Filter by log type (e.g., "application", "system").

  1. Click Submit to save the configuration.

After configuring the Kibana URL in DuploCloud, you can access the selected Kibana Logs data directly from the DuploCloud Portal:

  • Navigate to Administrator -> Observability -> Standard -> Logging.

kubernetes.namespace: Filter by Kubernetes namespace (e.g., "default").

  • log_level: Filter by log level (e.g., "error", "info").

  • time range: Adjust the time range (e.g., from:now-1h, from:now-7d).

  • Example: https://kibana.example.com/app/discover#/?_a=(query:(language:kuery,query:'log_type:"application"'))&_g=(time:(from:now-1h,to:now))

  • The Add Config pane
    The System Config page in the DuploCloud Platform

    Metrics

    Comprehensive metrics at a glance using the DuploCloud Standard Observability Suite

    In the DuploCloud Portal, navigate to Observability -> Standard -> Metrics to view various metrics per Tenant.

    circle-info

    Several out-of-the-box dashboards are available for various services. You can customize these, configure them, and add them to your standard view.

    Standard Metrics with Nodes tab selected

    Logs

    Logging in the DuploCloud Standard Observability Suite utilizing OpenSearch and Kibana

    DuploCloud Standard Observability Suite uses built-in OpenSearch for log storage and search. Filebeat is the collector deployed on each host to gather logs from containers and other system services.

    Logs can be accessed at the platform level (by Administrators), at the Tenant level, or directly from a specific Service using the Logs link.

    hashtag
    Viewing Platform Logs

    Use this option to view logs across all tenants or to troubleshoot issues at the platform level.

    1. Navigate to Administrator → Observability → Standard → Logging in the DuploCloud Portal.

    This page includes two tabs:

    • Main: Shows logs using an embedded OpenSearch dashboard provided by DuploCloud. You can select a Tenant and use filters to find the logs you need.

    • Kibana: Available if your organization has set up a custom Kibana (OpenSearch) URL. This lets you view logs with more advanced tools and features.

    hashtag
    Viewing Tenant Logs

    Use this view to access logs scoped to a single tenant.

    1. Navigate to Observability → Standard → Logging.

    In this view:

    • Logs are automatically filtered to show only data for the current tenant.

    • At the top, use the Service dropdown to filter logs for a specific service.

    • Alternatively, click Search by Filters to create custom filters for more precise log searching.

    hashtag
    Viewing Service-Specific Logs

    You can quickly access logs filtered for a Specific service directly from the Service details page. This shortcut helps you troubleshoot service issues faster without manually applying filters.

    To view logs for a Service:

    1. Navigate to Kubernetes → Services (or the relevant service list) in the DuploCloud Portal.

    2. Click the menu icon () next to the Service you want to view logs for.

    3. Select Logs. This will open a logs dashboard pre-filtered to show logs for that specific service.

    hashtag
    Application log retention policy

    DuploCloud retains application logs in two stages: comprehensive tracking and auditing.

    Initially, logs are available in OpenSearch for 60 days, ensuring immediate access for recent activity analysis.

    Logs are archived in an S3 Bucket for 365 days for long-term storage, meeting compliance, and historical data review needs. You can customize retention periods to meet specific requirements, offering flexibility in log management.

    Custom Kibana Audit URL

    Configure a custom Kibana URL for viewing audit logs from DuploCloud

    DuploCloud provides a default Kibana URL for viewing audit logs, but if you want to update filters or customize further, you can override the default with these settings.

    hashtag
    Configuring a Custom Kibana Audit URL

    1. Log in to the DuploCloud Portal.

    2. Navigate to Administrator -> System Settings -> System Config.

    3. Click Add. The Add Config pane displays.

    4. In the Config Type list box, select AppConfig.

    5. In the Key list box, choose one of the following keys:

      • Kibana Base URL: This URL points to the proxy or direct-access Kibana instance.

      • Kibana Audit URL Template: This URL template links to Kibana’s Audit logs.

    6. In the Value field, enter the custom URL for the selected key. For example:

      • For Kibana Base URL: /proxy/kibana

      • For Kibana Audit URL Template: https://kibana.example.com/app/discover#/?_a=(query:(language:kuery,query:'log_type:"audit"'))&_g=(time:(from:now-1h,to:now))

    circle-info

    You can customize the example URLs, replacing placeholders with your own values and modifying the query string to adjust the filters:

    • log_type: Filter by log type (e.g., "audit", "application").

    1. Click Submit to save the configuration.

    After configuring the Kibana URL in DuploCloud, you can access Kibana Audit data directly from the DuploCloud Portal:

    • Navigate to Administrator -> Observability -> Audit.

    time range: Adjust the time range (e.g., from:now-1h, from:now-7d).

    Example: https://kibana.example.com/app/discover#/?_a=(query:(language:kuery,query:'log_type:"audit"'))&_g=(time:(from:now-1h,to:now))

    The Add Config pane
    The System Config page in the DuploCloud Platform