.svg?alt=media&token=0f6df7c7-5b98-4a62-98b8-a54e38a8329e){kind=logo}
Set up central logging
Set up logging for the DuploCloud Portal
Last updated
Set up logging for the DuploCloud Portal
Last updated
© DuploCloud, Inc. All rights reserved. DuploCloud trademarks used herein are registered trademarks of DuploCloud and affiliates
If you need to make changes to the Control Plane Configuration, follow this procedure to do so, before enabling logging. Note that you cannot modify the Control Plane Configuration after you set up logging.
Docker applications use stdout for writing log files, collecting logs, placing them in the Host directory, mounting them into Filebeat containers, and sending them to AWS Elasticsearch. If you need to customize the log collection and you use folders other than stdout, for example, follow this procedure. Note that you cannot customize the log collection after you set up logging.
In the DuploCloud Portal, navigate to Administrator -> Observability -> Settings -> Logging.
From the Tenant list box at the top of the DuploCloud Portal, select the Default Tenant.
Click the Create Logging link. The Enable Logging page displays.
Use the Enable Logging page to deploy logging for the Control Plane, which uses OpenSearch and Kibana to retrieve and display log data for the Default Tenant. In the Cert ARN field, enter the ARN certificate for the Default Tenant. Find the ARN by selecting the Default Tenant from the Tenant list box at the top of the DuploCloud Portal; navigating to Administrator -> Plans; selecting the Plan that matches your Infrastructure Name; and clicking the Certificates tab.
Click Submit. Data gathering takes about fifteen (15) minutes. When data gathering is complete, graphical logging data is displayed in the Logging tab.
After logging has been enabled for the Control Plane, finish the logging setup by enabling the Log Collector to collect logs per Tenant. This feature is especially useful for Tenants that are spread across multiple regions. In the DuploCloud Portal, navigate to Administrator -> Observability -> Settings -> Logging.
In the Logging tab, on the Logging Infrastructure Tenants page, click Add.
Select the Tenants for which you want to configure logging, using the Select Tenants to enable logging area, as in the example below. The Control Plane configuration is deployed for each Tenant that you select in the Infrastructure, specified in Infrastructure Details.
The Log Collector uses Elastic Filebeat containers that are deployed within each Tenant.
When you enable a Tenant for logging, the Filebeat service starts up and begins log collection. View the Filebeat containers by navigating to Kubernetes -> Containers in the DuploCloud Portal. In the row of the container for which you want to view the logs, click on the menu icon and select Logs.
When you perform the steps above to configure logging, DuploCloud does the following:
An EC2 Host is added in the Default tenant, for example, duploservices-default-oc-diagnostics.
Services are added in the Default tenant, one for OpenSearch and one for Kibana. Both services are pinned to the EC2 host using allocation tags. Kibana is set up to point to ElasticSearch and exposed using an internal load balancer.
Security rules from within the internal network to port 443 are added in the Default Tenant to allow log collectors that run on Tenant hosts to send logs to ElasticSearch.
A Filebeat service (filebeat-duploinfrasvc) is deployed for each Tenant where central logging is enabled.
The /var/lib/docker/Containers are mounted from the Host into the Filebeat container. The Filebeat container references ElasticSearch, which runs in the Default Tenant. Inside the container, Filebeat is configured so that every log line is added with metadata information consisting of the Tenant name, Service names, Container ID, and Hostname, enabling ease of search using these parameters with ElasticSearch.
