Set up logging for the DuploCloud Portal
If you need to make changes to the Control Plane Configuration, follow this procedure to do so, before enabling logging. Note that you cannot modify the Control Plane Configuration after you set up logging.
Docker applications use stdout
for writing log files, collecting logs, placing them in the Host directory, mounting them into Filebeat containers, and sending them to AWS Elasticsearch. If you need to customize the log collection and you use folders other than stdout
, for example, follow this procedure. Note that you cannot customize the log collection after you set up logging.
In the DuploCloud Portal, navigate to Administrator -> Observability -> Settings -> Logging.
From the Tenant list box at the top of the DuploCloud Portal, select the Default Tenant.
Click the Create Logging link. The Enable Logging page displays.
Use the Enable Logging page to deploy logging for the Control Plane, which uses OpenSearch and Kibana to retrieve and display log data for the Default Tenant. In the Cert ARN field, enter the ARN certificate for the Default Tenant. Find the ARN by selecting the Default Tenant from the Tenant list box at the top of the DuploCloud Portal; navigating to Administrator -> Plans; selecting the Plan that matches your Infrastructure Name; and clicking the Certificates tab.
Click Submit. Data gathering takes about fifteen (15) minutes. When data gathering is complete, graphical logging data is displayed in the Logging tab.
After logging has been enabled for the Control Plane, finish the logging setup by enabling the Log Collector to collect logs per Tenant. This feature is especially useful for Tenants that are spread across multiple regions. In the DuploCloud Portal, navigate to Administrator -> Observability -> Settings -> Logging.
In the Logging tab, on the Logging Infrastructure Tenants page, click Add.
Select the Tenants for which you want to configure logging, using the Select Tenants to enable logging area, as in the example below. The Control Plane configuration is deployed for each Tenant that you select in the Infrastructure, specified in Infrastructure Details.
The Log Collector uses Elastic Filebeat containers that are deployed within each Tenant.
When you enable a Tenant for logging, the Filebeat service starts up and begins log collection. View the Filebeat containers by navigating to Kubernetes -> Containers in the DuploCloud Portal. In the row of the container for which you want to view the logs, click on the menu icon and select Logs.
When you perform the steps above to configure logging, DuploCloud does the following:
An EC2 Host is added in the Default tenant, for example, duploservices-default-oc-diagnostics.
Services are added in the Default tenant, one for OpenSearch and one for Kibana. Both services are pinned to the EC2 host using allocation tags. Kibana is set up to point to ElasticSearch and exposed using an internal load balancer.
Security rules from within the internal network to port 443 are added in the Default Tenant to allow log collectors that run on Tenant hosts to send logs to ElasticSearch.
A Filebeat service (filebeat-duploinfrasvc)
is deployed for each Tenant where central logging is enabled.
The /var/lib/docker/Containers
are mounted from the Host into the Filebeat container. The Filebeat container references ElasticSearch, which runs in the Default Tenant. Inside the container, Filebeat is configured so that every log line is added with metadata information consisting of the Tenant name, Service names, Container ID, and Hostname, enabling ease of search using these parameters with ElasticSearch.
Logging for AWS in the DuploCloud Platform
The DuploCloud Platform performs centralized logging for Docker-based applications. For the native and Kubernetes container orchestrations, this is implemented using OpenSearch and Kibana with Elastic Filebeat as the log collector.
For ECS Fargate, AWS Lambda, and AWS SageMaker Jobs, the platform integrates with CloudWatch, automatically setting up Log Groups and making them viewable from the DuploCloud Portal.
No setup is required to enable logging for ECS Fargate, Lambda, or AWS SageMaker Jobs. DuploCloud automatically sets up CloudWatch log groups and provides a menu next to each resource.
Display logs for the DuploCloud Portal, components, services, and containers
The central logging dashboard displays detailed logs for Service and Tenant. The dashboard uses Kibana and preset filters that you can modify.
In the DuploCloud Portal, navigate to Observability -> Logging.
Select the Tenant from the Tenant list box at the top of the DuploCloud Portal.
Select the Service from the Select Service list box.
Modify the DQL to customize Tenant selection, if needed.
Adjust the date range by clicking Show dates.
Add filters, if needed.
DuploCloud pre-filters logs per Tenant. All DuploCloud logs are stored in a single index. You can see any Tenant or combination of Tenants (using the DQL option) but the central logging control plane is shared, with no per-Tenant access.
Confirm that your Hosts and Services are running or runnable to view relevant log data.
See Kubernetes Containers for information on displaying logs per container.
Change configuration for the Control Plane, customize Platform Services
There are several use cases for customized log collection. The central logging stack is deployed within your environment, as with any other application, streamlining the customization process.
The version of OpenSearch, the EC2 host size, and the control plane configuration are all deployed based on the configuration you define in the Service Description. Use this procedure to customize the Service Description according to your requirements.
You must make Service Description changes before you enable central logging. If central logging is enabled, you cannot edit the description using the Service Description window.
In the DuploCloud Portal, navigate to Administrator -> System Settings.
In the Service Description tab, in the Name column, select duplo_svd_logging_opensearch. The Service Description window displays.
Edit the YAML in the Service Description window as needed.
Click Update when the configuration is complete to close the window and save your changes.
You can update the Control Plane configuration by editing the Service Description. If the control plane is already deployed using the Service Description specification, then updating the description is similar to making a change to any application.
Note that Control Plane Components are deployed in the DuploCloud Default Tenant. Using the Default Tenant, you can change instance size, Docker images, and more.
You can update the log retention period using the OpenSearch native dashboard by completing the following steps.
From the DuploCloud portal, navigate to Administrator -> Observability -> Logging.
Click Open New Tab to access the OpenSearch dashboard.
Navigate to Pancake -> Index management -> State management policies.
Edit the FileBeat YAML file and update the retention period.
For more information see the OpenSearch documentation.
The new retention period settings will only apply to logs generated after the retention period was updated. Older logs will still be deleted according to the previous retention period settings.
You can modify Elastic Filebeat logging configurations, including mounting folders other than /var/lib/docker
for writing logs to folders other than stdout
.
You need to customize the log collection before enabling logging for a Tenant.
If logging is enabled, you can update the Filebeat configuration for each tenant by editing the Filebeat Service Description (see the procedure in Defining Control Plane Configuration).
Alternately, delete the Filebeat collector from the Tenant and the platform automatically redeploys based on the newest configuration.
In the DuploCloud Portal, navigate to Administrator -> System Settings.
Select the Platform Services tab.
Click the Edit Platform Services button. The Platform Services window displays. Select the appropriate Filebeat service. For native container management, select filebeat; for Kubernetes container management, select filebeat-k8s.
Edit the YAML in the Platform Services window as needed.
Click Update to close the window and save your changes.
With DuploCloud, you have the choice to deploy third-party tools such as Datadog, Sumo Logic, and so on. To do this, deploy Docker containers that act as collectors and agents for these tools. Deploy and use these third-party app containers as you would any other container in DuploCloud.