.svg?alt=media&token=0f6df7c7-5b98-4a62-98b8-a54e38a8329e){kind=logo}
Central Logging Setup
To enable central logging go under Administrator --> Diagnostics --> Settings --> Logging.
Setup of Central Logging comprises two steps or components:
Control Plane: This comprises of Open Search and Kibana which are deployed in the Default tenant.
It takes about 10-15 mins for the control plane to setup and when ready you should see the screen under Administrator--.>Diagnostics --> Setting --> Logging Tab to look like the picture below
Log Collector: After the Control plane has been deployed and is active, logging per tenant has to be enabled as well. This can be done from under Administrator --> Diagnostics --> Settings and at the bottom of the page you see the option to enable Tenant logging. See figure above
Log collection is implemented by using filebeat containers that is deployed in each tenant. You can see these filebeat containers running within the tenant from under Devops --> Containers -->EKS/Native.
For the Log Collector to show meaningful logs, it is necessary for the docker applications to write the logs to stdout. Only then will Docker Engine will collect the logs and place them in the host directory from where those are first mounted into FileBeat container and from there sent to the Elastic Search
Following orchestration steps were performed behind the scenes:
- 1.Control plane deployment:
- 1.Add Hosts: The platform added an Ec2 instance under the default tenant. This Host is typically called duploservices-default-oc-diagnostics.
- 2.Added 2 services, under default tenant, one for open-search and other for Kibana. They were both pinned to the host added above using allocation-tags. The Kibana was setup to point to this elastic Search. The kibana was exposed via an internal load balancer
- 3.Security rules from within the internal network to port 443 were added under the default tenant to allow the log collectors that will run on Tenant hosts to send logs to the elastic search.
- 2.Log Collector: The file beat setup comprises of a service called filebeat-duploinfrasvc deployed for each tenant where the central logging in enabled. The service configuration includes mounting the /var/lib/docker/Containers from the host into the filebeat container. The filebeat container is pointing to the elastic search running in default tenant. Inside the container the file beat has been setup in such a way that each log line is added with the meta-data information comprising of Tenant Name, Service Names, Container ID and Host Name. Thus inside the elastic search it is easy to separate the logs by these parameters
By default, all the monitoring components are deployed and configured in the Default Tenant.
If you need a separate logging setup for the Tenant configured in a region other than Default, DuploCloud provides the ability to configure a separate logging infrastructure per Tenant. You can create it by going to the Logging Tab, click on Add, and select the Tenant where the logging setup needs to be configured. Control Plane will be deployed in the Tenant specified here.
Log Collector for Tenant in sharing the same Infrastructure will use the logging setup configured in the above step.
This feature is mainly configured, when the Tenants are spread across in multiple regions or the Administrator wants to create a separate logging setup.
Last modified 5mo ago