When creating an Infrastructure, use the Enable GKE toggle switch to enable GKE. In the Cluster Mode list box, select GKE Autopilot. Complete the remaining fields (GKE Version, GKE Endpoint Visibility, and Cluster IP CIDR). Click Create.

This takes about 20 minutes. Infrastructure status should move to Completed. Once the Infrastructure status shows Complete, navigate to Administrators -> Plans to verify that a Plan has been created with the same name (nonprod).

Configuring Tenant Config settings

You can configure settings to apply to all new Tenants under a Plan using the Config tab. Tenant Config settings will not apply to Tenants created under the Plan before the settings were configured.

1. From the DuploCloud portal, navigate to Administrator -> Plan.

2. Click on the Plan you want to configure settings under in the NAME column.

3. Select the Config tab.

4. Click Add. The Add Config pane displays.

5. From the Config Type field, select TenantConfig.

6. In the Name field, enter the setting that you would like to apply to new Tenants under this Plan. (In the example, the enable_alerting setting is entered.)

7. In the Value field, enter True.
8. Click Submit. The setting entered in the Name field (enable alerting in the example) will apply to all new Tenants added under the Plan.

Viewing Tenant Config settings

You can check that the Tenant Config settings are enabled for new Tenants on the Tenants details page, under the Settings tab.

1. From the DuploCloud portal, navigate to Administrator -> Tenants.

2. From the NAME column, select a Tenant that was added after the Tenant Config setting was enabled.

3. Click on the Settings tab.

4. Check that the configured setting is listed in the NAME column. (Enable Alerting in the example.)

DuploCloud provides a way to connect directly to the Cluster namespace using the kubectl token.

See kubectl Setup for available options.

When creating an Infrastructure, use the Enable GKE toggle switch to enable GKE. In the Cluster Mode list box, select GKE Standard. Complete the remaining fields (GKE Version, GKE Endpoint Visibility, and Cluster IP CIDR). Click Create.

This takes about 20 minutes. Infrastructure status should move to Completed. Once the Infrastructure status shows Complete, navigate to Administrators -> Plans to verify that a Plan has been created with the same name.

View GKE Cluster details

You can view the details and download the kubeconfig file to connect the cluster from GKE Tab available in the infrastructure created.

Google frequently updates the version of GKE based on new features available in the Kubernetes platform.

In future releases, upgrades will be available for customers to install.

Organization of use cases

Topics in this section are covered in the order of typical usage. Use cases that are foundational to DuploCloud such as Infrastructure, Tenant, and Hosts are listed at the beginning of this section; while supporting use cases such as Logs, Metrics, and Faults and alerts appear near the end.

Supported use cases for DuploCloud GCP

• Infrastructure and Plan

• Tenant

• Hosts

• Kubernetes Jobs and CronJobs

• Logs

• Metrics

• Faults and alerts

• Cost management for billing

Managing Tenant Expiration

In the DuploCloud Portal, configure an expiration time for a Tenant. At the set expiration time, the Tenant and associated resources are deleted.

1. In the DuploCloud Portal, navigate to Administrator -> Tenants. The Tenants page displays.

2. From the Name column, select the Tenant for which you want to configure an expiration time.
3. From the Actions list box, select Set Tenant Expiration. The Tenant - Set Tenant Expiration pane displays.
4. Select the date and time (using your local time zone) when you want the Tenant to expire.

5. Click Set. At the configured day and time, the Tenant and associated resources will be deleted.

In GCP, cloud features such as Resource Groups, Identity and Access Management (IAM), Security Groups, Cloud KMS, as well as Kubernetes Namespaces, are exposed in Tenants which reference their configurations.

For more information about DuploCloud Tenants, see the Tenants topic in the DuploCloud Common Components documentation.

Creating a Tenant

1. Navigate to Administrator -> Tenant in the DuploCloud Portal and click Add. The Create a Tenant pane displays.

2. In the Name field, enter a name for the Tenant. Choose unique names that are not substrings of one another, for example, if you have a Tenant named dev, you cannot create another named dev2. We recommend using distinct numerical suffixes like dev01 and dev02.

3. In the Plan list box, select the Plan to associate the Tenant with.

4. Click Create. The Tenant is created.

By exporting your Google Cloud Platform (GCP) billing data to BigQuery, you can leverage DuploCloud's dashboard to monitor and analyze your GCP billing effectively.

Prerequisites

To export to BigQuery you must have:

• A Google Cloud Platform account with billing enabled.

• Permission to access the Google Cloud Billing API and BigQuery.

  • Billing Account Administrator permissions

  • BigQuery Admin permissions

Creating a BigQuery Dataset

1. Navigate to the Console in your Google Cloud Platform account.

2. In GCP, select the Project where you want to create the dataset.

3. Click Create Dataset.
4. In the Create dataset window, configure your dataset with the following parameters:

   • Dataset ID: Enter a unique name for your dataset.

   • Location Type: Select Multi-Region.

   • Default table expiration: Select Enable table expiration and set a default expiration time for tables in this dataset, such as 60 days. Tables will be automatically deleted after this period.

   • Click Create Dataset.

   • Once the dataset is created, it appears in the BigQuery Console under your project. Select the dataset to view details.

Enabling billing export to BigQuery

2. Select Billing from the main menu or visit Google Cloud Billing.

3. Select the billing account for which you want to enable the billing export.

4. In the Billing Account Details page, select Billing Export from the left navigational pane.

5. In the Billing Export page, in the Detailed usage cost area, click Edit Settings.
6. In the BigQuery Export tab, configure Detailed usage cost.

   • Select the Project: Choose the project where you created the BigQuery dataset.

   • Select the Dataset: Choose the dataset you created for billing data.
7. Click Save.

8. Contact DuploCloud Support to complete additional steps to enable the billing dashboard.

Once we have the Infrastructure (Networking, Kubernetes cluster, and other common configurations) and an environment (Tenant) set up, the next step is to create VMs. These could be for:

• Compute Engine virtual machines in GCP

• Worker Nodes (Docker Hosts) if built-in container orchestration is used.

• Regular nodes that are not part of any container orchestration, where a user manually connects and installs applications.

In the DuploCloud Portal, you can create GCE VMs, Node Pools, or BYOH (bring your own Hosts) virtual machine setups.

Creating a GCE VM

To create a GCE VM (Virtual Machine), navigate to Cloud Services -> Hosts -> GCE VM, and click Add.

Creating a Node Pool

A GCP node pool is a group of VMs that share the same configuration, including machine type, disk size, and operating system. Node pools allow you to manage scaling and updates for groups of nodes collectively.

To create a GCP node pool, navigate to Cloud Services -> Hosts -> Node Pool, and click Add.

Creating a BYOH Host

While lower-level details such as IAM roles and security groups are abstracted, deriving instead from the Tenant, only the most application-centric inputs are required to set up Hosts.

Most of these inputs are optional and some are available as list box selections, set by the administrator in the Plan (for example, Image ID, in Host Advanced Options).

There is an additional parameter labeled Fleet Type. This is applicable if the VM is to be used as a host for container orchestration by the platform. The choices are:

• Linux Docker/Native: To be used for hosting Linux containers using the Built-in Container orchestration.

• None: To be used for non-Container Orchestration purposes and contents inside the VM are self-managed by the user.

Increasing minimum ports per VM instance (GKE Standard)

You can increase the number of available ephemeral ports per GKE Standard VM instance in the DuploCloud Portal using Infrastructure systems settings. More ports help handle high volumes of network traffic, especially for applications that require many simultaneous connections.

To increase the minimum ports per VM for your Infrastructure:

• Navigate to Administrator -> Infrastructure.

• In the NAME column, select your Infrastructure name.

• Select the Settings tab, and click Add. The Infra - Set Custom Data pane displays.

• From the Setting Name list box, select GKE Minimum Ports Per VM.

• In the Setting Value field, enter the minimum number of ports you want or each VM.

• Click Set. VMs in this Infrastructure will have at least the minimum number of ports configured.

Configuring a friendly image name

Set a friendly name for an image in your DuploCloud Plan. This name will display in the Image list box when creating a GCE VM in the DuploCloud Portal.

1. From the DuploCloud Portal, navigate to Administrator -> Plans.

2. Select the Plan from the NAME column.

3. Select the Images tab, and click Add. The Add Image pane displays.

1. Enter a friendly name and complete the remaining fields, as required. Click Submit. The image name will display in the Image list box when creating a GCE VM under the Plan.

Creating an Infrastructure

1. Click Administrator -> Infrastructure from the navigation menu.

2. Click Add.

3. Complete the fields on the Add Infrastructure form to define the infrastructure.

4. Optionally, click Enable GKE and complete the additional fields to create a GKE Autopilot or Standard Cluster.

5. Click Create. The Infrastructure is created and is listed on the Infrastructure page. DuploCloud automatically creates a Plan with the same name and configuration as the Infrastructure.

Usage costs for resources can be viewed and managed in the DuploCloud Portal, by month or week, and by Tenant. You can also explore historical resource costs.

To view the Billing page for GCP in the DuploCloud Portal, click Administrator -> Billing.

You can view usage by:

• Time

  • Select the Spend by Month tab and click More Details to display monthly and weekly spending options.

• Tenant

  • Select the Spend by Tenant tab.

Purpose

In Google Cloud Platform (GCP), billing data can be exported to a BigQuery dataset in only one project. However, when deploying instances of an application across multiple projects (e.g., dev, qa, stg, prod), it is necessary to replicate the billing dataset to enable billing monitoring on all DuploCloud dashboards in these projects. This documentation outlines the steps to configure automated replication of a BigQuery dataset from a source project to a destination project.

Prerequisites

• Two GCP projects: a source project where the original billing dataset resides, and a destination project where the dataset will be replicated.

• Appropriate permissions to create datasets and data transfer jobs in BigQuery.

• Google Cloud SDK installed and initialized.

• Enable BigQuery Data Transfer API from API and Services in Destination GCP Project.

Assumptions:

• Source Project: GCP project where the original billing dataset resides with billing export.

• Destination Project: New GCP project which has duplo-master running and dataset needs to be created.

Steps

1. Create a New Dataset in the Destination Project

• Open the BigQuery console in the source project: BigQuery Console

• Click on CREATE DATASET.

• Enter the dataset ID, choose a data location, and set other options as mentioned in the below screenshot.

• Click Create dataset.

2. Give required permissions

For the replication to work, you need to allow specific roles on the dataset in source project to the duplo-master GCP service account of the destination project

Following roles are needed:

• BigQuery Admin

• BigQuery Data Viewer

• BigQuery Data Editor

• BigQuery User

3. Set Up a Data Transfer Job in the Destination Project

• Open the BigQuery console in the destination project.

• In the left-hand menu, click on Data Transfers.

• Click on CREATE TRANSFER.

• Select Source Type as Dataset Copy

• Schedule options: Choose Start now. Set the frequency option to every 12 hours.

• Under the Destination Settings

  • Put destination project dataset as Dataset

  • Put source project dataset as Source Dataset

  • Put source project ID as Source Project

  • Enable checkbox Overwrite destination table

  • Under Service Account select the destination duplo-master service account (which has the permission to access the source project dataset)

  • Click SAVE

4. Monitor the Transfer Job

• In the BigQuery console of the destination project, go to the Transfers tab.

• You should see your transfer job listed. You can click on it to view details and monitor its progress.

Conclusion

By following these steps, you can set up automated replication of a BigQuery dataset from one GCP project to another, enabling billing monitoring on all DuploCloud dashboards across multiple projects. Ensure to monitor the transfer job periodically to make sure it is running as expected.