Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Using DuploCloud with Google Cloud Platform
The DuploCloud platform installs a Virtual Machine resource within your GCP Project. It can be accessed using a web interface, API, and a Terraform provider. Log in to the DuploCloud portal via SSO through your GSuite or O365 login.
Read through the DuploCloud Platform Overview and are familiar with DuploCloud terms such as Infrastructure, Plan, and Tenant.
Before you begin, ensure that:
DuploCloud Portal has been set up and you have access to it.
You have access to your individual Slack or Teams channel for 24x7 support from the DuploCloud team.
Behind the scenes, a topology is created similar to the following low-level configuration in GCP.
Key concepts for using DuploCloud with Docker and GCP
While deploying Dockerized applications, familiarize yourself with some key concepts and terminologies.
These are virtual machines. In GCP deployments, they are also called Worker nodes. By default, apps within a Tenant are pinned to VMs in the same Tenant. DuploCloud has the ability to deploy Hosts in a separate Tenant and apps in other Tenants that leverage these Hosts. This is called Shared Host Model and is not applicable to GCP.
Service is a DuploCloud term. DuploCloud Services are not Kubernetes Services. Services are microservices that are defined by a Name, DockerImage, and a number of replicas in addition to many other optional parameters. Behind the scenes, a DuploCloud Service maps 1:1 either to a Kubernetes deployment set or to a StatefulSet depending on whether the microservice has stateful volumes or not. There are many optional configurations associated with a DuploCloud Service that represent various ways Docker containers can be run. A few of these are:
Environment variables
Host Network Mode
Volume mounts
Entrypoint or command overrides
Resource caps
Health Checks
If a service needs to be pinned to run only a specific set of hosts, set an Allocation Tag on the Hosts as well as on the Service. The Allocation Tag is a case-insensitive substring match. For example, an Allocation Tag specified on a service is usually a substring of the tag specified on the host. A Host may be tagged as HighCpu;HighMem and the Service (if tagged highcpu) can be allocated on the Host. However, if the service is tagged highcpu;gpu then it won't be allocated and needs a host that has been tagged highcpu;gpu. If a Service does not have any tag set, it can be placed on any host.
If the Host is tagged with a specific value and you have Services with the same tag, the host is available for any Service which has no tags. If you want the exclusive assignment of a Host to a set of Services, ensure that every Service in the Tenant is tagged with some value.
In the case of Kubernetes deployments, the concept of Allocation Tags maps to labels on nodes, and on node selectors on the deployment set or StatefulSet.
Host Networking: By default, Docker containers have their own network addresses. you may want these containers to use the same network interface as the VM. This is called Host Network Mode.
Load Balancer: If a service must be accessed by other services, it needs to be exposed using internal and external load balancers.
Tasks to perform before you use GCP with DuploCloud
Before using DuploCloud, ensure the following prerequisites are met.
Read the Access Control section to ensure at least one person has administrator access.
Creating a Route 53 hosted zone to program DNS entries
The DuploCloud platform needs a unique GCP Cloud DNS zone to create DNS entries for services that you deploy. The domain must be created out-of-band and set in DuploCloud. The zone is a subdomain such as apps.[
MY-COMPANY
].com
.
Never use this subdomain for anything else, as DuploCloud owns all CNAME
entries in this domain and removes all entries it has no record of.
To create the Route53 hosted zone using the GCP Console:
Log in to the GCP console.
Navigate to Cloud DNS under Network Services.
Create a new zone with the desired domain name, for example, apps.acme.com
.
Access the zone and note the name server names.
Go to your root Domain Provider's site (for acme.com
, for example), and create a NS
record that references the domain name of the hosted zone you created (apps.acme.com
) and add the zone name to the name servers that you noted above.
Once this is complete, provision the Zone in every DuploCloud Plan, starting with the default plan. Add the zone Name and domain name, preceded with a dot (.).
Do not forget the dot (.) at the beginning of the DNS suffix, in the form as shown below.
Note that this domain must be set in each new Plan you create in your DuploCloud Infrastructure.
Set Docker registry credentials
In the DuploCloud Portal, navigate to Docker -> Services. Docker registry credentials are passed to the Kubernetes cluster as kubernetes.io/dockerconfigjson
.
Click the Docker item list in the upper right and select Docker Credentials. The Set Docker registry Creds pane displays.
Supply the credentials and click Submit.
Enable the Docker Shell Service by clicking Enable Docker Shell.
You can pull images from multiple Docker registries by adding multiple Docker Registry Credentials.
In the DuploCloud Portal, click Administrator -> Plan. The Plans page displays.
Select the Plan in the Name column.
Click the Config tab.
Click Add. The Add Config pane displays.
Creating the DuploCloud Infrastructure and a Plan
Each DuploCloud Infrastructure is a connection to a unique Virtual Private Cloud (VPC) network that resides in a region that can host Kubernetes clusters.
After you supply a few basic inputs, DuploCloud creates an Infrastructure for you within Google Cloud Platform (GCP) and DuploCloud, with a few clicks. Behind the scenes, DuploCloud does a lot with what little you supply — generating the VPC, Subnets, NAT Gateway, Routes, and Google Kubernetes Engine (GKE) cluster.
With the Infrastructure as your foundation, you can customize an extensible, versatile Platform Engineering development environment by adding Tenants, Hosts, Services, and more.
Estimated time to complete Step 1: 20 minutes. Much of this time is consumed by DuploCloud's creation of the Infrastructure and enabling your GKE cluster with Kubernetes.
Before starting this tutorial:
Learn more about DuploCloud Infrastructures, Plans, and Tenants.
Reference the Access Control documentation to create User IDs with the Administrator role. To perform the tasks in this tutorial, you must have Administrator privileges.
In the DuploCloud Portal, navigate to Administrator -> Infrastructure. The Infrastructure page displays.
Click Add. The Add Infrastructure page displays.
From the table below, enter the values that correspond to the fields on the Add Infrastructure page. Accept all other default values for fields not specified.
Use the toggle switch to Enable GKE.
Select either GKE Autopilot or GKE Standard options. You will follow different paths in the tutorial for creating clusters with GKE Autopilot or GKE Standard.
Click Create to create the Infrastructure. DuploCloud begins creating and configuring your Infrastructure and GKE Cluster using Kubernetes. It may take up to twenty (20) minutes to create the Infrastructure.
It may take up to twenty (20) minutes for your Infrastructure to be created and Kubernetes (GKE) enablement to be complete. Use the Kubernetes card in the Infrastructure screen to monitor the status, which should display as Enabled when completed. You can also monitor progress by using the Kubernetes tab, as DuploCloud generates your Cluster Name, Default VM Size, Server Endpoint, and Token.
Every DuploCloud Infrastructure generates a Plan. Plans are sets of templates that are used to configure the Tenants or workspaces, in your Infrastructure. You will set up Tenants in the next tutorial step.
Before proceeding, confirm that a Plan exists that corresponds to your newly created Infrastructure.
In the DuploCloud Portal, navigate to Administrator -> Plans. The Plans page displays.
Verify that a Plan exists with the name you gave to the Infrastructure you created.
You previously verified that your Infrastructure and Plan were created. Now verify that Kubernetes is enabled before proceeding to Create a Tenant.
In the DuploCloud Portal, navigate to Administrator -> Infrastructure. The Infrastructure page displays.
From the NAME column, click on the name of the Infrastructure you created.
Click the GKE tab. When Kubernetes has been Enabled for GKE, details are listed in the tab. The Infrastructure page displays the Enabled status on the Kubernetes card for GKE Clusters.
When an Infrastructure is created, a GKE Cluster is created by default. You can view the details and download the kubeconfig file from the DuploCloud portal.
From the DuploCloud portal, navigate to Administrator -> Infrastructure. Click on the name of the Infrastructure, and select the GKE tab. To download the kubeconfig file, click Download Kube Config.
Enabling shell access using Docker Native
DuploCloud allows shell access into the deployed containers.
In the DuploCloud Portal, navigate to Docker -> Services, displaying the Services page.
From the Platform list box, select Docker Native.
From the Certificate list box, select a certificate name.
From the Visibility list box, select Public.
Click Update.
A provisioned service named dockerservices-shell is created, enabling you to access the Service containers using SSH.
Connecting to the DuploCloud VPN with the OpenVPN client
DuploCloud integrates natively with OpenVPN by provisioning VPN users added in the Duplocloud portal. As a DuploCloud user, you can access resources in the private network by connecting to the VPN with the OpenVPN client.
The OpenVPN Access Server is set to forward only traffic destined for network resources in the DuploCloud-managed private networks. Traffic accessing other resources on the internet does not pass through the tunnel.
User VPN credentials are accessible on the User Profile page. It can be accessed through the menu on the upper right of the page or through the User menu option on the left.
Click the VPN URL link in the VPN Details section of your user profile. Browsers will call the link unsafe since it is using a self-signed certificate. Proceed to it.
Log in to the OpenVPN Access Server user portal using the credentials from the DuploCloud user profile section.
Open the .ovpn file and click OK in the Import .ovpn profile dialog.
Establish secure access to the DuploCloud portal with regional or global SSL certificates for GCP
Although DuploCloud supports both certificate configuration methods, we recommend avoiding using compute engine certificates, if possible. This is because compute engine certificates can't be validated until they're attached to a Load Balancer, which can make it hard to manage uptime. In contrast, certificate maps can be validated in advance, circumventing potential downtime.
Create a DNS authorization resource using the following command where YOUR_DOMAIN is your domain URL and MAP_NAME is your certificate name (a unique name you choose for your certificate map).
Manually create the DNS records shown in the output of the list
command. You'll usually do this in the certificate's domain zone in the Cloud DNS service for the same project, but it depends on how you set up DNS.
Create the certificate:
Create the certificate map and its entries:
Add the certificate map in the DuploCloud Plan. Navigate to Administrator -> Plans. Select the Certificates tab and click Add. The Add a Certificate pane displays.
In the Name field, create a name for the certificate (the name is arbitrary as it is only a display name to be used within DuploCloud).
In the GCP Certificate Type list box, select the certificate type. The certificate type must match the certificate entered in the gcloud certificate-manager maps entries create
command.
In the GCP Certificate Map field, enter the name of your map (in this example, MAP_NAME). Click Create.
Now you can use your certificate with your DuploCloud Services.
Add Infrastructure page field | Value |
---|---|
Click the Options Menu ( ) on the top row of the Services page, as in the example below. Select Enable Docker Shell. The Start Shell Service pane displays.
Install the OpenVPN Connect application on your local machine.
Download the OpenVPN user profile for your account from the link labeled Yourself (user-locked profile).
Click Connect.
SSL certificates secure connections between clients and servers or Load Balancers by encrypting information sent over the network using Transport Layer Security (TLS). GCP users have two options to configure SSL certificates: Compute Engine SSL certificates resource (compute engine certificates) and Certificate Manager (certificate maps). For more information, see the Google Cloud documentation about the different and .
Name
YOUR_INFRA_NAME
Account
YOUR_GOOGLE_ACCOUNT
VPC CIDR
10.10.0.0/16
Cloud
Google
Region
YOUR_GEOGRAPHIC_REGION
Subnet
CIDR 22
Get up and running with DuploCloud inside a Google Cloud Platform environment; harness the power of generating application infrastructures.
This quick-start tutorial shows you how to set up an end-to-end cloud deployment. You will create Google Cloud Platform infrastructure and Tenants. By the end of this tutorial, you should be able to view the deployed sample web application.
Estimated time to complete tutorial: 60-70 minutes.
When you complete the GCP Quick Start Tutorial, you have two options or paths, as shown in the table below.
Using GKE Autopilot - You create an app and service in DuploCloud using Google Kubernetes Engine-Autopilot and expose it using a load balancer within DuploCloud.
Using GKE Standard - You create an app, service and a node pool in DuploCloud using Google Kubernetes Engine-Standard and expose it using a load balancer within DuploCloud.
For beginners, we recommend you use GKE Autopilot. GKE Autopilot manages the infrastructure, including the nodes, node pools, and underlying infrastructure resources such as networking and storage. You do not need to manage or configure node pools, node instance types, or autoscaling policies.
GKE Standard offers more granular control over resource management, including the ability to configure node pools with specific types of instances, set scaling policies, and manage node upgrades.
For a high-level comparison of GKE Autopilot and GKE Standard and to help you choose which method best suits your needs, skills, and environments, see this Google Cloud documentation.
* - Optional Step
Click the card below to watch a DuploCloud GCP demo.
Creating a DuploCloud Tenant that segregates your workloads
Now that the Infrastructure and Plan exist and a Kubernetes GKE Cluster has been enabled, create one or more Tenants that use the configuration DuploCloud created.
Tenants in DuploCloud are similar to projects or workspaces and have a subordinate relationship to the Infrastructure. Think of the Infrastructure as a virtual "house" (cloud), with Tenants conceptually "residing" in the Infrastructure performing specific workloads that you define. As Infrastructure is an abstraction of a Virtual Private Cloud, Tenants abstract the segregation created by a Kubernetes Namespace, although Kubernetes Namespaces are only one component that Tenants can contain.
Estimated time to complete Step 2: 10 minutes.
DuploCloud customers often create at least two Tenants for their production and non-production cloud environments (Infrastructures).
For example:
Production Infrastructure
Pre-production Tenant - for preparing or reviewing production code
Production Tenant - for deploying tested code
Non-production Infrastructure
Development Tenant - for writing and reviewing code
Quality Assurance Tenant - for automated testing
In larger organizations, some customers create Tenants based on application environments, such as creating one Tenant for Data Science applications and another Tenant for web applications, and so on.
Tenants are sometimes created to isolate a single customer workload, allowing more granular performance monitoring, scaling flexibility, or tighter security. This is referred to as a single-Tenant setup.
Before creating a Tenant, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
An Infrastructure and Plan exist, both with the name you created.
The Infrastructure has Kubernetes (GKE) Enabled.
Create a Tenant for your Infrastructure and Plan:
In the DuploCloud Portal, navigate to Administrator -> Tenants.
Click Add. The Create a Tenant pane displays.
Enter a unique name for your Tenant in the Name field.
Select the Plan that you created in the previous step.
Click Create.
It may take 1-2 minutes for the Tenant to be set up. While the Tenant is setting up, a temporary fault may show up under Administrator -> Faults. This fault can be ignored, as it should clear within the first 2 minutes.
From the DuploCloud portal, navigate to Administrator -> Tenants, and verify that a Tenant exists with the name and Plan you created.
Creating a Load Balancer to configure network ports to access the application
Now that your DuploCloud Service is running, you have a mechanism to expose the containers and images in which your application resides. But because your containers are running inside a private network, you also need a load balancer to listen on the correct ports in order to access the application.
In this step, we add a Load Balancer Listener to complete this network configuration.
Estimated time to complete Step 4: 10 minutes.
Before creating a Load Balancer, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
All containers are running inside a private network and cannot be accessed from an external network. To do so one can create a load balancer.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
From the Name column, click on the name of your Service
Click the Load Balancers tab.
Click the Configure Load Balancer link. The Add Load Balancer Listener pane displays.
From the Type list box, select Application LB.
In the Container Port field, enter 80. This is the configured port on which the application inside the Docker Container Image is running.
In the External Port field, enter 80. This is the port through which users will access the web application.
From the Visibility list box, select Public.
From the Application Mode list box, select Docker Mode.
Type / (forward-slash) in the Health Check field to indicate that the cluster we want Kubernetes to perform Health Checks on is located at the root
level.
In the Backend Protocol list box, select HTTP.
Click Add. The Load Balancer is created and initialized. In approximately 2-3 minutes you will see the load balancer details available in the portal. When the Load Balancer is ready for use the LB Status card displays Ready.
From the DuploCloud portal, navigate to Kubernetes -> Services.
Click on the name of your Service.
Verify that the Load Balancer has a status of Ready on the LB Status card.
Creating a Kubernetes Service to run a Docker-containerized application
In this exercise, we will create a simple Google Cloud Nginx service. When you run the application, DuploCloud accesses Docker images in a preconfigured Docker Hub.
Estimated time to complete Step 3: 10 minutes.
Before creating a Service, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
Click Add. The Add Service page displays.
In the Service Name field, enter a name for the service (in the example below, the name is "myservice").
In the Docker image field, enter the docker image (nginx:latest
).
Click Next. The Advanced Options page is displayed.
At the bottom of the Advanced Options page, click Create. Your Service is created and initialized.
It may take approximately five (5) minutes for the Service to initialize. Use the Containers page (Kubernetes -> Containers) to monitor the Service creation status, between Desired (Running) and Current.
From the DuploCloud portal, navigate to Kubernetes -> Services, and verify that your DuploCloud Service has a Current status of Running.
Test the application to ensure you get the results you expect
You can test your application directly from the Services page.
Estimated time to complete Step 5 and finish tutorial: 10 minutes.
Before creating a Load Balancer, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
From the Name column, click on the Service you created.
Click the Load Balancers tab. The Application Load Balancer configuration is displayed.
Open a browser instance and Paste the IP Address in the URL field of your browser.
Press ENTER. A web page with the text Welcome to nginx! is displayed.
Congratulations! You have just launched your first web service on DuploCloud!
Step | GKE Autopilot | GKE Standard |
---|---|---|
An exist, both with the name you created.
The Infrastructure you created has .
A with the name you chose has been created.
A with the name you chose has been created.
When you run your own applications, you will choose a public image or provide credentials to access your private repository. Before you deploy your own applications, .
An exist, both with the name that you chose.
The Infrastructure has .
A Tenant with the .
An exist, both with the name you created.
The Infrastructure you created has .
A with the name you chose has been created.
A with the name you chose has been created.
An has been created.
In the LB Configuration card, click the Copy Icon ( ) to copy the IP Address displayed to your clipboard.
1
Create Infrastructure and Plan
Create Infrastructure and Plan
2
Create Tenant
Create Tenant
3
Create Service
Create Service
4
Create Load Balancer
Create a Node Pool
5
Test the app
Create Load Balancer
6
Test the app
Finish the Quick Start Tutorial by creating a Service using GKE Standard
In this tutorial for DuploCloud AWS, you have so far created a VPC network with configuration templates (Infrastructure and Plan) and an isolated workspace (Tenant).
Now you need to create a DuploCloud Service on top of your Infrastructure and configure the Service to run and deploy your application. In this tutorial path, we'll deploy using Docker containers, leveraging Google Cloud Platform's (GCE) Google Kubernetes Engine (GKE) Standard.
Alternatively, you can finish this tutorial by:
For a comparison of the benefits of GKE Autopilot vs. GKE Standard, consult this Google Cloud article.
Estimated time to complete remaining tutorial steps: 30-40 minutes
For the remaining steps in this tutorial, you will:
Create a GCE Virtual Machine (VM) or a Node Pool.
Create a Service and applications (webapp) using the premade Docker image nginx:latest.
Expose the Service by creating and sharing a load balancer and DNS name.
Test the application.
Creating a Kubernetes Service to run a Docker-containerized application
In this exercise, we will create a simple Google Cloud Nginx service. When you run the application, DuploCloud accesses Docker images in a preconfigured Docker Hub.
Estimated time to complete Step 4: 10 minutes.
Before creating a Service, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
Click Add. The Add Service page displays.
In the Service Name field, enter a name for the service (in the example below, the name is "myservice").
In the Docker image field, enter the docker image (for example "nginx:latest").
Click Next. The Advanced Options page is displayed.
At the bottom of the Advanced Options page, click Create. Your Service is created and initialized.
It may take approximately five (5) minutes for the Service to initialize. Use the Containers page (Kubernetes -> Containers) to monitor the Service creation status, between Desired (Running) and Current.
From the DuploCloud portal, navigate to Kubernetes -> Services, and verify that your DuploCloud Service has a Current status of Running.
When you run your own applications, you will choose a public image or provide credentials to access your private repository. Before you deploy your own applications, .
An exist, both with the name that you chose.
The Infrastructure has .
A with the name you chose has been created.
A had been created.
Create a Node Pool to run the operating system for your app.
A Node Pool is a group of Compute Engine VM instances within a GKE cluster that have the same configuration and provide the compute resources for running Kubernetes workloads. Node Pools allow you to customize and manage subsets of nodes within your GKE cluster to meet the requirements of your applications.
Estimated time to complete Step 3: 10 minutes.
Before creating a Service, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
An Infrastructure and Plan exist, both with the name that you chose.
The Infrastructure has GKE Enabled.
A Tenant with the name you chose has been created.
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
From the DuploCloud portal, navigate to Kubernetes -> Node Pools.
Click on the Node Pool tab and then click Add. The Add Node Pool page displays.
In the Name field, enter a unique name for your GCE Virtual Machine. Choose the same Availability Zone as your Infrastructure, and update the Instance Type, if needed.
Optionally, Enable Autoscaling.
If autoscaling is enabled, set the Initial Node Count, Min Node Count, Max Node Count, and Location Policy.
Click Create. The Node Pool is created.
Navigate to Kubernetes -> Nodes, and click on the Node Pool tab.
Click on the name of the Node Pool that you created.
Verify that the Status is Running.
For more advanced node pool functions, see the DuploCloud node pool documentation.
How Infrastructures and Plans work together to create a VPC
Infrastructures are abstractions that allow you to create a Virtual Private Cloud (VPC) instance in the DuploCloud Portal. When you create an Infrastructure, a Plan is automatically generated to supply the network configuration necessary for your Infrastructure to run.
DuploCloud creates a VNET with a default subnet and a default Network Security Group (NSG). The creation of an Infrastructure takes about ten (10) minutes.
When you create a DuploCloud Infrastructure, you create an isolated environment that maps to a Kubernetes cluster.
In DuploCloud, an Infrastructure maps one-to-one to a VPC in a specified region. It also maps to a Google Kubernetes Engine (GKE) cluster you use for container orchestration.
When creating an Infrastructure, specify the number of availability zones, the region, VPC Classless Inter-Domain Routing (CIDR), and a subnet mask. DuploCloud creates two subnets in each availability zone, one private and one public, and sets up routes and a NAT gateway.
Create a DuploCloud Infrastructure in the DuploCloud Portal:
Click Administrator -> Infrastructure from the navigation menu.
Click Add.
Define the Infrastructure by completing the fields on the Add Infrastructure form.
Click Enable GKE to enable GKE for the Infrastructure.
Click the Cluster Mode list box, and select either GKE Standard or GKE Autopilot.
Optionally, select Advanced Options to specify additional configurations (public and private subnets, for example).
Click Create. The Infrastructure is created and is listed on the Infrastructure page.
Up to one (0 or 1) GKS instance is supported for each DuploCloud Infrastructure.
When you create the Infrastructure, DuploCloud creates the following components:
VPC with 2 subnets (private, public) in each availability zone
Required security groups
NAT Gateway
Internet Gateway
Route tables
VPC peering with the master VPC, which is initially configured in DuploCloud
Cloud providers limit the number of Infrastructures that can run in each region. If you have completed the steps to create an Infrastructure and it doesn't show a Status of Complete, try selecting a different region.
Once the Infrastructure is created, a Plan (with the same Infrastructure name) is automatically created and populated with the Infrastructure configuration. The Plan is used to create Tenants.
Navigate from Administrator -> Infrastructure -> Add to create Infrastructure with GKE Standard Cluster.
Name: nonprod
Account: Google Cloud account
VPC CIDR: 10.11.0.0/16
Cloud: Google
Region: us-east1
Subnet CIDR: 22
Enable GKE: enabled
Cluster Mode: GKE Standard
This takes about 20 minutes. Infrastructure status should move to Completed. Once the Infrastructure status shows Complete, navigate to Administrators -> Plans to verify that a plan has been created with the same name (nonprod).
You can view the details and download the kubeconfig file to connect the cluster from GKE Tab available in the infrastructure created.
Navigate from Administrator -> Infrastructure -> Add to create Infrastructure with GKE Standard Cluster.
Name: nonprod
Account: Google Cloud account
VPC CIDR: 10.11.0.0/16
Cloud: Google
Region: us-east1
Subnet CIDR: 22
Enable GKE: enabled
Cluster Mode: GKE Autopilot
This takes about 20 minutes. Infrastructure status should move to Completed. Once the Infrastructure status shows Complete, navigate to Administrators -> Plans to verify that a plan has been created with the same name (nonprod).
Connect to the Cluster namespace using the kubectl token.
DuploCloud provides a way to connect directly to the Cluster namespace using the kubectl
token.
See kubectl Setup for available options.
Using Tenants in DuploCloud
In GCP, cloud features such as Resource Groups, Identity and Access Management (IAM), Security Groups, Cloud KMS, as well as Kubernetes Namespaces, are exposed in Tenants which reference their configurations.
When you create Tenants in an Infrastructure, a namespace is created in the Kubernetes cluster with the name duploservices-TENANT_NAME.
At the logical level, the Tenant is:
A Container of resources: All resources (except ones corresponding to the Infrastructure) are created within the Tenant. If a tenant is deleted, all the resources in the Tenant are terminated.
A Security Boundary: All resources within a Tenant can talk to each other. For example, a Docker container deployed in a GKE instance within the tenant will have access to Google Cloud Storage and Google Cloud databases within the same tenant. SQL database instances in another tenant cannot be reached, for example, by default. Tenants expose endpoints to each other using load balancers or explicit inter-Tenant security groups and identity management policies.
User Access Control: Self-service is the bedrock of the DuploCloud platform. To that end, users can be granted Tenant level access. For example, John and Jim are developers who can be granted access to the DEV01 tenant, Joe is an administrator who has access to all tenants, and Anna is a data scientist who has access only to the DATASCI tenant.
A Billing Unit: Because the Tenant is a container of resources, all resources in the Tenant are tagged with the Tenant's name in the cloud provider, making it easy to segregate usage by Tenant.
A mechanism for alerting: All alerts represent Faults in any resource within the Tenants.
A mechanism for logging: Each Tenant has its unique set of logs.
A mechanism for metrics: Each Tenant has its unique set of metrics.
DuploCloud customers usually create at least two Tenants for their production and non-production cloud environments (Infrastructures).
You can map Tenants in each or all of your development, testing, staging, Quality Assurance (QA), and production environments.
For example:
Production Infrastructure
Pre-production Tenant - for preparing or reviewing production code
Production Tenant - for deploying tested code
Non-production Infrastructure
Development Tenant - for writing and reviewing code
Quality Assurance Tenant - for automated testing
In larger organizations, some customers create Tenants based on application environments, such as creating a tenant for Data Science applications, another for web applications, etc.
Tenants are sometimes created to isolate a single customer workload, allowing more granular performance monitoring, scaling flexibility, or tighter security. This is referred to as a single-Tenant setup. In this case, a DuploCloud Tenant maps to an environment used exclusively by the end client.
When you have a large set of applications that different teams access, it is helpful to map Tenants to team workloads. For example, you could create Tenants for Dev-analytics, Stage-analytics, and so on.
While Infrastructure provides abstraction and isolation at the Virtual Private Cloud (VPC) and Kubernetes/cluster level, the Tenant supplies the next level of isolation implemented in GKS by segregating Tenants using the following construct per Tenant
A set of security groups
An identity management role and profile
A Kubernetes Namespace, a read-only service account, and a write service account
Cloud KMS
PEM file
GKS Worker nodes or virtual machines (VMs) created within a Tenant are given a label with the Tenant Name, as are the node selectors and namespaces. Consequently, even at the worker node level, two tenants achieve complete isolation and independence, even though they may be sharing the same Kubernetes cluster by a shared Infrastructure
Using Hosts in DuploCloud
Once we have the Infrastructure (Networking, Kubernetes cluster, and other common configurations) and an environment (Tenant) set up, the next step is to create VMs. These could be meant for:
Compute Engine virtual machines in GCP
Worker Nodes (Docker Hosts) if built-in container orchestration is used.
Regular nodes that are not part of any container orchestration, where a user manually connects and installs applications.
In GCP, you can use GCE VMs or BYOH (bring your own hosts) to get a Virtual Machine setup. Both of these are available through the Cloud Services -> Hosts menu
See the Services documentation for steps to create Hosts and configure Kubernetes storage options.
You can create a GCE VM by going to Cloud Services -> Hosts -> GCE VM.
While lower-level details such as IAM roles and security groups are abstracted, deriving instead from the Tenant, only the most application-centric inputs are required to set up Hosts.
Most of these inputs are optional and some are available as list box selections, set by the administrator in the Plan (for example, Image ID, in Host Advanced Options).
There is an additional parameter labeled Fleet Type. This is applicable if the VM is to be used as a host for container orchestration by the platform. The choices are:
Linux Docker/Native: To be used for hosting Linux containers using the Built-in Container orchestration.
None: To be used for non-Container Orchestration purposes and contents inside the VM are self-managed by the user.
If a VM is used for container orchestration, ensure that the Image ID corresponds to the Image in the container. Any name that begins with Duplo is an image that DuploCloud generates for Built-in container orchestration
Manage Tenant expiry settings in the DuploCloud Portal
In the DuploCloud Portal, configure an expiration time for a Tenant. At the set expiration time, the Tenant and associated resources are deleted.
In the DuploCloud Portal, navigate to Administrator -> Tenants. The Tenants page displays.
From the Name column, select the Tenant for which you want to configure an expiration time.
From the Actions list box, select Set Tenant Expiration. The Tenant - Set Tenant Expiration pane displays.
Select the date and time (using your local time zone) when you want the Tenant to expire.
Click Set. At the configured day and time, the Tenant and associated resources will be deleted.
The Set Tenant Expiration option is not available for Default or Compliance Tenants.
Export GCP billing data to BigQuery using DuploCloud
By exporting your Google Cloud Platform (GCP) billing data to BigQuery, you can leverage DuploCloud's dashboard to monitor and analyze your GCP billing effectively.
To export to BigQuery you must have:
A Google Cloud Platform account with billing enabled.
Permission to access the Google Cloud Billing API and BigQuery.
Billing Account Administrator permissions
BigQuery Admin permissions
Navigate to the BigQuery Console in your Google Cloud Platform account.
In GCP, select the Project where you want to create the dataset.
Click Create Dataset.
In the Create dataset window, configure your dataset with the following parameters:
Dataset ID: Enter a unique name for your dataset.
Location Type: Select Multi-Region.
Default table expiration: Select Enable table expiration and set a default expiration time for tables in this dataset, such as 60 days. Tables will be automatically deleted after this period.
Click Create Dataset.
Once the dataset is created, it appears in the BigQuery Console under your project. Select the dataset to view details.
In GCP, open the Google Cloud Console.
Select Billing from the main menu or visit Google Cloud Billing.
Select the billing account for which you want to enable the billing export.
In the Billing Account Details page, select Billing Export from the left navigational pane.
In the Billing Export page, in the Detailed usage cost area, click Edit Settings.
In the BigQuery Export tab, configure Detailed usage cost.
Select the Project: Choose the project where you created the BigQuery dataset.
Select the Dataset: Choose the dataset you created for billing data.
Click Save.
Contact DuploCloud Support to complete additional steps to enable the billing dashboard.
The exported billing data includes detailed information about your GCP usage and charges. Regularly monitor and analyze this data to keep track of your cloud spending.
Manage costs for resources
Usage costs for resources can be viewed and managed in the DuploCloud Portal, by month or week, and by Tenant. You can also explore historical resource costs.
To view the Billing page for GCP in the DuploCloud Portal, click Administrator -> Billing.
You can view usage by:
Time
Select the Spend by Month tab and click More Details to display monthly and weekly spending options.
Tenant
Select the Spend by Tenant tab.
In Google Cloud Platform (GCP), billing data can be exported to a BigQuery dataset in only one project. However, when deploying instances of an application across multiple projects (e.g., dev, qa, stg, prod), it is necessary to replicate the billing dataset to enable billing monitoring on all DuploCloud dashboards in these projects. This documentation outlines the steps to configure automated replication of a BigQuery dataset from a source project to a destination project.
Two GCP projects: a source project where the original billing dataset resides, and a destination project where the dataset will be replicated.
Appropriate permissions to create datasets and data transfer jobs in BigQuery.
Google Cloud SDK installed and initialized.
Source Project: GCP project where the original billing dataset resides with billing export.
Destination Project: New GCP project which has duplo-master running and dataset needs to be created.
Open the BigQuery console in the source project: BigQuery Console
Click on CREATE DATASET.
Enter the dataset ID, choose a data location, and set other options as mentioned in the below screenshot.
Click Create dataset.
For the replication to work, you need to allow specific roles on the dataset in source project to the duplo-master
GCP service account of the destination project
Following roles are needed:
BigQuery Admin
BigQuery Data Viewer
BigQuery Data Editor
BigQuery User
Open the BigQuery console in the destination project.
In the left-hand menu, click on Data Transfers.
Click on CREATE TRANSFER.
Select Source Type as Dataset Copy
Schedule options: Choose Start now. Set the frequency option to every 12 hours.
Under the Destination Settings
Put destination project dataset as Dataset
Put source project dataset as Source Dataset
Put source project ID as Source Project
Enable checkbox Overwrite destination table
Click SAVE
In the BigQuery console of the destination project, go to the Transfers tab.
You should see your transfer job listed. You can click on it to view details and monitor its progress.
By following these steps, you can set up automated replication of a BigQuery dataset from one GCP project to another, enabling billing monitoring on all DuploCloud dashboards across multiple projects. Ensure to monitor the transfer job periodically to make sure it is running as expected.
NOTE: This documentation is an extension of
Under Service Account select the destination duplo-master
service account (which has the )
Managing GCP services and related components
DuploCloud provides several configurable components when running Google Cloud Provider's Google Kubernetes Engine (GKE).
Applications involve GCP Services such as Cloud Armour, Redis and SQL databases, Storage Buckets, Load Balancers, and so on.
Using DuploCloud, you can create unlimited Services within each Tenant, using application-centric inputs. At the same time, the platform ensures that the lower-level nuances are programmed to best practices for security and compliance.
In addition to GKE standard and auto-pilot, the following services are supported. Supported Services are listed in alphabetical order.
Creating a Load balancer using GCP in DuploCloud
All containers are running inside a private network and cannot be accessed from an external network. To make them accessible from the an external network, create a Load Balancer.
If you need to create an Ingress Load Balancer, refer to the GKE Ingress page in the DuploCloud Kubernetes User Guide.
For an end-to-end example of deploying an application using a GCP Service, see the GCP Quick start.
In the DuploCloud Portal, navigate to Kubernetes -> Services.
On the Services page, select the Service name in the Name column.
Click the Load Balancers tab.
If no Load Balancers exist, click the Configure Load Balancer link. If other Load Balancers exist, click Add in the LB listeners card. The Add Load Balancer Listener pane displays.
From the Select Type list box, select a Load Balancer Listener type based on your Load Balancer.
Complete other fields as required and click Add to add the Load Balancer Listener.
DuploCloud allows no more than one (0 or 1) Load Balancer per DuploCloud Service.
Add GCP subscription details
The DuploCloud rules-based expert needs GCP Subscription details to manage cloud resources. Add Cloud Credentials in the DuploCloud Portal to add subscription details.
In the DuploCloud Portal, navigate to Administrator -> Cloud Credentials. The Cloud Credentials page displays.
Click Add.
In the Cloud list box, ensure Google is selected.
In the Project ID field, enter your Google Project ID.
In the Service Account Email field, enter the Service Account email. A service account is a special account used by an application or compute workload, rather than a person. Service accounts are managed by Identity and Access Management (IAM).
In the Service Account Private Key field, enter the private key associated with your service account.
Click Submit. Your credentials are displayed on the Cloud Credentials page.
Implement GCP Cloud Armour in DuploCloud
GCP Cloud Armour helps protect your applications and websites against denial of service, web breaches, and cyber-attacks.
Use DuploCloud to activate your GCP Cloud Armour software and monitor your cloud infrastructures and deployed services and applications.
Before you can use DuploCloud with Cloud Armour, define a Security Policy in the DuploCloud Plan that supports your DuploCloud Infrastructure.
In the DuploCloud Portal, navigate to Administrator -> Plan. The Plans page displays.
From the Name column, select the Plan that corresponds to your Infrastructure. When you create a DuploCloud Infrastructure, a Plan is created with the same name.
Click the Security Policy tab.
Click Add. The Add Security Policy pane displays.
In the Name field, enter an appropriate name for the Security Policy. This is the name used in the DuploCloud portal. It is convenient to keep it the same as the Security Policy ID, but not required.
In the Security Policy ID field, enter the name of your GCP Cloud Armour Security Policy. This is the name used in the GCP console.
Click Create. The Security Policy that you specified is displayed in the Security Policy tab.
Now that the Cloud Armour Security Policy has been defined in your DuploCloud Plan, add the policy to a Load Balancer so that it can monitor network traffic.
In the DuploCloud Portal, navigate to Kubernetes -> Services or Docker -> Services.
Select the Service to which your Load Balancer is attached.
Click the Load Balancer tab.
In the Other Settings card, click Edit. The Other Load Balancer Settings pane displays.
Select the Enable HTTP to HTTPS Redirect option.
Select Enable Access Logs to view rule evaluations.
In the Idle Timeout field, enter the number of minutes for timeout, in seconds.
Click Save.
The Security Policy displays in the Load Balancer's Other Settings card.
In the DuploCloud Portal, navigate to Administrator -> Plans. The Plans page displays.
From the Name column, select the Plan that corresponds to your Infrastructure.
Click the Security Policy tab.
Modify the Security Policy Name and the Security Policy ID as appropriate.
Click Update. The changes are saved and displayed in the Security Policy tab.
Logs will only be visible if you Enable Access Logs in the Load Balancer's Other Settings card.
To view Cloud Armor Security Policy logs:
Locate the Security Policy in the GCP Console.
Click the Logs tab.
Click the View policy logs link on the Logs tab to view logs of the policy's rule evaluations.
Configuration and Secret management in GCP
There are many ways to pass configurations to containers at run-time. Although simple to set up, using Environmental Variables can become complex if there are too many configurations, especially files and certificates.
From the Security Policy list box, select the .
To change your Cloud Armour configuration to use a different security policy, edit the Security Policy in the DuploCloud .
In the row listing your security policy, click the Edit Icon ( ) to change the Security Policy ID. The Update Security Policy pane displays.
Using Kubernetes, you can populate environment variables using .
Create Cloud Functions in GCP
In GCP, Cloud Functions are for serverless execution of code.
In the DuploCloud Portal, navigate to Cloud Services -> Storage. The Buckets page displays. Create a bucket and upload the code package.
Navigate to Cloud Services -> Functions, and click Add. The Add Function page displays. Fill out the appropriate fields and click Create.
Create cloud scheduler in GCP
Go under Cloud Services in the left nav bar and you can find the Cloud Scheduler menu. One can create a cloud scheduler to trigger from a pub/sub topic created in the previous section, an HTTP endpoint or an App Engine.
Adding SQL Databases in DuploCloud
Use this procedure to create:
MySQL databases
SQL databases with PostGres engines
SQL databases with SQLServer engines
In the DuploCloud Portal, navigate to Cloud Services -> Cloud SQL.
Click Add. The Add SQL DB page displays.
For MySQL databases and SQL databases with PostGres engines, provide the Name, SQL Version, and Tier (Machine Type/CPU). For SQL databases with SQLServer engines, provide the same inputs, in addition to Root Password and Disk Size in gigabytes (GB).
Click Create.
Select your database from the Name column in the SQL tab. The Details tab displays information about the database you created.
Refer to the graphics below for examples of creating and displaying the supported SQL databases.
Create a Firestore Database from within the DuploCoud platform.
Firestore is a flexible, scalable database for mobile, web, and server development from Google Cloud Platform. It's part of Firebase, a platform for developing mobile and web applications. Firestore is a NoSQL document database that simplifies storing, syncing, and querying data across multiple platforms and devices.
There are two Firestore Database modes to choose from:
Firestore Native Mode is the default mode for Firestore. It provides a richer feature set and supports more advanced querying capabilities, such as compound queries and real-time updates. Use Firestore Native for new projects and applications that require real-time updates and advanced querying features.
Datastore Mode provides a subset of Firestore's features and capabilities, supports a simpler data model, and lacks support for nested subcollections. Use Datastore Mode for migrating existing applications from Google Cloud Datastore to Firestore or for applications that do not require real-time updates or complex querying capabilities.
From the Tenant list box in the upper left, select your Tenant name.
From the DuploCloud portal, navigate to Cloud Services -> Firestore Database.
Click Add. The Add Firestore DB page displays.
In the Name field, enter a name for your database.
From the Type list box, select FIRESTORE_NATIVE or DATASTORE_MODE.
Select your location from the Location list box.
From the Point in Time Recovery Enablement list box, enable or disable point in time recovery, or lock your resources pessimistically.
From the Delete Protection State list box, enable or disable delete protection.
Click Create. Your Firestore Database is created.
Support for Redis database instances
DuploCloud supports Redis database instances. Redis stands for Remote Dictionary Server and is a fast, open-source, in-memory, key-value data store. Redis can function as a database, cache, message broker, and queue.
Redis delivers sub-millisecond response times, enabling millions of requests per second for real-time applications.
In the DuploCloud Portal, navigate to Cloud Services -> Redis.
Click Add. The Add Redis Instance page displays.
Enter the database Name.
In the Display Name field, enter a useful database name for reference.
From the Tier list box, select Basic for a Tier0 standalone instance; select Standard for a Tier1 High Availability primary/replica instance.
In the Memory Size field, enter memory size in gigabytes (GB).
In the Redis Config field, specify the Redis configuration.
In the Labels field, specify key
/value
pairs.
Select Enable Auth and Security to enable OSS Redis AUTH for the Redis instance.
Select Enable Encryption-in-Transit to select the TLS mode of the Redis instance.
Click Create. The Redis database Details tab displays on the Redis tab with Connectivity, General, and Security cards.
Create pub/sub in GCP
Creation of a pub sub topic is quite self explanatory with just a couple fields.
Orchestration across multiple Cloud providers
DuploCloud abstracts the complexity of container orchestration technologies, allowing you to focus on the deployment, updating, and debugging of your containerized application.
Among the technologies supported are:
Google Kubernetes Engine (GKE Autopilot): DuploCloud platform uses GKE Autopilot, providing you with a user-friendly interface that conceals the complexities of Kubernetes serverless workloads. Using the UI you can add K8S configurations around Pods, Containers, Secrets, and so on. See here on how to setup a Auto-Pilot cluster.
Google Kubernetes Engine (GKE Standard): DuploCloud platform uses GKE Standard, providing the same user-friendly interface to manage underlying Kubernetes Cluster and Node Pools. See here on how to setup a standard cluster.
Built-in (Docker Native): DuploCloud platform's built-in container management has the same interface as the docker run
command, except that it can be scaled to hundreds of containers across many hosts, providing capabilities such as associated load balancers, DNS, and more.
If you need other services, please get in touch with your DuploCloud support team. The typical turnaround time for creating a custom service is a business week.
Common questions about using DuploCloud GCP
DuploCloud typically runs Kubernetes services in GCP on GKE in Autopilot mode. Autopilot dynamically provisions nodes as needed to run your pods. This can add a couple of minutes to pod start time. You may see warnings from Kubernetes about being unable to place pods while autopilot hosts are starting, but they’ll clear once the hosts are available.
Use a self-signed certificate, because it enables you to control authentication at the IP address level. You cannot use Google Managed Certificates.
To give a user access to a specific Tenant, navigate the Users page. For a new user, click Add and enter the user's information. From the Role list box, select User. When the user role is selected, the Tenant list box displays. In the Tenant list box, select the Tenant(s) you would like to give the user access to. Click Submit. For an established user, navigate to the Users page and select the name of the user whose access you would like to update. From the Actions menu, click Update. From the Role list, select User, and from the Tenant list box, select the Tenant(s) to which you want to give them access. Click Submit.
To create a Google Managed certificate for use with DuploCloud, see the DuploCloud documentation on creating managed certificates with GCP.
Multiple container orchestration technologies for ease of consumption
DuploCloud abstracts the complexity of container orchestration technologies, allowing you to focus on the deployment, updating, and debugging of your containerized application.
Among the technologies supported are:
Google Kubernetes Engine (GKE Autopilot): DuploCloud platform uses GKE Autopilot, providing you with a user-friendly interface that conceals the complexities of Kubernetes serverless workloads. Using the UI you can add K8S configurations around Pods, Containers, Secrets, and so on.
Google Kubernetes Engine (GKE Standard): DuploCloud platform uses GKE Standard, providing the same user-friendly interface to manage underlying Kubernetes Cluster and Node Pools.
Built-in (DuploCloud): DuploCloud platform's built-in container management has the same interface as the docker run
command, except that it can be scaled to hundreds of containers across many hosts, providing capabilities such as associated load balancers, DNS, and more.
Use the feature matrix below to compare the features of the orchestration technologies that DuploCloud supports. DuploCloud helps you implement any option you choose through the Portal or the Terraform API.
One dot indicates a low rating, two dots indicate a medium rating, and three dots indicate a high rating. For example, Kubernetes has a low ease-of-use rating, but a high rating for stateful application support.
Use the definitions below to understand how each feature in the matrix above is rated in relation to each of the three listed technologies (Kubernetes, Built-In).
Ease of Use:
Kubernetes is extensible and customizable, but not without a cost in ease of use. The DuploCloud platform reduces the complexities of Kubernetes, making it comparable with other container orchestration technologies in ease of adoption.
DuploCloud's Built-in orchestration mirrors docker run
. You can SSH into a virtual machine (VM) and run docker
commands to debug and diagnose. If you have an application with a few stateless microservices; or configurations that use environment variables or Google Cloud Extensions, Google Cloud Storage, or GCP Secret Manager, consider using DuploCloud's Built-in container orchestration.
Features and Ecosystem Tools: Kubernetes is rich in many additional built-in features and ecosystem tools, most notably Secrets Management and ConfigMaps. While Kubernetes features have an equivalent in GCP, third parties tend to publish their software as Kubernetes packages (Helm Charts). Some examples are Influx DB, Time Series DB, Prefect, etc.
Suitability for Stateful apps: Stateful applications should be avoided in GCP. Instead, cloud-managed storage solutions should be leveraged for the best availability and SLA compliance. In scenarios where this is undesirable due to cost, Kubernetes offers the best solution. Kubernetes uses StatefulSets and Volumes to implicitly manage Google Cloud Storage volumes.
Stability and Maintenance: Although Kubernetes is highly stable, it is an open-source product. The native customizability and extensibility of Kubernetes can lead to points of failure when a mandatory cluster upgrade is needed, for example. This complexity often leads to support costs from third-party vendors. Maintenance can be costly with GKE, as versions are deprecated frequently and you are required to upgrade the control plane and data nodes. While DuploCloud automates this upgrade process, it still requires careful planning and execution.
GCP Cost: While the GCP control plane cost is relatively low, it is not recommended to operate a GKE environment without business support at an additional premium. If you are a small business, you may be able to add the support tier when you need it and then turn it off to reduce costs.
Multi-Cloud: For many enterprises and independent software vendors this is a requirement, either immediately or in the future. While Kubernetes provides this benefit, DuploCloud's implementation is much easier to maintain and easier to implement.
Feature | Kubernetes | Built-In |
---|---|---|
Ease of use
Features and ecosystem Tools
Suitability for stateful apps
Stability and maintenance
GCP cost
Multi-cloud (w/o DuploCloud)
Finish the Quick Start Tutorial by creating a Service using GKE Autopilot
In this tutorial for DuploCloud AWS, you have so far created a VPC network with configuration templates (Infrastructure and Plan) and an isolated workspace (Tenant).
Now you need to create a DuploCloud Service on top of your Infrastructure and configure the Service to run and deploy your application. In this tutorial path, we'll deploy using Docker containers, leveraging Google Cloud Platform's (GCE) Google Kubernetes Engine (GKE) Autopilot.
Alternatively, you can finish this tutorial by:
For a comparison of the benefits of GKE Autopilot vs. GKE Standard, consult this Google Cloud article.
Estimated time to complete remaining tutorial steps: 15-20 minutes
For the remaining steps in this tutorial, you will:
Create a Service and applications (webapp) using the premade Docker image nginx:latest.
Expose the Service by creating and sharing a load balancer and DNS name.
Test the application.
Integrate with OpenVPN by provisioning VPN users
DuploCloud integrates natively with OpenVPN by provisioning VPN users that you add to the Duplocloud Portal. OpenVPN setup is a two-step process.
Accept OpenVPN Free tier (Bring Your Own License) in the GCP marketplace:
Log into your GCP account. In the console, navigate to: https://console.cloud.google.com/marketplace?_ga=2.26702909.1494282976.1678740607-1491144562.1675196305&pli=1.
Accept the agreement.
In the DuploCloud Portal, navigate to Administrator -> System Settings.
Click the VPN tab.
Click Provision VPN.
After the OpenVPN is provisioned, it is ready to use. Behind the scenes, DuploCloud launches a cloud formation script to provision the OpenVPN.
You can find the OpenVPN admin password in the cloud formation stack in your GCP console.
Provision a VPN while creating a user:
In the DuploCloud Portal, navigate to Administrator -> Users.
Click Add. The Create User pane displays.
Enter a valid email address in the Username field.
In the Roles field, select the appropriate role for the User.
Select Provision VPN.
Click Submit.
For information about removing VPN access for a user, see Deleting a VPN user. To delete VPN access, you must have administrator privileges.
By default, users connected to a VPN can SSH or RDP into virtual machines (VMs). Users can also connect to internal load balancers and endpoints of the applications. However, to connect to other services, such as databases and elastic cache, you must open the port to the VPN:
In the DuploCloud Portal, navigate to Administrator -> Tenants.
Select the Tenant in the Name column.
Click the Security tab.
Click Add. The Add Tenant Security pane displays.
In the Source Type field, select Ip Address.
In the IP CIDR field, enter the name of your VPN.
Click Add.
Creating a Load Balancer to configure network ports to access the application
Now that your DuploCloud Service is running, you have a mechanism to expose the containers and images in which your application resides. But because your containers are running inside a private network, you also need a load balancer to listen on the correct ports in order to access the application.
In this step, we add a Load Balancer Listener to complete this network configuration.
Estimated time to complete Step 5: 10 minutes.
Before creating a Load Balancer, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
All containers are running inside a private network and cannot be accessed from an external network. To do so one can create a load balancer.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
From the NAME column, select the name of your Service
Click the Load Balancers tab.
Click the Configure Load Balancer link. The Add Load Balancer Listener pane displays.
From the Type list box, select Application LB.
In the Container Port field, enter 80. This is the configured port on which the application inside the Docker Container Image is running.
In the External Port field, enter 80. This is the port through which users will access the web application.
From the Visibility list box, select Public.
From the Application Mode list box, select Docker Mode.
Type / (forward-slash) in the Health Check field to indicate that the cluster we want Kubernetes to perform Health Checks on is located at the root
level.
In the Backend Protocol list box, select HTTP.
Click Add. The Load Balancer is created and initialized. In approximately 2-3 minutes you will see the load balancer details available in the portal. When the Load Balancer is ready for use the LB Status card displays Ready.
From the DuploCloud portal, navigate to Kubernetes -> Services.
Click on the name of your Service.
Verify that the Load Balancer has a status of Ready on the LB Status card.
An exist, both with the name you created.
The Infrastructure you created has .
A with the name you chose has been created.
A had been created.
A with the name you chose has been created.
Upgrade the Google Kubernetes Engine (GKE) version
Google frequently updates the version of GKE based on new features that are available in the Kubernetes platform.
DuploCloud pushes GKE upgrades to the DuploCloud Portal code, but we request that you contact the DuploCloud Support staff on your Slack channel or by email when upgrading, for the moment.
In future releases, this upgrade will be available for customers to install.
Use Cases supported for DuploCloud GCP
Topics in this section are covered in the order of typical usage. Use cases that are foundational to DuploCloud such as Infrastructure, Tenant, and Hosts are listed at the beginning of this section; while supporting use cases such as Logs, Metrics, and Faults and alerts appear near the end.
Infrastructure and Plan
Test the application to ensure you get the results you expect
You can test your application directly from the Services page.
Estimated time to complete Step 6 and finish tutorial: 10 minutes.
Before creating a Load Balancer, verify that you accomplished the tasks in the previous tutorial steps. Using the DuploCloud Portal, confirm that:
An Infrastructure and Plan exist, both with the name you created.
The Infrastructure you created has GKE Enabled.
A Tenant with the name you chose has been created.
A Node Pool has been created.
A Service with the name you chose has been created.
An Application Load Balancer has been created.
In the Tenant list box, on the upper-left side of the DuploCloud Portal, select the Tenant that you created.
In the DuploCloud Portal, navigate to Kubernetes -> Services. The Services page displays.
From the Name column, select demo-service.
Click the Load Balancers tab. The Application Load Balancer configuration is displayed.
Open a browser instance and Paste the IP Address in the URL field of your browser.
Press ENTER. A web page with the text Welcome to nginx! is displayed.
Congratulations! You have just launched your first web service on DuploCloud!
Creating and managing GCP Services using containers
Using the Services pages (Kubernetes -> Services or Docker -> Services) in the DuploCloud Portal, you can display and manage the Services you have defined.
You can deploy any native Docker container in a virtual machine (VM) with the DuploCloud platform.
In the DuploCloud Portal, select Docker -> Services from the navigation pane.
Click Add. The Add Service page displays.
Complete the fields on the page, including Service Name, Docker Image name, and number of Replicas. Use Allocation Tags to deploy the container in a specific set of hosts.
Do not use spaces when creating Service or Docker image names.
The number of Replicas defined must be less than or equal to the number of hosts in the fleet.
In the DuploCloud Portal, you can display and manage the containers you have defined.
Select the Tenant from the Tenant list box in the upper left, and navigate to Kubernetes -> Containers.
In the LB Configuration card, click the Copy Icon ( ) to copy the IP Address displayed to your clipboard.
Use the Options Menu ( ) in each container row to display Logs, State, Container Shell, Host Shell, and Delete options.
Option | Functionality |
---|---|
Logs
Displays container logs.
State
Displays container state configuration, in YAML code, in a separate window.
Container Shell
Accesses the Container Shell. To access the Container Shell option, you must first set up Shell access for Docker.
Host Shell
Accesses the Host Shell.
Delete
Deletes the container.
Set Docker registry credentials and Kubernetes secrets
In the DuploCloud Portal, navigate to Docker -> Services. Docker registry credentials are passed to the Kubernetes cluster as kubernetes.io/dockerconfigjson
.
Click the Docker list box in the upper right, and select Docker Credentials. The Set Docker registry Creds pane displays.
Supply the credentials and click Submit.
Enable the Docker Shell Service by clicking Enable Docker Shell.
You can pull images from multiple Docker registries by adding multiple Docker Registry Credentials.
In the DuploCloud Portal, click Administrator -> Plan. The Plans page displays.
Select the Plan in the Name column.
Click the Config tab.
Click Add. The Add Config pane displays.
You can pass Docker Credentials using the Environment Variables config field in the Add Service Basic Options page. See the Kubernetes Configs and Secrets section.
Create Cloud Storage Buckets in GCP
In GCP, Cloud Storage Buckets are containers that hold your data. Everything in Google Cloud Storage resides in a bucket. Learn more about GCP Cloud Storage and Cloud Storage Buckets.
In the DuploCloud Portal, navigate to Cloud Services -> Storage. The Buckets page displays.
In the Buckets tab, click Add. The Create a Bucket pane displays.
In the Name field, enter a bucket name.
Optionally, select Enable Versioning or Allow Public Access; enter a label string for your bucket in the Labels field.
Click Create.
Create Node Pool for GCE in the DuploCloud Portal
GCP Node Pools are useful when you need to schedule Pods requiring more resources than others, such as more memory or local disk space. Node Pools can be created for the DuploCloud Infrastructure with GKE Standard Cluster only.
Add a Tenant, specifying the DuploCloud Plan corresponding to a GKE Standard Cluster.
In the DuploCloud Portal, navigate to Kubernetes -> Nodes.
Click the Node Pool tab.
Click Add. The Add Node Pools page displays.
Provide Name, Availability Zone, Instance Type, and Node Counts.
Click Submit.
DuploCloud Portal provides additional options when configuring a Node Pool, as depicted below. To use Advanced Options select Advanced Options in the Add Node Pool page.
You can add Accelerator types for GPUs while creating a NodePool. From the Add Node Pool page, click Add Accelerator.
Accelerator Types are not available in all regions.
In the Add Service page, click Next for Advanced Options.
Enter command
, args
, and resources
in the Other Container Config field.
Click Create.
For additional details, refer to the documentation from Google Cloud here .
Select the Node Pool to which you want to add taints.
Click Actions and select Add Taint. The Add Taint pane displays.
Enter the Key/Value pair and select the Effect from the list box.
Click Add Taint.
For example, the following screen applies a taint to a Node Pool that has a Key/Value of dedicated=experimental
with a NoSchedule
effect.
You need to configure the correct tolerations
in the Service to schedule the Pod in a Node Pool.
To continue the examples above, create a Service with tolerations
using the Other Container Config field, as depicted below.
You can Edit or Delete a Taint by selecting the Node Pool Name, clicking the Actions menu, and selecting Edit or Delete. You edit the Node Pool using the Edit Node Pool page.
View Node Pools by clicking the Node Pool tab and selecting the Node Pool Name.
Nodes created as part of a Node Pool, are displayed in the GCE VM tab.
Taints configured to a Node Pool are displayed with a Tainted Status. Click the Tainted icon to display a window with a Taint List.
Enable access to the DuploCloud shell for your GCP account
Enabling DuploCloud shell access in GCP is part of a one-time DuploCloud portal setup process.
Create a DuploCloud Service in any Tenant.
From the DuploCloud portal, navigate to Kubernetes -> Services.
Click Add. The Add Service page displays.
From the table below, enter the values that correspond to the fields on the Add Service page. Accept all other default values for fields not specified.
In the Environment Variables field, enter the following YAML. Replace the flask app secret (b33d13ab-5b46-443d-a19d-asdfsd443 in this example) with a string of random numbers and letters in the same format and replace CUSTOMER_PREFIX with your customer URL prefix.
Click Next. The Advanced Options page displays.
Click Create. The Service is created.
Follow the steps on the GKE Ingress page to add Kubernetes Ingress, substituting the following values in the Name and Annotations fields:
Name: duplo-shell
Annotations: enter the following, replacing CERTIFICATE_NAME with your certificate name.
From the DuploCloud portal, navigate to Kubernetes -> Ingress.
Click on duplo-shell in the NAME column. The duplo-shell Ingress details page displays.
Select the Configuration tab.
From the DNS box, copy the DNS.
Navigate to Administrator -> Systems Settings.
Select the System Config tab, and click Add.
From the Config Type list box, select AppConfig.
From the Key list box, select Other.
In the second Key field, enter DuploShellfqdn
In the Value field, paste the DNS you copied from the Ingress details page.
Click Submit. DuploCloud shell access is enabled in GCP.
Add Service page field | Value |
---|---|
Name
YOUR_SERVICE_NAME
Cloud
Google
Platform
GKE Linux
Docker Image
duplocloud/shell:terraform_kubectl_v15