Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Each Infrastructure represents a network connection to a unique VPC/VNET, in a region with a Kubernetes. In case of AWS it can also include an ECS. Infrastructure can be created with 4 basic inputs: Name, VPC CIDR, Number of AZs, Region, and the option to enable or disable a K8S/ECS cluster. Behind the scenes, the system will automatically create the subnets, NAT gateway, routes, and clusters in the given region.
If the Infrastructure requirement includes custom Private/Public Subnet CIDR, it can be achieved using Advanced Options.
A common use for Infrastructure is having two Infrastructures, one for prod and one for non-prod. Another one is having an infrastructure in a different region either for DR or localized deployments for clients in that region.
The greatest capability of the DuploCloud platform is the application infrastructure centric abstraction created on top of the cloud provider which enables the user to deploy and operate their applications without knowledge of lower level DevOps nuances. Further, unlike a PAAS such as Heroku, the platform does not get in the way of users consuming cloud services directly from the cloud provider, as in a user directly operating on constructs like S3, DynamoDB, Lambda functions, GCP Redis, Azure SQL etc., while offering greater scale and unlimited flexibility.
Some concepts relating to security (DevSecOps) are hidden from the end user, for example IAM roles, KMS keys, Azure Managed Identities, GCP service accounts etc. However, even those are configurable for the operator and in any case since this is a self-hosted platform running in the customer's own cloud account, the platform is capable of working in tandem with direct changes on the cloud account by an administrator. This is explained with examples at https://duplocloud.com/white-papers/devops/
The following picture shows the high level abstractions within which applications are deployed and users operate.
While there are many concepts in the policy model, the following are the main ones to be aware of:
Infrastructure
Plan
Tenant
App and Cloud Services
Diagnostics
A Service could be a Kubernetes Deployment, Stateful set or a Daemon set. It can also be a Lambda function or an ECS task or service. It essentially captures a microservice. Each service (except Lambda) can be given a load balancer to expose itself and be assigned a DNS name.
DuploCloud Service should not be confused with a Kubernetes or a ECS service. By service we mean application components that can either be Docker-based or serverless.
Below is an image of some properties of a service:
Cloud Services: DuploCloud supports a simple application specific interface to configure dozens of cloud services like S3, SNS, SQS, Kafka, Elasticsearch, Data Pipeline, EMR, Sagemaker, Azure Redis, Azure SQL, Google Redis, etc. Almost all commonly used services are supported and new ones are constantly added. A typical request to support a new service takes the DuploCloud team a matter of days, based on the complexity of the service.
While users specify application level constructs for provisioning cloud resources, all the underlying DevOps and compliance controls are implicitly added by DuploCloud.
IMPORTANT: All services and cloud features are created within a Tenant.
Tenant is the most fundamental construct in DuploCloud which is essentially like a project or a workspace and is a child of the infrastructure. While Infrastructure is a VPC level isolation, Tenant is the next level of isolation implemented by segregating Tenants using Security Groups, IAM role, Instance Profile, K8S Namespace, KMS Key, etc., in case of AWS. Similar concepts are leveraged from other cloud providers like resource groups, managed identity, ASG, etc., in Azure.
A Tenant is fundamentally four things, at the logical level:
Container of resources: All resources (except ones corresponding to infrastructure) are created within the Tenant. If we delete the tenant then all resources within that are terminated.
Security Boundary: All resources within the tenant can talk to each other. For example a Docker container deployed in an EC2 instance within the tenant will have access to S3 buckets and RDS instances within the same tenant. RDS instances in another tenant cannot be reached, by default. Tenant can expose endpoints to each other either via ELBs or explicit inter-tenant SG and IAM policies.
User Access Control: Self-service is the bedrock of the DuploCloud platform. To that end, users can be granted Tenant level access. For example John and Jim are developers who can be granted access to Dev tenant, while Joe is an administrator who has access to all tenants, while Anna is a data scientist who has access only to the data science tenant.
Billing Unit: Since Tenant is a container of resources, all resources in the tenant are tagged with the Tenant's name in the cloud provider, making it easy to segregate usage by tenant.
A common use case for Tenant in an organization that contains 4 tenants: Dev and QA are under the non-prod infrastructure, while the Pre-prod and Prod tenants are under the Prod Infrastructure. In larger organizations, one could have tenants by groups as well like a tenant for Data Science, Tenant for web application, etc. We have seen companies creating dedicated tenants for each of their end user clients in cases where the application is single Tenant. Tenant is a logical concept that can be used either way.
Configure settings for all new Tenants under a Plan
You can configure settings to apply to all new Tenants under a Plan using the Config tab. Tenant Config settings will not apply to Tenants created under the Plan before the settings were configured.
From the DuploCloud portal, navigate to Administrator -> Plan.
Click on the Plan you want to configure settings under in the NAME column.
Select the Config tab.
Click Add. The Add Config pane displays.
From the Config Type field, select TenantConfig.
In the Name field, enter the setting that you would like to apply to new Tenants under this Plan. (In the example, the enable_alerting setting is entered.)
In the Value field, enter True.
Click Submit. The setting entered in the Name field (enable alerting in the example) will apply to all new Tenants added under the Plan.
You can check that the Tenant Config settings are enabled for new Tenants on the Tenants details page, under the Settings tab.
From the DuploCloud portal, navigate to Administrator -> Tenants.
From the NAME column, select a Tenant that was added after the Tenant Config setting was enabled.
Click on the Settings tab.
Check that the configured setting is listed in the NAME column. (Enable Alerting in the example.)
Corresponding to each Infrastructure is the concept of a Plan. A Plan is a placeholder or a template for configurations. These configurations are consistently applied to all Tenants within the Plan (or Infrastructure). Examples of such configurations are:
Certificates available to be attached to Load Balancers in Tenants of this Plan
Machine images
WAF web ACLs
Common IAM policies and SG rules to be applied to all resources in Tenants within the Plan
Unique or shared DNS domain name where applications provisioned in Tenants within the Plan can have a unique DNS name in this domain
Resource Quota: The plan also has a resource quota that is enforced in each of the Tenants within that Plan
DB Parameter Groups
Several policies and feature flags are to be applied at the infrastructure level on Tenants within the Plan
The figure below shows a screenshot of the plan constructs:
When creating DuploCloud Plans and DNS names, consider the following to prevent DNS issues:
Plans in different portals will delete each other's DNS records, so each portal must use a distinct subdomain for its Plans.
DuploCloud Plans in the same portal can share a DNS domain without deleting each other's records. Duplo-created DNS names will always include the Tenant name, which prevents collisions.
The recommended practice for most portals is to set all Plans to the same DNS name, including the default
Plan.
Ideally, custom subdomains will be set in the Plans before turning on shell, monitoring, or logging. If the DNS is changed later, those services may need to be updated.
The DuploCloud platform automatically orchestrates three main diagnostic functions:
Central Logging: A shared Elasticsearch cluster is deployed and file beat is installed in all worker nodes to fetch the logs from various applications across tenants. The logs are injected with metadata corresponding to Tenant name, service name, container ID, Host name, etc. Further, each tenant has the central logging dashboard which includes the Kibana view of the logs from applications within the service. See the screenshot below:
Metrics: Metrics are fetched from hosts, containers, and cloud services like ELB, RDS, Redis, etc., and displayed in Grafana. Behind the scenes, for cloud services, these are collected by calling cloud provider APIs like CloudWatch and Azure Mon, while for nodes and containers, this is done using Prometheus, Node Exporter, and cAdvisor. Again, the dashboards are Tenant-centric and segregated per application and cloud service as shown in the picture below:
Alarms and Faults: The platform creates faults for many failures automatically, such as Health check, container crashes, node crashes, deployment failures, etc. Further, users can easily set alarms on cloud services like CPU and Memory on EC2 instances, Free Disk space in RDS database, etc. All failures are displayed as faults per tenant. Sentry and Pager Duty projects can be linked to each tenant and DuploCloud will send these faults there so the user can set notification configurations.
Audit Trail: An audit trail of all changes made to the system are logged in Elasticsearch where these can be audited using high level constructs like changes by tenant, by service, by change type, by user and dozens of other such filters.