0% found this document useful (0 votes)

171 views148 pages

Alibaba Cloud Whitepaper - Ack k8s

The document discusses best practices for end-to-end cloud-native application management on Alibaba Cloud. It covers cloud-native application maturity models, CloudOps, DevOps/DevSecOps, SecOps, AIOps, and FinOps. It provides guidance on implementing these practices and managing applications throughout the development lifecycle.

Uploaded by

ChalermpanYenprasit

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

171 views148 pages

Alibaba Cloud Whitepaper - Ack k8s

Uploaded by

ChalermpanYenprasit

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 148

BEST PRACTICES

FOR END-TO-END
alibabacloud.com

CLOUD-NATIVE
© Alibaba Cloud 2022
All rights reserved

APPLICATION
MANAGEMENT
ON ALIBABA CLOUD
Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud ii


LEGAL NOTICES 

Alibaba Cloud reminds you to carefully read through and

completely understand all of the content in this section before
you read or use this document. If you read or use this document,
it is considered that you have identified and accepted all
contents declared in this section.

1. You shall download this document from the official website of

Alibaba Cloud or other channels authorized by Alibaba Cloud.
This document is only intended for legal and compliant
business activities. The contents in this document are
confidential, so you shall have the liability of confidentiality.
You shall not use or disclose all or part of the contents of this
document to any third party without written permission from
Alibaba Cloud.

2. Any sector, company, or individual shall not extract, translate,

reproduce, spread, or publicize, in any method or any channel,
all or part of the contents in this document without written
permission from Alibaba Cloud

3. This document may be subject to change without notice

due to product upgrades, adjustment, and other reasons.
Alibaba Cloud reserves the right to modify the contents in
this document without notice and to publish the document in
an authorized channel as and when required. You shall focus
on the version changes of this document, downloading and
acquiring the updated version from channels authorized by
Alibaba Cloud.

4. This document is only intended for product and service

reference. Alibaba Cloud provides this document for current
products and services with current functions, which may be
subject to change. Alibaba Cloud provides the document in
the context that Alibaba Cloud products and services are
provided on an "as is", "with all faults" and "as available" basis. 


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud iii


Alibaba Cloud makes the best efforts to provide an appropriate
introduction and operation guide on the basis of current 
technology, but Alibaba Cloud does not explicitly or implicitly
guarantee the accuracy, completeness, suitability, and reliability
of this document. Alibaba Cloud does not take any legal liability
for any error or loss caused by downloading, using, or putting
trust in this document by any sectors, company, or individuals.
In any case, Alibaba Cloud does not take any legal liability for
any indirect, consequential, punitive, occasional, incidental, or
penalized damage, including profit loss due to use of or trust
to this document (even if Alibaba Cloud has notified you it is
possible to cause this kind of damage).
The responsibilities and liabilities of Alibaba Cloud to its
customers are controlled by Alibaba Cloud agreements, and
this document is not part of, nor does it modify, any agreement
between Alibaba Cloud and its customers.

5. All content including, but not limited to, images, architecture

design, page layout, description text, and its intellectual property
(including, but not limited to, trademarks, patents, copyrights,
and business secrets) used in this document are owned by
Alibaba Cloud and/or its affiliates. You shall not use, modify,
copy, publicize, change, spread, release, or publish the content
from the official website, products, or programs of Alibaba Cloud
without the written permission from Alibaba Cloud and/or its
affiliates. Nobody shall use, publicize, or reproduce the name
of Alibaba Cloud for any marketing, advertisement, promotion,
or other purpose (including, but not limited to, a separate or
combined form to use the name, brand, logo, pattern, title,
product or service name, domain name, illustrated label, symbol,
sign, or similar description that may mislead readers and let
them identify that it comes from Alibaba Cloud and/or its
affiliates, of or from Alibaba Cloud, Aliyun, Wanwang, and/or its
affiliates) without the written permission from Alibaba Cloud.

6. If you discover any errors or mistakes within this document,

please contact Alibaba Cloud directly to raise this issue.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud iv


CONTRIBUTORS 

EDITORS
Xueting Zhou | Yanshun Lv


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud v


CONTENTS 

1. Overview 1

1.1 About this Solution 1

1.2 About This Lab 4

2. Procedure 5

2.1 Cloud-Native Application Maturity Model 5

2.2 CloudOps 8
2.3 DevOps/DevSecOps 55
2.4 SecOps 98
2.5 AIOps 128
2.6 FinOps 136

3. In Conclusion 142


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 1


1. OVERVIEW 

1.1 ABOUT THIS SOLUTION

BUSINESS NEED
With advancing digitization, enterprises now need to design, develop
and maintain their applications across multiple clouds and regions.
However, it can be challenging to provide the security, management
and reliability of these online applications. This is why many
companies use containerized workloads and services, using APIs to
democratize their application management tasks while growing their
online presence on a global basis.

OUR SOLUTION
The Alibaba Cloud Container Platform is a Kubernetes-based service
that runs containerized applications on the cloud.

Alibaba Cloud Container Service for Kubernetes (ACK) allows you to

deploy applications in high-performance and scalable containers and
provides full lifecycle management of enterprise-class containerized
applications. It is a comprehensive solution, integrating virtualization,
storage, networking, and security capabilities.

Alibaba Cloud was one of the first vendors to pass the Kubernetes
conformance certification tests globally. ACK was also recently
named as a leader by The Forrester Wave Public Cloud
Container Platforms.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 2




This lab covers the following architecture features:

ACR EE CloudOps
Application Center DevOps / DevSecOps
Image Scan SecOps
Image Signature AIOps
Image Replication Trigger FinOps
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Log Service ARMS
Cluster
Inspections Intelligence Service
Namespace Inspection Cluster Upgrade Logging Prometheus Monitoring
Node pool Solution to solve Check Tracing Cluster Monitoring
hystax
Application security issues Node/Pod/Network Ingress Access Center Cluster Topology
Inspection Report diagnosis Events & Alerting Application Monitoring
Multi-Cloud Bill Control … … … …


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 3

USE CASES
This lab focuses on the Alibaba Cloud Container Service for 
Kubernetes (ACK), providing an overview of its key features and
functionality to help you manage your applications with confidence.
This solution applies to the following user cases, covering best

practices for:

» CloudOps(including multi-cloud deployments)


» DevOps/DevSecOps
» SecOps
» AIOps
» FinOps

SecOps Auto Scaling CloudOps

Fine-grained Cloud Environment Build
Policy Cloud Department
security control Cluster management
Governance ARMS LogService IT Department
(Prometheus) (Logging)
Zero Trust

Cluster Terraform
Security Backup (Provisioning)
Center DevOps
Department
Cloud Na�ve

ACR Applica�on
EE Center Security
Cluster Security Department
AIOps Inspec�ons DevOps/DevSecOps
ASM
Prevent Risks (Service Mesh) Applica�on
Container
Solve Deployment, Image
abnormali�es Container management,
Intelligence Applica�on Security
Service MSE Department
(Microservice monitoring / tracing
Governance)

Cost Cost
Analysis Control Financial
Department

FinOps
Cost analysis & control


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 4

1.2 ABOUT THIS LAB


LAB OVERVIEW
This lab walks you through the best practices to set up and

manage the Alibaba Cloud Container Service for your containerized
applications. You learn how to:

» Scale, secure and monitor your clusters.
» Optimize your application delivery chain to log, monitor
and trace your ACK clusters and applications
» Configure automated, AI – based tools to boost your cluster
security and container service diagnostics.
» Enable cost analysis tools.

REQUIRED PRODUCTS
This lab requires the following Alibaba Cloud products:

» Elastic Compute Service (ECS)

» Alibaba Cloud Container Service for Kubernetes (ACK)
» ECS Bare Metal instances
» Auto Scaling
» Alibaba Cloud Service Mesh (ASM)
» Object Storage Service (OSS)
» Cloud Enterprise Network (CEN)
» Virtual Private Cloud (VPC)
» Key Management Service (KMS)
» Service Mesh Architecture
» Log Service
» Application Real-Time Monitoring Service (ARMS)
» DNS
» Security Center
» Microservices Engine (MSE)


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 5


2. PROCEDURE 

2.1 CLOUD-NATIVE APPLICATION

MATURITY MODEL
Cloud-native applications are built to run on a cloud infrastructure.
But individual organizations possess a different level of cloud native
maturity across their people, processes, policies and technologies,
leading to a range of business outcomes. As organizations increase
in cloud native maturity, they can realize improvements across their
business and achieve measurable results.

Level 1 (Build) Level 2 (Operate) Level 3 (Scale) Level 4 (Improve) Level 5 (Optimize)

You have a baseline for cloud-native implementation in place and are in pre-production.
ORGANIZATION BUSINESS OUTCOMES TECHNOLOGY
PEOPLE
• Business leaders don’t understand Initial experimentation and adoption
the benefits of cloud native. of Kubernetes. Building your cloud
infrastructure, including container
• Still in a POC phase or are focused
registry, RBAC policies, load balancer,
on one application.
cluster dashboards and container
• Teams are trained in 12 factor logging. Review microservice patterns
applications, microservice and and architecture.
cloud native patterns.
Define your KPIs from Container
PROCESS ACK
existing pain points such as
• Define your Git workflow and cost savings or other Container Registry
introduce this into your cloud native improved efficiencies. ACR EE
environment.
Increase security for the
• Consider security across all aspects Load Balancer
applications by automating CLB & ALB
of the implementation.
the identification of CVE in
your containers Security & Policies
POLICY RBAC Policy in ACK
• A limited set of documented Cluster Dashboards
policies are in place to support the Log Service
services being built in the cloud.
• Design SLOs and priorities for Auto Scaling
compliance. ACK


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 6


Level 1 (Build) Level 2 (Operate) Level 3 (Scale) Level 4 (Improve) Level 5 (Optimize)

The cloud native foundation is established and you are moving to production. 
ORGANIZATION BUSINESS OUTCOMES TECHNOLOGY
PEOPLE
• Create agile project groups to receive Consider moving applications suitable 
timely feedback / testing loops. for lift and shift. Introduce monitoring
• Automate deployments for some and observability into your workloads,
environments. start to evaluate application tracing.
Follow good practice with containers,
• Your security team should know how to
secrets and security. Set security
manage your Kubernetes cluster security.
policy management, resource
requests and limits.
PROCESS Understand what
• Build your CI/CD system. applications must be Observability
moved and why. DevOps, Logging / Monitoring / Tracing
• Auto trigger deployments.
SecOps and Ops teams
• Build security into your CI process Log Service Prometheus
including container scanning and
identify repeatable
patterns. One migrated ARMS Grafana
configuration scanning.
• Define log aggregation. application can seamlessly
be applied to another. Secrets

POLICY
Security & Policies
• Initial policies agreed as standard. Cluster Security Inspections in ACK
• Define initial resource metrics and
start collecting data. Resource Limits
Setting in ACK
• Initial auditing, carried out manually
or through simple scripts

Level 1 (Build) Level 2 (Operate) Level 3 (Scale) Level 4 (Improve) Level 5 (Optimize)

Your business competency is growing and you are defining the process for scaling.
ORGANIZATION BUSINESS OUTCOMES TECHNOLOGY
PEOPLE
• Cloud Native is now your first
strategy with teams showing
Monitoring, alerting and resource
excellence and expertise in this area.
usage capabilities. Automatic scan-
• Continuous delivery for all ning and have runtime observability of
environments, including for complex what occurs within your containers
releases and with built-in and clusters. Write Helm Charts for
compliance testing. your application releases, GitOps with
• Cloud native security training ArgoCD. Policy as code are threat
available. detection are also present.
Resource Usage
PROCESS
Cost Analysis in ACK
Faster time to market:
• Measuring container usage, (CPU/RAM).
Deploying a service takes Policy as code
• Implementing upgrades and patching, Policy Governance in ACK
particularly CVEs and critical updates.
minutes instead of days.
Reduced risk of databreaches Helm Charts
• Alerts and Filter noise are present.
thanks to monitoring for ACR EE
• CI and testing frequently carried out.
security attacks. Improved
• Implement automatic continuous GitOps with ArgoCD
customer experience, fast
scanning to flag misconfigurations or Application Center in ACK
security issues
responsiveness and improved
observability. Multi-cluster Management
ACK One for Multi-Cluster
POLICY (hybrid-cloud / multi-cloud)
• Implement policy-as-code and Management
build this into your CI/CD pipeline.
• Create policies based on metrics
refined around security, efficiency
and reliability.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 7


Level 1 (Build) Level 2 (Operate) Level 3 (Scale) Level 4 (Improve) Level 5 (Optimize)

Your business competency is growing and you are defining the process for scaling. 
ORGANIZATION BUSINESS OUTCOMES TECHNOLOGY
PEOPLE
• Cloud is now the default infrastructure
for all services.

• Development uses self-service.
• Developers can quickly test complex Familiar with Kubernetes and IaC
scenarios with many unknowns. tooling, Using ClusterAPI to deploy
• Cloud and application risks are easily and manage the lifecycle of your
and quickly identified and patched. clusters. Microservices are now the
• Kubernetes is widely adopted by multiple preferred pattern for applications,
groups in different business areas. open across the organization via a
service mesh. Gain visibility along
• Enforced security in the cloud native
with alerts allows you to respond
environment.
quickly to unplanned events.
PROCESS More time to focus on Identified Risks
• DevSecOps set policies around container your business instead of Security Center
usage, including auto-scaling policies. maintaining Kubernetes.
Container Intelligence
• Audit and alerts become mainstream Reporting covers Service in ACK
and are made mandatory across compliance, security,
applications. performance and cost. Event Center
• Release velocity increases. Security Strategy is aligned to the Event Center in LogService or ACK
remediation is automated and/or business goals Microservice Governance
identified automatically with established in Level 1.
remediation advice. Start to migrate your Service Mesh
other applications.
POLICY Microservice Engine

• Defined SLAs around policies and

remediation.
• Customize policies based on your
business needs and minimize
exceptions.
• Expand policy tooling to include
applications such as traffic proxies,
service mesh, message buses and Linux.

Level 1 (Build) Level 2 (Operate) Level 3 (Scale) Level 4 (Improve) Level 5 (Optimize)

You are revising the decisions made before and looking for
optimization for you applications and infrastructure.
ORGANIZATION BUSINESS OUTCOMES TECHNOLOGY
PEOPLE
• The organization has a mature strategy
and skilled workforce.
Focused on automation such as
• Strong ability to recover and maintain scanning, policy, security and testing.
throughput as individuals join and leave. Managing your complete
• Business decisions are well informed infrastructure lifecycle through
by rich and accurate data across all software and tooling. Builds,
teams. upgrades and decommissioning are
• Advanced testing and release patterns conduced through code. Onboard
are developed and in use, such as Measurable results are your existing applications to your
blue/green or canary. cloud native platform. Automate the
available for your leadership response to events. Full production
teams. Continue to optimize with GitOps operators and controls.
PROCESS
your workloads against new Incorporate machine learning as part
• Resource usage data will help you cost and performance of your threat detection practices.
optimize spend and provide business metrics. Revise your goals at
cost analysis.
this point, adjusting them Automation
• Enforcing audits will increase compared to what was Everything as Code
continuous deployment speed,
helping you ship new features faster.
achieved and what you want CloudOps
to achieve.
• The software supply chain is secured, DevOps / DevSecOps / GitOps
with clear code provenance and
Automate as much as
possible according to cloud SecOps
secured release pipelines.
native best practices to AIOps
POLICY remove human error and to
avoid security and
FinOps 
• Compliance never ends!
performance problems.
• Take advantage of technologies such
as machine learning in order to
improve detection and enforcement,
while ensuring visibility of anomalous 
conditions in a large volume of
compliance data.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 8

2.2 CLOUDOPS

The implementation of this lab focuses on cluster management.
This is an intuitive and seamless process using the Alibaba Cloud
Console. You no longer need to manually deploy and develop 
clusters. Instead, you can create clusters in the Console easily, and
use multiple instance types of ECS and ECS Bare Metal instances in
a cluster. You can also deploy your cluster across zones to ensure

high availability.

The Container Service provides optimized OS (operating system)

images for your Kubernetes containers. It offers Kubernetes versions
and Docker versions with high stability and reinforced security while
supporting multi-cluster management and deployments across zone
to ensure high availability.

This simplifies your Kubernetes development and monitoring

processes, removing the requirement for manual interventions.

You can use Terraform to automatic provision infrastructure.

Terraform is an open-source Infrastructure-as-Code (IaC) software
tool created by HashiCorp. Using Terraform, you can define
and manage your data center infrastructure using a declarative
configuration language known as HashiCorp Configuration
Language, or optionally JSON.

Alibaba Cloud recently released a Terraform Module Web GUI to

help developers use the Terraform Module with ease. This section
explains how you can use the GUI to manage your clusters. This
simplifies many cluster management processes, helping you scale,
monitor and secure your applications.

If you are not familiar with Terraform and need assistance with the
Terraform Module Web GUI, then there are help documents and more
information available.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 9

The implementation of this lab includes the following high-level steps:


Step Description

1 Cluster Scaling How to scale a Kubernetes cluster up (or down) on

your network. 
2 HTTPS Certification by How set up a HTTPS solution or TLS/SSL certificates for
Cert-Manager your website.

3 HTTPS Certification for How to use Cert-Manager to obtain HTTPS Certification for
ASM by cert-manager Alibaba Cloud Service Mesh (ASM).

4 External DNS How to use External-DNS component to automatic add

resolution records in DNS or a private zone.

5 Enabling Observability How to connect your LogService and ARMS monitoring

service to your ACK solution.

6 ACK Cluster Monitoring How to use ACK cluster monitoring to examine your
application performance across your containers, pods,
services, and the characteristics of the entire cluster.

7 Disaster Recovery How to implement disaster recovery and backup best

and Backup practices using the Object Storage Service (OSS) solution
and how to create a backup vault.

8 Multi-Cloud How to manage and maintain a multi-cloud infrastructure

Management in ACK from the Alibaba Console.

9 Cross Region How to set up a cross-region network and connect

network setting multiple clusters.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 10

2.2.1 CLUSTER SCALING


You will learn how to scale your node on your Kubernetes cluster.
This process uses the Auto Scaling solution, which automatically
adjusts your computing resources based on your user requests. 
ACR EE


Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Using Auto Scaling, you can either add nodes to a cluster or add pods
to a cluster.

Auto Scaling

Cluster HPA
Autoscaling (Pod Autoscaling)

Node Pool Resource Metrics Cloud Metrics Customer Metrics

Add nodes CPU / SLB / LogService /

Prometheus
Memory Usage CMS / AHAS

Add pods

For most of cases, a combination of cluster and HPA autoscaling will

meet your customer requirements. The best practices to scale your
Kubernetes cluster with Auto Scaling are:



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 11

1. Enable cluster auto scaling for a specific cluster by selecting

More and then Auto Scaling in the dropdown menu. 




2. This creates a node pool, which is a group of nodes within a

cluster with the same configuration. You should now see the
following screen, confirming your configuration.

3. Add a node label for the node pool.

4. If you add a node affinity with the same name as the node label
when you create a deployment, the pod will then deploy on the
special nodes.
5. Click the checkbox to enable an HPA (Horizontal Pod
Autoscaling), which will scale the pod according to your metrics.

6. The node will automatically add or remove nodes in the node

pool, according to the pod status.
7.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 12

2.2.2 HPA (HORIZONTAL POD AUTOSCALING)


You will learn how to set up Cloud Metrics and Customer Metrics.

For most of cases, a combination of cluster and HPA autoscaling will

meet your customer requirements. If additional metrics are required,

Cloud Metrics and Customer Metrics are also available, following
these steps: 
1. Enable HPA (Horizontal Pod Autoscaling) by clicking Create.

2. Set auto scaling to either CPU Usage or Memory Usage


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 13

To enable Cloud Metrics HPA, follow these steps:


1. Install ack-alibaba-cloud-metrics-adapter in Marketplace in
ACK console.



2. Deploy and configure HPA. An example yaml file is

available here.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 14

2.2.3 HTTPS CERTIFICATION BY CERT-MANAGER


You will learn how set up a HTTPS solution or TLS/SSL certificates
for your website.

To achieve this, you need to set up Cert-Manager. Cert-Manager is a


third-party Kubernetes controller that automates getting TLS/SSL
certificates from Let's Encrypt and refreshing them. 
ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Cert-Manager adds certificates and certificate issuers as resource

types in Kubernetes clusters, and simplifies the process of obtaining,
renewing and using those certificates.

It can issue certificates from a variety of supported sources,

including Let’s Encrypt, HashiCorp Vault, and Venafi as well as private
PKIs (Public Key Infrastructures). It also ensures your certificates
are valid and up to date, and will attempt to renew certificates at a
configured time before expiry.
Ietsencrypt-prod venafi-tpp

Issuers
hashicorp-vault

Ietsencrypt-staging venafi-as-a-service

cert-manager

foo.bar.com example.com 
Certificates
Issuer:venafi-tpp
www.example.com
Issuer: letsencrypt-prod


Kubernetes signed keypair signed keypair
Secrets


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 15

Cert-Manager


HTTP validation DNS validation


Create alidns webhook of
Create ClusterIssure or Issuer
cert-menager

Ingress Create ClusterIssure or Issuer
(Certificate and Secret
will be automatically created)

Create Certificate

Create Secret

Ingress ASM (Service Mesh)

2.2.3.1 HTTPS CERTIFICATION FOR INGRESS BY CERT-MANAGER

USING HTTP VALIDATION
You can use Cert-Manager for both HTTP validation and DNS
validation. In this section, you will learn how to gain HTTPS
certification for Ingress by Cert-manager using HTTP validation.

1. Install Cert-Manager and type the following:

kubectl apply -f https://github.com/jetstack/cert-
manager/releases/download/v1.6.1/cert-manager.yaml


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 16

2. Create ClusterIssuer using the following commands:


apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: nacos-http01 
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: #email

privateKeySecretRef:
name: nacos-https-cert-1
solvers:
- http01:
ingress:
class: nginx

3. Update Ingress, add annotations and TLS. The Certificate and

Secret will also be auto created.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 17

4. Now, nacos-http01-secret is set to True.






5. You can now access the website with HTTPS, and see
the certification.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 18

2.2.3.2 HTTPS CERTIFICATION USING DNS VALIDATION

You can use Cert-Manager for both HTTP validation and DNS 
validation. You will now learn how to gain HTTPS certification for
Ingress by Cert-Manager using DNS validation for Alibaba Cloud
Service Mesh (ASM) or Ingress.

Cert-Manager

HTTP validation DNS validation

Create alidns webhook of

Create ClusterIssure or Issuer
cert-menager

Ingress Create ClusterIssure or Issuer

(Certificate and Secret
will be automatically created)

Create Certificate

Create Secret

Ingress ASM (Service Mesh)

1. Install Cert-Manager and type the following:

kubectl apply -f https://github.com/jetstack/cert-
manager/releases/download/v1.6.1/cert-manager.yaml

2. Install alidns webhook of cert-manager using the

following commands: kubectl apply -f https://gitee.
com/godu/helminit/raw/master/cert-manager/alidns-cm-
webhook.yamll


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 19

3. Create a RAM account, which can be accessed by the

DNS service. 
4. Add the policy to the RAM account. To do this, replace
“#domain-name” to your real domain name, such as “*.abc.com” 
or “xxx.abc.com”

{
"Version": "1",

"Statement": [
{
"Action": "*",
"Resource": "acs:alidns:*:*:domain/#domain-name",
"Effect": "Allow"
},
{
"Action": [
"alidns:DescribeSiteMonitorIspInfos",
"alidns:DescribeSiteMonitorIspCityInfos",
"alidns:DescribeSupportLines",
"alidns:DescribeDomains",
"alidns:DescribeDomainNs",
"alidns:DescribeDomainGroups"
],
"Resource": "acs:alidns:*:*:*",
"Effect": "Allow"
}
]
}

5. Create the Secret, the AK/SK are from the above RAM account
Note:

a. AK and SK must be encoded by base64.

i.e.: echo -n #AK | base64
b. The Secret must be created in “cert-manager” namespace

apiVersion: v1
kind: Secret
metadata:
name: alidns-secret
namespace: cert-manager
data:
access-key: #your AK
secret-key: #your SK

6. Create ClusterIssuer and Certificate, and the Secret will be

auto created.
Note: if the certificate using in ASM, it must be created in
“istio-system” namespace. If the certificate using in Ingress,
the namespace is the same with deployment namespace
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata: 
name: letsencrypt-nacos
spec:
acme: 
# Replace to your letsencrypt email


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 20

email: #email
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:

name: letsencrypt-nacos-secret
solvers:
- dns01:
webhook:

groupName: acme.yourcompany.com
solverName: alidns
config:
region: ""

accessKeySecretRef:
name: alidns-secret
key: access-key
secretKeySecretRef:
name: alidns-secret
key: secret-key

apiVersion:
cert-manager.io/v1
kind: Certificate
metadata:
name: tls-domain-cert
namespace: istio-systemspec:
# The secretName will store certificate content
secretName: tls-ops-domain-secret
commonName:
dnsNames:
# Replace to your real DNS name
- "*.cloudfoundry.top"
issuerRef:
name: letsencrypt-nacos
kind: ClusterIssuer

7. Add SecretVolumes to ASM Gateway, the “secretName” is the

same as one created in the previous step.

8. Execute command in ACK cluster to verify cert already bound in

ACK cluster.

kubectl exec -it -n istio-system $(kubectl -n istio-system get

pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.
name}') -- ls -al /etc/istio/ingressgateway-certs

The result will look like this: 


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 21

9. Execute command in the ACK cluster to verify that the Subject

is right. 
kubectl exec -it -n istio-system $(kubectl -n istio-system get
pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.
name}') -- cat /etc/istio/ingressgateway-certs/tls.crt | openssl

x509 -text -noout | grep 'Subject:'

The result will look like this:



10. Execute the following command in ACK cluster to verify Ingress

Gateway can access the certificate.

kubectl exec -it -n istio-system $(kubectl -n istio-system get

pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.
name}') -- curl 127.0.0.1:15000/certs

The result will look like this:

11. Add tls in your VirtualService gateway and

istio-ingressgateway.

12. You can now access the website with HTTPS, and see
the certification.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 22

2.2.3.3 EXTERNAL DNS

In this section, you will learn how to add an external DNS to

your architecture.

ACR EE

Application Center
Image Scan
Image Signature

KMS
(Certification)
Image Replication
Image trigger
…
Trigger
synchronization
Deployment

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

This is a common external DNS set-up:

Annotation DNS Record Alibaba

Service
Cloud DNS

External-DNS

Alibaba Cloud
Ingress
Host DNS Record Private Zone

1. First, you need to add a Policy to the ACK worker role.

{
"Version": "1",
"Statement": [
{
"Action": "alidns:AddDomainRecord",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "alidns:DeleteDomainRecord",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "alidns:UpdateDomainRecord", 
"Resource": "*",
"Effect": "Allow"
},
{



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 23

"Action": "alidns:DescribeDomainRecords",
"Resource": "*",
"Effect": "Allow"

},
{
"Action": "alidns:DescribeDomains",
"Resource": "*",

"Effect": "Allow"
},
{
"Action": "pvtz:AddZoneRecord",

"Resource": "*",
"Effect": "Allow"
},
{
"Action": "pvtz:DeleteZoneRecord",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "pvtz:UpdateZoneRecord",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "pvtz:DescribeZoneRecords",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "pvtz:DescribeZones",
"Resource": "*",
"Effect": "Allow"
},
{
"Action": "pvtz:DescribeZoneInfo",
"Resource": "*",
"Effect": "Allow"
}
]
}

2. Click on Worker RAM Role.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 24

3. Click on Add Permissions.






4. Click on + Create Policy.

5. The ack-external-dns-policy should now appear in the

Policy List.

6. Next, you need to deploy an external DNS. Click Marketplace

and deploy the following command:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 25

Note: When you deploy, you will see the parameter

alibabaCloudZoneType. There are two options, public 
means using the Alibaba Cloud DNS service, private
means using the PrivateZone service

7. Next, you can either update the Service yaml.
8. Add an annotation where the key is external-dns.alpha. 
kubernetes.io/hostname: value is domain. Use the
following command:

9. Wait a few seconds, until the resolution record has been auto
added in the DNS.

10. Or you can update Ingress yaml, add domain in host,

ExternalDNS will auto add a record in the DNS. Use the
following command:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 26

11. Wait a few seconds, the resolution record has been auto added
in DNS. 




12. You need to set up the PrivateZone settings. Click

Add PrivateZone and add the zone name.

13. Next, add a resolution record manually using the

on-screen instructions.
14. Then, bind the VPC.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 27

15. You can use External DNS for private zones in Service or Ingress.
16. In Service, Update the Service yaml, add an annotation where

the key is external-dns.alpha.kubernetes.io/hostname:
value is private zone domain. 


17. Wait a few seconds, the resolution record has been auto added
in Private Zone.

18. In Ingress, update the Ingress yaml, add a private zone domain in
host, ExternalDNS will auto add resolution record in
Private Zone.

19. Wait a few seconds, the resolution record has been auto added
in Private Zone.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 28

2.2.4 ENABLING OBSERVABILITY


You will learn how to connect your LogService and ARMS monitoring
service to your ACK solution.

ACR EE

Application Center
Image Scan
Image Signature


Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

2.2.4.1 INTRODUCING THE DASHBOARDS

Our LogService provides a rich selection of out-of-the-box
dashboards with plenty of options to choose from. This includes
a K8s Cluster Dashboard, K8s Event Center Dashboard, Ingress
Dashboard, Mesh Ingress Dashboard and Log Data.

You will now learn the basics of some of these dashboards.

K8s Cluster Dashboard K8s Event Center Dashboard

Elastic Node Operation Operation Audit Resource Resource K8S Polaris Event Center Core Event Node Event Pod Event
AutoScale Audit for Accounts Operation Operation Check Dashboard Overview Components Query Monitoring Monitoring
Overview Details Events

Ingress Dashboard Mesh Ingress Dashboard

Ingerss Ingerss Ingerss Ingerss Ingerss

Access Access Monitoring Exception Monitoring Log Service
Mesh Ingress Mesh Ingress Mesh Ingress
Overview Center Center Center Center for
Access Overview Monitoring Center Access Center
Blue/Green
Deployment

Log Data

ACK ASM ARMS Application Logs


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 29

1. Click on the K8s Cluster Dashboard and the Ingress Access

Center tab and the following screen appears, providing key 
information on the status of your Kubernetes clusters:




2. Kubernetes events are a resource type in Kubernetes that are

automatically created when other resources have state changes,
errors, or other messages that should be broadcast to the
system. There is not a lot of documentation available for events,
so our EventCenter tab is an invaluable resource when
debugging issues in your Kubernetes cluster:

3. You can also query these events to analyze important events,

helping you gain further insights into your clusters:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 30

4. The Ingress Overview tab provides an overview of your AIP

objects and their routing rules, helping you to manage external 
users’ access to your services in a Kubernetes cluster:




5. You can also monitor your Ingress objects using the Ingress
Exceptions Center to identify and analyze your systems by
user-specified metrics:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 31

6. 6. Polaris Checks also allows you to monitor dozens of different

checks to help you discover Kubernetes misconfigurations that 
frequently cause security vulnerabilities, outages, scaling
limitations and more.



2.2.4.2 LOG COLLECTION

You will now learn some key Log Collection features.

1. Go to the Add-ons tab and make sure logtail-ds has

been installed.

2. In Collection Configuration, add the Logstore name and add the

Log Path to the Container as: stdout (if required).


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 32

3. The stdout logs will shortly be collected in LogService. This

usually takes around one minute to complete. You could also 
define index to do quick analysis. The tag app:jupiter can also
be seen, which was also automatically added to the
ACK deployment and automatically added in logging:



2.2.5 ACK CLUSTER MONITORING

To provide a reliable service, you need to understand how the

application behaves when it is deployed to scale that application.

Now, you will learn how ACK cluster monitoring can help and
understand how to examine your application performance across
your containers, pods, services, and the characteristics of the entire
cluster.

1. Go to the Container Monitoring tab and install arms-cmonitor

add-on:

2. Click on Cluster Resources to view the Cluster

Resource Overview:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 33

3. You can monitor the topology of your clusters using the

Cluster Topology tab: 




4. Finally, the Prometheus Monitoring tab allows you to view your

Prometheus monitored clusters. This visualizes the data,
including information on configurations, permissions and
services that allow Prometheus to access resources and pull
information by scraping the elements of your cluster.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 34

2.2.6 DISASTER RECOVERY AND BACKUP


You will learn how to implement disaster recovery and backup best
practices using the Object Storage Service (OSS) solution and how to
create a backup vault. 
A backup vault is a cloud repository that is used by OSS to store
backup data. 
ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

ACK Disaster Recovery Center

Backup/Restore Controller Migration Controller Policy Manager

Data Center Alibaba Cloud

K8s Cluster Backup Vault for ACK Cluster

K8s resources

Deployment ConfigMap Deployment ConfigMap

STS Svc/Ingress STS Svc/Ingress

Hybrid Backup Recovery（HBR）

Backup Vault
for data



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 35

1. Create an OSS/NAS policy using the Create Policy feature in

the Console. 




2. Under Permissions > Grants, grant access to the RAM user

within the Authorized Scope section by selecting the
Alibaba Cloud Account option.

3. Create an OSS Bucket using the prefix “cnfs-oss-”.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 36

4. Create an Application Backup by following the on-screen

instructions: 




5. Type a backup name.

6. When you first restore from a backup, you need to fill in the
backup name. The next time it will automatically be selected,
as shown below.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 37

7. When restore is complete, you will see that the deployments

have been restored. 




2.2.7 MULTI-CLOUD MANAGEMENT

Hybrid Cloud / Multi Cloud infrastructures are growing in popularity.

Now, you will learn how to manage and maintain a multi-cloud

infrastructure in ACK from the Alibaba Console.

ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 38

1. Click the Register Cluster tab.






2. Connect to your target cluster by:

a. Run the yaml file in the target cluster. The connection

command under the Connection Information tab should
show the following:

b. The ack-cluster-agent will then be added in your

target cluster. 


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 39

3. The registered cluster will now be shown in ACK console:






2.2.7.1 PULL IMAGES FROM ACR WITHOUT PASSWORD

You will learn how to create a container registry policy for a
RAM user.

Note: If required, a detailed policy statement is available.

1. You can create a Secret in the target register cluster using the
following commands:

kubectl
-n kube-system create secret generic alibaba- addon-secret
--from-literal='access-key-id=<your AccessKey ID>'
--from-literal='access-key-secret=<your AccessKey Secret>'

2. You can click the aliyun-acr-credential-helper in Security Tab to

check this. For more information, please see this Config
acr-configuration.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 40

2.2.7.2 ENABLE LOGSERVICE TO REGISTER K8S CLUSTER

LogService creates projects in the register cluster region. 
You will now learn how to create a logtail policy for a RAM user.

Note: you can read about this further in our detailed


policy statement.

1. Create a secret in the target register cluster using the
following commands:

kubectl
-n kube-system create secret generic alibaba- addon-secret
--from-literal='access-key-id=<your AccessKey ID>'
--from-literal='access-key- secret=<your AccessKey Secret>'

2. Install the add-on recordings to meet your requirements using:

a. logtail-ds: to enable a cluster operation log and

application log to register the cluster
b. ack-node-problem-detector: to enable the EventCenter
to register cluster


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 41

2.2.7.3 ENABLE K8S MONITORING TO REGISTER A K8S CLUSTER

You will learn how to enable K8s monitoring to register a K8s cluster. 

1. Click Add Permissions to add an RMS Policy for a RAM user:




2. Install the ack-arms-cmonitor component from the

Alibaba Cloud Marketplace.

You can now monitor your Kubernetes clusters and other cloud
clusters from the Alibaba Console.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 42

2.2.8 CROSS REGION SUPPORT


Alibaba Cloud operates 80 availability zones in 25 regions around the
world with more global regions set to follow.

You will now learn how to set up your network for a multi-cluster,

cross-region deployment.
ACR EE

Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Here’s a standard cross-region network overview, covering the

Singapore and China (Hong Kong) regions. This is used as an
example in this section.

This set-up relies on two solutions from Alibaba Cloud. First, the
Cloud Enterprise Network (CEN), which is a global network to rapidly
build a distributed, hybrid cloud infrastructure. This allows you to

create a network with enterprise-level scalability and global coverage.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 43

Second, Transit Routers, which are used to forward data within the
same region or across different regions. 
To manage this cross-region deployment, follow these steps:

1. You can create your Transit Routers by clicking the Create
Transit Router tab. One transit router per region is
recommended. The first time you connect a network instance in 
a region, the system automatically creates a transit router in
the region.

2. Next, create bandwidth connections across your regions. Again,

this can be achieved by simply selecting your desired regions
and bandwidth requirements in the dashboard:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 44

3. By default, a CEN is used as the backbone of your cross-region

deployment. Click the Cross-region box and select your desired 
regions and your transit routers are automatically connected.




4. You can also add a Virtual Private Cloud (VPC)] to different

regions. A virtual private cloud service that provides an isolated
cloud network to operate resources in a secure environment.

Simply click the VPC option and add your target regions from
the dropdown list. Then, the VPC will automatically connect
across regions via a transit router.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 45

2.2.9 THREE DATA CENTERS IN TWO REGIONS

BASED ON ACK ONE 
In this section, a disaster recovery architecture including three
centers in two regions is explained. Three business processing 
centers are used in two regions: a production center, a disaster
recovery center in the same region, and a disaster recovery center
in a different region.


User Cloud DNS Global Traffic Manager（GTM）

Normal traffic 50% Normal traffic 50% Failover traffic

Region 1 VPC 1 Region 1 VPC 2

AZ 1 AZ 1 AZ 1

SLB 1 SLB 2 SLB 1

Application Application Application

ACK Cluster 1 ACK Cluster 2 ACK Cluster 3

RDS 1 RDS 2 RDS 3

ACK One
Application distribution, Administrator
Aggregated monitoring

In this section, you will learn how to deploy two sets of environments
in a region (Region 1) to implement active-active mode in the same
region. You will learn how to deploy a set of environments in another
region (Region 2) as a remote disaster recovery center for data
backup. When two centers in the same region fail at the same time,
the remote disaster recovery center can switch to process services.
The disaster recovery solution based on ACK One can guarantee the
continuous operation of the business to a great extent.

ACK One's multi-cluster management application distribution

feature can help enterprises manage three K8S clusters in a unified
console, deploy and upgrade quickly on three Kubernetes clusters
configured by ACK One-Hub, and apply differentiated configuration
on 3 Kubernetes clusters. Using GTM (Global Traffic Management),
you can realize the automatic switching of business traffic among the
three K8s clusters when a fault occurs. Under normal circumstances,
all traffic is processed by the applications on the two clusters in
Region 1, and each cluster handles 50% of the traffic. When the

application on one cluster is abnormal, GTM routes all traffic to
another cluster for processing, and when the applications on the two
clusters in Region 1 are abnormal at the same time, GTM routes the 
traffic to the Region 2 cluster for processing.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 46

To achieve this, follow these steps:


1. Click on Distributed Container Platform ACK One to create an
ACKOne master instance.



2. Enable multi-cluster management

3. Add three K8X clusters to the master instance, which are

cluster1-hongkong, cluster2-hongkong and
cluster1-singapore.

4. Copy the following to the computer $HOME/.kube/config file toe

connect to the ACKOne master instance.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 47

apiVersion: apps/v1
kind: Deployment
metadata:

labels:
app: web-demo
name: web-demo
namespace: demo

spec:
replicas: 5
selector:
matchLabels:

app: web-demo
template:
metadata:
labels:
app: web-demo
spec:
containers:
- image: acr-multiple-clusters-registry.cn-hangzhou.
cr.aliyuncs.com/ack-multiple-clusters/web-demo:0.4.0
name: web-demo
env:
- name: ENV_NAME
value: cluster1-hongkong
volumeMounts:
- name: config-file
mountPath: "/config-file"
readOnly: true
volumes:
- name: config-file
configMap:
items:
- key: config.json
path: config.json
name: web-demo
---

5. Create app-meta.yaml under the demo namespace.

apiVersion: v1
kind: Service
metadata:
name: web-demo
namespace: demo
labels:
app: web-demo
spec:
selector:
app: web-demo
ports:
- protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-demo

namespace: demo
labels:
app: web-demo 
spec:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 48

rules:
- host: web-demo.example.com
http:

paths:
- path: /
pathType: Prefix
backend:

service:
name: web-demo
port:
number: 80

---
apiVersion: v1
kind: ConfigMap
metadata:
name: web-demo
namespace: demo
labels:
app: web-demo
data:
config.json: |
{
database-host: "hongkong-db.pg.aliyun.com"
}

6. Install ACK One's command line tools AMC.

wget http://ack-one.oss-cn-hangzhou.aliyuncs.com/kubectl-amc-linux
&& chmod +x kubectl-amc-linux && mv kubectl-amc-linux /usr/local/
bin/kubectl-amc

7. The below diagram shows the associated clusters managed by

the master instance.

8. Update the cluster name in policy ,include cluster1-hongkong,

cluster2-hongkong and cluster1-sinapore.
9. Create app.yaml under the demo namespace.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 49

10. Run the command in master instance, that will deploy

applications to cluster2-hongkong, cluster1-Singapore 
kubectl amc workflow resume web-demo -n demo

11. The below diagram shows the deployment status of 
an application.

12. The below diagram shows the running status of the application
on each cluster.

13. The below diagram shows the Ingress status of the application
on each cluster.

14. Next, activate and create a Global Traffic Management

instance (GTM).


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 50

15. On Basic Configuration, add the CNAME.





16. Edit the Record. Set DNS resolution to CNAME access domain
name for GTM. Record Value is the CNAME(Public Network)
of GTM.

17. In the created GTM example, create two address pools, enable
the Health Check. The Address is the Ingress IP address.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 51

18. Configure an access policy and set the Primary Address Pool to
be the Hong Kong address pool and the Secondary Address 
Pool to be the Singapore address pool. Normal traffic is handled
by the Hong Kong cluster application. When all the Hong Kong
cluster applications are unavailable, it will automatically switch to

the Singapore cluster application for processing.


19. Under normal circumstances, all traffic is processed by the

application on the two clusters in Hong Kong, and each cluster
handles 50% of the traffic.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 52

20. When the application on cluster1-hongkong is abnormal, GTM

routes all traffic to cluster2-hongkong for processing. 





Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 53

21. When the applications on the clusters cluster1-hongkong and

cluster2-hongkong are abnormal at the same time, GTM 
routes the traffic to the cluster1-singapore cluster
for processing.




Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 54





Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 55

2.3 DEVOPS/DEVSECOPS

DevOps covers the latest methodologies and practices such
as continuous integration/ continuous delivery (CI/CD), Image
Replication cross region, delivery chain etc. DevSecOps adds 
another layer of security to the DevOps environment.
ACR EE
Image Scan
Application Center

Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

The implementation of this lab includes the following high-level steps:

Step Description

1 Delivery Chain How to implement some best practices for several key tasks
on the DevOps and DevSecOps delivery chain.

2 Application Center How the Application Center streamlines the deployment

process, helping you release your applications with
confidence.

3 Service Mesh How to examine your Service Mesh Architecture and how this
fits in within the wider Container Service.

4 Tracing How to set up the Tracing Analysis Service and how

Tracing Analysis features allow you to quickly identify root
causes and analyze performance bottlenecks for your
distributed applications.

5 MSE as a Microservice How to set up and register MSE with Nacos and add a cloud-
Registry native gateway to your deployment.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 56

2.3.1 DELIVERY CHAIN


DevOps and DevSecOps practices are important to streamline
today’s software delivery chain, helping optimize and accelerate the
development and deployment of your applications. 
Fix CVEs Security Center
ACR EE
automatically

Pipeline

Detect

Lazy Pulling
Artifacts
Developer E.g. Docker Images, Scan Sign Geo Replicate ACR EE Accelerate ACK
Helm Charts

You will learn how to implement some best practices for several key
tasks on the DevOps and DevSecOps delivery chain.

2.3.1.1 SETTING PULL IMAGES WITHOUT A PASSWORD

You will learn how to set pull images without a password for
your instances.

1. Check “aliyun-acr-credentail-helper” has been installed.

2. Set your ACR instanceID and regionID (if you are using a cross
region deployment) in the acr-configuration area of
the ConfigMaps area.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 57

3. Click Policies and add your policy for the ACK cluster worker
role, using the following code: 




2.3.1.2 SET UP A SECURITY SCAN

You will learn how to set up a security scan across images.

1. Create a delivery chain and select Scope. The delivery chain will
now complete the scan.

2. Configure your node, selecting the Security Center option for

Security Engine.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 58

3. Click Target and select your region from the list to synchronize
the trigger. 




4. When you return to the Create Trigger window, the new image
version has been pushed into the trigger deployment. It is now
seen in the Trigger URL textbox.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 59

2.3.1.3 IMAGE SIGNATURE WORKFLOW

The Image Signature is a key security feature for Kubernetes. You will 
now learn how to set this up.
Deploy Image with Signature Deploy Image with Signature Deploy Image with Signature Deploy Image with Signature

KMS Security Center ACR ACK

Create a Key

Create Witness Key Configuration Cluster
(RSA_2048 Sign/Verify)

Create Security Policy Sign Configuration Name Space

1. Click Create Key in the Key Management Service (KMS) area of

the Console.

2. Create the RAM Role and update your Trust Policy, if required.

3. Create a Policy in the RAM area.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 60

4. Bind this Policy to a specific Role.






5. Create a Witness in the Container Signature area of the

Security Center.

6. Click to install the kritis-validation-hook component

in ACK.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 61

7. Add Policy in Security Center.






8. Create a signature rule in ACR using our simple two-step

process to set up the key and sign configurations.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 62

9. If you have deployed an image without a signature to a

namespace which has signature policy, the deployment will fail: 



10. However, if you deploy an image in a namespace that is not an
enabled image signature or an image that is signed in a special
namespace, the deployment will succeed.

2.3.2 APPLICATION CENTER

DevOps and DevSecOps teams are responsible for deploying

their applications on the cloud. Depending on your infrastructure,
this deployment may take place on a multi- cluster and/or multi-
cloud environment.

You will learn how our Application Center streamlines the deployment
process, helping you release your applications with confidence.

ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Log Service ARMS
Cluster
Inspections Intelligence Service

hystax
Namespace
Node pool
Inspection
Solution to solve
Cluster Upgrade
Check
Logging
Tracing
Prometheus Monitoring
Cluster Monitoring 
Application security issues Node/Pod/Network Ingress Access Center Cluster Topology
Inspection Report diagnosis Events & Alerting Application Monitoring
Multi-Cloud Bill Control … … … …


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 63

In this section, some typical deployment scenarios are examined

using the Application Center. As the diagram below demonstrates, 
Application Center sits at the heart of many application deployments,
bringing together a diverse range of solutions and teams.

Deployment/
Service

Developer Commit
Git Repo
App Stack 

Trigger to auto deploy new

Application Center
version image of application
ACR EE

GitOps GitOps

Register Cluster Other Cloud Provider New version

(K8S Cluster) Image push
ACK
Applications

2.3.2.1 INTRODUCING APPLICATION CENTER

The Application Center allows you to check the application status and
deploy applications using YAML or Helm. It is available under the
Multi-Cluster tab in the Console.

Before you start working in this area, the following preconditions

must be met:

1. Add the following policy to your worker role of your ACK cluster:

{
"Action": [
"cs:CreateTemplate",
"cs:DescribeTemplates",
"cs:DescribeTemplateAttribute",
"cs:UpdateTemplate",
"cs:DeleteTemplate"
],
"Resource": [
"*"
],
"Effect": "Allow" 
}


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 64

2. Install the appropriate component to the target cluster:






2.3.2.2 APPLICATION CENTER CONFIGURATION

You will learn how to configure your Application Center.

1. Select source, in git repository or yaml template, or deploy by

helm chart.

2. Assuming the source is git repository, Click Create Application

and set up your application’s repository, branch, path etc:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 65

3. Set your deployment clusters for the release configuration.






4. Create an application from the Application Center.

5. Here is an example deployment resources topology. If you

access this, it directly shows the health status of your resources.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 66

6. Once your configuration is set up, it’s easy to get a deployment

overview to monitor your environment. Simply click on your 
cluster and you should see a topology similar to the following:




2.3.2.3 DEPLOY YOUR APPLICATIONS IN THE MULTI-CLOUD

You will learn how to deploy your application in a multi-cloud
environment and across a full release.

1. This can be achieved with a few clicks. Simply click on a cluster

and click Deploy or Redeploy.

2. When the deployment is releasing and then complete, you can

view the health of your cluster.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 67

3. To monitor your deployments, click the Deployments tab

under Workloads. 




4. You can automatically deploy a new version of an application

from the Console too. When you create a trigger, the application
will automatically update to the new version of the application,
once the image version has been updated. Start by
clicking Trigger.

5. Fill in the Trigger’s credentials in the pop-up window.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 68

6. When you push a new version of image, the application will

automatically deploy, and the topology will show the revision has 
been changed.




2.3.3 SERVICE MESH

You will learn how to setting Service Mesh Architecture and how this
fits in within the wider Container Service.
ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Alibaba Cloud Service Mesh (ASM) is a fully managed service mesh

platform. ASM is compatible with the open source Istio service 
mesh of the Istio community. ASM allows you to manage services
in a simplified manner. For example, you can use ASM to route and
split inter-service traffic, secure inter-service communication with

authentication, and observe the behavior of services in meshes .

Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 69

The following figure shows the architecture of ASM.

GUI/Integrated Capability, Fully Compatible with Istio Community, Support KubeAPI


Service Mesh Control Plane

Managed Core – Unified Architecture (Stand/Pro Version)


Cloud Service Integration Capability Custom Expansion Capabilities
Flexible architecture, multi-version support, enhanced customization capabilities

Tracing
Analysis
Prometheus
Monitoring Managed Traffic Zero-Trust Hot upgrade Custom Envoy
Tracing
Analysis

Core Management Spiffe/Spire (with OpenKruise)

Inter-VPC Rate Limits Tracing Tracing

Cluster with AHAS Analysis Analysis

Lifecycle Mixerless Mesh

Diagnosis
Microservice Management Telemetry Expansion Tracing Tracing
Log Service
Engine Analysis Analysis

Provide Unified Grid Governance for Service on Heterogeneous Computing Infra

(Anywhere Service Mesh)

Service Mesh Data Plane Other Cloud Vender / IDC

Pod Pod ECI Pod ECS VM ENS Pod External Cluster

Ingress
ASM Proxy Proxy Proxy Proxy Proxy
Gateway
Egress
Envoy Expansion
Cert Management Service Service Service Service Service
Protocol Enhancement

ACK K8s Cluster & ECI Serverless K8s Cluster ECS VM Edge Cluster External Registered Cluster

The Console is incredibly easy to use whether setting up an ASM

instance, enabling tracing, logging or setting up the Prometheus
ability by GUI. Let’s examine each of these features now.

2.3.3.1 CREATE AN ASM INSTANCE

1. Click to Create ASM Instance and fill in the following information:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 70

2. Click OK and then add your ACK cluster to ASM by ticking the
target cluster. 




3. Assign permissions to the RAM user. If you need more

information on this topic, please see this document.

2.3.3.2 CREATE AN ASM GATEWAY

You will learn how to create an ASM Gateway.

1. Click to create an ASM Gateway.

2. Check the istio-ingressgateway option. This will then 

be automatically added to the ACK cluster.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 71

2.3.3.3 SERVICE MESH – SIDECAR INJECTION

You will now learn how to inject a sidecar proxy. This may be 
required when ASM detects a namespace label you configure for the
workload's Pod.

1. Create a namespace and enable automatic sidecar injection. If
the namespace is the same with ACK, the label will be added in 
namespace of ACK after synchronization.

2. If a pod is created before istio-injection, you need to

redeploy the deployment, and then check if istio-proxy
is present in the pods Container.

2.3.3.4 CREATE A SERVICE MESH CONNECTION TO SERVICE

REGISTRY (NACOS)
Nacos is an opensource and easy-to-use platform designed by the
Alibaba Cloud team for dynamic service discovery, configuration and
service management. It helps you build cloud native applications and
microservices platform easily.

You will now learn how to connect your ASM to Nacos.

1. Create ServiceEntry in ASM

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-nacos-svc
spec:
hosts:
#replace to real nacos host, i.e. mesh-nacos.**.com
- “NACOS_SERVER_HOST” 
location: MESH_EXTERNAL
ports:
- number: 8848
name: http

resolution: DNS


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 72

2. Create EnvoyFilter in ASM.


Please note: If EnvoyFilter was created after deployment,
don’t forget to redeploy your deployment in ACK.

apiVersion: networking.istio.io/v1alpha3

kind: EnvoyFilter
metadata:
labels: 
provider: "asm"
asm-system: "true"
name: nacos-subscribe-lua
namespace: istio-system
spec:
configPatches:
# The first patch adds the lua filter to the listener/http
connection manager
- applyTo: HTTP_FILTER
match:
proxy:
proxyVersion: "^1.*"
context: SIDECAR_OUTBOUND
listener:
portNumber: 8848
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value: # lua filter specification
name: envoy.lua
typed_config:
"@type": "type.googleapis.com/envoy.extensions.filters.
http.lua.v3.Lua"
inlineCode: |
-- copyright: ASM (Alibaba Cloud ServiceMesh)
function envoy_on_request(request_handle)
local request_headers = request_handle:headers()
-- /nacos/v1/ns/instance/list?healthyOnly=
false&namespaceId=public&clientIP=11.122.63.81&serviceName=
DEFAULT_GROUP%40%40service-provider&udpPort=53174&encoding=UTF-8
local path = request_headers:get(":path")
if string.match(path,"^/nacos/v1/ns/instance/list")
then
local servicename = string.
gsub(path,".*&serviceName.*40([%w.\\_\\-]+)&.*","%1")
request_
handle:streamInfo():dynamicMetadata():set("context", "request.
path", path)
request_
handle:streamInfo():dynamicMetadata():set("context", "request.
servicename", servicename)
request_handle:logInfo("subscribe for
serviceName: " .. servicename)
else
request_
handle:streamInfo():dynamicMetadata():set("context", "request.
path", "")
end
end 
function envoy_on_response(response_handle)
local request_path = response_
handle:streamInfo():dynamicMetadata():get("context")["request.
path"]



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 73

if request_path == "" then

return
end

local servicename = response_
handle:streamInfo():dynamicMetadata():get("context")["request.
servicename"]
response_handle:logInfo("modified response ip to

serviceName:" .. servicename)
local bodyObject = response_handle:body(true)
local body=
bodyObject:getBytes(0,bodyObject:length())

body = string.gsub(body,"%s+","")
body = string.gsub(body,"(ip\":\")
(%d+.%d+.%d+.%d+)","%1"..servicename)
response_handle:body():setBytes(body)
end

2.3.3.5 SERVICE MESH – CREATE DESTINATION RULE

You will learn how to create a Destination Rule for your
application deployment.

1. Fill in the basic information.

2. Add the HTTP Route information and select Matching URI to

automate this task.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 74

2.3.3.6 SERVICE MESH – CREATE GATEWAY AND SETTING ROUTE

You will learn how to create a service gateway in ASM and set the 
route for your application deployment.

1. Fill in the basic information. 



2. Set the HTTP Route information. Using Matching URI automates

this process.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 75

2.3.3.7 SERVICE MESH – SERVICE ROUTING

You will learn how to apply your service routing across your 
deployment.

1. Select Apply to All Sidecars and Matching URI to automate 

this process.


2.3.3.8 CROSS REGIONS TRAFFIC MANAGEMENT

ASM supports traffic management for any K8s cluster from Alibaba
Cloud, including edge, on-premises and other cloud deployments. It
provides full E2E observability for Service Mesh.

Alibaba DNS
HK Region Singapore Region
GTM (Global
gRPC REST Traffic Manager) REST gRPC

Service BFF Frontend Frontend BFF Service

ASM Gateway ASM Gateway

Deployment ConfigMap Deployment ConfigMap

CEN / SD-WAN

STS Svc/Ingress STS Svc/Ingress

Alibaba Cloud Service Mesh (ASM)


You will now learn some best practices when deploying your
applications across multiple regions.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 76

1. Add your ACK cluster in different regions to the ASM.






2. Deploy your applications across your ACK cluster in

different regions.
3. Create an ASM gateway in different regions, respectively.
4. Create a Gateway and VirtualService for a different region
in ASM.

5. Create Address Pool in Global Traffic Manager, it will be used in

next step.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 77

6. Create Global Traffic Manager in DNS, configuring your

Access Policy and Address Policy in the Console. 




7. Next, click on the location and select your desired location.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 78

2.3.3.9 OTHER SCENARIOS OF SERVICE MESH

Here are a handle of additional scenarios for your service mesh. 

1. Traffic distribution cross region, or failover services in the same

region, as shown below. 
1. CEN
2. Separately Vswitch IP range

3. Pod CIDR to vtb
4. Pod CIDR to Security group
70% Traffic 30% Traffic

HK Region Singapore Region HK Region HK Region

CEN CEN

VPC VPC VPC VPC

productpage productpage productpage productpage

Failover
reviews reviews reviews reviews

details details details or details

fallback

ratings ratings ratings ratings

Traffic Distribution Failover

2. You can also set up end-to-end A/B testing without any changes
to your application code, as shown below.

destination:
subset: base-env
ASM IngressGateway ASM IngressGateway
fallback:
case: noavailabled
target: dev-x-env

Base Env Dev-X Env Dev-Y Env Product Env Canary or A/B Testing Env

A A A Service A

fallback
B B Service B

fallback
C C Service C C

fallback
D D Service D

Multi-dev Environment Canary or A/B Testing


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 79

2.3.4 APPLICATION MONITORING


Application Monitoring allows you to monitor the health and status of
your applications.

You will learn how to enable our Application Monitoring service and

learn about some fundamental features of this service.
ACR EE

Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

To enable Application Monitoring:

1. Install ack-arms-pilot follow the guidance.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 80

2. Update Deployment, added corresponding labels.






3. You will see the applications monitoring dashboard in

ARMS console.

4. Click an application, the detail monitoring dashboard and

topology of the application looks like below:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 81

5. Click an Application Details in left side menu, and click the

TraceID link to access detailed Tracing information. 




6. The following information will appear.

7. Click on the Error Analysis tab and then the JVM Monitoring
tab to access additional information.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 82

8. The Interface Invocation Analysis tab gives you the invocation

link and response time for your application. 




9. If you want diagnosis an Application Issue Root Reason, you

could open Arthas in Application Settings.

Note: If the arms agent has to upgrade, go to “arms-pilot-

ack-arms-pilot” deployment in arms-pilot namespace in
ACK, add environment, key is ARMS_INIT_ARMS_AGENT_
DOWNLOAD_URL , value is http://arms-apm-ap-southeast.
oss-ap-southeast-1.aliyuncs.com/cloud_ap-
southeast-1/2.7.1.3/. Then restart application.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 83

10. Leverage Application Diagnosis to find the root cause for an

application’s issue. This provides analysis from the thread using 
either Object in a Class or special method.




11. Here is the objectViewer.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 84

12. You can also use the ‘Method execution analysis’ to get the
method with the highest response time. 




Note: method execution analysis will result during the method

is execution.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 85

2.3.5 TRACING

Log Service is an all-in-one service for your log-type data. It supports
the collection, consumption, shipping, search, and analysis of your
logs, while improving the capacity of your processing and analyzing 
capabilities for large amounts of logs.

ACR EE
Image Scan
Application Center

Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

OpenTelemetry is a set of APIs, SDKs, tooling and integrations

that are designed for the creation and management of telemetry
data such as traces, metrics, and logs, to help you analyze your
software’s performance and behavior. The project provides a vendor-
agnostic implementation that can be configured to send telemetry
data to the backend(s), it supports exporting data to a variety of
open-source and commercial back-ends.

Like the diagram shows, OpenTelemetry provides Specification,

Collection and SDKs features. But LogService is required to store and
analyze the trace data on Cloud, helping you find trace data quickly,
find log data to identify the root reasons for any issues. The data in
LogService also follows OpenTelemetry's rules.
LogService


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 86

You will now learn how to set up our Tracing Analysis Service using
the Trace Ability feature. To enable Trace Ability with LogService, we 
recommend the following steps:

1. To install Trace Ability, you can install the OpenTelemetry Probe 

without install agent or jar package in your applications using
the following command: 
kubectl apply -f https://github.com/jetstack/cert-manager/
releases/download/v1.6.1/cert-manager.yaml
kubectl apply -f https://github.com/open-telemetry/opentelemetry-
operator/releases/latest/download/opentelemetry-operator.yaml

2. Next, create a Trace Instance in LogService and deploy a CRD

(OpenTelemetryCollector) in ACK, setting the endpoint, project
name, logstore name, etc, using the following command: the
project name and endpoint you could reference 2.1–2.3.
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
name: otel
spec:
image: otel/opentelemetry-collector-contrib:latest
config: |
receivers:
otlp:
protocols:
grpc:
http:
exporters:
alibabacloud_logservice/logs:
endpoint: "cn-hangzhou.log.aliyuncs.com"
project: "demo-project"
logstore: "store-logs"
access_key_id: "access-key-id"
access_key_secret: "access-key-secret"
alibabacloud_logservice/metrics:
endpoint: "cn-hangzhou.log.aliyuncs.com"
project: "demo-project"
logstore: "store-metrics"
access_key_id: "access-key-id"
access_key_secret: "access-key-secret"
alibabacloud_logservice/traces:
endpoint: "cn-hangzhou.log.aliyuncs.com"
project: "demo-project"
logstore: "store-traces"
access_key_id: "access-key-id"
access_key_secret: "access-key-secret"
service:
pipelines:
traces:
receivers: [otlp]
exporters: [alibabacloud_logservice/traces]
logs:
receivers: [otlp]
exporters: [alibabacloud_logservice/logs]
metrics:
receivers: [otlp]

exporters: [alibabacloud_logservice/metrics]

The above OpenTelemetryCollector yaml file lacks endpoint 

and project, name. Follow the steps below to get the
related information.

Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 87

a. Get the Project name from the Trace list:



b. Find the project in the LogService home page. 

c. Get the endpoints from the project home page. These will
be used to configure the OpenTelemetryCollector.

3. Deploy a CRD (Instrumentation) in the ACK cluster, that will

auto collect service tracing data in log service, supports Java,
NodeJS, Python.

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
name: my-instrumentation
spec:
resource:
resourceAttributes:
service.name: SERVICE_NAME
service.namespace: SERVICE_NAMESPACE
exporter:
endpoint: http://otel-collector:4317
propagators:
- tracecontext
- baggage
- b3
java:
image: ghcr.io/open-telemetry/opentelemetry-operator/
autoinstrumentation-java:latest
nodejs:
image: ghcr.io/open-telemetry/opentelemetry-operator/
autoinstrumentation-nodejs:latest 
python:
image: ghcr.io/open-telemetry/opentelemetry-operator/
autoinstrumentation-python:latest



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 88

4. Add OTEL_RESOURCE_ATTRIBUTES environment in deployment

yaml file. 



5. Add instrumentation annotation in deployment yaml file.

6. When you click on Trace you should now see an overview

similar to the following:

7. You can also get an overview of your services by clicking the

Services tab:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 89

8. In Trace Analysis tab you could see all of the services and
latency of each span. 




9. Using our intuitive Common Query service you can access a

range of information, tailoring your queries.

10. It also provides query by business attribute, such as orderId,

to get trace data for a special order ID.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 90

11. Click the Details tab to get further information into your Traces.
Each Span has a different color line, making it easy to identify 
and work with.




12. From Topology Query, you could overview the microservices

topology and QPS some metrics.

We will now learn how our Tracing analysis features allows you to
quickly identify root causes and analyze performance bottlenecks for
your distributed applications.

1. Query errors in the Trace Analysis section::


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 91

2. To find the error method, you will receive some error information
from log tab: 




3. Click the Details tab, move to the “traceID” item, click the link
icon to see the detailed log of the trace.

4. This provides full details of the traceID:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 92

5. You also can use the traceID in Log detail link to trace console.





6. In the caller item you could add a event link to source code such
as github url.


Let’s summarize the scenario. As the above steps show when you
meet an error, you could find out the error method, drill down from a
traceID to find out the log details, and then drill down to find out the 
source code.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 93

7. Also, you can easily build your own dashboard from trace data,
according to your requirements. 




8. And you can access a Trace Quality Analysis report under the
Services tab in the Console.

2.3.6 MSE AS A MICROSERVICE REGISTRY

Our Microservices Engine (MSE) platform is compatible with a

range of open source ecosystems, providing a reliable, cost-effective
and efficient microservices API gateway that complies with the
Kubernetes Ingress standard.

Many service registration and configuration center solutions are

now available and we support ZooKeeper and Nacos. Alibaba Cloud
recommends the Nacos open-source platform by Alibaba.

Nacos makes it easy to build cloud-native applications and supports

dynamic service discovery, configuration management, and service
management. It represents the best practices of Alibaba. It can
be used in core scenarios, such as microservice registries and

configuration centers.

You will now learn how to set up and register MSE with Nacos using

the following steps:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 94

1. You can create a Microservice Registry Instance using engine

support from either Nacos or ZooKeeper. We would recommend 
using the Nacos platform.




2. Select your bandwidth requirements. If your ACK cluster and

MSE are in different VPCs, you should also open the
Internet endpoint.

3. When the MSE Registry instance is created, the access endpoint

is added to the list.

4. Update the application code, adding the Nacos discovery

server address.

5. Next, update the deployment yaml file, adding the Nacos

discovery environment using the following information:

name: spring.cloud.nacos.discovery.server-addr
value: #MSE registry endpoint
name: spring.cloud.nacos.discovery.metadata.version
value: #app version


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 95

6. When you create a deployment in ACK, the application will be

auto registered in the Services area of the MSE console. 




7. The health status is also shown in the Detailed Info section.

8. You can also add and view the application configuration details
in this area.

To add a cloud-native gateway to your deployment, we would

recommend the following steps:

1. Click Create Gateway to create a cloud-native gateway. Then,

you get the access IP information from the instance list.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 96

2. Set the Service Source. This it can be ACK or MSE Nacos.






3. Create the Service from Service Source.

4. Next, set the routes.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 97

5. Then, add the security group authorization. This is the same as

the security group of ACK nodes, which run your pod 
of applications.




6. Access the gateway url to check it is working.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 98

2.4 SECOPS

SecOps is formed from a combination of security and IT
operations. These highly skilled teams focus on monitoring and
assessing the risks with your digital solutions, while protecting 
your corporate assets.

The Alibaba Cloud Container Service provides a range of SecOps 

functionality, helping you monitor and assess the security of
your applications.

It covers the following four areas:

1. Infrastructure Security
» Computing Security
» Network Security
» Storage Security

2. Cloud-Native Architecture & Delivery Chain Security

» Artifact Security
» Deployment Security
» Runtime Security
» Cluster Security

3. Cloud-Native Application Security

» Authority Management
» Microservice Security

» Cloud-Native Security Operation

» Attack Awareness
» Security Report
» Security Inspection


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 99


Baseline Check Anti-DDoS EIP Anti-DDoS Pro Anti-DDoS Premium Game Shield 0day Emergency Response
Config Assessment Virus Defense
Anti-Ransomeware
Vulnerabiities

Proactive
Tamper Protection

AK Leak Intrusion/
Intelligent
Policy

Vulnerability Recommendation
Defense Detection
Prevention
Access Control


Security Data Leak Prevention
LifseaOS MFA Network
Center Resource Web Intrusion Prevention
(Multi-Factor Authentication) Boundaries
Security Access Protection
Hardening Management
(RAM) Node Pool Load Bot Threat Intelligence
Balance
Security Web
ALB/SLB APP Protection
Policies Security
ECS ECS
Management
ACK Security
Group
Anti- Backup and
DDoS Restore
Computing Cloud Data Disk
Security Firewall Bot Encryption
Network Management ACK TEE
Security
Security Base Image

WAF Image Immutable

ACK Image Security Scan

Storage Storage Image Security Fix
Security Image Signature
ACR
Infrastructure (Container
Artifact Delivery Chain
Security Registry)
Security
OPA Policy Governance

Deployment ACK Image Signature Verification

Security
Security Sandbox
ACK
ACK TEE

Base Architecture Runtime

Alibaba Cloud
Container Runtime Protection
Security
Security / Delivery Security Center Container Firewall
Chain Security
Cloud-Native
Proactive Defense for Containers
CIS Baseline
Check Cluster
Cluster Security API Server Auditing

Security
Inspections
Security Ingress Auditing
ACK CoreDNS Auditing
Logging

Overview Container MSE

and
Auditing
Event Alerting

Asset Exposure Analysis Zero Trust

Security Inspection Security
Attack Awareness

Cloud Honeypot Authorization Policies Application Monitoring

Log Analysis
Register Cluster for OPA sidecar
Security Report Hybrid-Cloud/Multi-Cloud/Edge

Request Authentication Web Intrusion Prevention


Configuration Management Confidential, Certification, Key Management


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 100

The implementation of this lab includes the following high-level steps:


Step Description

1 Secret Management How to add enable your Secret-Manager asset

Setting 
2 Policy Governance How to leverage OPA and Gatekeeper to enable some policies

3 Zero Trust Security How to add enable your ASM mTLS Authorization Policy.
You will learn how to:

» Block communication between namespaces
» Forbid a namespace access to a website
» Forbid a namespace access to RDS

2.4.1 SECRET MANAGEMENT SETTING

ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Sometimes, the sensitive data is stored in other services, rather than

secret in Kubernetes cluster. The KMS(key management service) is
an option to help you protect these assets.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 101

You will learn how to implement your Secret Management settings.


1. Add a policy to your ACK cluster worker role.

{
"Action": [

"kms:GetSecretValue"
],
"Resource": [ 
"*"
],
"Effect": "Allow"
}

2. Check the Worker RAM Role contains this information.

3. Deploy ack-secret-manager in the ACK cluster from the

Marketplace area.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 102

4. Create a Secret in the KMS.






5. Apply ExternalSecret in the ACK.

apiVersion: 'alibabacloud.com/v1alpha1'
kind: ExternalSecret
metadata:
name: jeecg-service-extsecret
spec:
data:
- key: jeecg-app-password
name: password
versionStage: ACSCurrent

Note, the key is the same with secret name as the KMS.

6. Mount a Secret in a volume, rather than in the environment

variables. This is because the volumes are automatically
removed from a mode when the pod on the node is deleted.
The values of the environment variables, however, may
accidentally appear in the logs.

apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1

kind: Deployment
metadata:
name: nginx-deployment-basic
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9 # replace it with your exactly <image_
name:tags>

ports:
- containerPort: 80
volumeMounts: 
- name: k8s-mysql-storage


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 103

mountPath: /tmp
resources:
limits:

cpu: "500m"
volumes:
- name: k8s-mysql-storage
secret:

secretName: jeecg-service-extsecret


2.4.2 POLICY GOVERNANCE

You will learn how to set up your OPA Gatekeeper in this section.

ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

OPA stands for Open Policy Agent. It is a unified toolset and

framework, policy-based control for cloud native environments,
embracing a policy-as-code method to decouple your policy from the
service's code. So, you can release, analyze, and review your policies
without sacrificing the availability or performance of your security
and compliance teams.

Gatekeeper is an extensible, parameterized policy library, a policy

engine for Cloud Native environments hosted by the Cloud Native
Computing Foundation (CNCF) and Kubernetes. Gatekeeper helps
you manage your OPA policies.

ACK integrated gatekeeper and OPA, providing an easy to install

solution, OPA Gatekeeper. OPA Gatekeeper provide various kind of
policy template by default, policy governance status overview, logging 
and searching to satisfaction business scenario, help security team
easy operation and governance cloud native environment.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 104




1. Click on Security, then Policy Governance.

Note: this feature is only supported on a Linux node.

2. ACK provides 35 policies by default, making it easy to enable

a special policy.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 105

3. For example, when you enable a policy item like

“ACKBlockLoadBalancer”, the Load Balancer Service cannot be 
deployed in specified namespaces. If you try to deploy the Load
Balancer Service, you receive an error.



2.4.3 ZERO TRUST SECURITY

You will now learn how to create an ASM mTLS Authorization Policy.

ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Using Alibaba Cloud Service Mesh (ASM), you can implement zero 
trust security controls using a range of methods. You will learn three
different ways to achieve this.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 106

Region A Region B

VPC VPC

ACK Cluster (Managed) ACK Cluster (Serverless) ACK Cluster (Managed)

CEN


Namespace A Namespace A Namespace A
Application Application Application Application Application

Routing External
Authorization Policy Request-Authentication Application Authorization
Namespace Path JWT Token


IPBlocks Hosts Username Role
… UserGroup …

Application Application

Namespace B

URL List
Guest /Product
Peer OPA
Authentication TLS Policy /Management
Admin

VPC Authorization
Policy

Namespace A
Application Application

Authorization
Policy
RDS

2.4.3.1 BLOCKING COMMUNICATION BETWEEN NAMESPACES

In this section, you will learn how to block communication between
two specific namespaces.
VPC

ACK Cluster

Node Pool A
Authorization policy
Namespace A Istio mTLS: Namespace B
source namespace
Test App Proxy Proxy Demo App

ASM (Service Mesh)

1. Create an ASM and then add an ACK Cluster to that ASM.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 107

2. Create a Namespace in ASM. The best practice is to name the

namespace with the same name as the ACK cluster. 
3. Click Synchronous Automatic Sidecar Injection, and the
namespace of ACK will auto inject the sidecar. 


4. Create the Authorization Policy in ASM, setting the target

namespace, action and source namespace.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 108

5. The yaml file should now look like this:






6. When you curl a server from “jeecg-frontend” namespace to

“jeecg-server” namespace, the result is ”RBAC: access
desnied403”.

2.4.3.2 FORBID A NAMESPACE ACCESS TO A WEBSITE

In this section, you will learn how to block communication from a
namespace to specific website.
VPC

ACK Cluster

Node Pool A

Namespace Authorization policy Egress gateway

Istio mTLS: 403
curl source namespace Forbidden
Test App Proxy Proxy www.aliyun.com
http://www.aliyun.com

ASM (Service Mesh)

1. Set the external access policy to REGISTRY_ONLY in ASM.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 109

2. Create ServiceEntry in the istio-systerm namespace in ASM.


apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: aliyuncom-ext 
namespace: istio-system
spec:
exportTo:
- istio-system

hosts:
- www.aliyun.com
location: MESH_EXTERNAL
ports:
- name: http
number: 80
protocol: HTTP
- name: tls
number: 443
protocol: TLS
resolution: DNS

3. If the application communicates with MySQL or MongoDB

or MQ etc, you should add ServiceEntry in ASM with the
corresponding protocol and port number, otherwise it cannot
connect. Below is a connect RDS(mysql) yaml example.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: jeecg-server-rds
namespace: istio-system
spec:
hosts:
- rm-gs59q6b8xg7w3kr1j.mysql.singapore.rds.aliyuncs.com
location: MESH_EXTERNAL
ports:
- name: tcp
number: 3306
protocol: TCP
resolution: DNS

4. Create an EgressGateway in ASM.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 110

5. Create ASM Gateway in the istio-system namespace, the

gateway pod selector value is egressgateway. Also enable TLS 
mode is Istio Mutual.




6. Create DestinationRule in ASM, where the host is the

egressgateway address.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: target-egress-gateway
namespace: jeecg-frontend
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: target-egress-gateway-mTLS
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
tls:
mode: ISTIO_MUTUAL

Note, the subsets name will be used in virtualservice subset.

7. Create VirtualService in ASM.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: example-com-through-egress-gateway
namespace: jeecg-frontend
spec:
exportTo: 
- istio-system
- jeecg-frontend
gateways:
- mesh 


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 111

- istio-system/istio-egressgateway
hosts:
- www.aliyun.com

http:
- match:
- gateways:
- mesh

port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.

local
port:
number: 80
subset: target-egress-gateway-mTLS
weight: 100
- match:
- gateways:
- istio-system/istio-egressgateway
port: 80
route:
- destination:
host: www.aliyun.com
port:
number: 80
weight: 100

8. Configure Global mTLS mode in ASM or create mTLS mode in

a namespace. The mTLS mode you require is STRICT.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 112

9. Create an Authorization Policy in the istio-system namespace in

ASM, setting a request source namespace. This cannot be done 
through egress.




10. To verify the result, go to a pod in ACK, click Terminal, select

the Container.

11. If you see the following, this is a forbidden result:

12. If you see the following, this is a without authorization

policy result:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 113

2.4.3.3 FORBID A NAMESCAPE ACCESS TO RDS

You will now learn how to block connections between a specific RDS 
and your VPC.
VPC

ACK Cluster

Node Pool A
Lost connection to


Namespace Authorization policy Egress gateway MySQL server at 'reading
Istio mTLS: authorization packet'
mysql –h hos t –u root source namespace
Test App Proxy Proxy
–p password
RDS

ASM (Service Mesh)

1. Check your external access policy is set to REGISTRY_ONLY

in ASM.

2. Create ServiceEntry in istio-systerm namespace in ASM.

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: jeecg-server-rds
namespace: jeecg-server
spec:
endpoints:
- address: rm-gs59q6b8xg7w3kr1jlo.mysql.singapore.rds.
aliyuncs.com
ports:
tcp: 3306
hosts:
- rrm-gs59q6b8xg7w3kr1jlo.mysql.singapore.rds.aliyuncs.com
location: MESH_EXTERNAL
ports:
- name: tcp
number: 3306
protocol: TCP
resolution: DNS


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 114

3. Create an EgressGateway in ASM, adding it to the 13306

service port. 




4. Create an ASM Gateway in the istio-system namespace, the

gateway pod selector value is egressgateway, enable TLS
mode is Istio Mutual.

5. Create a DestinationRule in the istio-systerm namespace in

ASM, where:
» Host is egressgateway address.
» The sni is rds connection URL. 
» The subsets name will be used in VirtualService subset.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 115

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:

name: jeecg-server-egress-gateway
namespace: istio-system
spec:
host: istio-egressgateway.istio-system.svc.cluster.local

subsets:
- name: mysql-gateway-mTLS
trafficPolicy:
loadBalancer:

simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 13306
tls:
mode: ISTIO_MUTUAL
sni: rm-gs59q6b8xg7w3kr1jlo.mysql.singapore.rds.
aliyuncs.com

6. Create VirtualService in ASM.

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: jeecg-server-through-egress-gateway
namespace: jeecg-server
spec:
exportTo:
- istio-system
- jeecg-server
gateways:
- mesh
- istio-system/istio-egressgateway
hosts:
- rm-gs59q6b8xg7w3kr1jlo.mysql.singapore.rds.aliyuncs.com
tcp:
- match:
- gateways:
- mesh
port: 3306
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.
local
port:
number: 13306
subset: mysql-gateway-mTLS
weight: 100
- match:
- gateways:
- istio-system/istio-egressgateway
port: 13306
route:
- destination:
host: rm-gs59q6b8xg7w3kr1jlo.mysql.singapore.rds.
aliyuncs.com
port:
number: 3306
weight: 100


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 116

7. Configure Global mTLS mode in the ASM or create mTLS mode

in a namespace, where the mTLS mode is STRICT. 




8. Create an Authorization Policy in the istio-system namespace in

ASM, you cannot set a request source namespace
through egress.

9. Verify the result, login the container which in jeecg-service

namespace, and run mysql connection command, you will see
“Lost Connection to MySQL**”

10. The Without authorization policy will result.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 117

2.4.4 RAM ROLES FOR SERVICE ACCOUNTS (RRSA)


You can use the RAM Roles for Service Accounts (RRSA) feature
to enforce access control on pods that are deployed in a Container
Service for Kubernetes (ACK) cluster. This feature allows you to 
authorize pods to call the APIs of different cloud resources. Only
ACK clusters that run Kubernetes 1.22 or later versions support
RRSA. The ACK clusters include ACK standard clusters, ACK Pro

clusters, Serverless Kubernetes (ASK) standard clusters, and ASK
Pro clusters.

The following two scenarios illustrate how to use RRSA to specify

that a user only has the rights to a certain KMS Secret. This avoids
the risk of exposing all KMS Secrets and achieves fine-grained
permission control.

1. Scenario 1: Specify the Secret of a KMS you need to get through

the RAM Role. The RAM Role is specified by ServiceAccount, so
the Secret you get from the Pod is accessible only to the users
defined in the RAM role.
2. Scenario 2: Specify the Secret of a KMS you need to get through
the RAM Role. You can specify multiple RAM Roles, and each
Pod can use OIDC Token to call KMS OpenAPI to get the Secret
which is accessible only to the users defined in the RAM roles. In
this scenario, you can configure a different Secret for each pod.
RRSA of ACK Ack-secret-manager

ServiceAccount RRSA
(ack-secret-manager)

Namespace

ServiceAccount RAM Role Secret Secret Secret

ACK KMS
Scenario 1 (ack-secret-manager)
GetSecretValue
VolumeMounts
Pod Secret1
Secret Secret Secret1

Use the OIDC Token bound

to the Pod to get

Pod Pod STS Token1 RAM Role1

ACK
Scenario 2 Use the OIDC Token bound
to the Pod to get
Docker Image Docker Image
STS Token2 RAM Role2

Get secret1 Get secret1

KMS OpenAPI KMS OpenAPI


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 118

2.4.5 SECURITY CENTER


In this lab, we will explore the Security Center are of the Console.

2.4.5.1 ASSETS 
1. The Assets page in the Security Center console displays the
statistics and security status of protected assets.


2. Click on Synchronize Asset.

3. The feature of asset exposure analysis automatically analyzes

the exposures of your Elastic Compute Service (ECS) instances
on the Internet and visualizes the communication links between
your ECS instances and the Internet.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 119

2.4.5.2 CONTAINER SECURITY

4. Security Center provides the container firewall feature. The

feature delivers firewall capabilities to protect containers(pods).
If attackers exploit vulnerabilities or malicious images to intrude 
into clusters, the container firewall feature generates alerts or
blocks attacks.
Only Security Center Ultimate supports this feature. If you do

not use the Ultimate edition, you must upgrade Security Center
to the Ultimate edition before you can use this feature. Click
here to upgrade to the Ultimate edition.

5. Add a source network object.

AppName’s value is tag value of deployment in ACK.

6. Add a destination network object.

AppName’s value is tag value of deployment in ACK.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 120

7. You can now see it displayed here.



8. Add the network object names. 

9. Open the Defensive status. After you create a defense rule for a
cluster and enable the defense rule, the defense rule allows,
blocks, or generates alerts for the traffic destined for the cluster.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 121

10. Initiate a request from the source network object to the

destination network object. 




11. You will see an alert prompt.

2.4.5.3 PROACTIVE DEFENSE FOR CONTAINERS

12. The feature of proactive defense for containers provides
common security rules and allows you to create custom security
rules. You can follow the instructions to create a custom defense
rule. Only Security Center Ultimate supports this feature. If you
do not use the Ultimate edition, you must upgrade Security
Center to the Ultimate edition before you can use this feature.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 122

13. Install gatekeeper,policy-template-controller,and logtail-ds

in ACK. 




14. Create policy of Defend against unscanned images.

15. You have successfully created Defend against

unscanned images.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 123

16. A trigger alert appears when pulled the against

unscanned images. 

2.4.5.4 ALERTS 
17. The Security Center generates different types of alerts for your
assets in real time. The types of alerts include the alerts for web
tampering, suspicious processes,Container Runtime Detection,
webshells, unusual logons, and malicious processes.
The web tamper proofing feature and the cloud threat detection
feature must upgrade Security Center to the Ultimate.

2.4.5.5 ATTACK AWARENESS

18. Security Center supports the Attack Awareness feature. The
feature lists and analyzes the attacks against your assets.
Only Security Center Ultimate supports this feature. If you do
not use the Ultimate edition, you must upgrade Security Center
to the Ultimate edition before you can use this feature.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 124

2.4.5.6 CLOUD HONEYPOT

19. Next, you will learn how to Create Cloud Honeypot. The Cloud

Honeypot function of the Security Center can provide you with
attack detection and attack countermeasures inside and outside 
the cloud. You can create a cloud honeypot instance on ana
Alibaba Cloud VPC or a server instance connected to the
Security Center to defend against real attacks on your server 
inside and outside the cloud and strengthen your server's
security protection.

20. Create a Management Node.

21. Next, Create a Honeypot.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 125

22. The changes appear in Config Management.




23. Create a Host Probe.

24. This is what a normal deployment looks like.

25. When the SSH service is attacked, it will be transferred to the

honeypot, and then an alarm will be triggered. From the alarm,
you can see the attacker's source address and
other information.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 126

26. You can view this in the Event Log.






2.4.5.7 LOGSTORE
27. Full logs of Security Center are stored in a dedicated Logstore.
You can find the Logstore in the project that stores Security
Center logs in the Log Service console. The name of the project
is in the sas-log-ID of your Alibaba Cloud account-Region
ID format.

2.4.5.8 SECURITY REPORT

28. Security Center provides the security report feature. You can
create security reports and specify the email addresses to
which security reports are sent on a regular basis. This way,
you can monitor the security status of your assets at the
earliest opportunity.
Only Security Center Ultimate supports this feature. If you do
not use the Ultimate edition, you must upgrade Security Center
to the Ultimate edition before you can use this feature.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 127




2.4.5.9 PLAYBOOK
29. Security Center provides automatic orchestration and response
capabilities on the Playbook page. This allows you to orchestrate
the logic of repetitive tasks in the response to security events
into automatic processing policies and helps you reinforce the
security of your system.
You can create only automatic vulnerability fixing tasks on the
Playbook page.

Only Security Center Ultimate supports this feature. If you do

not use the Ultimate edition, you must upgrade Security Center
to the Ultimate edition before you can use this feature.

30. Set up your scan tasks.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 128




31. Automatic vulnerability fixing tasks completed.

2.5 AIOPS
ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Scale
up/down
Auto Scaling Region (HK) Region (SGP) Other External
KMS External Cloud Vender DNS
(Secrets Manager) Secret D N S DNS Private Zone
Secret-Manager ACK ACK
Policy
Cluster Backup Scaling Cross-region Cluster Governance
OSS OPA Gatekeepper
SSL Management K8S
Nginx-Ingress
Cost Analysis Multi-Cloud Cluster
HTTPS
Cert-Manager Cluster Diagnosis Management
Zero Trust
Istio Ingress Gateway … ASM mTLS
HTTPS Service Mesh Authorization Policy
ASM
Traffic shifting FinOps
GTM
Multi-cluster
(Global Traffic Observability
Distribution & Failover AIOps
Manager) Cross-region (support multi-cloud)
Traffic Manager End-to-end A/B Testing
Gateway
…

Cost Analysis Cluster Security Container
Log Service ARMS
Cluster
Inspections Intelligence Service
Namespace Inspection Cluster Upgrade Logging Prometheus Monitoring
Node pool Solution to solve Check Tracing Cluster Monitoring
hystax
Application security issues
Inspection Report
Node/Pod/Network
diagnosis
Ingress Access Center
Events & Alerting
Cluster Topology
Application Monitoring

Multi-Cloud Bill Control … … … …


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 129

Our Container Service provides a range of AIOps functionality, helping

you monitor and optimize the security of your applications and 
diagnose any service issues.

Some 20% of common problems in cloud environments happen 

in high frequency, such as Resource Quotas reaching their upper
limit, or Resource Pressure reaching the upper limit, causing errors
when creating services. For these problems, you can use the regular

inspection function that can resolve 70%-80% of them.

The other 80% of the problems happen with low frequency, they
are closely related to the conditions and status of your cloud
environments, networks, and application, etc. For these problems,
we provide diagnostic tools to locate errors in nodes, pods,
and networks.

In addition, the Managed Node Pool will repair problematic nodes

automatically. You can also use its FAQ documents for quick
troubleshooting.
Container Intelligence Service
Regular Inspection
Cluster Inspection enter
Abnormal Event Alerting
from ACK cluster list menu Abnormality
Found Resource Pressure
20% high-frequency
issues addressed by
regular inspection Resource Quotas

K8s Versions and Certification

Cluster Risk

CIS
(Container
Intelligence
Service)
Knowledge Base(FAQ) Abnormality Abnormality Diagnosis
Node Pool Self-Healing Fix Diagnosis
Typical issues fixed 80% uncommon
with Node Self-Healing issues addressed
CVE Vulnerability and FAQ by diagnosis
Node Diagnosis

Docker Pod Diagnosis

OS Network Diagnosis

Systemd

The implementation of this lab includes the following high-level steps:

Step Description

1 Cluster Security How to use the Reports area to inspect the security of your
Inspection clusters and use automated tools to monitor cluster health.

2 Cluster Regular How to view the detailed information of the diagnostic results
Inspection including the diagnostic items and troubleshooting advice for
common errors to locate and resolve the issues quickly.

3 Node/Pod/Network How to manage the alerts from the Log Service.

Diagnosis

4 Node Pool Self- How to use the self-healing functionality and see the relevant
Healing events in the Event Center of ACK.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 130

2.5.1 CLUSTER SECURITY INSPECTIONS


You will learn how to use the out-of-the-box security features to
identify and address any security flaws and compliance issues.

1. Click the Inspections section to detect security risks in the


configuration of your workloads.


2. Click the Reports tab of the Inspections area. This will provide
you with an inspection report from the ACK console, which
provides a visualization of the health of your Kubernetes
Polaris instances.

3. Here, you get a suggestions list in the Console of issues to fix.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 131

4. Click Detail to understand how to solve this issue.






2.5.2 CLUSTER REGULAR INSPECTION

ACK provides basic diagnostic capabilities for clusters based on the

best practices of Kubernetes and our experience. In this section, you
learn how to view the detailed information of diagnosis result
including the diagnostic items and troubleshooting advice for
common errors to locate and resolve the issues quickly.

1. Click on Cluster Inspection.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 132

2. Create inspection reports based on your requirements and set

the maximum memory resources that can be used to run 
containers. This helps you to prevent malicious processes in
your containers.



3. ACK provides resource pressure and quota inspection capability

to examine issues such as insufficient quota on VPC route
entries, or excessive SLB connections. You can resolve these
issues without looking up the threshold values. Click on the
Resource Quotas and Resource Level tabs to see the
report details.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 133

4. Click the Versions and Certificates and Cluster Risk tabs to see
further information. 




Log Service (SLS) allows you to set alerts to monitor the operations
that are performed on specific resources in real time. Alerts
can be sent via SMS messages, DingTalk chatbots, emails,
custom webhooks, and the Message Center of the Alibaba Cloud
Management Console. In this section, you learn how to manage
your alerts.

5. In the event of an abnormal event, an alert is given. Click on

View More Log Applications in the Log Service area.

6. Now, click on K8s Event Center.

7. Click on the Alert Configuration under

the cluster name in the left menu to see
all events.



Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 134




2.5.3 MODE/POD/NETWORK DIAGNOSIS

ACK can also perform diagnosis on nodes, pods, networks,

kubeDNS, etc., with troubleshooting advice. You can specify a node
or pod for small-scale diagnosis to minimize O&M workload.

1. Click on Diagnosis in the Cluster Check dropdown to view the

status of your clusters.

2. You can choose to see the node diagnosis result.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 135

3. Or you can click on the Network Diagnosis result.






4. Or you can click on the Pod Diagnosis result.

2.5.4 NODE POOL SELF-HEALING

1. The node pool self-healing is automatically triggered if you have

a managed pool node and the Node status is “NotReady was
during more than 180 seconds” or “DockerHung was during
90 seconds”

2. Check the nodes again, the status is Ready.


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 136

3. After Self-Healing, you could see relevant repair event

in EventCenter. 




There is more information available on this online.

2.6 FINOPS
FinOps is a cloud-based financial management discipline. It allows
organizations to analyze costs and provides valuable insights so
their multidisciplinary teams can collaborate on data-driven
spending decisions.
ACR EE
Application Center
Image Scan
Image Signature
Image Replication Trigger
KMS Image trigger Deployment
synchronization
(Certification) …

New version
Security Center
(Witness/Security
Policy) Image signature

Cost Analysis Cluster Security Container

Hystax is a third-party provider that builds FinOps and cloud cost

management software for Kubernetes and Alibaba Cloud workloads.

The implementation of this lab includes the following high-level steps:

Step Description 
1 Enable Cost Analysis How to enable cost analyses of your Kubernetes containers.

2 Introducing the Cost Learn some of the key features of the Cost 
Analysis Dashboard analysis dashboard.

3 Introducing OptScale Learn some of the key features of the OptScale dashboard.

Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 137

2.6.1 ENABLE COST ANALYSIS


You will learn how to enable cost analyses of your
Kubernetes containers.

1. Go to Cost Analysis in the ACK console and upgrade your


components, if required.


2. Add the below policy to the Worker RAM Role in the

Cluster Resources section.

{
"Action": [
"bssapi:QueryInstanceBill"
],
"Resource":"*",
"Effect":"Allow"
– },
{
"Action":[
"ecs:DescribeSpotPriceHistory",
"ecs:DescribeInstances",
"ecs:DescribePrice"
],
"Resource":"*",
"Effect":"Allow"
}


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 138

2.6.2 INTRODUCING THE COST ANALYSIS DASHBOARD


To make an effective and accurate cost analysis, there are many
dimensions to consider.
Daily/Weekly/Monthly cost CPU/Memory Request Utilization

Cost proportion by namespace Cost by a specific namespace
Cost Trending CPU/Memory Request & Usage Trending
Request & Usage Utilization Proportion
help you to do Cost Efficiency
Cost Trending
Top Resource Request & Usage ordered

by Pods

Cluster Namespace

Application NodePool

Cost by application label

Proportion of Node Charge Type
Pod Amount
(PAYG/Subscription/Spot)
Utilization
Cost Trending
Detailed billing by per pod

The Cost Analysis dashboard visualizes this wealth of information,

presenting it in an intuitive manner to help you understand and gain
insights into your Container Service.

You will now learn some of the key features of the Cost Analysis
dashboard.

1. You can analyze cost efficiencies:

2. Compare the cost of individuals namespaces:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 139

3. View individual bills by date, product type, payment type, number

of instances and the amount of tax paid: 



4. Analyze CPU, memory and costs of individual namespaces
and applications:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 140

2.6.3 INTRODUCING OPTSCALE


For multi-cloud bill control, the OptScale dashboard is a useful tool. It
allows you to manage hybrid clouds along with the virtual machines,
volumes and network settings. Custom and mandatory tagging 
is supported. Custom TTL rules can also be created to clear out
workloads according to your policies.

You will now learn some of the key features of the OptScale
dashboard.

1. Analyze and manage costs of your resources:

2. Identify where money is being spent using the Cost

Explorer option:

3. Identify costs by geographical region using the Cost Map

visualization tab:


Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 141

4. Create a FinOps checklist to monitor your costs across

specific timeframes: 





Alibaba Cloud | Best Practices for End-to-End Cloud-Native Application Management on Alibaba Cloud 142


3. IN CONCLUSION 

The Alibaba Cloud Container Platform is a leading

Kubernetes-based service that ensures high efficiency for
enterprises by running containerized applications on the cloud.

The Alibaba Cloud Container Service for Kubernetes (ACK)

provides end-to-end application management for today’s
enterprises, integrating Alibaba Cloud’s extensive virtualization,
storage, networking, and security capabilities. ACK allows you to
deploy applications in high-performance and scalable containers
while providing full lifecycle management of your enterprise-class,
containerized applications.

In this lab, you have learnt some of the best practices to help
you make the most of ACK, identifying the key elements across
your CloudOps, DevOps/DevSecOps, SecOps, AIOps and
FinOps strategies.


ABOUT
Established in 2009, Alibaba Cloud is a global leader in cloud computing
and artificial intelligence (AI). It is the technological backbone of the
Alibaba Group, which aims to make it easy to do business anywhere.
As one of the significant business units of the group, Alibaba Cloud
provides the best-in-class cloud computing infrastructure and cloud
services for global enterprises across industries to operate more
efficiently, effectively, and securely. Alibaba Cloud also supports
Alibaba Group’s groundbreaking Double 11 Global Shopping Festival,
which generated USD $74.1 billion (RMB ¥498.2 billion) in gross
merchandise value (GMV) in 2020 with zero downtime.

Headquartered in Hangzhou, China, the Alibaba Group provides

vital technology infrastructure and marketing capabilities to help
businesses grow their products and services online. The Alibaba
Group encompasses commerce, cloud computing, digital media,
and innovation. In addition to Alibaba Cloud – the cloud computing
division of the Group – other business units include AliExpress, the
global consumer marketplace, and Alipay, the mobile and online
payment platform.

In 2017, Alibaba established the DAMO Academy (Academy for

Discovery, Adventure, Momentum and Outlook), dedicated to scientific
and technological research and innovation. Over the last three years,
Alibaba has invested billions of dollars in research and development,
and has built research labs in seven cities worldwide, looking into
topics including data intelligence, FinTech, quantum computing, IoT,
and human-machine interaction.

www.alibabacloud.com/contact-sales

Dynatrace Associate VILT Day 1
No ratings yet
Dynatrace Associate VILT Day 1
122 pages
DevOps Cheat Sheet
No ratings yet
DevOps Cheat Sheet
297 pages
Static Timing Analysis
No ratings yet
Static Timing Analysis
40 pages
Acronis #CyberFit Cloud Tech Foundation C21.03
No ratings yet
Acronis #CyberFit Cloud Tech Foundation C21.03
252 pages
Storage Ceph 5 Documentation IBM
No ratings yet
Storage Ceph 5 Documentation IBM
1,252 pages
Timur Akhmadeev - Patching With Ansible - ConTech 2022
No ratings yet
Timur Akhmadeev - Patching With Ansible - ConTech 2022
150 pages
Aws Networking Interview Questions
No ratings yet
Aws Networking Interview Questions
10 pages
Aws Dev Ops Shells Rip T Interview Questions
No ratings yet
Aws Dev Ops Shells Rip T Interview Questions
6 pages
Openshift - Container - Platform 4.10 Monitoring en Us
No ratings yet
Openshift - Container - Platform 4.10 Monitoring en Us
107 pages
Aws SDK Java DG
No ratings yet
Aws SDK Java DG
197 pages
Microsoft Remote Desktop Services Deployment Guide
No ratings yet
Microsoft Remote Desktop Services Deployment Guide
62 pages
Openstack Horizon
No ratings yet
Openstack Horizon
89 pages
Wireguard PI VPN Tested (06.07.2023)
No ratings yet
Wireguard PI VPN Tested (06.07.2023)
16 pages
AWS+Partner+ +Containers+on+AWS+ (Technical) + +v1.0.1 Compressed
No ratings yet
AWS+Partner+ +Containers+on+AWS+ (Technical) + +v1.0.1 Compressed
170 pages
Open VPN
No ratings yet
Open VPN
92 pages
Docker: Rui Couto António Nestor Ribeiro
No ratings yet
Docker: Rui Couto António Nestor Ribeiro
20 pages
Thesis - Online - Shoppping Portal - SRM - GSK
No ratings yet
Thesis - Online - Shoppping Portal - SRM - GSK
69 pages
Red Hat Enterprise Linux-8-Building Running and Managing containers-en-US
No ratings yet
Red Hat Enterprise Linux-8-Building Running and Managing containers-en-US
127 pages
Imperva - 15 Ways Your Website Is Under Attack
No ratings yet
Imperva - 15 Ways Your Website Is Under Attack
12 pages
4-Goolge Cloud Platform (GCP) Compute, Storage and Network Services-20-08-2024
No ratings yet
4-Goolge Cloud Platform (GCP) Compute, Storage and Network Services-20-08-2024
71 pages
MBU - Red Hat Ansible Automation Platform Technical Deck
100% (1)
MBU - Red Hat Ansible Automation Platform Technical Deck
95 pages
Nutanix Licensing Quick Reference Guide2
No ratings yet
Nutanix Licensing Quick Reference Guide2
5 pages
C Panel
No ratings yet
C Panel
3 pages
Ex 200
No ratings yet
Ex 200
5 pages
Alwasail Electrofusion Catalogue
No ratings yet
Alwasail Electrofusion Catalogue
24 pages
Ansible Cheatsheet PDF
0% (1)
Ansible Cheatsheet PDF
291 pages
OpenVPN 2.1
No ratings yet
OpenVPN 2.1
68 pages
Red Hat Openstack Platform-16.1-Networking Guide-En-Us
No ratings yet
Red Hat Openstack Platform-16.1-Networking Guide-En-Us
208 pages
RH066x EdX Lab Instructions-RHEL8
No ratings yet
RH066x EdX Lab Instructions-RHEL8
13 pages
Sudo Apt-Get Install Docker-Ce 17.12.0 cd-0 Ubuntu
No ratings yet
Sudo Apt-Get Install Docker-Ce 17.12.0 cd-0 Ubuntu
5 pages
Get Certified-04 2024.03.19 Cert Journey Kickoff
No ratings yet
Get Certified-04 2024.03.19 Cert Journey Kickoff
7 pages
Amazon - Transcender.aws Solution Architect Associate - Pdf.exam.2020 Nov 18.by - Emmanuel.146q.vce
No ratings yet
Amazon - Transcender.aws Solution Architect Associate - Pdf.exam.2020 Nov 18.by - Emmanuel.146q.vce
29 pages
Podman Part2
No ratings yet
Podman Part2
5 pages
Visualisation Grafana Most Important 20
No ratings yet
Visualisation Grafana Most Important 20
7 pages
Sensu + Graphite + Grafana Install
No ratings yet
Sensu + Graphite + Grafana Install
16 pages
DDWRT WireGuard Client Setup Guide v14
No ratings yet
DDWRT WireGuard Client Setup Guide v14
10 pages
Solace Essentials Activity Guide
No ratings yet
Solace Essentials Activity Guide
38 pages
Install Guide
No ratings yet
Install Guide
155 pages
Microsoft Azure DevOps Engineer Certification Training
No ratings yet
Microsoft Azure DevOps Engineer Certification Training
12 pages
Session 14 Alerting
No ratings yet
Session 14 Alerting
24 pages
Build Share: Docker Cheat Sheet
No ratings yet
Build Share: Docker Cheat Sheet
1 page
Adobe Creative Cloud Collection (2014)
No ratings yet
Adobe Creative Cloud Collection (2014)
3 pages
BS100 Installation Manual
No ratings yet
BS100 Installation Manual
33 pages
CICD For Mobile
No ratings yet
CICD For Mobile
3 pages
Docker Basic Commands
No ratings yet
Docker Basic Commands
6 pages
Build Public/private Cloud Using Openstack: by Framgia Server Lab
No ratings yet
Build Public/private Cloud Using Openstack: by Framgia Server Lab
32 pages
Ma Ansible Automation Platform 2 Ebook f30107 202109 en 1
No ratings yet
Ma Ansible Automation Platform 2 Ebook f30107 202109 en 1
14 pages
OpenStack Cheat Sheet 1
No ratings yet
OpenStack Cheat Sheet 1
3 pages
Speed Control of 3 Phase Induction Motor
No ratings yet
Speed Control of 3 Phase Induction Motor
10 pages
Lab 9.2.7 Troubleshooting Using Network Utilities
No ratings yet
Lab 9.2.7 Troubleshooting Using Network Utilities
5 pages
Devops: Detailed Course Syllabus
No ratings yet
Devops: Detailed Course Syllabus
4 pages
Getting Started With ADCC For PIC18 PDF
No ratings yet
Getting Started With ADCC For PIC18 PDF
23 pages
Extending SaltStack - Sample Chapter
No ratings yet
Extending SaltStack - Sample Chapter
15 pages
IMPV Path To Your Data Infographic
No ratings yet
IMPV Path To Your Data Infographic
1 page
Health Facility Registry
No ratings yet
Health Facility Registry
15 pages
Bind9Serverhowto: Search
No ratings yet
Bind9Serverhowto: Search
10 pages
Ace Multicloud Network Professional Comprehensive Guide
No ratings yet
Ace Multicloud Network Professional Comprehensive Guide
2 pages
Online Matatu Management Project
No ratings yet
Online Matatu Management Project
44 pages
OSP8 Roadmap
No ratings yet
OSP8 Roadmap
84 pages
Oca - Openstack PDF
No ratings yet
Oca - Openstack PDF
5 pages
Grammar: Answer Key
No ratings yet
Grammar: Answer Key
2 pages
07-Design and Simulation of Traffic Engineering Using MPLS in GNS3 Environment.2018
No ratings yet
07-Design and Simulation of Traffic Engineering Using MPLS in GNS3 Environment.2018
5 pages
LinuxCBT Firewall Notes
No ratings yet
LinuxCBT Firewall Notes
6 pages
Linux Learning Centre
No ratings yet
Linux Learning Centre
1 page
LinuxCBT Squid Notes
No ratings yet
LinuxCBT Squid Notes
3 pages
VC15B+/VC17B+ Digital Multimeter User Manual: NOT Exceed The "Maximum Value" Indicated in The Specification
No ratings yet
VC15B+/VC17B+ Digital Multimeter User Manual: NOT Exceed The "Maximum Value" Indicated in The Specification
2 pages
? 90-Days Data Analyst Learning & Project Plan For Priyanka
No ratings yet
? 90-Days Data Analyst Learning & Project Plan For Priyanka
2 pages
Grammaticalization vs. Reanalysis, A SemanticPragmatic Account of Functional Change in Grammar
No ratings yet
Grammaticalization vs. Reanalysis, A SemanticPragmatic Account of Functional Change in Grammar
45 pages
Aoop
No ratings yet
Aoop
13 pages
Tema 5. Coac
No ratings yet
Tema 5. Coac
12 pages
Kali Linux 2 Windows Penetration Testing Paperba
No ratings yet
Kali Linux 2 Windows Penetration Testing Paperba
1 page
Proxmox Pricing
No ratings yet
Proxmox Pricing
2 pages
DS8500 Kit
No ratings yet
DS8500 Kit
29 pages
04 Linear Model
No ratings yet
04 Linear Model
25 pages
MODULE 3 Brute Force Attack Through John The Ripper PDF
No ratings yet
MODULE 3 Brute Force Attack Through John The Ripper PDF
9 pages
Note Minimalism
No ratings yet
Note Minimalism
6 pages
Prediction of Construction Cost Overrun in Tamil Nadu-A Statistical Fuzzy Approach
No ratings yet
Prediction of Construction Cost Overrun in Tamil Nadu-A Statistical Fuzzy Approach
9 pages
Bind Shell With Netcat
No ratings yet
Bind Shell With Netcat
5 pages
01-43 DTC Memory PDF
No ratings yet
01-43 DTC Memory PDF
3 pages
LH 450 Ctte With Options
No ratings yet
LH 450 Ctte With Options
4 pages
IEEE Conference-template-A4
No ratings yet
IEEE Conference-template-A4
5 pages
Course Objectives:: University of Mumbai, Information Technology (Semester V and VI) (Rev-2012)
No ratings yet
Course Objectives:: University of Mumbai, Information Technology (Semester V and VI) (Rev-2012)
5 pages
Niagara Edge 10 Data Sheet - Tridium
No ratings yet
Niagara Edge 10 Data Sheet - Tridium
2 pages
Amway Preferred Customer Application: Personal Details
No ratings yet
Amway Preferred Customer Application: Personal Details
2 pages
ER Diagram Rider Reward System
No ratings yet
ER Diagram Rider Reward System
1 page
Ultimate Microsoft Intune for Administrators: Master Enterprise Endpoint Security and Manage Devices, Apps, and Cloud Security with Expert Microsoft Intune Strategies (English Edition)
From Everand
Ultimate Microsoft Intune for Administrators: Master Enterprise Endpoint Security and Manage Devices, Apps, and Cloud Security with Expert Microsoft Intune Strategies (English Edition)
Paul Winstanley
No ratings yet
Unix / Linux FAQ: with Tips to Face Interviews
From Everand
Unix / Linux FAQ: with Tips to Face Interviews
Prof. N.B. Venkateswarlu
No ratings yet
How to a Developers Guide to 4k: Developer edition, #3
From Everand
How to a Developers Guide to 4k: Developer edition, #3
Xinc Cyberwizard
No ratings yet
Cloud: Get All The Support And Guidance You Need To Be A Success At Using The CLOUD
From Everand
Cloud: Get All The Support And Guidance You Need To Be A Success At Using The CLOUD
John Hawkins
No ratings yet
Configuration of Microsoft ISA Proxy Server and Linux Squid Proxy Server
From Everand
Configuration of Microsoft ISA Proxy Server and Linux Squid Proxy Server
Dr. Hidaia Mahmood Alassouli
No ratings yet

Alibaba Cloud Whitepaper - Ack k8s

Uploaded by

Alibaba Cloud Whitepaper - Ack k8s

Uploaded by

BEST PRACTICES

Alibaba Cloud reminds you to carefully read through and

1. You shall download this document from the official website of

2. Any sector, company, or individual shall not extract, translate,

3. This document may be subject to change without notice

4. This document is only intended for product and service

5. All content including, but not limited to, images, architecture

6. If you discover any errors or mistakes within this document,

1.1 About this Solution 1

2.1 Cloud-Native Application Maturity Model 5

1.1 ABOUT THIS SOLUTION

Alibaba Cloud Container Service for Kubernetes (ACK) allows you to

This lab covers the following architecture features:

Cost Analysis Cluster Security Container

» CloudOps(including multi-cloud deployments)

SecOps Auto Scaling CloudOps

1.2 ABOUT THIS LAB

» Elastic Compute Service (ECS)

2.1 CLOUD-NATIVE APPLICATION

• Defined SLAs around policies and

The Container Service provides optimized OS (operating system)

This simplifies your Kubernetes development and monitoring

You can use Terraform to automatic provision infrastructure.

Alibaba Cloud recently released a Terraform Module Web GUI to

The implementation of this lab includes the following high-level steps:

1 Cluster Scaling How to scale a Kubernetes cluster up (or down) on

4 External DNS How to use External-DNS component to automatic add

5 Enabling Observability How to connect your LogService and ARMS monitoring

7 Disaster Recovery How to implement disaster recovery and backup best

8 Multi-Cloud How to manage and maintain a multi-cloud infrastructure

9 Cross Region How to set up a cross-region network and connect

2.2.1 CLUSTER SCALING

Cost Analysis Cluster Security Container

Node Pool Resource Metrics Cloud Metrics Customer Metrics

Add nodes CPU / SLB / LogService /

For most of cases, a combination of cluster and HPA autoscaling will

1. Enable cluster auto scaling for a specific cluster by selecting

2. This creates a node pool, which is a group of nodes within a

3. Add a node label for the node pool.

6. The node will automatically add or remove nodes in the node

2.2.2 HPA (HORIZONTAL POD AUTOSCALING)

For most of cases, a combination of cluster and HPA autoscaling will

2. Set auto scaling to either CPU Usage or Memory Usage

To enable Cloud Metrics HPA, follow these steps:

2. Deploy and configure HPA. An example yaml file is

2.2.3 HTTPS CERTIFICATION BY CERT-MANAGER

To achieve this, you need to set up Cert-Manager. Cert-Manager is a

Cost Analysis Cluster Security Container

Cert-Manager adds certificates and certificate issuers as resource

It can issue certificates from a variety of supported sources,

HTTP validation DNS validation

Ingress ASM (Service Mesh)

2.2.3.1 HTTPS CERTIFICATION FOR INGRESS BY CERT-MANAGER

1. Install Cert-Manager and type the following:

2. Create ClusterIssuer using the following commands:

3. Update Ingress, add annotations and TLS. The Certificate and

4. Now, nacos-http01-secret is set to True.

2.2.3.2 HTTPS CERTIFICATION USING DNS VALIDATION

Create alidns webhook of

Ingress Create ClusterIssure or Issuer

Ingress ASM (Service Mesh)

1. Install Cert-Manager and type the following:

2. Install alidns webhook of cert-manager using the

3. Create a RAM account, which can be accessed by the

a. AK and SK must be encoded by base64.

6. Create ClusterIssuer and Certificate, and the Secret will be

7. Add SecretVolumes to ASM Gateway, the “secretName” is the

8. Execute command in ACK cluster to verify cert already bound in

kubectl exec -it -n istio-system $(kubectl -n istio-system get

The result will look like this: 

9. Execute command in the ACK cluster to verify that the Subject

The result will look like this:

10. Execute the following command in ACK cluster to verify Ingress

kubectl exec -it -n istio-system $(kubectl -n istio-system get

The result will look like this:

11. Add tls in your VirtualService gateway and

1.1 About this Solution 1

2.1 Cloud-Native Application Maturity Model 5