template-gke-resources

Purpose and Scope

This project defines the configuration templates for the resources required to deploy applications on Google Kubernetes Engine (GKE) within the LATAM ecosystem. Its purpose is to standardize and simplify the creation of the base infrastructure in GKE, including network, security, and environment variable configuration.

The scope includes the definition of:

• Backend and Frontend configurations for services.
• ConfigMaps for variable injection.
• Ingress resources for service exposure and traffic routing.
• Basic security policies through Cloud Armor.
• Use of TLS certificates and HTTPS redirection.

This template is designed to be used with the gke-krane-deploy deployment pipeline.

Architecture

The project is based on Kubernetes manifest templates in YAML format, using ERB (Embedded Ruby) to allow customization of values at deployment time. These manifests describe the resources that will be created or updated in a GKE cluster.

The main components defined are:

• BackendConfig: Defines specific configurations for service backends, such as timeouts and security policies (e.g., Cloud Armor).
• FrontendConfig: Defines frontend configurations, such as automatic redirection to HTTPS.
• ConfigMap: Stores non-sensitive configuration data in key-value pairs that can be consumed by pods.
• Ingress: Manages external access to services within the cluster, typically HTTP/HTTPS, configuring load balancing, SSL termination, and host- and path-based routing.

Deployment of these resources is performed through a Jenkins pipeline that uses Krane.

Technologies and Dependencies

• Orchestration Platform: Google Kubernetes Engine (GKE)
• Resource Definition: Kubernetes YAML
• Template Engine: ERB (Embedded Ruby)
• Continuous Integration/Continuous Deployment (CI/CD): Jenkins (using the gke-krane-deploy pipeline)
• Version Control: Git (GitLab)
• Perimeter Security (optional): Google Cloud Armor (referenced in BackendConfig)

Local Configuration

Prerequisites

• Git installed.
• Access to the project repository.

Steps to Review the Templates

1. Clone the repository: Follow the steps below

2. Explore the templates: The main configuration files are located in the deploy/gke/ directory. The key file is:

   • deployment.yaml.erb: Contains the templates for BackendConfig, FrontendConfig, ConfigMap, and Ingress.

   Variables enclosed in <%= ... %> (e.g., <%= owner_name %>, <%= environment %>) are replaced by the deployment pipeline with environment- and application-specific values.

Deployment

Deployment of these GKE resources is performed through the Jenkins pipeline specified in the Jenkinsfile of the application that consumes this template.

Jenkinsfile

The Jenkinsfile of the application that uses this template must invoke the LATAM global pipeline for GKE deployments with Krane:

pipelineLatam("gke-krane-deploy")

This pipeline will process the .erb templates in deploy/gke/deployment.yaml.erb, substituting the variables and applying the resulting YAML manifests to the corresponding GKE cluster.

Deployment Variables

The templates use variables that are injected during the deployment process. Some of the key variables are:

• owner_name: Owner or application name.
• environment: Deployment environment (e.g., dev, qa, prod).
• git_commit: Git commit hash.
• build_id: Build identifier.
• dns_record: DNS record for the Ingress.
• static_ip_name: Global static IP resource name.
• is_prod_deploy: Boolean indicating whether it is a production deployment.

API Endpoints

This project does not define APIs directly, but rather the infrastructure (Ingress) that exposes the endpoints of the deployed applications. The Ingress resource configured in deploy/gke/deployment.yaml.erb defines how external traffic is routed to internal services.

Ingress Route Configuration

The Ingress defines path-based routes for different services:

spec:
  tls:
    - secretName: <%= owner_name %>-<%= environment %>-certificate # TLS certificate
  rules:
    - host: <%= dns_record %> # Primary DNS
      http:
        paths:
          - path: / # Route for the main frontend
            pathType: Prefix
            backend:
              service:
                name: <%= owner_name %>-template-angular-fe-service
                port:
                  number: 80
          - path: /api/bl/ # Route for the Business Logic service
            pathType: Prefix
            backend:
              service:
                name: <%= owner_name %>-template-spring-bl-service
                port:
                  number: 80
          - path: /authorization-service/ # Route for the authorization service
            pathType: Prefix
            backend:
              service:
                name: <%= owner_name %>-auth-service-bl-service
                port:
                  number: 80
          - path: /web/ # Route for the web service (e.g., traditional Spring MVC)
            pathType: Prefix
            backend:
              service:
                name: <%= owner_name %>-template-spring-web-service
                port:
                  number: 80

• Traffic is directed to the corresponding service (backend.service.name) based on the request path.
• A secret name (secretName) is used for TLS configuration, ensuring HTTPS communication.
• A global static IP is associated (kubernetes.io/ingress.global-static-ip-name).

Specific Components (Kubernetes Resources)

BackendConfig

• Purpose: Configure advanced parameters for backend services, such as timeouts and security policies.
• Configuration Fragment (deploy/gke/deployment.yaml.erb):

  apiVersion: cloud.google.com/v1
  kind: BackendConfig
  metadata:
    name: "<%= owner_name %>-<%= environment %>-backendconfig"
    labels:
      app: "<%= owner_name %>"
      git_commit: "<%= git_commit %>"
      build_id: "<%= build_id %>"
  spec:
    timeoutSec: 120 # Timeout for backend responses
    securityPolicy:
      name: "cloudarmor-corp" # Cloud Armor policy

• Usage: Associated with Kubernetes services through annotations in the service definition (not included in this template, but is the expected usage).

FrontendConfig

• Purpose: Specify configurations for the Ingress frontend, primarily HTTP to HTTPS redirection.
• Configuration Fragment (deploy/gke/deployment.yaml.erb):

  apiVersion: networking.gke.io/v1beta1
  kind: FrontendConfig
  metadata:
    name: "<%= owner_name %>-<%= environment %>-frontendconfig"
    labels:
      app: "<%= owner_name %>"
      git_commit: "<%= git_commit %>"
      build_id: "<%= build_id %>"
  spec:
    redirectToHttps:
      enabled: true
      responseCodeName: MOVED_PERMANENTLY_DEFAULT # 301 response code

• Usage: Associated with the Ingress through the networking.gke.io/v1beta1.FrontendConfig annotation.

ConfigMap

• Purpose: Store and manage configuration variables that can be consumed by applications.
• Configuration Fragment (deploy/gke/deployment.yaml.erb):

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: "<%= owner_name %>-<%= environment %>-configmap"
    labels:
      app: "<%= owner_name %>"
      git_commit: "<%= git_commit %>"
      build_id: "<%= build_id %>"
  data:
    ENVIRONMENT: "<%= environment %>"
    APPLICATION: "<%= owner_name %>"
    DNS_RECORD: "<%= dns_record %>"
    IS_PROD_DEPLOY: "<%= is_prod_deploy %>"

• Usage: Pods can mount this data as environment variables or files.

Ingress

• Purpose: Manage incoming traffic, providing load balancing, SSL termination, and routing based on host names and paths.
• Configuration Fragment (deploy/gke/deployment.yaml.erb):

  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: <%= owner_name %>-<%= environment %>-ingress
    labels:
      app: "<%= owner_name %>"
      git_commit: "<%= git_commit %>"
      build_id: "<%= build_id %>"
    annotations:
      kubernetes.io/ingress.global-static-ip-name: <%= static_ip_name %>
      networking.gke.io/v1beta1.FrontendConfig: <%= owner_name %>-<%= environment %>-frontendconfig
  spec:
    tls:
      - secretName: <%= owner_name %>-<%= environment %>-certificate
    rules:
    # ... (see API Endpoints section for rule details)

• Usage: Directs external traffic to the correct services within the cluster, based on the defined rules.

Testing

This project, being a set of infrastructure templates, does not contain unit or integration tests in the traditional software sense. "Testing" consists of:

1. YAML/ERB Syntax Validation: Ensuring the templates are valid before deployment. This is usually part of the linting process or Krane itself.
2. Successful Deployment: Verifying that the Jenkins pipeline applies the manifests to the GKE cluster without errors.
3. GKE Resource Verification: Confirming that the resources (BackendConfig, FrontendConfig, ConfigMap, Ingress) have been created or updated correctly in the cluster.
4. Connectivity and Routing Tests: Once the applications using these resources are deployed, test access through the Ingress, the correct application of security policies, and HTTPS redirection.

Tools for these verifications include:

• kubectl to inspect resources in the GKE cluster.
• Google Cloud Platform Console.
• Endpoint testing tools (e.g., Postman, curl) to verify Ingress routing.

Security Considerations

• Mandatory HTTPS: The FrontendConfig is configured to redirect all HTTP traffic to HTTPS.
• TLS Certificates: The Ingress uses a Kubernetes secret (secretName: <%= owner_name %>-<%= environment %>-certificate) for SSL/TLS termination. Management of this certificate (creation, renewal) is external to this template but crucial.
• Cloud Armor: The BackendConfig references a Cloud Armor security policy (cloudarmor-corp). This implies that traffic to the backends may be protected by WAF and other security rules defined in that policy.
• Static IP: Using a global static IP (kubernetes.io/ingress.global-static-ip-name) facilitates whitelist configuration and DNS records.
• ConfigMaps: Should only be used for non-sensitive configuration data. Secrets should be managed using Kubernetes Secrets.
• Pipeline Permissions: The Jenkins pipeline (gke-krane-deploy) must have the appropriate permissions in GCP to manage these resources in GKE.