Grafana Alloy Agent Installation Procedure

Objective

Install the Grafana Alloy observability stack in a GKE cluster (or corresponding environment), ensuring the generation of metrics, logs, and traces to Grafana Cloud through automation provided in the GitLab repositories of the LATAM Airlines Resilience and Observability team.

Scope

This procedure applies to the following environments:

• Development (dev)
• Integration (intg)
• Production (prod)

Each environment has its own GitLab repository and configuration file.

Prerequisites

Before starting the installation, ensure you meet the following requirements:

• Have access to the GitLab project corresponding to the environment.
• Have gcloud installed and configured on your local machine.
• Have authenticated with GCP: gcloud auth login
• Have read/write permissions on the repository and the ability to create branches and merge requests.
• Have access to the GCP projects where Grafana Alloy will be deployed.

Procedure

1. Clone the repository for the corresponding environment

   Depending on the environment where the stack will be installed:

Environment	GitLab Repository
Production (prod)	alloy-prod-clusters
Development (dev)	alloy_dev_clusters
Integration (intg)	alloy_gke_intg

Clone the corresponding repository:

    git clone <REPOSITORY_URL>

Then create a new branch for your installation:

    git checkout -b feature/install_<cluster_name>

2. Edit the environment configuration file

Locate the clusters_ENVIRONMENT.json file within the cloned repository. (Replace ENVIRONMENT with dev, intg, or prod as appropriate).

Example names:

clusters_prod.json clusters_dev.json clusters_intg.json

3. Add the new cluster definition

   The JSON file has the following base structure:

{
  "repositories": [
    {
      "project": "xxxxxxx",
      "location": "xxxxxxx",
      "cluster_id": "xxxxxxx",
      "autopilot": "no"
    },
    {
      "project": "yyyyyyyy",
      "location": "yyyyyy",
      "cluster_id": "yyyyyy",
      "autopilot": "no",
      "stack": "stack_name"
    }
  ]
}

To add a new cluster:

Add a comma (,) at the end of the last existing object. Insert a new object with the required fields (project, location, cluster_id, autopilot, and optionally squad).

Example addition:

{
  "project": "marketingdomain-prod-9o7z",
  "location": "us-east1",
  "cluster_id": "gke-01-us-east1-prod",
  "autopilot": "yes",
  "stack": "eloyalty"
}

The stacks available in the templates are:

• eloyalty
• ecarga
• monobs
• ANDES

4. Validate GCP access

Verify that you have access to GCP and that gcloud is authenticated correctly:

    gcloud auth list
    gcloud config list project

If you are not authenticated:

    gcloud auth login

5. Run the installer

   Once the JSON file has been modified, run the installation script install.sh:

    ./install.sh clusters_ENVIRONMENT.json

This script:

• Reads the configuration from the JSON file.
• Automatically generates the required values.yaml files for each cluster.
• Prepares the Helm charts for: Stackdriver exporter Grafana Alloy OpenTelemetry Operator

6. Validate file generation

   After running the script, the following files should have been generated:

Expected structure:

• deploy/
  • helm/
    • k8s-monitoring/
      • values_<project_id><cluster_id>.yaml
    • stackdriver/
      • values_<project_id><cluster_id>.yaml

You must also add the Helm charts to install in these files.

7. Create Merge Request

Once the generated files have been validated:

1. Push the changes:

       git add .
       git commit -m "Add new cluster <cluster_name> configuration"
       git push origin feature/install_<cluster_name>

2. Create a Merge Request from your branch to master.

3. Request review from the Observability and Resilience team.