Verificando autenticación…

Skip to main content

Python Cloud Function

Purpose and Scope​

This project is a basic template designed to facilitate the development and deployment of Google Cloud Functions Generation 2 using Python. The main goal is to provide a project structure that allows developers to get started quickly and that is adaptable to various cloud development needs.

Problem it solves: It simplifies the initial setup and structure of a project for Google Cloud Functions Gen 2, promoting good organization and deployment practices.

Scope:

  • Provides a base directory structure.
  • Includes a simple HTTP function example (main.py).
  • Defines dependency management (requirements.txt).
  • Configuration for deployment in multiple environments (dev, intg, prod) through YAML files.
  • Integration with CI/CD pipelines (GitLab CI and Jenkins).
  • Basic configuration for the Cloud Function (runtime, memory, timeout, etc.).

This template is intended for a simple function, but can be adapted for more complex functions or integrated with other Google Cloud services, such as Pub/Sub, Firestore, etc.

Architecture​

The project is designed to be deployed as a Google Cloud Function Second Generation (Gen 2). This type of function runs in a containerized environment on Cloud Run, offering greater flexibility and capabilities than the first generation.

The project architecture is organized as follows:

.
├── deploy
│   ├── env
│   │   ├── dev.yaml                    # Environment variables for development
│   │   ├── intg.yaml                   # Environment variables for integration
│   │   └── prod.yaml                   # Environment variables for production
│   └── function                        # Cloud Function configuration folder
│       ├── environment.yaml.erb        # Environment configuration for the function (template)
│       └── runtime-config.yaml.erb     # Runtime configuration (template)
├── src                                 # Main code folder
│   ├── main.py                         # Main function code
│   └── requirements.txt                # Python dependencies
├── .gitlab-ci.yml                      # Pipeline configuration for GitLab CI
├── Jenkinsfile                         # Jenkins integration pipelines
└── README.md                           # Project documentation

The function entry point is defined in src/main.py. Deployment-specific and Cloud Function runtime configurations are managed through the .yaml and .yaml.erb files in the deploy/ directory.

Technologies and Dependencies​

  • Language: Python 3.12 (defined in deploy/function/runtime-config.yaml.erb)
  • Cloud Platform: Google Cloud Functions Gen 2
  • Functions Framework: functions-framework==3.8.1
  • Main Libraries:
    • MarkupSafe~=2.1.5 (for safe HTML escaping)
    • (Other dependencies are added in src/requirements.txt as needed)
  • CI/CD:
    • GitLab CI (configured in .gitlab-ci.yml using $LATAM_PIPELINE_CLOUD_FUNCTION_DEPLOY)
    • Jenkins (referenced by Jenkinsfile)
  • Dependency Management: Pip (through requirements.txt)

Local Setup​

Prerequisites​

  • Python 3.12
  • Pip (usually comes with Python)
  • Google Cloud SDK (gcloud CLI) installed and configured.
  • (Optional) A Python virtual environment (e.g., venv, conda).

Setup Steps​

  1. Clone the repository: Follow these steps

  2. (Optional but recommended) Create and activate a virtual environment:

    python -m venv venv
    source venv/bin/activate  # On Linux/macOS
    # venv\Scripts\activate    # On Windows

  3. Install dependencies:

    pip install -r src/requirements.txt

  4. Configure src/main.py: Modify the entry_point function (or the function you define as the entry point) in src/main.py according to your business logic needs.

    # src/main.py
    import functions_framework
    from markupsafe import escape

    @functions_framework.http
    def entry_point(request):
        """HTTP Cloud Function.
        Args:
            request (flask.Request): The request object.
        Returns:
            The response text, or any set of values that can be turned into a
            Response object using `make_response`.
        """
        request_json = request.get_json(silent=True)
        request_args = request.args

        if request_json and "name" in request_json:
            name = request_json["name"]
        elif request_args and "name" in request_args:
            name = request_args["name"]
        else:
            name = "World"
        return f"Hello {escape(name)}!"

  5. Run locally (using Functions Framework): To test the function locally, you can use the Functions Framework:

    functions-framework --target=entry_point --source=src/main.py --port=8080 --debug

    Then, you can access http://localhost:8080 in your browser or with curl.

Deployment​

Cloud Function deployment is managed through CI/CD pipelines and gcloud scripts.

Deployment Configuration​

The key files for deployment configuration are:

  • deploy/function/runtime-config.yaml.erb: Defines the Cloud Function runtime properties.

    runtime: python312
    cpu: 1
    memory: 512MB
    timeout: 3600s
    region: "<%= region %>" # Interpolated from environment files
    entry-point: entry_point
    gen2: ~
    source: src
    max-instances: 2
    trigger-http:
    update-labels: "cmdb-<%= cmdb_it_element %>=<%= cmdb_it_element %>"
    service-account: "<%= service_account %>"

  • deploy/function/environment.yaml.erb: Defines environment variables that will be passed to the function.

    BRANCH: "<%= branch_name %>"
    IS_PROD: "<%= is_prod_deploy %>"
    GIT_COMMIT: "<%= git_commit %>"
    BUILD_ID: "<%= build_id %>"

    TEST_ENV: "<%= test_env %>" # Example, interpolated from environment files

  • deploy/env/*.yaml: Contain environment-specific variables for each environment (dev, intg, prod). Example dev.yaml:

    test_env: "test in dev"
    region: ${{ REGION }} # These may be CI/CD variables
    service_account: ${{ SERVICE_ACCOUNT }}
    project_id: ${{ PROJECT_ID }}
    cmdb_it_element: ${{ IT_ELEMENT }}

CI/CD Pipelines​

  • GitLab CI: The .gitlab-ci.yml file uses a predefined pipeline component for deployment:

    include:
      - component: $LATAM_PIPELINE_CLOUD_FUNCTION_DEPLOY

    This pipeline handles build, test, security, release, deploy, and other steps.

  • Jenkins: A Jenkinsfile is included in the project root, suggesting the possibility of integration with Jenkins for pipelines.

gcloud Commands (Manual Example)​

Although deployment is usually automated, a manual gcloud command to deploy a Gen 2 function might look like (adapted from the configuration):

gcloud functions deploy YOUR_FUNCTION_NAME \
  --gen2 \
  --runtime=python312 \
  --region=YOUR_REGION \
  --source=./src \
  --entry-point=entry_point \
  --trigger-http \
  --allow-unauthenticated \ # Or configure authentication as needed
  --service-account=YOUR_SERVICE_ACCOUNT_EMAIL \
  --memory=512MB \
  --cpu=1 \
  --timeout=3600s \
  --max-instances=2 \
  --set-env-vars=VAR1=value1,VAR2=value2 \ # Variables from environment.yaml.erb
  --update-labels=cmdb-YOUR_CMDB_ELEMENT=YOUR_CMDB_ELEMENT

Variables (YOUR_FUNCTION_NAME, YOUR_REGION, etc.) and configuration files (--env-vars-file) would be handled by the CI/CD pipeline from the deploy/env/*.yaml and deploy/function/*.yaml.erb files.

API Endpoints​

The example function in src/main.py exposes an HTTP endpoint.

MethodEndpointDescriptionRequest Parameters (JSON or Query)Request Example (Query)Success Response ExampleAuthentication
GET/POST/ (relative to the function URL)Greets the provided name or "World" if no name is provided. The function name is part of the base URL.name (string, optional)curl "FUNCTION_URL?name=User"Hello User!--trigger-http (by default allows unauthenticated if specified, or requires IAM)

Request Example (JSON with curl):

curl -X POST \
  FUNCTION_URL \
  -H "Content-Type: application/json" \
  -d '{"name": "World"}'

Response:

Hello World!

Note: The function base URL (FUNCTION_URL) is provided by Google Cloud after deployment.

Specific Components​

This base template does not include direct integrations with components such as Pub/Sub, Storage, Redis, Kafka, etc. However, it is designed to be easily extended.

To integrate with other Google Cloud services or external services:

  1. Add Dependencies: Add the necessary client libraries to the src/requirements.txt file. For example:

    • For Google Cloud Storage: google-cloud-storage
    • For Google Cloud Pub/Sub: google-cloud-pubsub

  2. Modify src/main.py: Implement the logic to interact with the desired service within your function.

    # Conceptual example to interact with Storage
    from google.cloud import storage

    def interact_with_storage():
        storage_client = storage.Client()
        bucket = storage_client.bucket("my-bucket-name")
        blob = bucket.blob("my-file.txt")
        # ... perform operations with the blob ...

  3. Environment Configuration: If necessary, add environment variables (bucket names, topics, etc.) in the deploy/function/environment.yaml.erb and deploy/env/*.yaml files.

  4. Permissions: Ensure that the service account (service-account specified in runtime-config.yaml.erb) associated with the Cloud Function has the necessary IAM permissions to interact with the other services.

Testing​

This template does not include a testing framework configured by default (such as pytest or unittest). It is recommended to add unit and integration tests according to project needs.

Suggested Steps to Add Tests:​

  1. Choose a Testing Framework: For example, pytest.

    • Add pytest to src/requirements.txt (or to a requirements-dev.txt).

  2. Write Tests: Create a tests/ directory at the project root or inside src/ and write the test files.

    # example tests/test_main.py
    # from src.main import entry_point # Adjust import according to structure
    # import flask

    # def test_entry_point_no_name():
    #     req = flask.Request.from_values() # Basic simulation
    #     # Mock get_json and args if needed for a more robust test
    #     # Or use the Flask test client with Functions Framework
    #     response = entry_point(req)
    #     assert response == "Hello World!"

    # def test_entry_point_with_name_query():
    #     req = flask.Request.from_values(args={'name': 'TestUser'})
    #     response = entry_point(req)
    #     assert response == "Hello TestUser!"

  3. Run Tests:

    python -m pytest

  4. CI/CD Integration: Add a step in the CI/CD pipeline (.gitlab-ci.yml or Jenkinsfile) to run tests automatically.

Security Considerations​

  • Service Account Permissions: The service account specified in deploy/function/runtime-config.yaml.erb (service_account) should follow the principle of least privilege. Grant only the IAM roles and permissions strictly necessary for the function to operate.
  • Function Access:
    • For HTTP functions, consider whether they should be publicly accessible (--allow-unauthenticated) or require IAM authentication.
    • For event-triggered functions (e.g., Pub/Sub, Storage), authentication is handled by the invoking service.
  • Secret Handling: Avoid hardcoding secrets (API keys, passwords) in the code. Use environment variables (configured through environment.yaml.erb and the env/*.yaml files, whose values may come from a secrets manager in the CI/CD pipeline) or Google Secret Manager.
  • Dependencies: Keep dependencies in src/requirements.txt up to date and review regularly for known vulnerabilities.
  • Input Validation: For HTTP functions, validate and sanitize all user inputs (e.g., request.args, request.get_json()) to prevent vulnerabilities such as XSS (although MarkupSafe helps with output escaping in the example).
  • Logging and Monitoring: Configure adequate logging for auditing and debugging. Use Cloud Logging and Cloud Monitoring.