Transfer Image

By Raj Marni. March 27, 2025. Revised. Version: 0.0.01

Layer 2: Transfer Functionality Overview

1. Purpose

The Transfer functionality provides a seamless way for developers to re-deploy Docker images into different environments (dev → test, test → prod, etc.) while preserving chart definitions, environment metadata, and lifecycle logs. It orchestrates updates to multiple back-end resources (ImageHarbor in C16, ChartStore in C20, and ViewPort in C32) via calls from the k8or Portal (C8).

Orbit Transfer Image Diagram

Users initiate these “promotions” via the Portal (C8), whose back-end logic ensures all relevant back-end resources are updated consistently. This design greatly reduces manual overhead and promotes a more auditable, secure, and automated approach to multi-environment container deployments.

2. Key Functions

  1. Image Reference & Environment Transition

    • Re-labeling / Re-tagging: The Transfer logic updates references to the Docker image so that it’s recognized under the new environment.

    • Cross-Environment Consistency: Ensures the image is valid and properly tracked in the next stage (test, prod), reflecting environment-specific configurations.

  2. Metadata & Chart Updates

    • Helm Chart Adjustments: The system modifies or creates a new Helm chart entry (stored in C20 – ChartStore) that points to the new environment.

    • Logging & Event Recording: Transfer events (success/failure) are recorded in user-event or error-log tables, enabling audit trails.

  3. State Tracking & Lifecycle

    • ViewPort (C32) might receive updates tracking the Docker image’s journey (e.g., from dev to test) for governance or user-facing progress displays.

    • ImageHarbor (C16): Ensures the actual Docker images or references remain accessible in the new environment context.

  4. Portal-Driven Orchestration

    • User-Initiated Transfer: A developer triggers a transfer from the k8or Portal (C8), e.g., clicking a “Promote to Test” button.

    • Portal Transfer Logic (BE): A back-end microservice (e.g., c8bms18) coordinates all required calls to C16, C20, and C32, guaranteeing a consistent environment migration.

3. Architecture & Interactions

3.1 Major Components

  • C8 – k8or Portal

    • Frontend (FE MSV): User interacts with the UI to initiate a transfer.

    • Backend (BE MSV): Executes the Transfer logic, collecting or updating data from the relevant resources.

  • C16 – ImageHarbor

    • Stores or references the Docker images, typically in an S3-like bucket.

    • The Transfer operation may re-tag images or confirm their existence under a new environment label.

  • C20 – ChartStore

    • Holds Helm charts and deployment metadata for each environment.

    • On Transfer, the system may create or update a new chart referencing the same Docker image but assigned to test/prod, etc.

  • C32 – ViewPort

    • Tracks extended lifecycle data (e.g., environment transitions, user logs, environment fields).

    • The Transfer logic updates or inserts relevant records about the new environment state.

3.2 Process Flow (Generic)

  1. User Action: Developer selects an image in dev environment, clicks “Transfer to Test”.

  2. Portal Backend (C8-BE) receives the request, checks environment rules, and prepares calls to:

    • C16 (ImageHarbor) to confirm the Docker image is valid or create a new environment-based reference.

    • C20 (ChartStore) to update or insert a Helm chart record referencing the new environment.

    • C32 (ViewPort) to log the environment change or store relevant metadata.

  3. Database & S3 Updates: Each resource (C16, C20, C32) receives calls with unique Process IDs for tracking, ensuring event logs and chart definitions are fully updated.

  4. Success/Failure Confirmation: The Portal reflects the final status—successful environment transfer or logs any errors in user-event or error logs.

3.3 Error Handling & Validation

  • Checks: Verifies the image’s existence in dev, ensures that the test environment is authorized for the user, and checks available resource definitions.

  • Rollbacks: If any step (e.g., ChartStore update) fails, the Transfer logic can revert partial changes or mark the Transfer as incomplete, showing an error in the Portal.

4. Benefits & Impact

  1. Streamlined Promotion Flow

    • Eliminates the manual overhead of re-pushing images or re-writing chart files for each environment.

    • Offers a single Portal-based button or command to shift an application from dev → test → production.

  2. Consistency & Auditability

    • Chart definitions, Docker references, and environment logs are consistently updated across C16, C20, and C32.

    • Users have a clear audit trail in ViewPort (C32) for who transferred what, from which environment to which.

  3. Reduced Human Error

    • Automated calls (Portal → Transfer Logic → C16/C20/C32) mitigate the risk of mis-tagging images or forgetting to update associated Helm charts.

    • Validation checks along the way help detect missing references or environment constraints.

  4. Scalable Workflow

    • As the number of environments grows (dev, test, stage, prod), the underlying logic remains the same—Portal orchestrates environment re-labeling and metadata updates.

    • Additional environment-specific customizations can be integrated (e.g., storing environment configs in ChartStore).

Layer 3: Architecture for Transfer Functionality

1. Overview

Transfer functionality provides a structured workflow that takes a Docker image and its related configurations from one environment (like dev) to another (like test). It is Portal-driven (C8) and requires synchronous or asynchronous updates in ImageHarbor (C16), ChartStore (C20), and ViewPort (C32). Each diagram in the set highlights a segment of the flow:

  1. C8–C16: Focuses on image references, uploads, and environment tagging in ImageHarbor.

  2. C8–C20: Handles Helm chart creation, user/event logging, and environment-based chart versions in the ChartStore.

  3. C8–C32: Tracks extended environment data changes (e.g., environment fields, WP-like postmeta tables) in ViewPort.

  4. Unified (C8–C16–C20–C32): Shows the entire cross-resource orchestration in one main diagram.

2. Internal Modules & Key Flows

2.1 Portal (C8)

  • Frontend Microservice (c8fmsXX)

    • User Interface: Developer clicks a “Transfer to Test” for an image in Dev environment.

    • Validation: Confirms user privileges, environment source/target constraints.

    • API Calls: Forwards the request to the back-end microservice with relevant data (image ID, environment ID, user context).

  • Backend Microservice (c8bmsXX)

    • Transfer Logic:

      1. Checks environment rules (e.g., dev → test is allowed) and gathers chart / image references.

      2. Constructs process flow steps (PIDs) to call external resources in the correct order.

    • Orchestration:

      • Coordinates calls to C16 (ImageHarbor) for re-tagging or referencing images,

      • C20 (ChartStore) for chart creation/update,

      • C32 (ViewPort) for environment lifecycle logging.

2.2 ImageHarbor (C16)

  • Backend Resource (c16breXX)

    • S3-like Storage: Holds or references the Docker images (app-image-1206, app-image-1209, etc.).

    • Image Re-Tagging: Transfer might create new environment tags or copies, depending on how the system organizes environment-based images.

    • PID Example: c8bms18-c16bre06-e40 / c8bms18-c16bre09-e50 – describing calls to store or verify images for dev/test references.

2.3 ChartStore (C20)

  • Backend Resource (c20breXX)

    • Helm Chart & Metadata: On a successful “transfer”, the system either modifies an existing chart or creates a new chart version pointing to the same Docker image but labeled for the new environment.

    • User/Event Log Tables: Stores user actions (e.g., user-event-log-1621) or error logs (user-error-log-1624) with references to the environment shift.

    • Kustomize Tables: If the system uses Kustomize overlays, new environment overlay entries might be inserted or updated.

2.4 ViewPort (C32)

  • Relational Store (c32bre03)

    • Environment Fields & Lifecycle: Possibly structured like WordPress postmeta, capturing environment transitions.

    • Logging: The system updates references to the new environment, so the user can see a chronological record of environment promotions (e.g., dev → test) in a web UI or admin panel.

    • PID Example: c8bms18-c32bre03-e70 – signifying an update to environment fields or postmeta data.

3. Detailed Process Flow

  1. Developer Action (Frontend MSV - c8fms06)

    • Initiates “Transfer” from the Portal UI, specifying source environment (dev) and target environment (test) plus any additional data (version, notes).

  2. Portal Transfer Logic (Backend MSV - c8bms18)

    • Step A: Validates user privileges, environment existence, and image references.

    • Step B: Forwards requests to the correct environment-based resources:

      • To C16 for verifying or re-tagging Docker images.

      • To C20 for Helm chart creation/updating, user logs, Kustomize tables.

      • To C32 for environment lifecycle or metadata updates.

  3. ImageHarbor (C16)

    • Check: Is this image present under dev?

    • Create/Update: Tag or reference for test environment.

    • Confirm: Returns a success/failure response to c8bms18, possibly with new image references.

  4. ChartStore (C20)

    • Update: The system modifies or creates a new chart referencing the newly updated Docker image for the test environment.

    • Log: Writes an event record (user-event-log-1621) capturing “User X transferred Image Y to environment test” or any errors in user-error-log-1624.

    • Link: Kustomize references the new overlay or environment label in k8or-kustomize-1609, etc.

  5. ViewPort (C32)

    • Record: The environment change or postmeta is updated to reflect the new environment assignment.

    • UI: The Portal can read back from c32bre03 to show environment statuses or transitions.

  6. Portal Confirmation

    • If all calls succeed, the Portal acknowledges the successful transfer.

    • In case of partial failures (e.g., chart update fails), the system logs errors and reverts environment references.

4. Internal Communications & PIDs

Each arrow in the diagrams often has a Process ID (PID) that follows a pattern like startResource-endResource-eXX:

  • Example: c8bms18-c16bre06-e40 or c8bms18-c20bre24-911.

  • Execution Order: The suffix (e40, e911) indicates a step in the overall flow.

  • RIDs & MIDs: c16breXX, c20breXX, c32breXX, c8bmsXX identify specific microservices or resources for logs, S3 storage, or DB tables.

The Transfer logic typically orchestrates these calls in sequence:

  1. Check image existence in dev environment.

  2. Update environment references.

  3. Write chart or Kustomize metadata.

  4. Log environment transition.

  5. Confirm to Portal.

5. Error Handling & Validation

  • Environment Constraints: Ensures dev → test is permitted, test → prod might require additional approvals, etc.

  • Image Mismatch: If the source Docker image doesn’t exist, or the target environment is locked, the system logs an error and halts.

  • Atomicity: If any step fails (e.g., ChartStore can’t update the chart), the system may roll back or mark the Transfer as incomplete.

6. Advantages & Outcomes

  1. Single Action from the user’s perspective: “Transfer Image”, hidden behind multiple automated calls to C16, C20, and C32.

  2. Audit & Traceability: Because each step logs to user-event or error logs, administrators can track exactly when/where a transfer was initiated or failed.

  3. Consistent Deployments: The environment definition (Helm chart, environment fields) stays in sync with the Docker image.

  4. Scalability: The same logic extends to any number of environments, as long as the Transfer rules are well-defined (e.g., dev → test → staging → prod).

PreviousDeploy ImageNextOpsBoard: Using k8or Common CRDs for AWS Cloud Service Integration

Last updated