Upload Image

By Raj Marni. March 27, 2025. Revised. Version: 0.0.03

Layer 2: Upload Functionality Overview

1. Purpose

The Upload functionality provides a streamlined workflow for developers to push/upload a Docker image into the k8or Orbit environment via the OpsBoard. Users can specify metadata (title, version, environment notes), and the system ensures:

  1. Image Validation: The Docker image is valid, tagged, and meets any security checks.

  2. Metadata Generation: The system captures all necessary details (Helm chart references, user logs, environment fields, etc.).

  3. Storage & Logging: If needed, image or related data is placed into S3-like buckets (C16) or recorded in ChartStore (C20).

  4. Portal Integration: The user sees an updated status in the Portal, verifying a successful or failed upload.

Orbit Upload Image Diagram

2. Key Functions

  1. User-Initiated Image Upload

    • Portal FE (c8fms03): Allows the user to pick a Docker image file or specify an existing image reference, fill out metadata, and trigger “Upload”.

    • Portal BE (c8bms06): Coordinates the next steps—validating or storing the image, updating logs, and collecting additional data.

  2. Manifestor Components (C12)

    • Image Validator (c12bms03): Checks that the image meets naming or versioning requirements, runs basic checks.

    • Chart Generator (c12bms06): Creates or updates a chart/manifest entry for the new image, referencing environment data or user input metadata.

  3. Storage & Metadata

    • ImageHarbor or Buckets (C16): May store the actual binary or references (e.g., temporary-app-image-1203-bkt-k8d-aws).

    • ChartStore (C20):

      • Maintains user-event logs (user-event-log-1621), error logs (user-error-log-1624), or tables for metadata (app-image-metadata-1615, user-input-manifest-1606).

      • Possibly integrates with Kustomize base references or environment definitions.

    • ViewPort (C32):

      • Contains WordPress-like tables (wp_posts, wp_postmeta) that track environment associations or user-owned image references.

  4. Logging & Event Handling

    • If the upload or chart generation fails at any stage, the system logs an error in user-error-log.

    • On success, the system records a new event referencing the user’s upload action, accessible in the Portal UI.

3. Architecture & Interactions

3.1 Typical Sequence

  1. User Action: Developer in the Portal FE (c8fms03) selects “Upload Image”, picks a Docker file or references an existing Docker Hub image.

  2. Portal Upload Logic (c8bms06) receives the request:

    • Metadata Gathering: Asks for title, environment tags, or additional fields from the user.

    • Calls: Possibly calls C12 (Manifestor) to validate or create chart definitions.

  3. Image Validation (c12bms03):

    • Checks Docker references, ensures no naming conflict, or logs issues if the image is invalid.

  4. Storage & Logging:

    • C16: If needed, stores the raw image data or references in an S3-like bucket.

    • C20:

      • Writes user-event logs, environment data, or Kustomize references (table-based).

    • C32: Optionally records environment or user fields in WP-like tables.

  5. Success/Failure:

    • The Portal presents a “Upload Successful” or “Failed” message. The logs in C20 or C32 reflect the final state.

3.2 Main Components

  • Portal FE (c8fms03): The user-facing upload board.

  • Portal Upload Logic (c8bms06): Orchestrates calls to other microservices, ensures user data is consolidated.

  • Manifestor (C12):

    • Image Validator (c12bms03)

    • Chart Generator (c12bms06)

  • Storage:

    • ImageHarbor / S3 (C16) for image files.

    • ChartStore (C20) for logs, metadata, event info.

    • ViewPort (C32) for environment or user references.

4. Error Handling & Validation

  1. Invalid Docker Image: If the system can’t parse the Docker file or the user inputs the wrong reference, the system logs an error (user-error-log-1624) and halts.

  2. Metadata Conflicts: If the chart name is already taken, or environment references conflict, the chart generator can fail, leading to an error event.

  3. Storage Failure: Issues storing large image files in S3-like buckets or saving logs to the ChartStore can produce partial states, but the system attempts to log or revert incomplete data.

5. Benefits & Impact

  • Simplified Onboarding: Less guesswork or manual Docker commands; a developer can drag-and-drop an image or specify a path.

  • Automated Chart Creation: The user doesn’t manually craft Helm charts; the Manifestor does so behind the scenes.

  • Traceable: The system logs each step, enabling quick debugging if an image fails to upload or a chart fails to build.

  • Extensible: Additional validations, scanning, or security checks can be inserted around the “Image Validator” stage without major UI changes.

Layer 3: Upload Functionality

1. Overview

The Upload functionality lets developers submit a Docker image and its associated metadata into the orbit-plane environment, behind a simple Portal interface. The back-end orchestrates calls to:

  • Manifestor (C12) microservices for validating the image and generating or updating Helm chart definitions,

  • ImageHarbor (C16) or S3-like storage buckets for potentially storing the binary layers or references,

  • ChartStore (C20) to persist logs, chart info, user-provided metadata, and environment references,

  • ViewPort (C32) for capturing environment-lifecycle data or user fields in WordPress-like tables.

By abstracting these tasks, the user experiences a simple “Upload Image” workflow, while the system ensures data consistency, auditing, and potential for future expansion (security scans, more advanced validations, etc.).

2. Internal Modules & Responsibilities

2.1 Portal Front-End: c8fms03

  • User Interface:

    • Provides an upload board (drag-and-drop or text-based references to a Docker image).

    • Prompts the user for optional environment metadata (e.g., dev, test, etc.) or tags (image version, custom labels).

  • Request Formation:

    • Packages the data (image name, environment details, user ID) into a structured request and sends it to the back-end microservice.

2.2 Portal Upload Logic (Back-End: c8bms06)

  • Coordinator:

    • Receives the user’s upload request, logs the initial event, and calls the relevant Manifestor microservices.

  • Metadata Consolidation:

    • Gathers user-supplied fields, merges them with any auto-generated data (timestamps, user ID, etc.).

  • Writes to ChartStore:

    • If partial or final data must be recorded (e.g., user-event-log-1621), c8bms06 inserts or updates relevant tables in C20.

2.3 Manifestor Components (C12)

  1. Image Validator (c12bms03)

    • Docker Image Checks: Confirms the image syntax (e.g., “myapp:1.0”), checks repository existence or naming rules.

    • Security / Integrity: Potentially could run quick checks (or connect to a scanning pipeline) to verify the image meets organizational policies.

  2. Chart Generator (c12bms06)

    • Helm Chart / Kustomize Creation: Builds or updates a chart that references the new image.

    • Metadata: Ingests environment details from c8bms06, then commits a new chart version or overlay references to ChartStore (C20).

    • Error Handling: If chart creation fails, logs an error event (user-error-log-1624).

2.4 Storage in ImageHarbor (C16)

  • Optional S3 Bucket:

    • If the system physically stores the image layers or a temporary reference, c12bms03 might push them to an S3 bucket like temporary-app-image-XXXX-bkt-k8d-aws.

    • The system logs the final location or reference in ChartStore so other processes (like Deploy or Transfer) can fetch it.

2.5 ChartStore (C20)

  • Logging & Metadata:

    • A variety of tables (e.g., user-event-log-1621, user-error-log-1624, app-image-metadata-1615, user-input-manifest-1606) track states, errors, user input.

  • Kustomize References:

    • If the new image needs environment overrides, base-kustomize-1603 or k8or-kustomize-1609 might be updated to reflect environment overlays.

2.6 ViewPort (C32)

  • Environment or WP-lifecycle Data:

    • If the user or environment needs to be associated with additional WP-like meta, c8bms06 or the Manifestor might write a record referencing WP tables in C32.

  • Portal UI:

    • The system can read from these tables to show “Your Uploaded Images” or environment statuses.

3. Data Flow & Process IDs

A typical flow (the diagram references various eXX or eYYY calls):

  1. User Initiates Upload

    • c2usr03-c8fms03-e05: Developer to front-end microservice with basic image info.

  2. Portal FE → Portal BE

    • c8fms03-c8bms06-e10: The front-end sends the request (image name, environment selection, user metadata) to Upload Logic.

  3. Upload Logic → Manifestor

    • c8bms06-c12bms03-eXX: The system calls “Image Validator” or “Chart Generator”.

    • For example, c12bms03-c16bre03-e50 might store references in a S3 bucket or verify them.

    • Meanwhile, c12bms06-c20bre12-e80 might write chart metadata to ChartStore.

  4. Event Logging

    • The system writes logs or references to user-event-log-1621 (C20). If an error occurs, it writes to user-error-log-1624.

  5. Feedback to User

    • The front-end poll or simply receives a final success or error.

    • If successful, the new image can appear in the “My Images” list or be recognized for future Deploy or Transfer steps.

4. Error Handling & Validation

  • Image Syntax: The image might have an invalid tag or reference. The system logs an immediate error.

  • Name Collisions: If a helm chart or an environment entry already exists with the same name, the system can refuse or prompt for a rename.

  • Partial Writes: If the system fails mid-process (e.g., chart generator fails), partial data might remain. The system logs a partial or incomplete state, letting users see errors and possibly retry.

5. Benefits & Impact

  1. Automated Helm Chart Generation: Developers no longer craft charts manually; the Manifestor does it, ensuring consistency.

  2. Audit Trails: The system logs who uploaded which image, and when, plus environment details, helping with compliance or debugging.

  3. Ease of Use: A single “Upload” action hides multiple behind-the-scenes steps, so developers do not juggle DockerHub, Helm, or S3 individually.

  4. Future Extensibility: Additional scanning, naming rules, or environment checks can be integrated into the Manifestor steps without changing the user’s flow.

PreviousComponent C132 – NATSNextSearch Image

Last updated