Update Egg Env Vars: Replace & Merge Modes

by Admin 43 views
Update Egg Environment Variables: Replace and Merge Modes

Hey there, code enthusiasts! Ever found yourself wrestling with environment variables when tweaking your applications? It's a common pain point, especially when you need to update an "egg" (or any running process) without causing downtime or losing its current state. Let's dive into a solution that allows you to seamlessly update your egg's environment variables using either a full replacement or a partial merge approach. This is all about making your development workflow smoother and more efficient.

The Current Pain Point and its Limitations

Currently, the process of updating an egg's environment variables can be a bit of a hassle. You're essentially forced to go through these steps:

  1. Remove the egg: Using a command like kurv remove <egg>. This is the first step which leads to downtime.
  2. Update the configuration file: Modify your config file with the new environment variables, whether it's the .kurv or .egg file.
  3. Re-collect the egg: Using kurv collect <config-file>, which essentially re-initializes everything.

This isn't ideal, right? The main issues are the downtime caused by removing and re-collecting, and the loss of the egg's runtime state. All the progress like the uptime or retry counts are lost, which can be frustrating.

Proposed Solution: A Better Way to Update Environment Variables

We are going to introduce a new API endpoint and a CLI command that makes updating environment variables a breeze. This new approach will eliminate the need to remove and re-collect eggs.

1. New API Endpoint: POST /eggs/{egg_id}/env

We'll create a new API endpoint specifically for updating environment variables. This will handle the "magic" behind the scenes. This endpoint will accept a JSON payload with the new environment variables and a mode specifying how to apply the changes.

Request Body:

{
  "env": {
    "PORT": "8080",
    "DEBUG": "true"
  },
  "mode": "merge"  // or "replace"
}

2. Modes: Flexibility in How You Update

We're providing two modes for maximum flexibility:

  • replace: This mode will completely replace the existing environment variables with the ones provided in the request body. Think of it as a full reset of the environment.
  • merge: This mode will merge the provided environment variables with the existing ones. It will update or add the keys specified in the request, while keeping any existing keys that aren't mentioned. This is ideal for adding a single new variable without changing the others.

3. Implementation Details

To make this happen, we need to create some new code and tweak existing parts. This involves a new module, route registration, and CLI command implementation.

Creating src/api/env.rs

Let's start by creating a new module to handle the API logic. We will be creating src/api/env.rs file. This module will handle the logic for updating the environment variables. This involves parsing the request body, validating the mode, getting the egg, updating the environment, and restarting the egg if needed.

use { 
    super::{Context, err}, 
    crate::{ 
        common::tcp::{Request, Response, json}, 
        kurv::EggStatus, 
    }, 
    anyhow::{Result, anyhow}, 
    serde::{Deserialize, Serialize}, 
    std::collections::HashMap, 
};

#[derive(Serialize, Deserialize, Debug)]
pub struct UpdateEnvRequest {
    pub env: HashMap<String, String>,

    #[serde(default = "default_mode")]
    pub mode: String, // "replace" or "merge"
}

fn default_mode() -> String {
    "merge".to_string()
}

pub fn update_env(request: &Request, ctx: &Context) -> Result<Response> {
    // Parse request body
    let update_req: UpdateEnvRequest = serde_json::from_str(&request.body)
        .map_err(|e| anyhow!("Invalid request body: {}", e))?;

    // Validate mode
    if update_req.mode != "replace" && update_req.mode != "merge" {
        return Ok(err(400, format!("Invalid mode '{}', must be 'replace' or 'merge'", update_req.mode)));
    }

    // Get egg by ID/name/pid
    if let Some(token) = request.path_params.get("egg_id") {
        let state = ctx.state.clone();
        let mut state = state.lock().map_err(|_| anyhow!("failed to lock state"))?;

        let id = state.get_id_by_token(token);

        if let Some(id) = id {
            if let Some(egg) = state.get_mut(id) {
                // Check if egg is a plugin (plugins can't have env updated)
                if egg.plugin.unwrap_or(false) {
                    return Ok(err(403, "Cannot update environment of plugins".to_string()));
                }

                // Update env based on mode
                match update_req.mode.as_str() {
                    "replace" => {
                        egg.env = Some(update_req.env);
                    }
                    "merge" => {
                        let current_env = egg.env.clone().unwrap_or_default();
                        let mut new_env = current_env;
                        new_env.extend(update_req.env);
                        egg.env = Some(new_env);
                    }
                    _ => unreachable!(),
                }

                // If egg is running, restart it to apply new env
                if egg.is_running() {
                    egg.set_status(EggStatus::Restarting);
                }

                return Ok(json(200, egg.clone()));
            }
        }

        return Ok(err(404, format!("Egg not found: {}", token)));
    }

    Ok(err(400, "Missing egg_id parameter".to_string()))
}

Register Route in src/api/mod.rs

We need to register the new endpoint in our API's routing configuration. This ensures that requests to /eggs/{egg_id}/env are correctly handled by our update_env function.

fn routes(&self) -> Vec<RouteDef> {
    vec![
        // ... existing routes
        ("POST", "/eggs/(?P<egg_id>.*)/env", env::update_env),
        ("POST", "/eggs/(?P<egg_id>.*)/stop", eggs::stop),
        // ... rest
    ]
}

Add a New CLI Command

Next, we need to add a CLI command. This command will allow users to update environment variables from the command line. We'll support both merge and replace modes, as well as the ability to load variables from a file.

# Merge (add/update specific vars)
kurv env <egg> --set KEY=VALUE --set KEY2=VALUE2

# Replace entire env
kurv env <egg> --replace KEY=VALUE KEY2=VALUE2

# From file (TOML format)
kurv env <egg> --from-file ./env.toml

Implementation in src/cli/cmd/env.rs

use anyhow::Result;
use serde_json::json;

pub fn run(arguments: &mut Arguments) -> Result<()> {
    let egg_token = arguments.free_from_str()?;
    let replace = arguments.contains("--replace");
    let from_file = arguments.opt_value_from_str("--from-file")?;
    let set_vars: Vec<String> = arguments.opt_values_from_str("--set")?;

    // Build env map from --set flags or file
    let env_map = if let Some(file_path) = from_file {
        // Load from TOML file
        load_env_from_file(file_path)?
    } else {
        // Parse KEY=VALUE pairs
        parse_env_pairs(set_vars)?
    };

    let mode = if replace { "replace" } else { "merge" };

    // Send request to API
    let request_body = json! {
        "env": env_map,
        "mode": mode
    };

    // ... make TCP request to server ...
}

Register Command in src/cli/mod.rs

We'll need to register our new command so that the CLI knows about it and can execute it when called.

match subcmd.as_ref() {
    // ... existing commands
    "env" => cmd::env::run(&mut arguments).map(|_| DispatchResult::Dispatched),
    // ... rest
}

4. Automatic Restart Behavior

When you update the environment variables of a running egg, the server will need to restart it to apply the changes. Here's how this is going to work:

  1. Server sets egg status to Restarting: When the env is updated, the server will mark the egg as restarting.
  2. Main loop detects Restarting status: The main loop in the application monitors the status of the eggs.
  3. Kills current process: The server will terminate the existing process.
  4. Spawns new process with updated env: A new process is created with the updated environment variables.
  5. Status changes to Running: Once the new process is up and running, the egg's status is updated to Running.

This ensures the new environment variables take effect immediately.

5. State Persistence

All the changes are automatically persisted to your .kurv state file. This ensures that the environment variable updates are saved, even if the server restarts. The state sync mechanism in src/kurv/mod.rs:run() takes care of this.

Implementation Checklist

Let's break down the implementation steps:

Backend

  • [x] Create src/api/env.rs with update_env() function
  • [x] Add UpdateEnvRequest struct with mode validation
  • [x] Implement "replace" mode (full env replacement)
  • [x] Implement "merge" mode (partial env update)
  • [x] Add route POST /eggs/{egg_id}/env in src/api/mod.rs
  • [x] Handle automatic restart when egg is running
  • [x] Prevent env updates for plugins (403 error)
  • [x] Test state persistence after env update

CLI

  • [x] Create src/cli/cmd/env.rs
  • [x] Implement --set KEY=VALUE flag parsing
  • [x] Implement --replace flag
  • [x] Implement --from-file option for TOML env files
  • [x] Add TCP client logic to send request to server
  • [x] Register command in src/cli/mod.rs
  • [x] Add help text and examples

Documentation

  • [x] Update README.md with env update examples
  • [x] Document API endpoint in comments
  • [x] Add to .github/copilot-instructions.md

Usage Examples: Let's Get Practical

Time to see this in action! Here are some examples to show you how to use this feature.

CLI - Merge Mode (default):

# Update PORT, keep other env vars
kurv env myapp --set PORT=9090

# Update multiple vars
kurv env myapp --set PORT=9090 --set DEBUG=true

This will merge the new variables with the existing ones, only modifying the variables you specify, without affecting the other configurations.

CLI - Replace Mode:

# Replace entire env (removes all previous env vars)
kurv env myapp --replace PORT=9090 DEBUG=true

This is a complete reset of the environment variables. Any existing variables will be removed and replaced with the ones you specify.

CLI - From File:

# Load env from TOML file
kurv env myapp --from-file ./myapp.env.toml

Load environment variables directly from a TOML file. This makes managing multiple environment variables much easier.

API - Merge:

curl -X POST http://127.0.0.1:58787/eggs/1/env \
  -H "Content-Type: application/json" \
  -d '{"env": {"PORT": "9090"}, "mode": "merge"}'

This is a direct API call that merges the new PORT variable with any existing variables. It's great for automation or integrating with other services.

API - Replace:

curl -X POST http://127.0.0.1:58787/eggs/1/env \
  -H "Content-Type: application/json" \
  -d '{"env": {"PORT": "9090", "DEBUG": "true"}, "mode": "replace"}'

This API call replaces the entire environment. All existing variables are removed, and only the PORT and DEBUG variables will remain.

Edge Cases: Handling the Unexpected

We also need to consider some edge cases to make this feature robust:

  • Plugin eggs: If an egg is a plugin, it cannot have its environment updated. We will return a 403 error in this case.
  • Stopped eggs: If an egg is stopped, the environment variables will still be updated, but the egg will not be restarted. The new environment will be used when the egg is started again.
  • Restarting eggs: If an egg is already restarting, the environment update will be queued and applied after the restart is complete. This avoids any conflicts.
  • Empty env: We will allow the {"env": {}, "mode": "replace"} to clear all env variables.

References

Here are some references to the key parts of the code:

  • Current egg structure: src/kurv/egg/mod.rs
  • API routing: src/api/mod.rs
  • Egg spawning: src/kurv/spawn.rs (where the environment variables are passed to the Command)
  • CLI commands: src/cli/cmd/
  • Restart logic: Already implemented in the code.

That's it, guys! With this new feature, you'll be able to update environment variables with ease and efficiency, making your development process smoother than ever. Happy coding!