me/ncm
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

chore: blue/green documentation updates

Browse source
This commit is contained in:
Adriano Caloiaro 2026-07-18 07:33:34 -06:00
parent c6476a7657
commit 50457205e6
No known key found for this signature in database
2 changed files with 22 additions and 29 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
BLUE_GREEN_DEPLOYMENT.md
View file

@ -15,9 +15,11 @@ The NCM cluster now supports incremental node additions and removals, allowing y
Each node is defined in `deploy/compute_providers/linode/nodes.tf` with:
- **Unique name**: Stable identifier (e.g., `ncm-0`, `ncm-3`)
- **Generation label**: Version tag (e.g., `blue`, `green`, `v1`, `v2`)
- **Base image**: Can differ between nodes, allowing gradual image rollouts
- **Base image**: Can differ between nodes, allowing gradual image rollouts; defaults to `var.compute_provider_instance_base_image`
- **Node type**: Optional override for different instance sizes
> **Important**: Each node's private IP is deterministically derived from its name via a hash (`node_private_ips` in `nodes.tf`). Renaming a node changes its private IP, so treat node names as permanent once deployed.
## Workflow
### Phase 1: Add New Nodes (Green)
@ -47,21 +49,21 @@ locals {
# Existing blue generation (DON'T MODIFY THESE)
"ncm-0" = {
generation = "blue"
base_image = "private/11111111" # old image
base_image = var.compute_provider_instance_base_image
}
"ncm-1" = {
generation = "blue"
base_image = "private/11111111"
base_image = var.compute_provider_instance_base_image
}
"ncm-2" = {
generation = "blue"
base_image = "private/11111111"
base_image = var.compute_provider_instance_base_image
}
# NEW: Green generation nodes
"ncm-3" = {
generation = "green"
base_image = "private/12345678" # new image (or reuse old)
base_image = "private/12345678" # pin to new image, or use var for same image
}
"ncm-4" = {
generation = "green"
@ -87,23 +89,14 @@ tofu apply
#### 4. Prepare New Node Configurations
Update the `ncm-prepare` script to handle only new nodes. For now, manually prepare:
Run `ncm-prepare` from the repo root. It fetches all nodes from OpenTofu, skips any that already have a config directory (existing blue nodes), and prepares only the new ones:
```bash
# Get new node details
cd deploy/
tofu output -json cluster_node_details | jq '.value[] | select(.generation == "green")'
# For each new node, create config directory
for node in ncm-3 ncm-4 ncm-5; do
cp -r operate/templates/* operate/hosts/$node/
# Follow the ncm-prepare steps manually for each node
done
# Or run the full ncm-prepare (will regenerate all nodes)
ncm-prepare
```
It will scan SSH host keys, generate configs from templates, rekey secrets, and stage the changes with `git add` automatically.
#### 5. Deploy NixOS Configurations to New Nodes
```bash

24
deploy/compute_providers/linode/nodes.tf
View file

@ -10,18 +10,18 @@ locals {
# - node_type: override default node type (optional)
nodes = {
# Current generation nodes
"ncm-3" = {
generation = "green"
base_image = var.compute_provider_instance_base_image
}
"ncm-4" = {
generation = "green"
base_image = var.compute_provider_instance_base_image
}
"ncm-5" = {
generation = "green"
base_image = var.compute_provider_instance_base_image
}
# "ncm-3" = {
# generation = "green"
# base_image = var.compute_provider_instance_base_image
# }
# "ncm-4" = {
# generation = "green"
# base_image = var.compute_provider_instance_base_image
# }
# "ncm-5" = {
# generation = "green"
# base_image = var.compute_provider_instance_base_image
# }
# New blue generation nodes
"ncm-0" = {
generation = "blue"