ncm/jobs/README.md

Jobs

This module deploys workloads to the Nomad cluster using Terraform and Nomad job specifications.

Overview

The jobs module:

• Manages Nomad job lifecycle via Terraform
• Deploys containerized applications
• Configures Nomad namespaces and variables
• Provisions Nomad ACL policies

Structure

jobs/
├── jobspecs/         # Nomad job specifications (HCL)
├── nomad_policies/   # Nomad ACL policies
├── main.tf           # Terraform configuration
└── variables.tf      # Terraform variables

Available Jobs

Job	Description	Ports
caddy	Reverse proxy and HTTPS termination	80, 443
postgres	PostgreSQL database	5432
miniflux-rss-reader	RSS feed reader	-
backups	Automated backup service	-
linode-csi-controller	Linode block storage CSI driver	-
opencloud	Custom application	-

Prerequisites

1. Nomad cluster running (deployed via operate/ module)
2. Nomad server accessible
3. Required secrets configured in Nomad variables

Quick Start

1. Configure Environment

Set Nomad connection details:

export TF_VAR_nomad_server_url="http://<nomad-node-ip>:4646"

Or create a .env file with required variables.

2. Initialize Terraform

cd jobs/
tofu init

3. Deploy Jobs

# Plan deployment
tofu plan

# Apply changes
tofu apply

4. Verify Deployment

# List jobs via Nomad CLI
nomad job status

# Check specific job
nomad job status caddy

# View allocations
nomad alloc status <alloc-id>

Managing Jobs

Deploy Single Job

Edit main.tf to include only desired jobs, then:

tofu apply -target=nomad_job.caddy

Update Job Specification

1. Edit the job file in jobspecs/
2. Apply changes:

   tofu apply
   

Remove Job

Comment out or remove the job resource in main.tf, then:

tofu apply

Or stop directly via Nomad:

nomad job stop <job-name>

View Job Logs

nomad alloc logs -f <alloc-id>

Job Specifications

Caddy (Reverse Proxy)

File: jobspecs/caddy.nomad.hcl

Features:

• Automatic HTTPS with Let's Encrypt
• Reverse proxy for services
• Static file serving
• Configuration via Caddyfile

PostgreSQL

File: jobspecs/postgres.hcl

Features:

• Persistent storage with Nomad volumes
• Database initialization
• Network-accessible on private network

Miniflux (RSS Reader)

File: jobspecs/miniflux-rss-reader.hcl

Features:

• Lightweight RSS reader
• PostgreSQL backend
• Web interface

Backups

File: jobspecs/backups.nomad.hcl

Features:

• Scheduled backup jobs
• Cloud storage integration
• Automated retention policies

Linode CSI Controller

File: jobspecs/linode-csi-controller.nomad.hcl

Features:

• Linode block storage integration
• Dynamic volume provisioning
• Volume snapshots

Terraform Configuration

State Backend

Job state is stored in S3-compatible object storage (Linode Object Storage).

Configure credentials:

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"

Remote State

The module reads infrastructure details from the deploy module's remote state:

data "terraform_remote_state" "compute" {
  backend = "s3"
  # ... configuration
}

Nomad Variables

Secrets and configuration are stored as Nomad variables. Create via Terraform:

resource "nomad_variable" "example" {
  path = "nomad/jobs/my-app"
  items = {
    api_key = "secret-value"
  }
}

Access in job specs:

env {
  API_KEY = {{ with nomadVar "nomad/jobs/my-app" }}{{ .api_key }}{{ end }}
}

Troubleshooting

Job Placement Failures

Check node resources:

nomad node status

View job constraints:

nomad job inspect <job-name>

Container Image Pull Failures

Verify internet connectivity from nodes:

nomad alloc logs <alloc-id>

Service Registration Issues

Check Nomad logs:

nomad alloc logs <alloc-id>
journalctl -u nomad -f  # on the node

Volume Mount Failures

Ensure CSI controller is running:

nomad job status linode-csi-controller

Check volume status:

nomad volume status