Ansible Project Initialization Script

Table of Contents

• Overview
• Prerequisites
• Quickstart (TL;DR)
• Getting Started (Step-by-step)
  • 1) Clone the repo
  • 2) Initialize a project using the script
  • 3) Review the project layout
  • 4) Configure Ansible basics
  • 5) Discover hosts (dynamic inventory)
  • 6) Bootstrap the ansible user (first run)
  • 7) Switch to normal operations
  • 8) Optional: Vault setup
  • 9) Optional: Non-interactive/CI usage
  • 10) Verify connectivity and run a playbook
• Major Features and Workflow
• Typical Usage
  • Overriding the SSH User for Initial Bootstrap
  • Non-Interactive / CI Mode
  • Dry-Run Behavior
• Advanced Features
• Ansible Vault Integration & Password Handling
• Security and Best Practices
• Using the Ansible Project after initialization
• Dynamic Inventory
  • Running the Dynamic Inventory
  • Advanced Scanner Flags (v1.0.1)
• Using the Setup Playbook
  • Prerequisites
  • Basic Usage
  • Connecting without password auth (SSH certs/keys)
  • Using Vault-Encrypted Secrets
  • Running Playbooks with Vault
  • Using the Root/Admin SSH Password from the Vault
  • Security Notes
  • Command Options
  • Examples
  • What the Playbook Does
  • Troubleshooting
• Git HTTPS Credentials Helper
• CI Docker Image Publishing
• Best Practices
• Troubleshooting
• License
• Contributing
• Roadmap

Overview

Back to TOC

This script bootstraps a new, production-ready Ansible project directory with best practices, robust idempotency, and user-friendly automation. It automates the creation of all required directories, files, SSH keys, and a dynamic inventory system, ensuring a consistent, secure, and ready-to-use Ansible environment.

Prerequisites

Back to TOC

• Ansible and Python 3 installed locally
• nmap (for host discovery used by inventory/discover_hosts.py)
• OpenSSH client
• Optional: python-dotenv if using --use-env with discovery (pip install python-dotenv)

Quickstart (TL;DR)

Back to TOC

1) Download Setup Script

curl -LO https://onedev.clfyed.stapel.io/ANIT/~raw/main/setup.sh

Add execute permissions:

chmod +x setup.sh

Pinned version example:

curl -LO https://onedev.clfyed.stapel.io/ANIT/~raw/v0.1.3-2/setup.sh
chmod +x setup.sh

1.2) Alternatively, Clone Source Repo

git clone https://onedev.clfyed.stapel.io/ANIT


<!-- ## Github
<!--   git clone https://github.com/Stapel-LLC/ANIT.git (Coming Soon) -->

cd ANIT

Installer usage (recommended)

Back to TOC

Use the installer setup.sh for both fresh installs and updates. It fetches the desired version, performs non-destructive updates by default, creates backups, and can run post-checks.

# Local execution (already cloned repo)
chmod +x ./setup.sh
./setup.sh --name MyProject --with-vault --dry-run   # preview
./setup.sh --name MyProject --with-vault             # execute

# Update an existing project
./setup.sh --project-dir /srv/infra --channel stable --dry-run
./setup.sh --project-dir /srv/infra --channel stable --post-check

# Lock to a specific tag (when tags are published)
./setup.sh --project-dir /srv/infra --version v0.1.3-3

# Remote one-liner (consider checksum verification in production)
curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/main/setup.sh -o /tmp/anit-setup.sh
sha256sum /tmp/anit-setup.sh   # verify manually or against a published checksum
bash /tmp/anit-setup.sh --name MyProject

Pinned one-liner:

curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/v0.1.3-3/setup.sh -o /tmp/anit-setup.sh
sha256sum /tmp/anit-setup.sh   # verify against published checksum when available
bash /tmp/anit-setup.sh --name MyProject

Checksum verification examples:

# Main latest
curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/main/setup.sh -o /tmp/anit-setup.sh
curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/main/setup.sh.sha256 -o /tmp/anit-setup.sh.sha256 || true
sha256sum -c /tmp/anit-setup.sh.sha256 || echo "No published checksum for main; compare manually if required"

# Pinned version
curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/v0.1.3-3/setup.sh -o /tmp/anit-setup.sh
curl -fsSL https://onedev.clfyed.stapel.io/ANIT/~raw/v0.1.3-3/setup.sh.sha256 -o /tmp/anit-setup.sh.sha256
sha256sum -c /tmp/anit-setup.sh.sha256

Initializer options via setup.sh

• The installer can optionally run the initializer after fetching/copying files.
• Use these flags to control behavior:
  • --run-init: run the initializer at the end
  • --skip-init: do not run the initializer (overrides --run-init)
  • --init-args "<args>": args to pass to the initializer, e.g. "-u admin -k /tmp/keys --with-vault"

Examples:

# Prompt at end whether to run initializer (interactive)
./setup.sh --project-dir /srv/infra --channel stable

# Run initializer with specific options
./setup.sh --project-dir /srv/infra \
  --run-init --init-args "-u admin -k /tmp/keys --with-vault"

# CI/non-interactive: require explicit args
./setup.sh --project-dir /srv/infra --non-interactive \
  --run-init --init-args "-u admin --with-vault"

# Skip initializer and run later manually
./setup.sh --project-dir /srv/infra --skip-init
(cd /srv/infra && ./scripts/init_ansible_project.sh MyProject -u admin)

Plugins on fresh install

• You can choose to copy the repository's plugins/ directory into your project during a fresh install.
• Flags:
  • --include-plugins: copy plugins/ without prompting.
  • --exclude-plugins: do not copy plugins/ (overrides include).
• Defaults:
  • Interactive: if no flag is provided, you will be prompted.
  • Non-interactive: defaults to not copying unless --include-plugins is provided.

Examples:

# Copy plugins on fresh install (interactive or CI)
./setup.sh --project-dir /srv/infra --include-plugins

# Explicitly do not copy (and suppress prompt)
./setup.sh --project-dir /srv/infra --exclude-plugins

Signed tags and verification

• Stable channel resolves to the latest signed tag by default; the installer verifies tag signatures.
• If your environment cannot verify signatures yet, use --insecure-verify to bypass verification temporarily.
• Recommended: configure GPG on your release host and use signed tags for all releases.

Getting started with plugins (no repo clone)

If you used the installer only (did not clone this repo), your fresh install initially contains just the scaffolding and initializer. To fetch the plugins and tooling into your project:

1. Sync plugins into your project (non-destructive)

# After a fresh install created /srv/infra (it contains a .anit marker)
./setup.sh --project-dir /srv/infra --channel stable --post-check

# Notes:
# - This update step syncs the repository contents into your project while protecting
#   user-owned areas (inventory/, group_vars/, host_vars/) by default.
# - Use --force if you want to overwrite protected files (not recommended).

2. Install Ansible collections for plugins

cd /srv/infra
ansible-galaxy collection install -r plugins/collections/requirements.lock.yml || \
  ansible-galaxy collection install -r plugins/collections/requirements.yml

3. Explore plugins with pluginctl

cd /srv/infra
python3 plugins/tools/pluginctl.py list --columns name,category,summary --sort name
python3 plugins/tools/pluginctl.py show configuration/traefik

4. Run a plugin

cd /srv/infra
# Example: deploy Traefik (see the plugin README for variables/labels and TLS options)
ansible-playbook -i inventory/hosts.ini plugins/configuration/traefik/playbook.yml \
  -e traefik_deploy=true

# Tip: standard tags are available: preflight, render, deploy, health
ansible-playbook -i inventory/hosts.ini plugins/configuration/traefik/playbook.yml \
  --tags deploy

5. Keep plugins up to date

# Non-destructive sync from the upstream repository
./setup.sh --project-dir /srv/infra --channel stable --dry-run    # preview
./setup.sh --project-dir /srv/infra --channel stable              # apply

For plugin development details, see plugins/README.md.

Post-install checklist

• Ensure plugins are present (if you chose not to copy on fresh install):
  • ./setup.sh --project-dir /srv/infra --channel stable (add --include-plugins to copy plugins on fresh)
• Install Ansible collections:
  • ansible-galaxy collection install -r plugins/collections/requirements.lock.yml
• Verify plugin tooling:
  • python3 plugins/tools/pluginctl.py list --columns name,category,summary --sort name
• Run the initializer if you deferred it:
  • (cd /srv/infra && ./scripts/init_ansible_project.sh MyProject -u admin)
• Configure Ansible basics:
  • Review ansible.cfg and group_vars/all.yml (see the “Configure Ansible basics” section)

2) Init a new project (see: Getting Started – Step 2)

chmod +x ./scripts/init_ansible_project.sh ./scripts/init_ansible_project.sh MyProject

3) Discover hosts

cd MyProject ./inventory/discover_hosts.py --network 192.168.1.0/24

4) Bootstrap ansible user (first run)

Recommended: wrapper installs Python first, then bootstraps user as root

ansible-playbook -i inventory/hosts.ini playbooks/bootstrap.yml
--limit <your_group> -u root --private-key ~/.ssh/<your_key>

Alternative (if Python already present):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml
--limit <your_group> -u root --private-key ~/.ssh/<your_key>

5) Normal ops

ansible-playbook -i inventory/hosts.ini playbooks/site.yml

Getting Started (Step-by-step)

Back to TOC

1) Download setup.sh or clone repo

Back to TOC

curl -LO https://onedev.clfyed.stapel.io/ANIT/~raw/main/setup.sh
chmod +x setup.sh
./setup.sh --name MyProject

or

git clone https://onedev.clfyed.stapel.io/ANIT
cd ANIT
chmod +x ./scripts/init_ansible_project.sh

2) Initialize a project using the script

Back to TOC

• Minimal interactive run:

./scripts/init_ansible_project.sh MyProject

• Common options:
  • -u <ssh-user> override initial SSH user
  • -k <key-dir> SSH key directory
  • -o <output-dir> parent output path
  • --with-vault create encrypted vault
  • --dry-run preview only
  • --force overwrite generated assets

Additional Examples:

# Basic project initialization
./scripts/init_ansible_project.sh my_project

# Custom key directory
./scripts/init_ansible_project.sh my_project -k /tmp/keys

# Custom SSH username
./scripts/init_ansible_project.sh my_project -u admin

# Custom SSH port
./scripts/init_ansible_project.sh my_project -p 2222

# Dry run mode
./scripts/init_ansible_project.sh my_project --dry-run

# Force overwrite existing keys/files
./scripts/init_ansible_project.sh my_project --force

# Vault integration (prompt for vault password)
./scripts/init_ansible_project.sh my_project --with-vault

# Vault integration (non-interactive, with env)
VAULT_PASSWORD=secret ./scripts/init_ansible_project.sh my_project --with-vault

# Vault + root pass via env (non-interactive)
VAULT_PASSWORD=secret ROOT_PASS=adminpass ./scripts/init_ansible_project.sh my_project --with-vault

# CI mode (suppress all prompts, all secrets via env)
CI=1 VAULT_PASSWORD=secret ./scripts/init_ansible_project.sh my_project --with-vault

# --- More Combined Option Examples ---

# Custom user, key directory, port, vault, and force overwrite
./scripts/init_ansible_project.sh my_project -u deployer -k /tmp/keys -p 2200 --with-vault --force

# CI mode with custom network, output dir, and vault
CI=1 VAULT_PASSWORD=secret ./scripts/init_ansible_project.sh -n 10.10.10.0/24 -o /tmp/ci_out my_project --with-vault

# Dry run with vault integration
./scripts/init_ansible_project.sh my_project --with-vault --dry-run

# Custom user, custom SSH key dir, force, vault, and debug
./scripts/init_ansible_project.sh my_project -u ansibleadmin -k /opt/sshkeys --with-vault --force --debug

# Fully loaded: CI, custom user, key dir, network, port, output dir, vault, force, debug, all secrets from env
CI=1 VAULT_PASSWORD=supersecret ROOT_PASS=rootpw ./scripts/init_ansible_project.sh -u admin -k /tmp/ci_keys -n 192.168.56.0/24 -p 2222 -o /tmp/ci_project my_project --with-vault --force --debug

# Create a test project in the testing directory
./scripts/init_ansible_project.sh -o testing/test_project test_project

# Create test project with a custom network range
./scripts/init_ansible_project.sh -o testing/test_project -n 192.168.1.0/24 test_project

Environment file (.env) and option precedence

Back to TOC

Use a .env file at the repo root (next to scripts/init_ansible_project.sh) to supply defaults without typing flags each run. You can start from .env.example and adjust values.

• Precedence (highest to lowest)

  • CLI flags (e.g., -u, -k, -n, -o, --with-vault)
  • Environment variables exported in the shell (e.g., export ANSIBLE_USER=...)
  • .env file values
  • Script built-in defaults

• Supported environment variables

  • PROJECT_NAME: Project directory name
  • ANSIBLE_USER: SSH username to configure and use
  • SSH_KEY_PATH: Directory for SSH keys; file will be <ANSIBLE_USER>_ed25519
  • OUTPUT_DIR: Parent directory where the project folder is created (default: .)
  • NETWORK_RANGE: One or more CIDR blocks (comma or space separated)
  • SSH_PORT: SSH port used for initial connectivity
  • CI or NO_INTERACTIVE: When set to 1, disables prompts (non-interactive)
  • VAULT_PASSWORD: Vault password for --with-vault flows
  • ROOT_PASS: Optional initial root/admin password to encrypt into vault
  • ENABLE_VAULT: Set to 1 to enable vault without passing --with-vault (flags still take precedence)

• Examples

cp .env.example .env
echo 'ANSIBLE_USER=admin' >> .env
echo 'NETWORK_RANGE="192.168.1.0/24 10.0.0.0/24"' >> .env

# Values from .env are picked up automatically
./scripts/init_ansible_project.sh MyProject

# CLI flags override .env
./scripts/init_ansible_project.sh -u deploy -o /srv/ansible MyProject

Option reference (initializer)

• Project and paths

  • PROJECT_NAME (positional): name of target project folder
  • -o, --output-dir <dir>: parent directory for the project (default: .)
  • -k, --key-path <dir>: directory where SSH key pair is stored (default: ~/.ssh)

• Connectivity

  • -u, --user <name>: initial SSH user name (default: ansible)
  • -p, --port <number>: SSH port (default: 22)
  • -n, --network <cidr[,cidr...]>: CIDR(s) for discovery

• Behavior

  • --with-vault | --no-vault: enable/disable vault scaffolding and encryption steps
  • --dry-run: preview actions; do not write files
  • --force: overwrite generated assets where applicable
  • -v, --verbose: more informational logs
  • -d, --debug: most detailed logs (implies verbose)

Non-interactive/CI variables

• Set CI=1 or NO_INTERACTIVE=1 to suppress prompts.
• Provide secrets via env vars:
  • VAULT_PASSWORD for vault initialization
  • ROOT_PASS to encrypt an initial root/admin password into vault (optional)

Example:

export CI=1
VAULT_PASSWORD=secret ROOT_PASS=adminpass \
  ./scripts/init_ansible_project.sh -n 10.0.0.0/24 -o /srv/ansible MyProject --with-vault

Variables and flags that are not read from environment

Back to TOC

• These are controlled by CLI flags only; setting them via env has no effect because the script initializes defaults internally:
  • --dry-run
  • --force
  • -v, --verbose
  • -d, --debug

3) Review the project layout

Back to TOC

The script creates a project directory with the following structure, take some time to become familiar with it:

• inventory/, playbooks/, roles/, group_vars/, host_vars/, ansible.cfg, etc.

4) Configure Ansible basics

Back to TOC

The script creates a basic ansible.cfg and group_vars/all.yml. You can use the following as a helpful starter and then iterate based on your workflow. For the full catalog of options, see the Ansible reference: https://docs.ansible.com/ansible/latest/reference_appendices/config.html#ansible-configuration-settings

Starter ansible.cfg (safe, readable defaults)

# ansible.cfg
[defaults]
inventory = inventory/hosts.ini
host_key_checking = False           # set True in hardened environments and manage known_hosts
forks = 20                          # tune for your control host and target size
timeout = 30
gathering = smart
stdout_callback = yaml               # human-friendly output
bin_ansible_callbacks = True
retry_files_enabled = False
roles_path = roles
collections_paths = collections
filter_plugins = filter_plugins

# Vault handling (choose one strategy)
# vault_password_file = .vault_pass.txt
# vault_identity_list = dev@~/.vault_pass_dev.txt, prod@/usr/local/bin/askpass-prod

[ssh_connection]
pipelining = True
control_path = %(directory)s/ansible-ssh-%%h-%%p-%%r
control_persist = 60s
# Optional SSH hardening/compat flags
# ssh_args = -o ControlMaster=auto -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

Project variables in group_vars/all.yml

# group_vars/all.yml
project_dir: /abs/path/to/project
ssh_key_dir: "{{ project_dir }}/.ssh"
ssh_private_key_dir: /abs/path/to/private/keys

# Managed account to create/configure during bootstrap
managed_user: ansible
managed_user_shell: /bin/bash
managed_user_public_key: "{{ lookup('file', ssh_key_dir + '/ansible_ed25519.pub') }}"

# Convenience variables some roles/plugins use
ansible_ssh_private_key_file: "{{ ssh_private_key_dir }}/ansible_ed25519"
ansible_ssh_public_key: "{{ lookup('file', ssh_key_dir + '/ansible_ed25519.pub') }}"

# Store sensitive values encrypted in group_vars/all/vault.yml (created with --with-vault)
# Example (encrypted):
# ansible_become_password: "supersecret"

Notes

• Do not define ansible_user globally here; set it per-inventory after bootstrap, e.g. inventory/group_vars/<group>.yml:

  ansible_user: "{{ managed_user }}"
  

• The installer creates a symlink inventory/group_vars -> ../group_vars so variables are always discovered properly.

Tips

• Keep secrets in group_vars/all/vault.yml (encrypted) and never in ansible.cfg. Configure vault unlocking via vault_password_file or vault_identity_list.
• For CI/non-interactive runs, prefer vault_identity_list scripts that fetch the secret from your secret manager.
• Set host_key_checking = True in hardened setups and manage known_hosts explicitly.
• Increase forks for larger fleets; decrease if your control host is resource constrained.

Vault identities (single vs multi-environment)

[defaults]
# Option A: simple local file (development)
# vault_password_file = .vault_pass.txt

# Option B: multiple identities (recommended for teams/environments)
vault_identity_list = dev@~/.vault_pass_dev.txt, prod@/usr/local/bin/askpass-prod

Manage known_hosts safely

# Add known hosts explicitly (recommended when host_key_checking = True)
- name: Add Git host to known_hosts
  known_hosts:
    name: github.com
    key: "{{ lookup('pipe', 'ssh-keyscan -t rsa github.com 2>/dev/null') }}"

Interpreter and fact caching

[defaults]
interpreter_python = auto_silent
fact_caching = jsonfile
fact_caching_connection = .ansible_cache
fact_caching_timeout = 7200

Strategy and forks

[defaults]
# linear: respect dependencies between tasks/hosts; free: more concurrency
strategy = linear
forks = 20

Collections install convention

# Install collections declared by the project into the local collections/ dir
ansible-galaxy collection install -r collections/requirements.yml -p collections

Debugging configuration

ansible --version
ansible-config view
ansible-config dump --only-changed

For examples of storing become/SSH passwords securely with Vault, see docs/USING_VAULT.md.

Security best practices

• Enable host_key_checking = True in production and maintain known_hosts (see example above).
• Use SSH keys, not passwords, and set strict permissions on private keys (600).
• Keep vault passwords out of source control. Prefer vault_identity_list referencing scripts that securely fetch secrets.
• Protect generated backups and .backup/ directories (permissions 700).
• Run playbooks with --check and --diff first on sensitive environments.

Per-environment configuration

• Organize variables per environment and pair them with vault identities:

group_vars/
  all.yml            # global defaults (non-secret)
  all/
    vault.yml        # global secrets (encrypted)
  dev/
    vars.yml         # dev-specific vars (non-secret)
    vault.yml        # dev secrets (encrypted)
  prod/
    vars.yml         # prod vars (non-secret)
    vault.yml        # prod secrets (encrypted)

Example vault identities mapping to envs:

[defaults]
vault_identity_list = dev@~/.vault_pass_dev.txt, prod@/usr/local/bin/askpass-prod

Then include env-specific files via inventory or playbooks (e.g., inventory groups for dev/prod).

5) Discover hosts (dynamic inventory)

Back to TOC

./inventory/discover_hosts.py --network 192.168.1.0/24
./inventory/discover_hosts.py --network 192.168.1.0/24 --tenant "Acme" --site "NYC-DC1"

6) Bootstrap the ansible user (first run)

Back to TOC

# Recommended wrapper (installs Python via raw, then runs bootstrap role)
ansible-playbook -i inventory/hosts.ini playbooks/bootstrap.yml \
  --limit <your_group> -u root --private-key ~/.ssh/<your_key>

# Alternative if Python is already available on targets
ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  --limit <your_group> -u root --private-key ~/.ssh/<your_key>

7) Switch to normal operations

Back to TOC

ansible-playbook -i inventory/hosts.ini playbooks/site.yml

8) Optional: Vault setup

Back to TOC

• If you used --with-vault, edit group_vars/all/vault.yml or see docs/USING_VAULT.md.

9) Optional: Non-interactive/CI usage

Back to TOC

export CI=1
VAULT_PASSWORD=secret ./scripts/init_ansible_project.sh MyProject --with-vault

10) Verify connectivity and run a playbook

Back to TOC

ansible -i inventory/hosts.ini all -m ping
ansible-playbook -i inventory/hosts.ini playbooks/site.yml -v

Major Features and Workflow

Back to TOC

1. Environment Setup & Defaults

Back to TOC

• Uses strict error handling (set -euo pipefail) for safety.
• Initializes and exports key option flags (DEBUG, VERBOSE, DRY_RUN, FORCE) for consistent behavior throughout the script.
• Loads environment variables from a .env file if present. Example file provided as .env.example.

2. Output Formatting and Logging

Back to TOC

• Provides color-coded log output for different levels (INFO, WARN, ERROR, DEBUG).
• Supports verbosity and debug flags for detailed troubleshooting.
• All actions and errors are logged with context-aware messages.

3. Argument Parsing

Back to TOC

• Accepts options for project name, SSH username, SSH key path, output directory, network range, SSH port, and feature flags (--force, --dry-run, --debug, --verbose).
• Validates and sanitizes all user input.

4. Project Directory Structure Creation

Back to TOC

• Creates a full, standardized Ansible project directory tree, including:
  • inventory/ for static and dynamic inventories
  • group_vars/ and host_vars/ for variable scoping
  • roles/common/ with all standard subdirectories (tasks/, handlers/, defaults/, vars/, files/, templates/)
  • playbooks/, collections/, filter_plugins/, and other best-practice folders
• Generates sample files such as ansible.cfg, site.yml, and variable YAMLs for groups and hosts.
• Copies .env.example into the project if present.

5. SSH Key Management

Back to TOC

• Generates a new Ed25519 SSH key pair for the project if one does not exist.
• Respects --force to overwrite existing keys, and --dry-run to preview actions.
• Ensures secure file permissions for all key material.

6. Dynamic Inventory Script Creation

Back to TOC

• Inlines a robust Python script (discover_hosts.py) into inventory/.
• This script:
  • Scans specified network ranges for live hosts and open ports using nmap.
  • Groups discovered hosts into categories (ssh_hosts, webservers, databases, unknown) based on open ports.
  • Outputs the inventory in JSON (for Ansible dynamic inventory use).
  • Intelligently updates hosts.ini: Merges newly discovered hosts into the correct groups, preserving existing entries and never removing or re-categorizing existing hosts.
  • Supports --debug, --verbose, and --dry-run for safe and transparent operation.
  • Provides robust error handling and logging.

7. Idempotency and Safety

Back to TOC

• All operations are idempotent: running the script multiple times will not duplicate or corrupt existing data.
• No destructive changes are made unless explicitly requested with --force.
• Dry-run mode allows users to see what would happen without making changes.

8. Backups

Back to TOC

• Before major changes, the script can create timestamped backups of itself for rollback and auditing.

Back to TOC

Typical Usage

Back to TOC

Overriding the SSH User for Initial Bootstrap

Back to TOC

Why override the user?

The ansible user (or your chosen automation user) does not exist on managed hosts until you run the initial setup playbook. For this first run, you must connect as an existing privileged user (like root or another user with sudo). After the user is created and keys are installed, switch to using the automation user for all future plays.

Example: Initial Bootstrap as root

# First run as root or sudoer
ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  -u root --private-key ~/.ssh/id_rsa -K

Back to TOC

Example: Bootstrap as an existing non-root user with sudo

# First run as an existing non-root user with sudo
ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  -u myexistinguser --private-key ~/.ssh/other_id -K

Back to TOC

Example: Normal usage (after ansible user is created)

# Normal usage after ansible user is created
ansible-playbook -i inventory/hosts.ini playbooks/site.yml

Back to TOC

Tip: You can always override the SSH user and key at runtime using -u <user> and --private-key <path> with any playbook.

Non-Interactive / CI Mode

Back to TOC

To run the script in non-interactive mode (for CI/CD or automation), set the environment variable CI=1 or NO_INTERACTIVE=1 before invoking the script:

export CI=1
./scripts/init_ansible_project.sh ...

• All interactive prompts will be suppressed (vault password, root/admin SSH password, vault password save prompt).
• Required secrets (e.g., VAULT_PASSWORD, ROOT_PASS) must be provided via environment variables.
• If a required secret is missing, the script will exit with an error instead of prompting.
• This is the recommended approach for all automated test and CI/CD environments.

bash ./scripts/init_ansible_project.sh -u ansibleuser -k ~/.ssh/ -o ./my_ansible_project MyProject --force --debug

• This will create a new project in ./my_ansible_project/MyProject, generate SSH keys, sample files, and a dynamic inventory system ready for immediate use.
• See more examples in Getting Started – Step 2.

Dry-Run Behavior

Back to TOC

• When invoked with --dry-run, the script performs a non-destructive preview.
• No files or directories are created, modified, or removed. The script logs actions it would take.
• Use --output-dir to indicate where the project would be scaffolded, without creating it during dry-run.

Examples:

# Preview a project setup without making any changes
./scripts/init_ansible_project.sh MyProject --dry-run

# Preview with a custom output directory (no files will be written)
./scripts/init_ansible_project.sh -o /tmp/out MyProject --dry-run

• See more examples in Getting Started – Step 2.

Advanced Features

Back to TOC

• Network Discovery: The dynamic inventory script can discover hosts on any specified network range and update the static inventory accordingly.
• Role Skeletons: Automatically creates a common role with all required subfolders and main.yml files for rapid role development.
• Extensible: The script is modular and can be extended to create additional roles, documentation templates, or CI/CD integration.

Ansible Vault Integration & Password Handling

Back to TOC

• When initializing with --with-vault, the script securely creates an encrypted vault.yml for secrets management.
• All vault operations (including root/admin SSH password encryption) use a secure in-memory password file for all ansible-vault commands; no static password file is required unless you choose to save one.
• The script is robust for both interactive and CI/CD (automation) workflows.
• The .vault_pass.txt file is only created if you choose to save the password for development; it is not required for automation.
• Root/admin SSH password encryption uses the same secure mechanism.
• Vault Password Handling:
  • If the VAULT_PASSWORD environment variable is not set, the script will prompt for a password only if running interactively (in a terminal).
  • If running in automation (non-interactive), and no password is supplied, the script will exit with an error and instructions to set VAULT_PASSWORD or provide a password file.
  • This ensures secure, robust, and automation-friendly behavior.
• Best Practices for Automation:
  • Set the VAULT_PASSWORD environment variable in your CI/CD or automation environment (never in source control).
  • Or, use a password file and the --vault-password-file option with Ansible:

    ansible-playbook --vault-password-file .vault_pass.txt playbook.yml
    

  • Or, configure vault_password_file = /secure/path/to/.vault_pass.txt in your ansible.cfg.
• Security Warning:
  • Never commit vault passwords or password files to source control.
  • Use environment variables or secure secrets management solutions in production.

Security and Best Practices

Back to TOC

• All sensitive files (e.g., SSH keys) are created with strict permissions.
• No existing keys or data are overwritten unless explicitly allowed.
• The script is safe to use in both development and production environments.

Using the Ansible Project after initialization

Back to TOC

Important: SSH must be running and accessible on the remote host for Ansible to connect and manage it. The initial playbook includes logic to bootstrap Python if it is missing (using the raw module), but you must be able to connect via SSH first.

After initializing an Ansible project with this script, you can use it to manage your infrastructure. The script provides a turnkey, best-practice Ansible project setup with dynamic and static inventory management, role scaffolding, secure SSH key handling, and robust idempotency. It is ideal for teams and individuals looking to standardize and accelerate their Ansible automation workflows.

├── filter_plugins/      # Custom filter plugins
└── .env.example        # Example environment variables

Dynamic Inventory

Back to TOC

The script includes a powerful dynamic inventory system that can:

• Discover hosts on your network using nmap
• Categorize hosts based on open ports
• Automatically update the inventory
• Support both static and dynamic inventory formats

Running the Dynamic Inventory

Back to TOC

# Run discovery and update inventory
./inventory/discover_hosts.py --network 192.168.1.0/24

# Optionally assign tenant and site variables to all discovered hosts:
./inventory/discover_hosts.py --network 192.168.1.0/24 --tenant "AcmeCorp" --site "NYC-DC1"
# The --tenant and --site options tag all discovered hosts with these variables for inventory and NetBox integration.

# Generate inventory in JSON format (for testing)
./inventory/discover_hosts.py --list

Advanced Scanner Flags (v1.0.1)

Back to TOC

The scanner now supports include/deny lists, concurrency, rate control, and lightweight caching:

• --include <IPs...>: Only process these IPs after sweep.
• --include-file <path>: File with IPs to include (one per line).
• --exclude <IPs...>: Skip these IPs.
• --exclude-file <path>: File with IPs to exclude (one per line).
• --use-env [ENV_PATH]: Load NETWORK/NETWORK_RANGE and other defaults from a .env file. If ENV_PATH is omitted, uses the repo/project .env located one directory above inventory/ (install python-dotenv).
• --max-workers <int>: Max concurrent host scans (default: 8).
• --rate <float>: Submission rate in hosts/second (default: 10.0).
• --cache-file <path>: Cache file path (default: inventory/.scan_cache.json).
• --cache-ttl <seconds>: Reuse cached port results within TTL.
• --no-cache: Disable cache even if TTL > 0.

Examples:

# Faster parallel scan with rate control
./inventory/discover_hosts.py --network 192.168.1.0/24 --max-workers 16 --rate 20.0

# Use include/exclude from files
./inventory/discover_hosts.py --use-env \
  --include-file inventory/include.txt \
  --exclude-file inventory/exclude.txt

# Reuse cached results for 1 hour
./inventory/discover_hosts.py --network 192.168.1.0/24 --cache-ttl 3600

# Force fresh scan even if cache exists
./inventory/discover_hosts.py --network 192.168.1.0/24 --cache-ttl 3600 --no-cache

Using the Setup Playbook

Back to TOC

After discovering your hosts using the dynamic inventory, you can use the setup_ansible_user.yml playbook to set up an Ansible user with passwordless sudo access on your target hosts.

Prerequisites

Back to TOC

1. Run the dynamic inventory to discover and populate your hosts
2. Ensure you have SSH access to the target hosts with a user that has sudo privileges
3. Your SSH public key is available at ~/.ssh/ansible_ed25519.pub (or update the playbook with your key path)

Basic Usage

Back to TOC

# Navigate to your project directory
cd /path/to/your/ansible_project

# First, update your inventory (if needed)
./inventory/discover_hosts.py --network 192.168.1.0/24

# Then run the setup playbook
ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml -K

The -K flag will prompt you for the sudo password of the remote user.

Connecting without password auth (SSH certs/keys)

Back to TOC

Some environments disable SSH password authentication entirely. In that case, run the bootstrap playbook using key-based authentication and, if applicable, OpenSSH certificates. Examples:

1. Use a specific private key (non-default path):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  -u root --private-key /secure/keys/root_id_ed25519

2. Use a non-root privileged user with sudo (passwordless sudo already configured):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  -u admin --private-key /secure/keys/admin_id_ed25519 --become

3. Use an OpenSSH certificate in a non-standard location together with a private key:

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  -u root \
  --private-key /secure/keys/root_id_ed25519 \
  -e 'ansible_ssh_common_args=-o CertificateFile=/secure/certs/root_id_ed25519-cert.pub'

4. Declare key/cert once via inventory or group vars (preferred for teams):

In group_vars/all.yml:

ansible_user: root
ansible_ssh_private_key_file: /secure/keys/root_id_ed25519
ansible_ssh_common_args: -o CertificateFile=/secure/certs/root_id_ed25519-cert.pub

Then run:

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml

5. Use SSH agent instead of passing key paths:

ssh-add /secure/keys/root_id_ed25519
ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml -u root

Notes:

• If you connect as root, you generally do not need -K/--become for the initial bootstrap.
• If connecting as a non-root user, ensure that user already has sudo privileges. Use --become and add -K only if sudo requires a password. When password auth is disabled, prefer users with passwordless sudo or run as root.
• You can also reference SSH settings via ~/.ssh/config; Ansible respects OpenSSH config by default.

Using Vault-Encrypted Secrets

Back to TOC

If you enabled vault integration during project initialization, secrets (such as the root/admin SSH password) are stored in group_vars/all/vault.yml and encrypted with your vault password.

Running Playbooks with Vault

Back to TOC

Interactive (prompt for vault password):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml --ask-vault-pass -K

Non-interactive/automation (provide vault password via file):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml --vault-password-file .vault_pass.txt -K

Non-interactive/automation (provide vault password via env):

VAULT_PASSWORD=your_vault_pass ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml --vault-password-file <(echo "$VAULT_PASSWORD") -K

Using the Root/Admin SSH Password from the Vault

Back to TOC

If you stored the root/admin SSH password in the vault during initialization, you can access it in your playbooks as the variable root_ssh_password.

Example usage in a playbook:

- name: Use root password from vault
  ansible.builtin.debug:
    msg: "Root password is {{ root_ssh_password }}"

Security Notes

Back to TOC

• Never commit .vault_pass.txt or your vault password to version control.
• For automation, always provide secrets via environment variables or secure vault password files.
• See .env.example for all supported environment variables.
• See group_vars/all/vault.yml for encrypted secrets structure.

Command Options

Back to TOC

Examples

Back to TOC

Run on specific hosts only:

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  --limit webservers -K

Dry run (no changes made):

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  --check -v

Use a different SSH key:

ansible-playbook -i inventory/hosts.ini playbooks/setup_ansible_user.yml \
  --extra-vars 'ansible_ssh_private_key_file=~/.ssh/custom_key' -K

What the Playbook Does

Back to TOC

1. Installs required packages (python3, sudo, openssh-server)
2. Creates the specified Ansible user
3. Configures passwordless sudo access
4. Sets up SSH key-based authentication

Troubleshooting

Back to TOC

• Permission Denied: Ensure your SSH key is properly configured and the remote user has sudo privileges
• Package Installation Fails: Check internet connectivity and package manager configuration on the target host
• SSH Connection Issues: Verify network connectivity and firewall settings

Git HTTPS Credentials Helper

Back to TOC

To securely store Git HTTPS credentials in your system keyring using the libsecret credential helper, this repo includes a helper script:

• Path: scripts/setup-git-https-credentials.sh
• Prerequisite: Configure Git to use libsecret (git config --global credential.helper libsecret).

Examples:

# Configure helper and save GitHub credentials for all repos
./scripts/setup-git-https-credentials.sh --set-helper --host github.com \
  --username YOUR_USERNAME --token YOUR_GITHUB_PAT

# Scope credentials to a single repository path
./scripts/setup-git-https-credentials.sh --host github.com --path OWNER/REPO \
  --username YOUR_USERNAME --token YOUR_GITHUB_PAT

# Show or remove stored credentials
./scripts/setup-git-https-credentials.sh --show --host github.com --username YOUR_USERNAME
./scripts/setup-git-https-credentials.sh --remove --host github.com --username YOUR_USERNAME

Notes:

• Use a Personal Access Token (PAT) as the password for GitHub/GitLab.
• If --token is omitted, the script will securely prompt for it.
• Credentials are stored by your desktop keyring via libsecret, not in plaintext files.

CI Docker Image Publishing

Back to TOC

This repo includes a reusable script to build and publish the CI agent Docker image to OneDev Packages.

• Path: ci/publish-image.sh
• Registry coordinates (two-level project path):
  • REGISTRY=onedev.clfyed.stapel.io
  • PROJECT=ANIT/anit-ci-agent
  • IMAGE_NAME=anit-ci-agent
  • Full image path: onedev.clfyed.stapel.io/ANIT/anit-ci-agent/anit-ci-agent:<tag>

Minimal local usage:

export REGISTRY=onedev.clfyed.stapel.io
export PROJECT=ANIT/anit-ci-agent
export IMAGE_NAME=anit-ci-agent
export TAG=v0.1.0

# Optional if not already logged in
export DOCKER_USER=YOUR_ONEDEV_USERNAME
export DOCKER_PAT=YOUR_ONEDEV_PAT

bash ci/publish-image.sh

Options (env):

• PUSH_LATEST=1 also pushes :latest.
• EXTRA_TAGS="stable,dev" adds more tags.
• BUILD_ARGS="KEY=VALUE" forwards build args to Docker.
• DOCKERFILE (default ci/Dockerfile) and CONTEXT (default .) are overridable.
• DRY_RUN=1 prints without executing.

OneDev CI wiring (summary):

• A job named Publish CI Image triggers on tag creation matching v* and runs the script.
• It derives TAG from the git tag and pushes:
  • onedev.clfyed.stapel.io/ANIT/anit-ci-agent/anit-ci-agent:<tag>
  • onedev.clfyed.stapel.io/ANIT/anit-ci-agent/anit-ci-agent:latest
• Required project secrets (create in OneDev):
  • onedev-docker-user
  • onedev-docker-pat

Pulling the image:

docker pull onedev.clfyed.stapel.io/ANIT/anit-ci-agent/anit-ci-agent:v0.1.0
docker pull onedev.clfyed.stapel.io/ANIT/anit-ci-agent/anit-ci-agent:latest

Notes:

• Ensure the runner has Docker access and trust for the registry TLS (install custom CA if applicable).
• Tags are treated as immutable; prefer creating a new tag for changes.

Best Practices

Back to TOC

1. Version Control: Initialize a git repository and commit your changes
2. Secrets Management: Never commit sensitive data to version control
3. Documentation: Update the README with project-specific information
4. Testing: Test playbooks in a development environment first
5. Backup: Regularly backup your inventory and configuration

Troubleshooting

Back to TOC

Common Issues

Back to TOC

1. Permission Denied

   • Ensure the script is executable: chmod +x scripts/init_ansible_project.sh
   • Run with sudo if needed for package installation

2. SSH Connection Issues

   • Verify SSH keys are properly configured
   • Check firewall settings and network connectivity

3. Package Installation Failures

   • Update your package lists: sudo apt update (Debian/Ubuntu) or sudo yum update (RHEL/CentOS)
   • Ensure you have internet connectivity

License

Back to TOC

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Back to TOC

Contributions are welcome! Please read our contributing guidelines before submitting pull requests.

Contributing / Pre-commit

To ensure plugin metadata stays valid and generated artifacts remain up to date, this repo includes pre-commit hooks.

Setup:

python3 -m pip install --upgrade pip pre-commit
pre-commit install

Run on all files:

pre-commit run -a

What the hooks do:

• plugins: validate metadata
  • Runs python plugins/tools/validate_plugins.py
• plugins: generate requirements and index
  • Runs python plugins/tools/generate_requirements_yml.py and python plugins/tools/generate_plugin_index.py
• plugins: ensure generated artifacts committed
  • Fails if diffs in plugins/collections/requirements.yml or plugins/PluginDirectory.md

Fixing failures:

• Re-run the generators locally and commit changes:

python plugins/tools/generate_requirements_yml.py
python plugins/tools/generate_plugin_index.py
git add plugins/collections/requirements.yml plugins/PluginDirectory.md
git commit -m "chore(plugins): regenerate requirements and index"

Roadmap

Back to TOC

See our ROADMAP.md for planned features and improvements.

Created with ❤️ by Stapel Dev