Merge branch 'caddy' into 'main'

Caddy

See merge request ujamii_dev/auto-dev-reverse-proxy!1
This commit is contained in:
Kevin Baensch 2023-01-18 09:13:35 +00:00
commit 6db91f6502
15 changed files with 694 additions and 251 deletions

6
.gitignore vendored
View file

@ -1,2 +1,4 @@
/etc/ssh/*key*
/etc/ssh/.ssh
etc/ssh/*key*
etc/ssh/.ssh/
config/
caddy_data/

139
README.md
View file

@ -1,23 +1,59 @@
# Starten des Reverse-Proxies
# First Steps
## Initiales Setup
Öffne ein Terminal und führe die `setup.sh` Datei aus.
Das Script:
1. Behebt Berechtigungsprobleme und legt alle nötigen Ordner/Dateien an, die zum starten des Proxies benötigt werden.
2. Installiert den `myssh` befehl nach `$HOME/bin` und fügt diesen ggf zur PATH variable hinzu
3. Erstellt SSH Keys für den SSH Docker Container (wenn diese nicht bereits existieren)
4. Erstellt und Konfiguriert einen Client SSH Key, dessen public Key wird dem SSH Docker Container hinzugefügt
## Starten des Reverse-Proxies
Der Proxy kann über docker compose gestartet werden
```bash
docker compose up -d
```
# Hinzufügen von Docker Containern
# Proxy Konfiguration
## HTTP-Proxy
### Hinzufügen von Docker Containern (Hosts)
Container werden automatisch vom reverse proxy/host manager aufgegriffen wenn sie:
1. Die Umgebungsvariable `VIRTUAL_HOST` gesetzt haben.
1. Das Label `local.web.host` gesetzt haben.
2. Im gleichen Docker-Netzwek (Default: `proxy`) sind.
Optional kann über die Umgebungsvariable `VIRTUAL_PORT` der gebundene Port gesetzt werden (Default: 80)
Optional kann über das Label `local.web.port` der gebundene Port gesetzt werden (Default: 80)
### SSL Zertifikat Konfiguration
Nach starten des docker containers ist das von caddy erstellte SSL Zertifikat in './caddy_data/pki/authorities/local/root.crt' gefunden werden.
Relevante Dokumentation:
- [MacOS](https://support.apple.com/guide/keychain-access/add-certificates-to-a-keychain-kyca2431/mac)
- [Linux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/sec-shared-system-certificates)
## SQL Proxy
### Hinzufügen von DB Docker Containern
Container werden automatisch vom reverse proxy/host manager aufgegriffen wenn sie:
1. Die Umgebungsvariable `local.db.type` (`mysql` oder `postgres`) und `local.db.host` gesetzt haben.
2. Im gleichen Docker-Netzwek (Default: `proxy`) sind.
Optional kann über das Label `local.web.port` der gebundene Port gesetzt werden (Default mysql: 3306, Default postgres: 5432)
## Minimale Beispiel-Konfiguration
In diesem Beispiel sind zwei Container, nur der `app` Kontainer ist teil des proxy Netzwerks, der `db` Container ist nicht im proxy Netzwerk.
In diesem Beispiel sind zwei Container, beide sind Teil des `proxy` Netzwerks.
Der `app` Container hat das Label `local.web.host` und wird deshalb als HTTP Proxy Target registriert.
Der `db` Container hat die Label `local.db.type` und `local.db.host` und wird deshalt als SQL Proxy Target registriert.
```yaml
version: "3.4"
services:
db:
image: mariadb:10.4
labels:
# SQL-Proxy Configuration
# Required labels
local.db.type: "mysql"
local.db.host: "db.myapp.localhost"
# Optional labels
local.db.port: "3306"
local.db.user: "db_user"
local.db.password: "db_user_pass"
environment:
MYSQL_DATABASE: db_name
MYSQL_USER: db_user
@ -26,22 +62,23 @@ services:
restart: unless-stopped
volumes:
- db-data:/var/lib/mysql
networks:
- default
- proxy
app:
image: some_base/image
depends_on:
- db
environment:
DB_TYPE: mysql
DB_HOST: db
DB_PORT: 3306
DB_USER: db_user
DB_PASS: db_user_pass
DB_NAME: db_name
VIRTUAL_HOST: app_host
VIRTUAL_PORT: 3000
labels:
# HTTP-Proxy Configuration
# Required Label
local.web.host: "myapp.localhost"
# "Optional" Label to set the correct port
local.web.port: "3000"
restart: unless-stopped
networks:
- default
- proxy
networks:
@ -58,26 +95,42 @@ networks:
external: true
```
## SQL Proxy
### Initiales Setup
Öffne ein Terminal und führe die `sqlproxy_setup.sh` Datei aus.
Das Script:
1. Installiert den `myssh` befehl nach `$HOME/bin` und fügt diesen ggf zur PATH variable hinzu
2. Erstellt SSH Keys für den SSH Docker Container (wenn diese nicht bereits existieren)
3. Erstellt und Konfiguriert einen Client SSH Key, dessen public Key wird dem SSH Docker Container hinzugefügt
## Fortgeschrittene Optionen (für docker compose)
### Hostman Script Umgebugsvariablen
- `DOCKER_SOCK_PATH`
- DEFAULT: `"/tmp/docker.sock"`
- DESCRIPTION: Docker Socket Pfad
- `NETWORK_NAME`
- DEFAULT: `"proxy"`
- DESCRIPTION: Docker netzwerk in dem nach Containern gesucht wird
### Starten des Reverse-Proxies
Der SQL Proxy kann über docker compose gestartet werden
```bash
docker compose -f docker-compose.yml -f docker-compose-sqlproxy.yml up -d
```
### Template Spezifische Umgebungsvariablen
#### Caddy
- `DOCKER_CADDY_NAME`
- DEFAULT: `proxy`
- DESCRIPTION: Der Caddy Container-/Host-Name unter dem das Caddy Admin Interface erreichbar ist.
- `DOCKER_CADDY_PORT`
- DEFAULT: `2020`
- DESCRIPTION: Der Port des Caddy Admin Interfaces.
### Hinzufügen von DB Docker Containern
Container werden automatisch vom reverse proxy/host manager aufgegriffen wenn sie:
1. Die Umgebungsvariable `DB_VHOST` gesetzt haben.
2. Im gleichen Docker-Netzwek (Default: `proxy`) sind.
#### Hosts
- `HOST_CONF_PATH`
- DEFAULT: `"/config/hosts"`
- DESCRIPTION: hosts Datei-Pfad
- `RESOLVE_DOCKERHOST`
- DEFAULT: `false`
- DESCRIPTION: Setzt ob IP Addressen in der hosts Datei auf die der Docker Container (true) oder 127.0.0.1 (false) aufgelöst werden.
- `DOCKER_HOSTNAME_VAR`
- DEFAULT: `"LOCAL_WEB_HOST"` (entspricht: `local.web.host`)
- DESCRIPTION: Docker Container Umgebungsvariable die den Hostnamen bestimmt (das auto generierte Öabel ist lower case und verwendet Punkte statt Unterstriche)
### myssh cli
#### SQLProxy
- `EXCLUDE_USERPASS`
- DEFAULT: `false`
- DESCRIPTION: Ob Nutzername und Passwort für die Authentifizierung ausgelassen werden soll.
# myssh cli
Der SQL Proxy Client hat folgende Optionen:
```bash
ls: Gibt eine Liste an verfügbaren DB Hosts zurück
@ -85,31 +138,11 @@ connect $DB_HOST [-u $USERNAME ] [-p $PASSWORD]: Erstellt einen Tunnel zum DB Ho
disconnect: schließt die SSH Multiplex Session und damit auch alle aktuellen Verbindungen
```
#### myssh Umgebungsvariablen
## myssh Umgebungsvariablen
- `SQL_PROXY_HOST`
- DEFAULT: `"localhost"`
- DESCRIPTION: Setzt den Target Proxy Host
- `SQL_PROXY_DB_PORT`
- DEFAULT: `"3306"`
- DESCRIPTION: Setzt den DB Host Target Port
- `SQL_CLI_TEMPLATE`
- (DEPRECATED) `SQL_CLI_TEMPLATE`
- DEFAULT LINUX: `'mysql --protocol=TCP -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -h localhost -P 3306'`
- DEFAULT MACOS: `'open \"mysql://$MYSQL_USERNAME:$MYSQL_PASSWORD@localhost:3306\" -a \"Sequel Ace\"'`
- DESCRIPTION: Setzt den auszuführenden Datenbank-Client Befehl
## Hostman Umgebugsvariablen (für docker compose)
- `DOCKER_SOCK_PATH`
- DEFAULT: `"/tmp/docker.sock"`
- DESCRIPTION: Docker Socket Pfad
- `NETWORK_NAME`
- DEFAULT: `"proxy"`
- DESCRIPTION: Docker netzwerk in dem nach Containern gesucht wird
- `RESOLVE_DOCKERHOST`
- DEFAULT: `false`
- DESCRIPTION: Setzt ob IP Addressen in der hosts Datei auf die der Docker Container (true) oder 127.0.0.1 (false) aufgelöst werden.
- `HOST_CONF_PATH`
- DEFAULT: `"/tmp/hosts"`
- DESCRIPTION: hosts Datei-Pfad
- `DOCKER_HOSTNAME_VAR`
- DEFAULT: `"VIRTUAL_HOST"`
- DESCRIPTION: Docker Container Umgebungsvariable die den Hostnamen bestimmt

View file

@ -1,25 +0,0 @@
version: "3.4"
services:
sshd:
build:
context: .
dockerfile: ./Dockerfile
command: ["./sqlproxy.sh", "&", "wait", "$!" ]
ports:
- 3022:22
volumes:
- ./etc/ssh:/etc/ssh/
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./script/hostman.sh:/hostman.sh:ro
- ./script/sqlproxy.sh:/sqlproxy.sh:ro
- ./script/sqlproxy_cli.sh:/sqlproxy_cli.sh:ro
environment:
DISABLE_KEYGEN: true
DISABLE_CONFIG_GEN: true
HOST_CONF_PATH: /etc/hosts
RESOLVE_DOCKERHOST: true
DOCKER_HOSTNAME_VAR: DB_VHOST
networks:
- proxy
restart: unless-stopped

View file

@ -2,21 +2,47 @@ version: "3.4"
services:
hostman:
image: apteno/alpine-jq
image: julienlecomte/docker-make
command: ["./hostman.sh", "&", "wait", "$!" ]
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /etc/hosts:/tmp/hosts:rw
- ./script/hostman.sh:/hostman.sh:ro
nginx-proxy:
image: jwilder/nginx-proxy
- ./script/docker-templater.sh:/docker-templater.sh:ro
- ./templates:/templates:ro
- ./config:/config:rw
- /etc/hosts:/config/hosts:rw
networks:
- proxy
restart: unless-stopped
proxy:
image: caddy:2.6.2-alpine
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./config/Caddyfile:/etc/caddy/Caddyfile:ro
- ./caddy_data:/data/caddy
networks:
- proxy
restart: unless-stopped
sshd:
build:
context: .
dockerfile: ./Dockerfile
command: ["./sqlproxy.sh", "&", "wait", "$!" ]
ports:
- 3022:22
volumes:
- ./etc/ssh:/etc/ssh/
- ./script/sqlproxy.sh:/sqlproxy.sh:ro
- ./script/sqlproxy_cli.sh:/sqlproxy_cli.sh:ro
- ./config:/config
environment:
DISABLE_KEYGEN: true
DISABLE_CONFIG_GEN: true
networks:
- proxy
restart: unless-stopped
networks:
proxy:

132
script/docker-templater.sh Executable file
View file

@ -0,0 +1,132 @@
#!/usr/bin/env bash
# Export Variables so they can be picked up by envsubst
set -ae
TEMPLATE_SRC=$1;
DOCKER_DATA=$2;
NETWORK_NAME="${NETWORK_NAME:-proxy}"
get_date() {
printf '%s' "$(date +'%Y.%m.%d_%H:%M:%S')"
}
# Import Template
# Required Variables:
# - TEMPLATE :: The template string which will be read by envsubst
# Optional Variables:
# - WRAP_START :: String to prepend in front of the mapped template result
# - WRAP_END :: String to append to the mapped template result
# - SEPARATOR :: String/Charater to separate TEMPLATE lines (DEFAULT: \n)
# - OUT :: if set the result will be written to $OUT instead of stdout
# Optional Functions:
# - label_hook :: modify/override label query results
# - template_hook :: hook to verify/modify template rsults (0 => OK; 1 => SKIP)
# - finally_hook :: gets executed after (and also only if) the final result has been written to disk
if [ -f "${TEMPLATE_SRC}" ]
then
source "${TEMPLATE_SRC}"
if [ -z "${TEMPLATE}" ]
then
printf '%s | [ERR]: Template "%s" does define a template string.\n' "$(get_date)" "${TEMPLATE_SRC}"
exit 1
fi
else
printf '%s | [ERR]: No such File "%s".\n' "$(get_date)" "${TEMPLATE_SRC}";
exit 1
fi
# Check Hooks
if type label_hook 2>/dev/null | grep -q "label_hook is a function"
then
RUN_LABEL_HOOK=true
else
RUN_LABEL_HOOK=false
fi
if type template_hook 2>/dev/null | grep -q "template_hook is a function"
then
RUN_TEMPLATE_HOOK=true
else
RUN_TEMPLATE_HOOK=false
fi
if type finally_hook 2>/dev/null | grep -q "finally_hook is a function"
then
RUN_FINALLY_HOOK=true
else
RUN_FINALLY_HOOK=false
fi
if [ -z "${SEPARATOR+.}" ]
then
SEPARATOR='\n'
fi
for row in $(jq -r '.[] | @base64' <<< "${DOCKER_DATA}")
do
CONTAINER_DATA=$(base64 -d <<< "${row}");
# Set non Label values
if [ -z "${LOCAL_IP}" ]
then
LOCAL_IP=$(jq -r '.NetworkSettings.Networks | if has($ENV.NETWORK_NAME) then .[$ENV.NETWORK_NAME].IPAddress else first(.[].IPAddress) end' <<< "${CONTAINER_DATA}");
fi
# Read Label values
CONTAINER_LABELS=$(jq '.Labels' <<< "${CONTAINER_DATA}");
for var in $(envsubst -v "${TEMPLATE}")
do
if [ -z "${!var}" ]
then
VAR_LABEL=$(tr '[:upper:]_' '[:lower:].' <<< "${var}")
LABEL_VAL=$(jq --arg KEY "${VAR_LABEL}" -r 'if has($KEY) then .[$KEY] else "" end' <<< "${CONTAINER_LABELS}")
if [ -n "${LABEL_VAL}" ]
then
declare "${var}"="${LABEL_VAL}";
fi
fi
done
if ${RUN_LABEL_HOOK}
then
label_hook
fi
PARTIAL_RESULT="$(envsubst <<< "${TEMPLATE}")";
# Verify template result if check_template is defined
if ${RUN_TEMPLATE_HOOK}
then
if template_hook
then
# FIX with printf
RESULT=$(printf '%s' "${RESULT}" "${RESULT:+$SEPARATOR}" "${PARTIAL_RESULT}")
fi
else
RESULT=$(printf '%s' "${RESULT}" "${RESULT:+$SEPARATOR}" "${PARTIAL_RESULT}")
fi
# Unset label-environment variables for next container
for var in $(envsubst -v "${TEMPLATE}")
do
unset "${var}"
done
done
RESULT="$(printf "%s" "${WRAP_START}" "${RESULT}" "${WRAP_END}")"
if [ -z "${OUT}" ]
then
printf "%b" "${RESULT}";
else
if [ "$(cat "${OUT}")" != "$(printf '%b'"${RESULT}")" ]
then
printf "%s | Template Task: '%s' has been written to: '%s'\n" "$(get_date)" "${TEMPLATE_SRC}" "${OUT}"
printf "%b" "${RESULT}" > "${OUT}";
if ${RUN_FINALLY_HOOK}
then
finally_hook
fi
fi
fi

View file

@ -1,54 +1,54 @@
#!/usr/bin/env sh
# Configurable Variables
[ -z $DOCKER_SOCK_PATH ] && DOCKER_SOCK_PATH="/tmp/docker.sock"
[ -z $NETWORK_NAME ] && NETWORK_NAME="proxy"
[ -z $RESOLVE_DOCKERHOST ] && RESOLVE_DOCKERHOST=false
[ -z $HOST_CONF_PATH ] && HOST_CONF_PATH="/tmp/hosts"
[ -z $DOCKER_HOSTNAME_VAR ] && DOCKER_HOSTNAME_VAR="VIRTUAL_HOST"
DOCKER_SOCK_PATH="${DOCKER_SOCK_PATH:-/tmp/docker.sock}"
NETWORK_NAME="${NETWORK_NAME:-proxy}"
# Path to Docker Templater and Templates
TEMPLATER_PATH="/docker-templater.sh"
TEMPLATE_FOLDER_PATH="/templates"
get_date() {
printf '%s' "$(date +'%Y.%m.%d_%H:%M:%S')"
}
query_docker () {
curl --unix-socket $DOCKER_SOCK_PATH --silent -g http://v1.41/$1$2
}
get_host_list() {
PROXY_HOST_CONF=""
CONTAINER_LIST=$(query_docker "containers/json" "?filters={%22network%22:[%22${NETWORK_NAME}%22],%22status%22:[%22running%22]}" | jq -cr '.[].Id')
for id in $CONTAINER_LIST
do
# Query individual container to access relevant data
CONTAINER_DATA=$(query_docker "containers/${id}/json")
if $RESOLVE_DOCKERHOST
update_templates() {
CONTAINER_LIST=$(query_docker "containers/json" "?filters={%22network%22:[%22${NETWORK_NAME}%22],%22status%22:[%22running%22]}")
LABELS_NEW="$(echo "${CONTAINER_LIST}" | jq '.[].Labels | to_entries | map(select(.key | startswith("local.")) | .key + .value) | sort | .[] | @base64')"
if [ "${LABELS_NEW}" != "${LABELS_OLD}" ]
then
if [ -n "${LABELS_OLD}" ]
then
HOST_IP=$(echo $CONTAINER_DATA | jq -cr '.NetworkSettings.Networks.proxy.IPAddress')
else
HOST_IP="127.0.0.1"
printf '%s | Container label list change detected.\n' "$(get_date)"
fi
# Filter Env for HOSTNAME, remove list parenthesis and split/only keep values
HOST_NAMES=$(echo $CONTAINER_DATA | jq -cr ".Config.Env[] | select(contains(\"$DOCKER_HOSTNAME_VAR=\")) | split(\"=\")[1]")
for hostname in $HOST_NAMES
LABELS_OLD="${LABELS_NEW}"
for template in "${TEMPLATE_FOLDER_PATH}"/*
do
PROXY_HOST_CONF="$PROXY_HOST_CONF\n$HOST_IP $hostname # Added by hostman"
if ! "${TEMPLATER_PATH}" "${template}" "${CONTAINER_LIST}"
then
printf '%s | Error Processing Template: %s\n' "$(get_date)" "${template}"
fi
done
done
echo $PROXY_HOST_CONF
fi
}
update_host_list() {
FILTERED_HOSTS=$(grep -ve "# Added by hostman$" $HOST_CONF_PATH)
echo -e "$FILTERED_HOSTS$(get_host_list)" > $HOST_CONF_PATH
}
# Initial Generation
update_templates
update_host_list
# cannot filter because reailine no longer recognized lines otherwise (check how IFS changes)
query_docker "events" | while true
LAST_CHECK="$(date +%s)"
while true
do
read -r;
# wait for related events to finish
while [ $? -eq 0 ]
EVENTS=""
printf '%s | Listening for docker events\n' "$(get_date)"
while [ -z "${EVENTS}" ]
do
read -t 5 -r;
NEXT_CHECK="$(($(date +%s) + 5))"
EVENTS=$(query_docker 'events' "?since=${LAST_CHECK}&until=${NEXT_CHECK}&filters={%22event%22:[%22die%22,%22kill%22,%22oom%22,%22pause%22,%22restart%22,%22start%22,%22stop%22,%22unpause%22,%22update%22]}")
LAST_CHECK="${NEXT_CHECK}"
done
update_host_list
printf '%s | Checking for changes in container label list.\n' "$(get_date)"
update_templates
done

View file

@ -1,25 +1,8 @@
#!/usr/bin/env bash
if [ -z "$SQL_CLI_TEMPLATE" ]
then
if [ $(uname -s) = "Linux" ]
then
SQL_CLI_TEMPLATE='mysql --protocol=TCP -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -h localhost -P 3306'
else
SQL_CLI_TEMPLATE='open \"mysql://$MYSQL_USERNAME:$MYSQL_PASSWORD@localhost:3306\" -a \"Sequel Ace\"'
fi
else
echo -e "Warning, custom client string:\n$SQL_CLI_TEMPLATE"
read -r -p "Continue [Y/n] " CONTINUE
if [[ ! $CONTINUE =~ "^[Yy]" ]]
then
exit
fi
fi
[ -z $SQL_PROXY_HOST ] && SQL_PROXY_HOST="localhost"
[ -z $SQL_PROXY_DB_PORT ] && SQL_PROXY_DB_PORT="3306"
CONNECTION_CACHE="$HOME/.cache/sqlproxy_$SQL_PROXY_HOST"
set -e
SQL_PROXY_HOST="${SQL_PROXY_HOST:-localhost}"
SSH_SQL_PROXY_HOST="sqlproxy.${SQL_PROXY_HOST}"
CONNECTION_CACHE="$HOME/.cache/sqlproxy_${SQL_PROXY_HOST}"
HELP="Usage: myssh [ls|connect]\n
SUBCOMMANDS:\n
@ -30,84 +13,197 @@ SUBCOMMANDS:\n
SYNTAX connect host [-u user] [-p password] [-c client]
"
get_template_string() {
if [ -z "$SQL_CLI_TEMPLATE" ]
then
if [ "$(uname -s)" = "Linux" ]
then
SQL_CLI_TEMPLATE='mysql --protocol=TCP -u $MYSQL_USERNAME -p$MYSQL_PASSWORD -h localhost -P 6033'
else
SQL_CLI_TEMPLATE='open \"mysql://$MYSQL_USERNAME:$MYSQL_PASSWORD@localhost:6033\" -a \"Sequel Ace\"'
fi
else
echo -e "Warning, custom client string:\n$SQL_CLI_TEMPLATE"
read -r -p "Continue [Y/n] " CONTINUE
if [[ ! $CONTINUE =~ "^[Yy]" ]]
then
exit
fi
fi
printf '%s' "${SQL_CLI_TEMPLATE}"
}
ssh_status() {
ssh -O check -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST > /dev/null 2>&1
echo $?
ssh -n -O check -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" > /dev/null 2>&1
}
connect() {
mkdir -p $HOME/.ssh/controlmasters
mkdir -p "${HOME}/.ssh/controlmasters"
if [ $(ssh_status) -ne 0 ]
if ! ssh_status
then
echo "" > $CONNECTION_CACHE
ssh -o "ControlPersist=10m" -M -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST q
echo "" > "${CONNECTION_CACHE}"
ssh -n -o "ControlPersist=10m" -M -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" q
fi
}
disconnect() {
if [ $(ssh_status) -eq 0 ]
if ssh_status
then
ssh -O stop -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST q
ssh -n -O stop -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" q
fi
}
# Establishes a port forward to the target host
# args:
# $1 - target ip
# $2 - target port
port_forward() {
ACTIVE_HOST=$(cat $CONNECTION_CACHE)
if [ -z $ACTIVE_HOST ] || [ $ACTIVE_HOST != $1:$SQL_PROXY_DB_PORT ]
ACTIVE_HOST=$(cat "${CONNECTION_CACHE}")
if [ -z "${ACTIVE_HOST}" ] || [ "${ACTIVE_HOST}" != "$1:$2" ]
then
echo "Reconnect"
if [ ! -z $ACTIVE_HOST ]
if [ -n "${ACTIVE_HOST}" ]
then
ssh -O cancel -L 3306:$ACTIVE_HOST -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST q
ssh -n -O cancel -L "6033:${ACTIVE_HOST}" -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" q
fi
ssh -O forward -L 3306:$1:$SQL_PROXY_DB_PORT -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST
ssh -n -O forward -L "6033:$1:$2" -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}"
fi
echo $1:$SQL_PROXY_DB_PORT > $CONNECTION_CACHE
echo "$1:$2" > "${CONNECTION_CACHE}"
}
ls_hosts() {
ssh -S $HOME/.ssh/controlmasters/%r@%h:%p $SQL_PROXY_HOST ls
ssh -n -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" ls
}
get_host() {
if [ "$1" = '' ]
then
printf 'Please specify the host to connect to.\nRun "myssh ls" to list all available hosts.\n'
exit 1
else
TARGET_HOST_DATA=$(ssh -n -S "${HOME}/.ssh/controlmasters/%r@%h:%p" "${SSH_SQL_PROXY_HOST}" "get $1")
if [ "${TARGET_HOST_DATA}" = '' ]
then
printf 'No such host: "%s"\n' "$1"
exit 1
fi
fi
}
# Checks and sets sql login variables
# args:
# $1 - sql type (mysql or psql)
# $2 - target ip
# $3 - target port
# $4 - username (optional)
# $5 - password (optional)
set_host_env() {
if [ "$1" = 'mysql' ] || [ "$1" = 'postgres' ]
then
TARGET_HOST_TYPE="$1"
else
printf 'Invalid Database type: "%s"\n' "$1"
exit 1
fi
if [[ "$2" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]
then
TARGET_HOST_IP="$2"
else
printf 'Invalid Host IP "%s" given.\n' "$2"
exit 1
fi
if [[ "$3" =~ ^[0-9]+$ ]]
then
TARGET_HOST_PORT="$3"
else
printf 'Invalid Host Port "%s" given.\n' "$3"
exit 1
fi
if [ -z "${TARGET_HOST_USERNAME}" ] && [ -n "$4" ]
then
TARGET_HOST_USERNAME="$4"
fi
if [ -z "${TARGET_HOST_PASSWORD}" ] && [ -n "$5" ]
then
TARGET_HOST_PASSWORD="$5"
fi
}
# Runs the sql client for an active connection
# args:
# $1 - sql type (mysql or psql)
# $2 - username
# $3 - password
run_client() {
$(eval echo $SQL_CLI_TEMPLATE)
if [ "$1" = 'mysql' ]
then
if which ace >/dev/null 2>&1
then
ace
elif which mysql >/dev/null 2>&1
then
mysql --protocol=TCP -u "$2" -p"$3" -h "${SQL_PROXY_HOST}" -P 6033
else
SHOW_CLI_HELP=true
fi
elif [ "$1" = 'postgres' ]
then
if which psql >/dev/null 2>&1
then
psql "postgresql://$2:$3@${SQL_PROXY_HOST}:6033/postgres"
else
SHOW_CLI_HELP=true
fi
fi
if [ "${SHOW_CLI_HELP}" = true ]
then
printf 'No %s client binary found.\nYou can maually establish a connection using the following data.\n' "$1"
printf 'host:\t%s\nport:\t%s\nuser:\t%s\npassword:\t%s\n' "${SQL_PROXY_HOST}" '6033' "$2" "$3"
fi
}
MAIN_OPTION=$1
shift
MAIN_OPTION="$1"
if [ -n "$1" ]
then
shift
fi
# ensure connection
connect
case $MAIN_OPTION in
case "${MAIN_OPTION}" in
ls)
ls_hosts;;
connect)
# check if host is valid
TARGET_HOST=$1
shift
ls_hosts | grep -qe "^$TARGET_HOST$"
GREP_EXIT_CODE=$?
if [ $GREP_EXIT_CODE -eq 0 ]
TARGET_HOST="$1"
if [ -n "$1" ]
then
while getopts "u:p:" o
do
case "$o" in
u) MYSQL_USERNAME="$OPTARG" ;;
p) MYSQL_PASSWORD="$OPTARG" ;;
esac
done
port_forward $TARGET_HOST
if [ ! -z $MYSQL_USERNAME ] && [ ! -z $MYSQL_PASSWORD ]
then
run_client
fi
shift
else
echo "Invalid Hostname: $TARGET_HOST."
printf 'No host specified.\n'
exit 1
fi
while getopts "u:p:" o
do
case "$o" in
u) TARGET_HOST_USERNAME="$OPTARG" ;;
p) TARGET_HOST_PASSWORD="$OPTARG" ;;
esac
done
get_host "${TARGET_HOST}"
# Do not quote this.
set_host_env ${TARGET_HOST_DATA}
port_forward "${TARGET_HOST_IP}" "$TARGET_HOST_PORT"
if [ -n "${TARGET_HOST_USERNAME}" ] && [ -n "${TARGET_HOST_PASSWORD}" ]
then
run_client "${TARGET_HOST_TYPE}" "${TARGET_HOST_USERNAME}" "${TARGET_HOST_PASSWORD}"
fi
;;
disconnect)
disconnect;;
*)
echo -e $HELP;;
echo -e "${HELP}";;
esac

View file

@ -5,4 +5,4 @@ chown -R sqlproxy:sqlproxy /etc/ssh/.ssh
chmod 0700 /etc/ssh/.ssh
chmod 0600 /etc/ssh/.ssh/authorized_keys
source ./hostman.sh
sleep infinity

View file

@ -1,18 +1,27 @@
#!/usr/bin/env sh
DB_DATA_FILE="${DB_DATA_FILE:-/config/sqlproxy.json}"
ls_hosts() {
# the containers version of grep does not support perl regex so "[^ ]*(?= # Added by hostman)" does not work
grep -e "# Added by hostman" /etc/hosts | grep -oe "^[^ ]* [^ ]*" | grep -oe "[^ ]*$"
jq -r '.[].host' < "${DB_DATA_FILE}"
}
get_host() {
export HOST=$(echo "${SSH_ORIGINAL_COMMAND}" | cut -d ' ' -f2)
if [ "${HOST}" != 'get' ]
then
jq -r 'first(.[] | select(.host == $ENV.HOST)) | [ .type, .ip, .port, .user, .password ] | join(" ")' < "${DB_DATA_FILE}"
fi
}
idle() {
echo "Press CTRL C to quit this connection"
printf "Press CTRL C to quit this connection\n"
sleep infinity
}
case "$SSH_ORIGINAL_COMMAND" in
case "${SSH_ORIGINAL_COMMAND}" in
"") idle;;
ls) ls_hosts;;
get*) get_host;;
q|quit) exit 0;;
*) exit 1;;
esac

130
setup.sh Executable file
View file

@ -0,0 +1,130 @@
#!/usr/bin/env bash
set -e
PROJECT_PATH=$(dirname $0)
WHOAMI="$(id -un)"
if [ "$(uname -s)" = "Linux" ]
then
MYGROUP="${WHOAMI}"
else
MYGROUP="staff"
fi
if [ $(id -u) -eq 0 ]
then
printf 'Do not run this script as root.\n'
exit 1
fi
check() {
read -r -p "$1" ANSWER
if [[ "${ANSWER}" =~ ^[Yy] ]]
then
return 0
fi
return 1
}
setup_base() {
printf 'Change ownership of "%s" to "%s"? (setup may fail otherwise)\n' "${PROJECT_PATH}" "${WHOAMI}"
printf 'running: "sudo chown -R %s %s"\n' "${WHOAMI}:${MYGROUP}" "${PROJECT_PATH}"
if check 'Continue? [Y/n] '
then
sudo chown -R "${WHOAMI}:${MYGROUP}" "${PROJECT_PATH}"
fi
mkdir -p "${PROJECT_PATH}/config" "${PROJECT_PATH}/caddy_data" "${PROJECT_PATH}/etc/ssh/.ssh"
touch "${PROJECT_PATH}/config/Caddyfile" "${PROJECT_PATH}/etc/ssh/.ssh/authorized_keys"
if [ "$(uname -s)" = 'Darwin' ] && [ ! -w '/etc/hosts' ]
then
printf 'On MacOS docker is run by your local user (not root).\nYour user has no write permission for "/etc/hosts".\nRunning: "sudo chown %s /etc/hosts"\n' "${WHOAMI}"
if check 'Continue? [Y/n] '
then
sudo chown "${WHOAMI}" '/etc/hosts'
fi
fi
}
setup_myssh() {
# Always copy newest version to bin
mkdir -p "${HOME}/bin"
cp "${PROJECT_PATH}/script/myssh" "${HOME}/bin/myssh"
# Detect Shell Init Path
if [[ "${SHELL}" =~ bin/bash$ ]]
then
RC_FILE=".bashrc"
elif [[ "${SHELL}" =~ bin/zsh$ ]]
then
RC_FILE=".zshrc"
else
printf 'Unable to detect Shell Configuration.\nPlease add %s to your PATH variable.\n' "${HOME}/bin"
return 0
fi
touch "${HOME}/${RC_FILE}"
if [ -f "${HOME}/${RC_FILE}" ] && [[ ! "${PATH}" =~ "${HOME}/bin" ]] && ! grep -qe '^PATH="${PATH}:${HOME}/bin"$' "${HOME}/${RC_FILE}" 2> /dev/null
then
printf 'PATH="${PATH}:${HOME}/bin"\n' >> "${HOME}/${RC_FILE}"
fi
}
setup_sqlproxy() {
if [ ! -f "${PROJECT_PATH}/etc/ssh/ssh_host_ed25519_key" ]
then
printf "Generating sqlproxy SSHD keys\n"
ssh-keygen -f "${PROJECT_PATH}" -A
fi
if check 'Auto generate client keys+config? [Y/n] '
then
mkdir -p "${HOME}/.ssh"
read -r -p 'Key Name (default: sqlproxy): ' KEY_NAME
KEY_NAME="${KEY_NAME:-sqlproxy}"
# Only add key if it does not already exist
if [ ! -f "${HOME}/.ssh/${KEY_NAME}" ]
then
ssh-keygen -t ed25519 -f "${HOME}/.ssh/${KEY_NAME}" -C "$(date +'%Y.%m.%d')_${WHOAMI}@${HOSTNAME}"
else
printf 'Key "%s" already exists. Using existing key.\n' "${HOME}/.ssh/${KEY_NAME}"
fi
read -r -p 'Target Host (default: "localhost"): ' HOST_NAME
HOST_NAME="${HOST_NAME:-localhost}"
# Check if there is an entry for $HOST_NAME in the users ssh config
if ! grep -qe "$(printf '^Host %s$' "sqlproxy.${HOST_NAME}")" "${HOME}/.ssh/config" 2>/dev/null
then
printf '\nHost sqlproxy.%s\n HostName %s\n Port 3022\n User sqlproxy\n IdentityFile ~/.ssh/%s' "${HOST_NAME}" "${HOST_NAME}" "${KEY_NAME}" >> "${HOME}/.ssh/config"
else
printf 'User ssh configuration located in "%s" already has a configuration for host "%s".\nMake sure your configuration matches the following:\n' "${HOME}/.ssh/config" "${HOST_NAME}"
printf '"""\nHost sqlproxy.%s\n HostName %s\n Port 3022\n User sqlproxy\n IdentityFile ~/.ssh/%s\n"""\n' "${HOST_NAME}" "${HOST_NAME}" "${KEY_NAME}"
fi
# Check if public key is already in the containers authorized_keys file
PUB_KEY="$(cat ${HOME}/.ssh/${KEY_NAME}.pub)"
if ! grep -qe "$(printf '%s$' "${PUB_KEY}")" "${PROJECT_PATH}/etc/ssh/.ssh/authorized_keys"
then
printf 'command="/sqlproxy_cli.sh" %s\n' "${PUB_KEY}" >> "${PROJECT_PATH}/etc/ssh/.ssh/authorized_keys"
fi
else
printf 'Not generating client ssh key.\nPlease put your desired public keys into %s\nAlso add %s in front of your key\n' "${PROJECT_PATH}/etc/ssh/.ssh/authorized_keys" "'command=\"/sqlproxy_cli.sh\" '"
fi
}
setup_base
if check 'Install myssh binary? [Y/n] '
then
setup_myssh
fi
if check 'Configure sql proxy? [Y/n] '
then
setup_sqlproxy
fi
printf 'Restarting sql proxy (if running) to fix permissions.\n'
docker compose --project-directory "${PROJECT_PATH}" -f "${PROJECT_PATH}/docker-compose.yml" restart sshd

View file

@ -1,65 +0,0 @@
#!/usr/bin/env bash
PROJECT_PATH=$(dirname $0)
# Always copy newest version to bin
mkdir -p $HOME/bin
cp $PROJECT_PATH/script/myssh $HOME/bin/myssh
# Detect Shell Init Path
if [[ $SHELL =~ bin/bash$ ]]
then
RC_FILE=.bashrc
elif [[ $SHELL =~ bin/zsh$ ]]
then
RC_FILE=.zshrc
fi
grep -qe '^PATH=$PATH:$HOME/bin$' $HOME/$RC_FILE 2> /dev/null
if [ ! -z $HOME/$RC_FILE ] && [[ ! $PATH =~ $HOME/bin ]] && [ $? -ne 0 ]
then
echo -e 'PATH=$PATH:$HOME/bin' >> $HOME/$RC_FILE
fi
if [ ! -f $PROJECT_PATH/etc/ssh/ssh_host_ed25519_key ]
then
echo "Generating sqlproxy SSHD keys"
ssh-keygen -f $PROJECT_PATH -A
fi
read -r -p "Auto generate client keys+config? [Y/n] " GEN_KEYS
case $GEN_KEYS in
[yY]*)
mkdir -p $HOME/.ssh
read -r -p "Key Name (will not be overridden if it already exists in ~/.ssh): " KEY_NAME
# Only add key if it does not already exist
if [ ! -f $HOME/.ssh/$KEY_NAME.key ]
then
ssh-keygen -t ed25519 -f $HOME/.ssh/$KEY_NAME.key -C "$(date --iso-8601)_$(whoami)@$HOSTNAME"
fi
read -r -p "Target Host: " HOST_NAME
# Check if there is an entry for $HOST_NAME in the users ssh config
grep -qe "^Host $HOST_NAME$" $HOME/.ssh/config
if [ $? -ne 0 ]
then
echo -ne "\nHost $HOST_NAME\n Port 3022\n User sqlproxy\n IdentityFile ~/.ssh/$KEY_NAME.key" >> $HOME/.ssh/config
fi
# Fix permssions if necessary
if [[ ! -w $PROJECT_PATH/etc/ssh/.ssh ]] || [[ ! $PROJECT_PATH/etc/ssh/.ssh/authorized_keys ]]
then
WHOAMI=$(id -un)
echo -e "Missing file permissions for authorized key file\nrunning: 'sudo chown -R $WHOAMI:$WHOAMI $PROJECT_PATH'"
sudo chown -R $WHOAMI:$WHOAMI $PROJECT_PATH
fi
# Check if public key is already in the containers authorized_keys file
grep -qe "$(cat $HOME/.ssh/$KEY_NAME.key.pub)$" $PROJECT_PATH/etc/ssh/.ssh/authorized_keys
if [ $? -ne 0 ]
then
echo -e command=\"/sqlproxy_cli.sh\" $(cat $HOME/.ssh/$KEY_NAME.key.pub) >> $PROJECT_PATH/etc/ssh/.ssh/authorized_keys
fi
# Restart sshd if permissions were changed
if [ ! -z $WHOAMI ]
then
docker compose --project-directory $PROJECT_PATH -f $PROJECT_PATH/docker-compose.yml -f $PROJECT_PATH/docker-compose-sqlproxy.yml restart sshd
fi;;
*) echo -e "Not generating client ssh key.\nPlease put your desired public keys into $PROJECT_PATH/etc/ssh/.ssh/authorized_keys\nAlso add 'command=\"/sqlproxy_cli.sh\" ' in front of your key";;
esac

31
templates/caddy.sh Normal file
View file

@ -0,0 +1,31 @@
#!/usr/bin/env bash
set -e
DOCKER_CADDY_NAME="${DOCKER_CADDY_NAME:-proxy}"
DOCKER_CADDY_PORT="${DOCKER_CADDY_PORT:-2020}"
WRAP_START='{\n\tadmin :2020\n}\n'
WRAP_END='\n'
TEMPLATE='${LOCAL_WEB_HOST} {\n\treverse_proxy ${LOCAL_IP}:$LOCAL_WEB_PORT\n}'
SEPARATOR='\n'
OUT='/config/Caddyfile'
label_hook() {
LOCAL_WEB_PORT="${LOCAL_WEB_PORT:-80}"
}
template_hook() {
if grep -q '^[^ ]\+ {\\n\\treverse_proxy \(?::\|[.0-9a-e]\)\+\:[0-9]\+\\n}$' <<< "${PARTIAL_RESULT}"
then
return 0;
fi
return 1;
}
finally_hook() {
if curl --silent "${DOCKER_CADDY_NAME}:${DOCKER_CADDY_PORT}/load" -H "Content-Type: text/caddyfile" --data-binary "@${OUT}"
then
printf '%s | Updated Caddy Config\n' "$(get_date)"
else
printf '%s | Failed to update Caddy Config\n' "$(get_date)"
fi
}

40
templates/hosts.sh Normal file
View file

@ -0,0 +1,40 @@
#!/usr/bin/env bash
set -e
HOST_CONF_PATH="${HOST_CONF_PATH:-/config/hosts}"
RESOLVE_DOCKERHOST="${RESOLVE_DOCKERHOST:-false}"
DOCKER_HOSTNAME_VAR="${DOCKER_HOSTNAME_VAR:-LOCAL_WEB_HOST}"
if [ -f "${HOST_CONF_PATH}" ]
then
WRAP_START=$(grep -ve "# Added by hostman$" "${HOST_CONF_PATH}")
WRAP_START+="\n"
else
printf "[WARN]: No such file or directory: %s\n" "${HOST_CONF_PATH}"
printf "Creating %s" "${HOST_CONF_PATH}"
touch "${HOST_CONF_PATH}"
fi
if ! $RESOLVE_DOCKERHOST
then
TEMPLATE='127.0.0.1 '
else
TEMPLATE='$LOCAL_IP '
fi
TEMPLATE="${TEMPLATE}\${${DOCKER_HOSTNAME_VAR}} # Added by hostman"
# Allow overriding out for debugging and testing purposs
if [ -z "${HOST_CONF_OUT+.}" ]
then
OUT="${HOST_CONF_PATH}"
else
OUT="${HOST_CONF_OUT}"
fi
template_hook() {
if grep -q '^[:.0-9a-e]\+ [^ ]\+ # Added by hostman$' <<< "${PARTIAL_RESULT}"
then
return 0;
fi
return 1;
}

34
templates/sqlproxy.sh Normal file
View file

@ -0,0 +1,34 @@
#!/usr/bin/env bash
EXCLUDE_USERPASS="${EXCLUDE_USERPASS:-false}"
set -e
WRAP_START='[\n'
if ${EXCLUDE_USERPASS}
then
TEMPLATE=' { "ip": "${LOCAL_IP}", "type": "${LOCAL_DB_TYPE}", "host": "${LOCAL_DB_HOST}", "port": "${LOCAL_DB_PORT}" }'
else
TEMPLATE=' { "ip": "${LOCAL_IP}", "type": "${LOCAL_DB_TYPE}", "user": "${LOCAL_DB_USER}", "password": "${LOCAL_DB_PASSWORD}", "host": "${LOCAL_DB_HOST}", "port": "${LOCAL_DB_PORT}" }'
fi
SEPARATOR=',\n'
WRAP_END='\n]'
OUT="/config/sqlproxy.json"
label_hook() {
if [ -z "${LOCAL_DB_PORT}" ]
then
if [ "${LOCAL_DB_TYPE}" = "mysql" ]
then
LOCAL_DB_PORT='3306'
elif [ "${LOCAL_DB_TYPE}" = "postgres" ]
then
LOCAL_DB_PORT='5432'
fi
fi
}
template_hook() {
if [ "$(jq '((.type == "mysql") or (.type == "postgres")) and (.host != "")' <<< "${PARTIAL_RESULT}" 2> /dev/null)" = true ]
then
return 0
fi
return 1
}