Nomad (Server)

Overview

This is a flavour containing the nomad service orchestrator.

This flavour is configured as orchestration server. You need one or more nomad client instances that connect to this server and actually run the workload.

Since clients need to run jobs e.g. via pot, you need to install the client directly on a host; therefore you will not find a nomad client flavour on potluck.

Together with consul and the traefik pot flavours on this site, you can easily set up a virtual datacenter.

Please note that a specific network configuration is suggested (see Installation-chapter) as this jail does not work behind the NAT of a public-bridge.

Installation

  • Create your local jail from the image or the flavour files.
  • This jail does not work with a public bridge, so clone it to use an IP address directly on your host:
    sudo pot clone -P <nameofimportedjail> -p <clonejailname> -N alias -i "<interface>|<ipaddress>"
    e.g. sudo pot clone -P nomad-server-amd64-12_1_1_0 -p my-nomad-server -N alias -i "em0|10.10.10.11"
  • Adjust to your environment:
    sudo pot set-env -p <clonejailname> -E DATACENTER=<datacentername> -E IP=<IP address of this nomad instance> -E CONSULSERVER=<IP or hostname of consulserver>

Usage

You can connect to the dashboard on port 4646 of your jail IP address.

To run a new job, connect to the jail via pot term <jailname> and run a nomad job description via nomad run -address=http://<jailip>:4646 <jobfile> or place the job via the dashboard.

How To Use The Ready-Made Image

FreeBSD 12.1:
pot import -p nomad-server-amd64-12_1 -t 1.0.1 -U https://potluck.honeyguide.net/nomad-server

FreeBSD 11.4:
pot import -p nomad-server-amd64-11_4 -t 1.0.1 -U https://potluck.honeyguide.net/nomad-server

Note: Some images might require specific network configuration, double check the Overview-chapter at the top.

Alternatively: Create a Jail With This Flavour Yourself

1. Create Flavour Files

Create the following
/usr/local/etc/pot/flavours/nomad-server.sh
and
chmod ugo+x /usr/local/etc/pot/flavours/nomad-server.sh

#!/bin/sh

# EDIT THE FOLLOWING FOR NEW FLAVOUR:
# 1. RUNS_IN_NOMAD - yes or no
# 2. Adjust package installation between BEGIN & END PACKAGE SETUP
# 3. Adjust jail configuration script generation between BEGIN & END COOK

# Set this to true if this jail flavour is to be created as a nomad (i.e. blocking) jail.
# You can then query it in the cook script generation below and the script is installed
# appropriately at the end of this script 
RUNS_IN_NOMAD=false

# -------------- BEGIN PACKAGE SETUP -------------
[ -w /etc/pkg/FreeBSD.conf ] && sed -i '' 's/quarterly/latest/' /etc/pkg/FreeBSD.conf
ASSUME_ALWAYS_YES=yes pkg bootstrap
touch /etc/rc.conf
sysrc sendmail_enable="NO"
sysrc nomad_enable="YES"
sysrc nomad_user="root"
sysrc nomad_env="PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin"

# Install packages
pkg install -y nomad 
pkg clean -y
# -------------- END PACKAGE SETUP -------------

#
# Create configurations
#

#
# Now generate the run command script "cook"
# It configures the system on the first run by creating the config file(s) 
# On subsequent runs, it only starts sleeps (if nomad-jail) or simply exits 
#

# ----------------- BEGIN COOK ------------------ 
echo "#!/bin/sh

# No need to change this, just ensures configuration is done only once
if [ -e /usr/local/etc/pot-is-seasoned ]
then
    # If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
    # created by pot and we block indefinitely
    if [ ! -e /tmp/environment.sh ]
    then
        tail -f /dev/null 
    fi
    exit 0
fi

# ADJUST THIS: STOP SERVICES AS NEEDED BEFORE CONFIGURATION
/usr/local/etc/rc.d/nomad stop  || true

# No need to adjust this:
# If this pot flavour is not blocking, we need to read the environment first from /tmp/environment.sh
# where pot is storing it in this case
if [ -e /tmp/environment.sh ]
then
    . /tmp/environment.sh
fi

#
# ADJUST THIS BY CHECKING FOR ALL VARIABLES YOUR FLAVOUR NEEDS:
# Check config variables are set
#
if [ -z \${DATACENTER+x} ]; 
then 
    echo 'DATACENTER is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${IP+x} ]; 
then 
    echo 'IP is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${CONSULSERVER+x} ];
then
    echo 'CONSULSERVER is unset - see documentation how to configure this flavour'
    exit 1
fi

# ADJUST THIS BELOW: NOW ALL THE CONFIGURATION FILES NEED TO BE CREATED:
# Don't forget to double(!)-escape quotes and dollar signs in the config files

# Create nomad server config file 
echo \"
bind_addr = \\\"\$IP\\\"
plugin_dir = \\\"/usr/local/libexec/nomad/plugins\\\"
datacenter = \\\"\$DATACENTER\\\"

advertise {
  # This should be the IP of THIS MACHINE and must be routable by every node
  # in your cluster
  http = \\\"\$IP:4646\\\"
}

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  # The address to the Consul agent.
  address = \\\"\$CONSULSERVER:8500\\\"

  # The service name to register the server and client with Consul.
  server_service_name = \\\"\$DATACENTER-server\\\"

  # Enables automatically registering the services.
  auto_advertise = true

  # Enabling the server and client to bootstrap using Consul.
  server_auto_join = true
}

enable_syslog=true
log_level=\\\"INFO\\\"
syslog_facility=\\\"LOCAL1\\\"\" > /usr/local/etc/nomad/server.hcl
echo \"nomad_args=\\\"-config=/usr/local/etc/nomad/server.hcl -network-interface=\$IP\\\"\" >> /etc/rc.conf

# ADJUST THIS: START THE SERVICES AGAIN AFTER CONFIGURATION
/usr/local/etc/rc.d/nomad start

# Do not touch this:
touch /usr/local/etc/pot-is-seasoned

# If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
# created by pot and we now after configuration block indefinitely
if [ ! -e /tmp/environment.sh ]
then
    tail -f /dev/null
fi
" > /usr/local/bin/cook

# ----------------- END COOK ------------------


# ---------- NO NEED TO EDIT BELOW ------------

chmod u+x /usr/local/bin/cook

#
# There are two ways of running a pot jail: "Normal", non-blocking mode and
# "Nomad", i.e. blocking mode (the pot start command does not return until
# the jail is stopped).
# For the normal mode, we create a /usr/local/etc/rc.d script that starts
# the "cook" script generated above each time, for the "Nomad" mode, the cook
# script is started by pot (configuration through flavour file), therefore
# we do not need to do anything here.
# 

# Create rc.d script for "normal" mode:
echo "#!/bin/sh

#
# PROVIDE: cook 
# REQUIRE: LOGIN
# KEYWORD: shutdown
#

. /etc/rc.subr

name=cook
rcvar=cook_enable

load_rc_config $name

: ${cook_enable:=\"NO\"}
: ${cook_env:=\"\"}

command=\"/usr/local/bin/cook\"
command_args=\"\"

run_rc_command \"\$1\"
" > /usr/local/etc/rc.d/cook

chmod u+x /usr/local/etc/rc.d/cook

if [ $RUNS_IN_NOMAD = false ]
then
    # This is a non-nomad (non-blocking) jail, so we need to make sure the script
    # gets started when the jail is started:
    # Otherwise, /usr/local/bin/cook will be set as start script by the pot flavour
    echo "cook_enable=\"YES\"" >> /etc/rc.conf
fi

2. Create Jail From Flavour

Run
pot create -b <FreeBSD Version> -p <jailname> -t single -N public-bridge -f fbsd-update -f nomad-server

with your FreeBSD version (e.g. 12.1) and the name your jail should get.

Note: Some images might require specific network configuration, double check the Overview-chapter at the top.

Version History

1.0.1

  • Fixed typo because of which DATACENTER was not set correctly

1.0

  • Initial commit

These images were built on Mon Jul 27 11:51:21 UTC 2020

Manual Image Download Links

nomad-server-amd64-12_1_1.0.1.xz ( 214.062 MB )
nomad-server-amd64-12_1_1.0.1.xz.skein ( 0.250977 KB )

nomad-server-amd64-11_4_1.0.1.xz ( 178.863 MB )
nomad-server-amd64-11_4_1.0.1.xz.skein ( 0.250977 KB )

Jenkins Pot Creation Logs

nomad-server-amd64-12_1_1.0.1:


nomad-server/nomad-server:
nomad-server/nomad-server.sh:
#!/bin/sh

# EDIT THE FOLLOWING FOR NEW FLAVOUR:
# 1. RUNS_IN_NOMAD - yes or no
# 2. Adjust package installation between BEGIN & END PACKAGE SETUP
# 3. Adjust jail configuration script generation between BEGIN & END COOK

# Set this to true if this jail flavour is to be created as a nomad (i.e. blocking) jail.
# You can then query it in the cook script generation below and the script is installed
# appropriately at the end of this script 
RUNS_IN_NOMAD=false

# -------------- BEGIN PACKAGE SETUP -------------
[ -w /etc/pkg/FreeBSD.conf ] && sed -i '' 's/quarterly/latest/' /etc/pkg/FreeBSD.conf
ASSUME_ALWAYS_YES=yes pkg bootstrap
touch /etc/rc.conf
sysrc sendmail_enable="NO"
sysrc nomad_enable="YES"
sysrc nomad_user="root"
sysrc nomad_env="PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin"

# Install packages
pkg install -y nomad 
pkg clean -y
# -------------- END PACKAGE SETUP -------------

#
# Create configurations
#

#
# Now generate the run command script "cook"
# It configures the system on the first run by creating the config file(s) 
# On subsequent runs, it only starts sleeps (if nomad-jail) or simply exits 
#

# ----------------- BEGIN COOK ------------------ 
echo "#!/bin/sh

# No need to change this, just ensures configuration is done only once
if [ -e /usr/local/etc/pot-is-seasoned ]
then
    # If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
    # created by pot and we block indefinitely
    if [ ! -e /tmp/environment.sh ]
    then
        tail -f /dev/null 
    fi
    exit 0
fi

# ADJUST THIS: STOP SERVICES AS NEEDED BEFORE CONFIGURATION
/usr/local/etc/rc.d/nomad stop  || true

# No need to adjust this:
# If this pot flavour is not blocking, we need to read the environment first from /tmp/environment.sh
# where pot is storing it in this case
if [ -e /tmp/environment.sh ]
then
    . /tmp/environment.sh
fi

#
# ADJUST THIS BY CHECKING FOR ALL VARIABLES YOUR FLAVOUR NEEDS:
# Check config variables are set
#
if [ -z \${DATACENTER+x} ]; 
then 
    echo 'DATACENTER is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${IP+x} ]; 
then 
    echo 'IP is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${CONSULSERVER+x} ];
then
    echo 'CONSULSERVER is unset - see documentation how to configure this flavour'
    exit 1
fi

# ADJUST THIS BELOW: NOW ALL THE CONFIGURATION FILES NEED TO BE CREATED:
# Don't forget to double(!)-escape quotes and dollar signs in the config files

# Create nomad server config file 
echo \"
bind_addr = \\\"\$IP\\\"
plugin_dir = \\\"/usr/local/libexec/nomad/plugins\\\"
datacenter = \\\"\$DATACENTER\\\"

advertise {
  # This should be the IP of THIS MACHINE and must be routable by every node
  # in your cluster
  http = \\\"\$IP:4646\\\"
}

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  # The address to the Consul agent.
  address = \\\"\$CONSULSERVER:8500\\\"

  # The service name to register the server and client with Consul.
  server_service_name = \\\"\$DATACENTER-server\\\"

  # Enables automatically registering the services.
  auto_advertise = true

  # Enabling the server and client to bootstrap using Consul.
  server_auto_join = true
}

enable_syslog=true
log_level=\\\"INFO\\\"
syslog_facility=\\\"LOCAL1\\\"\" > /usr/local/etc/nomad/server.hcl
echo \"nomad_args=\\\"-config=/usr/local/etc/nomad/server.hcl -network-interface=\$IP\\\"\" >> /etc/rc.conf

# ADJUST THIS: START THE SERVICES AGAIN AFTER CONFIGURATION
/usr/local/etc/rc.d/nomad start

# Do not touch this:
touch /usr/local/etc/pot-is-seasoned

# If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
# created by pot and we now after configuration block indefinitely
if [ ! -e /tmp/environment.sh ]
then
    tail -f /dev/null
fi
" > /usr/local/bin/cook

# ----------------- END COOK ------------------


# ---------- NO NEED TO EDIT BELOW ------------

chmod u+x /usr/local/bin/cook

#
# There are two ways of running a pot jail: "Normal", non-blocking mode and
# "Nomad", i.e. blocking mode (the pot start command does not return until
# the jail is stopped).
# For the normal mode, we create a /usr/local/etc/rc.d script that starts
# the "cook" script generated above each time, for the "Nomad" mode, the cook
# script is started by pot (configuration through flavour file), therefore
# we do not need to do anything here.
# 

# Create rc.d script for "normal" mode:
echo "#!/bin/sh

#
# PROVIDE: cook 
# REQUIRE: LOGIN
# KEYWORD: shutdown
#

. /etc/rc.subr

name=cook
rcvar=cook_enable

load_rc_config $name

: ${cook_enable:=\"NO\"}
: ${cook_env:=\"\"}

command=\"/usr/local/bin/cook\"
command_args=\"\"

run_rc_command \"\$1\"
" > /usr/local/etc/rc.d/cook

chmod u+x /usr/local/etc/rc.d/cook

if [ $RUNS_IN_NOMAD = false ]
then
    # This is a non-nomad (non-blocking) jail, so we need to make sure the script
    # gets started when the jail is started:
    # Otherwise, /usr/local/bin/cook will be set as start script by the pot flavour
    echo "cook_enable=\"YES\"" >> /etc/rc.conf
fi

nomad-server/nomad-server+1:
nomad-server/nomad-server+1.sh:

nomad-server/nomad-server+2:
nomad-server/nomad-server+2.sh:

nomad-server/nomad-server+3:
nomad-server/nomad-server+3.sh:

nomad-server/nomad-server+4:
nomad-server/nomad-server+4.sh:
Password:=====>  -i auto: assigned 10.192.0.3
===>  Creating a new pot
===>  pot name : nomad-server-amd64-12_1
===>  type : single
===>  base : 12.1
===>  pot_base :
===>  level : 0
===>  network-type: public-bridge
===>  ip : 10.192.0.3
===>  bridge :
===>  dns : inherit
===>  flavours : fbsd-update nomad-server nomad-server+1 nomad-server+2 nomad-server+3 nomad-server+4
===>  Fetching FreeBSD 12.1
===>  Extract the tarball
=====>  Flavour: fbsd-update
=====>  Starting nomad-server-amd64-12_1 pot for the initial bootstrap
=====>  mount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp
defaultrouter: NO -> 10.192.0.1
===>  Starting the pot nomad-server-amd64-12_1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Starting Network: lo0 epair0b.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:e4:6a:67:90:0b
	inet 10.192.0.3 netmask 0xffc00000 broadcast 10.255.255.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.192.0.1
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Generating host.conf.
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Updating motd:.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting cron.

Mon Jul 27 11:46:43 UTC 2020
/usr/local/etc/pot/flavours/fbsd-update.sh -> /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp/fbsd-update.sh
=====>  Executing fbsd-update script on nomad-server-amd64-12_1
src component not installed, skipped
freebsd-update fetch should not be run non-interactively.
Run freebsd-update cron instead.
src component not installed, skipped
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
=====>  Stop the pot nomad-server-amd64-12_1
=====>  Remove epair0[a|b] network interfaces
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/dev
=====>  Flavour: nomad-server
=====>  Executing nomad-server pot commands on nomad-server-amd64-12_1
=====>  Starting nomad-server-amd64-12_1 pot for the initial bootstrap
=====>  mount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp
defaultrouter: 10.192.0.1 -> 10.192.0.1
===>  Starting the pot nomad-server-amd64-12_1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Starting Network: lo0 epair0b.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:12:31:6b:de:0b
	inet 10.192.0.3 netmask 0xffc00000 broadcast 10.255.255.255
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.192.0.1
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Updating motd:.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting cron.

Mon Jul 27 11:46:48 UTC 2020
/usr/local/etc/pot/flavours/nomad-server.sh -> /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp/nomad-server.sh
=====>  Executing nomad-server script on nomad-server-amd64-12_1
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] Installing pkg-1.14.6...
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] Extracting pkg-1.14.6: .......... done
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
sendmail_enable: NO -> NO
nomad_enable:  -> YES
nomad_user:  -> root
nomad_env:  -> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin
Updating FreeBSD repository catalogue...
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] Fetching meta.conf: . done
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] Fetching packagesite.txz: .......... done
Processing entries: .......... done
FreeBSD repository update completed. 32005 packages processed.
All repositories are up to date.
Updating database digests format: . done
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	nomad: 0.11.3

Number of packages to be installed: 1

The process will require 75 MiB more space.
15 MiB to be downloaded.
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] [1/1] Fetching nomad-0.11.3.txz: .......... done
Checking integrity... done (0 conflicting)
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] [1/1] Installing nomad-0.11.3...
===> Creating groups.
Creating group 'nomad' with gid '472'.
===> Creating users
Creating user 'nomad' with uid '472'.
===> Creating homedir(s)
[nomad-server-amd64-12_1.vsf00001.cpt.za.honeyguide.net] [1/1] Extracting nomad-0.11.3: ....... done
Nothing to do.
=====>  Stop the pot nomad-server-amd64-12_1
=====>  Remove epair0[a|b] network interfaces
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/tmp
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-12_1/m/dev
=====>  Flavour: nomad-server+1
=====>  Executing nomad-server+1 pot commands on nomad-server-amd64-12_1
=====>  No shell script available for the flavour nomad-server+1
=====>  Flavour: nomad-server+2
=====>  Executing nomad-server+2 pot commands on nomad-server-amd64-12_1
=====>  No shell script available for the flavour nomad-server+2
=====>  Flavour: nomad-server+3
=====>  Executing nomad-server+3 pot commands on nomad-server-amd64-12_1
=====>  No shell script available for the flavour nomad-server+3
=====>  Flavour: nomad-server+4
=====>  Executing nomad-server+4 pot commands on nomad-server-amd64-12_1
=====>  No shell script available for the flavour nomad-server+4

nomad-server-amd64-11_4_1.0.1:


nomad-server/nomad-server:
nomad-server/nomad-server.sh:
#!/bin/sh

# EDIT THE FOLLOWING FOR NEW FLAVOUR:
# 1. RUNS_IN_NOMAD - yes or no
# 2. Adjust package installation between BEGIN & END PACKAGE SETUP
# 3. Adjust jail configuration script generation between BEGIN & END COOK

# Set this to true if this jail flavour is to be created as a nomad (i.e. blocking) jail.
# You can then query it in the cook script generation below and the script is installed
# appropriately at the end of this script 
RUNS_IN_NOMAD=false

# -------------- BEGIN PACKAGE SETUP -------------
[ -w /etc/pkg/FreeBSD.conf ] && sed -i '' 's/quarterly/latest/' /etc/pkg/FreeBSD.conf
ASSUME_ALWAYS_YES=yes pkg bootstrap
touch /etc/rc.conf
sysrc sendmail_enable="NO"
sysrc nomad_enable="YES"
sysrc nomad_user="root"
sysrc nomad_env="PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin"

# Install packages
pkg install -y nomad 
pkg clean -y
# -------------- END PACKAGE SETUP -------------

#
# Create configurations
#

#
# Now generate the run command script "cook"
# It configures the system on the first run by creating the config file(s) 
# On subsequent runs, it only starts sleeps (if nomad-jail) or simply exits 
#

# ----------------- BEGIN COOK ------------------ 
echo "#!/bin/sh

# No need to change this, just ensures configuration is done only once
if [ -e /usr/local/etc/pot-is-seasoned ]
then
    # If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
    # created by pot and we block indefinitely
    if [ ! -e /tmp/environment.sh ]
    then
        tail -f /dev/null 
    fi
    exit 0
fi

# ADJUST THIS: STOP SERVICES AS NEEDED BEFORE CONFIGURATION
/usr/local/etc/rc.d/nomad stop  || true

# No need to adjust this:
# If this pot flavour is not blocking, we need to read the environment first from /tmp/environment.sh
# where pot is storing it in this case
if [ -e /tmp/environment.sh ]
then
    . /tmp/environment.sh
fi

#
# ADJUST THIS BY CHECKING FOR ALL VARIABLES YOUR FLAVOUR NEEDS:
# Check config variables are set
#
if [ -z \${DATACENTER+x} ]; 
then 
    echo 'DATACENTER is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${IP+x} ]; 
then 
    echo 'IP is unset - see documentation how to configure this flavour'
    exit 1
fi
if [ -z \${CONSULSERVER+x} ];
then
    echo 'CONSULSERVER is unset - see documentation how to configure this flavour'
    exit 1
fi

# ADJUST THIS BELOW: NOW ALL THE CONFIGURATION FILES NEED TO BE CREATED:
# Don't forget to double(!)-escape quotes and dollar signs in the config files

# Create nomad server config file 
echo \"
bind_addr = \\\"\$IP\\\"
plugin_dir = \\\"/usr/local/libexec/nomad/plugins\\\"
datacenter = \\\"\$DATACENTER\\\"

advertise {
  # This should be the IP of THIS MACHINE and must be routable by every node
  # in your cluster
  http = \\\"\$IP:4646\\\"
}

server {
  enabled = true
  bootstrap_expect = 1
}

consul {
  # The address to the Consul agent.
  address = \\\"\$CONSULSERVER:8500\\\"

  # The service name to register the server and client with Consul.
  server_service_name = \\\"\$DATACENTER-server\\\"

  # Enables automatically registering the services.
  auto_advertise = true

  # Enabling the server and client to bootstrap using Consul.
  server_auto_join = true
}

enable_syslog=true
log_level=\\\"INFO\\\"
syslog_facility=\\\"LOCAL1\\\"\" > /usr/local/etc/nomad/server.hcl
echo \"nomad_args=\\\"-config=/usr/local/etc/nomad/server.hcl -network-interface=\$IP\\\"\" >> /etc/rc.conf

# ADJUST THIS: START THE SERVICES AGAIN AFTER CONFIGURATION
/usr/local/etc/rc.d/nomad start

# Do not touch this:
touch /usr/local/etc/pot-is-seasoned

# If this pot flavour is blocking (i.e. it should not return), there is no /tmp/environment.sh
# created by pot and we now after configuration block indefinitely
if [ ! -e /tmp/environment.sh ]
then
    tail -f /dev/null
fi
" > /usr/local/bin/cook

# ----------------- END COOK ------------------


# ---------- NO NEED TO EDIT BELOW ------------

chmod u+x /usr/local/bin/cook

#
# There are two ways of running a pot jail: "Normal", non-blocking mode and
# "Nomad", i.e. blocking mode (the pot start command does not return until
# the jail is stopped).
# For the normal mode, we create a /usr/local/etc/rc.d script that starts
# the "cook" script generated above each time, for the "Nomad" mode, the cook
# script is started by pot (configuration through flavour file), therefore
# we do not need to do anything here.
# 

# Create rc.d script for "normal" mode:
echo "#!/bin/sh

#
# PROVIDE: cook 
# REQUIRE: LOGIN
# KEYWORD: shutdown
#

. /etc/rc.subr

name=cook
rcvar=cook_enable

load_rc_config $name

: ${cook_enable:=\"NO\"}
: ${cook_env:=\"\"}

command=\"/usr/local/bin/cook\"
command_args=\"\"

run_rc_command \"\$1\"
" > /usr/local/etc/rc.d/cook

chmod u+x /usr/local/etc/rc.d/cook

if [ $RUNS_IN_NOMAD = false ]
then
    # This is a non-nomad (non-blocking) jail, so we need to make sure the script
    # gets started when the jail is started:
    # Otherwise, /usr/local/bin/cook will be set as start script by the pot flavour
    echo "cook_enable=\"YES\"" >> /etc/rc.conf
fi

nomad-server/nomad-server+1:
nomad-server/nomad-server+1.sh:

nomad-server/nomad-server+2:
nomad-server/nomad-server+2.sh:

nomad-server/nomad-server+3:
nomad-server/nomad-server+3.sh:

nomad-server/nomad-server+4:
nomad-server/nomad-server+4.sh:
Password:=====>  -i auto: assigned 10.192.0.4
===>  Creating a new pot
===>  pot name : nomad-server-amd64-11_4
===>  type : single
===>  base : 11.4
===>  pot_base :
===>  level : 0
===>  network-type: public-bridge
===>  ip : 10.192.0.4
===>  bridge :
===>  dns : inherit
===>  flavours : fbsd-update nomad-server nomad-server+1 nomad-server+2 nomad-server+3 nomad-server+4
===>  Fetching FreeBSD 11.4
===>  Extract the tarball
=====>  Flavour: fbsd-update
=====>  Starting nomad-server-amd64-11_4 pot for the initial bootstrap
=====>  mount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp
defaultrouter: NO -> 10.192.0.1
===>  Starting the pot nomad-server-amd64-11_4
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Starting Network: lo0 epair0b.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:1b:f5:b9:85:0b
	hwaddr 02:1b:f5:b9:85:0b
	inet 10.192.0.4 netmask 0xffc00000 broadcast 10.255.255.255
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	groups: epair
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.192.0.1
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Generating host.conf.
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Updating motd:.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting cron.

Mon Jul 27 11:49:21 UTC 2020
/usr/local/etc/pot/flavours/fbsd-update.sh -> /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp/fbsd-update.sh
=====>  Executing fbsd-update script on nomad-server-amd64-11_4
src component not installed, skipped
freebsd-update fetch should not be run non-interactively.
Run freebsd-update cron instead.
src component not installed, skipped
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
=====>  Stop the pot nomad-server-amd64-11_4
=====>  Remove epair0[a|b] network interfaces
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/dev
=====>  Flavour: nomad-server
=====>  Executing nomad-server pot commands on nomad-server-amd64-11_4
=====>  Starting nomad-server-amd64-11_4 pot for the initial bootstrap
=====>  mount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp
defaultrouter: 10.192.0.1 -> 10.192.0.1
===>  Starting the pot nomad-server-amd64-11_4
ELF ldconfig path: /lib /usr/lib /usr/lib/compat
32-bit compatibility ldconfig path: /usr/lib32
Starting Network: lo0 epair0b.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:c9:44:76:fe:0b
	hwaddr 02:c9:44:76:fe:0b
	inet 10.192.0.4 netmask 0xffc00000 broadcast 10.255.255.255
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	groups: epair
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.192.0.1
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Creating and/or trimming log files.
Starting syslogd.
Clearing /tmp (X related).
Updating motd:.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting cron.

Mon Jul 27 11:49:26 UTC 2020
/usr/local/etc/pot/flavours/nomad-server.sh -> /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp/nomad-server.sh
=====>  Executing nomad-server script on nomad-server-amd64-11_4
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] Installing pkg-1.14.6...
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] Extracting pkg-1.14.6: .......... done
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:11:amd64/latest, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
sendmail_enable: NO -> NO
nomad_enable:  -> YES
nomad_user:  -> root
nomad_env:  -> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/sbin:/bin
Updating FreeBSD repository catalogue...
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] Fetching meta.conf: . done
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] Fetching packagesite.txz: .......... done
Processing entries: .......... done
FreeBSD repository update completed. 30797 packages processed.
All repositories are up to date.
Updating database digests format: . done
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
	nomad: 0.12.1

Number of packages to be installed: 1

The process will require 71 MiB more space.
15 MiB to be downloaded.
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] [1/1] Fetching nomad-0.12.1.txz: .......... done
Checking integrity... done (0 conflicting)
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] [1/1] Installing nomad-0.12.1...
===> Creating groups.
Creating group 'nomad' with gid '472'.
===> Creating users
Creating user 'nomad' with uid '472'.
===> Creating homedir(s)
[nomad-server-amd64-11_4.vsf00001.cpt.za.honeyguide.net] [1/1] Extracting nomad-0.12.1: ....... done
Nothing to do.
=====>  Stop the pot nomad-server-amd64-11_4
=====>  Remove epair0[a|b] network interfaces
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/tmp
=====>  unmount /mnt/data/pot/jails/nomad-server-amd64-11_4/m/dev
=====>  Flavour: nomad-server+1
=====>  Executing nomad-server+1 pot commands on nomad-server-amd64-11_4
=====>  No shell script available for the flavour nomad-server+1
=====>  Flavour: nomad-server+2
=====>  Executing nomad-server+2 pot commands on nomad-server-amd64-11_4
=====>  No shell script available for the flavour nomad-server+2
=====>  Flavour: nomad-server+3
=====>  Executing nomad-server+3 pot commands on nomad-server-amd64-11_4
=====>  No shell script available for the flavour nomad-server+3
=====>  Flavour: nomad-server+4
=====>  Executing nomad-server+4 pot commands on nomad-server-amd64-11_4
=====>  No shell script available for the flavour nomad-server+4

This site © Honeyguide Group (Pty) Ltd, all the hosted software their respective license owners 2020 - Disclaimer