Overcast is a tiny command line program designed to make it easy to spin up, configure and manage clusters of machines across multiple cloud providers.
Create, reboot and destroy instances on DigitalOcean or locally using Virtualbox.
# Spin up a new Ubuntu 20.04 instance on DigitalOcean:
$ overcast digitalocean create db-01
# Perform a full system upgrade and then install Redis:
$ overcast run db-01 install/core install/redisRun multiple commands or multiple scripts on any of your instances over SSH. Commands can be run sequentially or in parallel. Run and test your deployment scripts across different providers.
# Create a LAMP stack using bundled install scripts:
$ overcast run lamp-01 install/core install/apache install/mysql install/php
# Run your own scripts using relative or absolute paths:
$ overcast run app-cluster ./my-app/my-install-script /path/to/another-script
# Run sequences of commands and scripts across multiple machines in parallel:
$ overcast run app-* ./script.sh uptime "free -m" "df -h" --parallelQuickly SSH in to any instance by name.
$ overcast ssh app-01Push and pull files between your local machine and any of your instances.
$ overcast push app-01 nginx/myapp.conf /etc/nginx/sites-enabled/myapp.conf
$ overcast pull app-01 /var/log/syslog ./my-local-syslog-copyOvercast is a thin layer on top of your SSH client. It doesn't install or leave anything on the servers you communicate with, so Overcast itself has no real attack surface.
A library of scripts and recipes are bundled to make it easy to deploy a number of common software stacks and applications.
-
Install Node.js if not already installed.
-
Install Overcast using npm.
$ npm -g install overcast
-
You can now use Overcast from any directory. Running any overcast command from anywhere will create the
~/.overcastconfig directory if it doesn't already exist. Add your API keys to~/.overcast/variables.jsonto use their respective commands, either manually or using thevarcommand:$ overcast var set DIGITALOCEAN_API_TOKEN abc123 -
To make working with Overcast easier, you can add bash tab completion by adding the following to your
.bash_profile:# Overcast Tab completion _overcast_completions() { local cur=${COMP_WORDS[COMP_CWORD]} COMPREPLY=($(compgen -W "`overcast completions`" -- "$cur")) return 0 } complete -F _overcast_completions overcast
Using Overcast on Windows is possible, but unsupported.
Since Overcast is just a wrapper around SSH, there is nothing on your remote machines to uninstall. To uninstall Overcast from your local machine:
# To remove the Overcast package:
$ npm -g remove overcast
# Optionally delete your Overcast SSH keys and configuration files:
$ rm -rf ~/.overcastOvercast looks for an .overcast directory in the current directory, a parent directory, or ~/.overcast, in that order. This means you can have multiple configurations and treat your server infrastructure like source code.
The command overcast init will create a new configuration in the current directory. The config directory looks like this:
/.overcast
/keys # SSH keys, can be your own or auto-generated by overcast
overcast.key
overcast.key.pub
clusters.json # Cluster/instance definitions (see example.clusters.json)
variables.json # API keys, etc (see example.variables.json)There are many server management frameworks out there already, but they generally involve a complex server-client implementation, a steep learning curve, or a giant, monolithic conceptual framework that requires taking a course to understand.
I wanted something that had little to no learning curve, that did only what you asked it to do on the remote machines and nothing more, that just focused on multi-server provisioning and communication and leaves problems like process/state management and system monitoring to tools designed specifically for those problems.
Usage:
overcast aliases
Description:
Return a list of bash aliases for SSHing to your instances.
To use, add this to your .bash_profile:
test -f $HOME/.overcast_aliases && source $HOME/.overcast_aliases
And then create the .overcast_aliases file:
overcast aliases > $HOME/.overcast_aliases
Or to automatically refresh aliases in every new terminal window
(which will add a couple hundred milliseconds to your startup time),
add this to your .bash_profile:
overcast aliases > $HOME/.overcast_aliases
source $HOME/.overcast_aliases
Usage:
overcast cluster count [name]
Description:
Return the number of instances in a cluster.
Examples:
$ overcast cluster count db
> 0
$ overcast instance create db.01 --cluster db
> ...
$ overcast cluster count db
> 1
Usage:
overcast cluster add [name]
Description:
Adds a new cluster.
Examples:
$ overcast cluster add db
Usage:
overcast cluster rename [name] [new-name]
Description:
Renames a cluster.
Examples:
$ overcast cluster rename app-cluster app-cluster-renamed
Usage:
overcast cluster remove [name]
Description:
Removes a cluster from the index. If the cluster has any instances
attached to it, they will be moved to an "orphaned" cluster.
Examples:
$ overcast cluster remove db
Usage:
overcast completions
Description:
Return an array of commands, cluster names, and instance names for use
in bash tab completion.
To enable tab completion in bash, add this to your .bash_profile:
_overcast_completions() {
local cur=${COMP_WORDS[COMP_CWORD]}
COMPREPLY=($(compgen -W "`overcast completions`" -- "$cur"))
return 0
}
complete -F _overcast_completions overcast
Usage:
overcast digitalocean boot [name]
Description:
Boot up an instance if powered off, otherwise do nothing.
Usage:
overcast digitalocean create [name] [options...]
Description:
Creates a new instance on DigitalOcean.
Options: Defaults:
--cluster CLUSTER default
--ssh-port PORT 22
--ssh-key PATH overcast.key
--ssh-pub-key PATH overcast.key.pub
--region REGION
--image IMAGE
--size SIZE
--backups false
--monitoring false
--private-networking false
--vpc-uuid
--with-droplet-agent false
Examples:
# Match using slugs:
$ overcast digitalocean create vm-01 --size 2gb --region sfo1
# Match using IDs or names:
$ overcast digitalocean create vm-02 --region "London 1" --image 6374128
Usage:
overcast digitalocean destroy [name] [options...]
Description:
Destroys a DigitalOcean droplet and removes it from your account.
Using --force overrides the confirm dialog.
Options: Defaults:
--force false
Examples:
$ overcast digitalocean destroy vm-01
Usage:
overcast digitalocean images
Description:
List all images, including snapshots.
Usage:
overcast digitalocean instances
Description:
List all instances in your account.
Usage:
overcast digitalocean reboot [name]
Description:
Reboot an instance using the provider API.
Usage:
overcast digitalocean regions
Description:
List all available regions.
Usage:
overcast digitalocean rebuild [name] [image]
Description:
Rebuilds an existing instance on DigitalOcean, preserving the IP address.
[image] can be image ID, name or slug.
Examples:
# Rebuild an instance using a readymade image:
$ overcast digitalocean rebuild vm-01 ubuntu-14-04-x64
# Rebuild an instance using a snapshot:
$ overcast digitalocean rebuild vm-01 "vm-01 backup"
Usage:
overcast digitalocean resize [name] [size] [options...]
Description:
Shutdown, resize, and reboot a DigitalOcean instance.
[size] must be a valid size slug.
If the --skip-boot flag is used, the instance will stay powered off.
Options: Defaults:
--skip-boot false
Examples:
# Resize an instance to 2gb:
$ overcast digitalocean resize vm-01 2gb
Usage:
overcast digitalocean snapshot [name] [snapshot-name]
Description:
Creates a named snapshot of a droplet. This will reboot the instance.
Examples:
$ overcast digitalocean snapshot vm-01 vm-01-snapshot
Usage:
overcast digitalocean snapshots
Description:
List all available snapshots in your account.
Usage:
overcast digitalocean shutdown [name]
Description:
Shut down an instance using the provider API.
Usage:
overcast digitalocean sizes
Description:
List all available instance sizes.
Usage:
overcast digitalocean sync [name]
Description:
Fetch and update instance metadata.
Usage:
overcast expose [instance|cluster|all] [port...] [options]
Description:
Reset the exposed ports on the instance or cluster using iptables.
This will fail if you don't include the current SSH port.
Specifying --allowlist will restrict all ports to the specified address(es).
These can be individual IPs or CIDR ranges, such as "192.168.0.0/24".
Expects an Ubuntu server, untested on other distributions.
Options:
--user USERNAME
--password PASSWORD
--allowlist "IP|RANGE"
--allowlist-PORT "IP|RANGE"
Examples:
Allow SSH, HTTP and HTTPS connections from anywhere:
$ overcast expose app 22 80 443
Allow SSH from anywhere, only allow Redis connections from 1.2.3.4:
$ overcast expose redis 22 6379 --allowlist-6379 "1.2.3.4"
Only allow SSH and MySQL connections from 1.2.3.4 or from 5.6.7.xxx:
$ overcast expose mysql 22 3306 --allowlist "1.2.3.4 5.6.7.0/24"
Usage:
overcast exposed [instance|cluster|all]
Description:
List the exposed ports on the instance or cluster.
Expects an Ubuntu server, untested on other distributions.
Options:
--user USERNAME
--password PASSWORD
--machine-readable, --mr
Usage:
overcast help
Description:
Provides help about Overcast and specific commands.
Usage:
overcast info
overcast info [name]
Description:
Prints the complete .overcast/clusters.json file.
Optionally display only instances matching [name].
Usage:
overcast init
Description:
Create an .overcast config directory in the current working directory.
No action taken if one already exists.
Usage:
overcast instance get [instance|cluster|all] [attr...] [options...]
Description:
Returns the attribute(s) for the instance or cluster, one per line,
or space-delimited using the --single-line option.
Deeply nested arrays and objects are supported.
"origin" is a compound attribute that returns user@ip:ssh-port.
Options: Defaults:
--single-line, -s false
Examples:
$ overcast instance get app-01 origin
root@1.2.3.4:22
$ overcast instance get app-cluster ip
127.0.0.1
127.0.0.2
127.0.0.3
$ overcast instance get app-01 digitalocean.image.id
103510828
Usage:
overcast instance add [name] [ip] [options...]
Description:
Adds an existing instance to a cluster.
Options: Defaults:
--cluster CLUSTER default
--ssh-port PORT 22
--ssh-key PATH overcast.key
--user USERNAME root
--password PASSWORD
Examples:
$ overcast instance add app.01 127.0.0.1 --cluster app \
--ssh-port 22222 --ssh-key $HOME/.ssh/id_rsa
Usage:
overcast instance list [cluster...]
Description:
Returns all instance names, one per line.
Optionally limit to one or more clusters.
Examples:
$ overcast instance list
$ overcast instance list app-cluster db-cluster
Usage:
overcast instance remove [name]
Description:
Removes an instance from the index.
The server itself is not affected by this action.
Examples:
$ overcast instance remove app-01
Usage:
overcast instance update [instance|cluster|all] [options...]
Description:
Update any instance property. Specifying --cluster will move the instance
to that cluster. Specifying --name will rename the instance.
Options:
--name NAME
--cluster CLUSTER
--ip IP
--ssh-port PORT
--ssh-key PATH
--user USERNAME
--password PASSWORD
Examples:
# Update the user and ssh-key of an instance:
$ overcast instance update app.01 --user myuser --ssh-key /path/to/key
# Update ssh-port of a cluster:
$ overcast instance update app-cluster --ssh-port 22222
Usage:
overcast list
Description:
List your cluster and instance definitions.
Usage:
overcast ping [instance|cluster|all] [options]
Description:
Display the average ping time for an instance or cluster.
Options: Defaults:
--count N, -c N 3
Examples:
$ overcast ping app-01
$ overcast ping db --count 5
Usage:
overcast port [instance|cluster|all] [port]
Description:
Change the SSH port for an instance or a cluster. This command fails
if the new port has not been previously opened by iptables.
See also the "expose" and "exposed" commands.
Examples:
# Expose only necessary ports:
$ overcast expose vm-01 22 55522 80 443
# Update SSH port from 22 to 55522:
$ overcast port vm-01 55522
# Close port 22:
$ overcast expose vm-01 55522 80 443
Usage:
overcast pull [instance|cluster|all] [source] [dest] [options...]
Description:
Pull a file or directory from an instance or cluster using scp by default,
or using rsync if the --rsync flag is used. Source is absolute or relative
to the home directory. Destination can be absolute or relative to the
.overcast/files directory. Any reference to {instance} in the destination
will be replaced with the instance name.
Options: Defaults:
--rsync false
--user USERNAME
--password PASSWORD
Examples:
Assuming instances "app.01" and "app.02", this will expand to:
- .overcast/files/app.01.bashrc
- .overcast/files/app.02.bashrc
$ overcast pull app .bashrc {instance}.bashrc
Usage:
overcast push [instance|cluster|all] [source] [dest] [options...]
Description:
Push a file or directory to an instance or cluster using scp by default,
or rsync if the --rsync flag is used. Source can be absolute or relative
to the .overcast/files directory. Destination can be absolute or relative
to the home directory. Any reference to {instance} in the source will be
replaced with the instance name. The --exclude flag only works with rsync.
Options: Defaults:
--rsync false
--user USERNAME
--password PASSWORD
--exclude FILE_OR_DIRECTORY
Examples:
Assuming instances "app.01" and "app.02", this will expand to:
- .overcast/files/app.01.bashrc
- .overcast/files/app.02.bashrc
$ overcast push app {instance}.bashrc .bashrc
Usage:
overcast run [instance|cluster|all] [command|file...]
Description:
Execute commands or script files on an instance or cluster over SSH.
Commands will execute sequentially unless the --parallel flag is used.
An error will stop execution unless the --continueOnError flag is used.
Script files can be either absolute or relative path.
Options: Defaults:
--env "KEY=VAL KEY='1 2 3'"
--user USERNAME
--password PASSWORD
--ssh-key PATH
--ssh-args ARGS
--continueOnError false
--machine-readable, --mr false
--parallel, -p false
--shell-command "COMMAND" bash -s
--only-once false
Examples:
# Run arbirary commands and files in sequence across all instances:
$ overcast run all uptime "free -m" "df -h" /path/to/my/script
# Setting environment variables:
$ overcast run app --env "foo='bar bar' testing=123" env
# Use machine-readable output (no server prefix):
$ overcast run app-01 uptime --mr
# Run bundled and custom scripts in sequence:
$ overcast run db-* install/core install/redis ./my/install/script
# Pass along arbitrary SSH arguments, for example to force a pseudo-tty:
$ overcast run all /my/install/script --ssh-args "-tt"
Usage:
overcast scriptvar [instance|cluster|all] [filename] [key] [value]
Description:
Set a named variable in a remote file on an instance or cluster.
Expects a shell variable format, for example MY_VAR_NAME="my_value"
Options: Defaults:
--user USERNAME
--password PASSWORD
--continueOnError false
--machine-readable, --mr false
--parallel, -p false
Examples:
$ overcast scriptvar app-01 /path/to/file.sh MY_API_TOKEN abc123
Usage:
overcast slack [message] [options...]
Description:
Sends a message to a Slack channel.
Requires a SLACK_WEBHOOK_URL property to be set in variables.json.
You can set that with the following command:
overcast var set SLACK_WEBHOOK_URL https://foo.slack.com/blah
Options: Defaults:
--channel NAME #alerts
--icon-emoji EMOJI :cloud:
--icon-url URL
--user NAME Overcast
--KEY VALUE
Examples:
$ overcast slack "Deploy completed." --icon-emoji ":satelite:"
$ overcast slack "Server stats" --channel "#general" --cpu "0.54 0.14 0.09"
Usage:
overcast ssh [instance] [options...]
Description:
Opens an interactive SSH connection to an instance.
Options:
--user USERNAME
--password PASSWORD
--ssh-key PATH
Examples:
$ overcast ssh instance-01
# To use a personal username and key in variables.json:
$ overcast vars set OVERCAST_SSH_USER my-username
$ overcast vars set OVERCAST_SSH_KEY /path/to/my.key
$ overcast ssh instance-01 # will use the above variables to attempt a connection
Usage:
overcast sshkey create [name]
Description:
Creates a new SSH key in the current .overcast config.
Examples:
$ overcast sshkey create myKeyName
New SSH key "myKeyName" created.
- /path/to/.overcast/keys/myKeyName.key
- /path/to/.overcast/keys/myKeyName.key.pub
Usage:
overcast sshkey delete [name]
Description:
Deletes SSH public/private key files from the current .overcast config.
Examples:
$ overcast sshkey delete myKeyName
SSH key "myKeyName" deleted.
Usage:
overcast sshkey get [name] [option]
Description:
Display the requested SSH key data or path from the current .overcast config.
Defaults to displaying the public key data if no option found.
Options:
--public-data
--private-data
--public-path
--private-path
Examples:
$ overcast sshkey get myKeyName
[public key data]
$ overcast sshkey get myKeyName --private-data
[private key data]
Usage:
overcast sshkey list
Description:
List the found SSH key names in the current .overcast config.
Examples:
$ overcast sshkey list
myKeyName
overcast
Usage:
overcast sshkey push [instance|cluster|all] [name|path] [options...]
Description:
Push a public SSH key to an instance or cluster. Accepts a key name,
filename, or full path. This will overwrite the existing authorized_keys
file, unless you use --append.
Options: Defaults:
--user USERNAME
--append, -a false
Examples:
# Generate new SSH key pair:
$ overcast sshkey create newKey
# Push public key to instance, update instance config to use private key:
$ overcast sshkey push vm-01 newKey
$ overcast instance update vm-01 --ssh-key newKey.key
# Same as above but using key path instead of key name:
$ overcast sshkey push vm-02 "~/.ssh/id_rsa.pub"
$ overcast instance update vm-02 --ssh-key "~/.ssh/id_rsa"
# Push public key to instance using arbitrary user:
$ overcast sshkey push vm-03 newKey --user myOtherUser
# Append public key to authorized_keys instead of overwriting:
$ overcast sshkey push vm-04 newKey --append
Usage:
overcast tunnel [instance] [local-port((:hostname):remote-port)...]
Description:
Opens an SSH tunnel to the port(s) specified.
If only one port is specified, assume the same port for local/remote.
If no remote host is specified, assume the remote host itself (127.0.0.1).
Multiple tunnels can be opened over a single connection.
Options:
--user USERNAME
--password PASSWORD
--ssh-key PATH
Examples:
# Tunnel local 5984 to remote 5984
$ overcast tunnel app-01 5984
# Tunnel local 8000 to remote 5984, local 8001 to remote 3000
$ overcast tunnel app-01 8000:5984 8001:3000
# Tunnel local 3000 to otherhost.com:4000
$ overcast tunnel app-01 3000:otherhost.com:4000
Usage:
overcast vars list
Description:
List all variables in your current config.
Usage:
overcast vars set [name] [value]
Description:
Set a variable in your current config.
Examples:
$ overcast vars set AWS_KEY myawskey12345
$ overcast vars set MY_CUSTOM_VARIABLE_NAME foo
Usage:
overcast vars get [name]
Description:
Get a variable from your current config.
Examples:
$ overcast vars get AWS_KEY
> myawskey12345
$ overcast vars get MY_CUSTOM_VARIABLE_NAME
> foo
Usage:
overcast vars delete [name]
Description:
Delete a variable from your current config.
Examples:
$ overcast vars delete MY_CUSTOM_VARIABLE_NAME
Usage:
overcast virtualbox boot [name]
Description:
Boot up a Virtualbox instance.
Usage:
overcast virtualbox create [name] [options...]
Description:
Creates a new Virtualbox instance.
Options: Defaults:
--cluster CLUSTER default
--cpus COUNT 1
--image NAME trusty64
--ram MB 512
--ip ADDRESS 192.168.22.10
--ssh-key PATH overcast.key
--ssh-pub-key PATH overcast.key.pub
Examples:
$ overcast virtualbox create vm-01
$ overcast virtualbox create vm-02 --ram 1024 --image precise64
Usage:
overcast virtualbox destroy [name] [options...]
Description:
Destroys a Virtualbox instance.
Using --force overrides the confirm dialog.
Options: Defaults:
--force false
Examples:
$ overcast virtualbox destroy vm-01
Usage:
overcast virtualbox reboot [name]
Description:
Reboots a Virtualbox instance.
Usage:
overcast virtualbox shutdown [name]
Description:
Shut down a Virtualbox instance.
Usage:
overcast wait [seconds]
Description:
Show a progress bar for a specified number of seconds.
Examples:
$ overcast wait 30
npm testnpm -g update overcastConfiguration files are left alone during an upgrade.
Contributions are welcome. If you've got an idea for a feature or found a bug, please open an issue. If you're a developer and want to help improve Overcast, open a pull request with your changes.
git clone git@github.com:andrewchilds/overcast.git
cd overcast
# To use local development version:
node overcast [args...]
# To use published version:
overcast [args...]- Standardize Recipes
- Tagging
- Google Compute Engine support
- More comprehensive script/recipe library
- More test coverage
MIT. Copyright © 2014-2022 Andrew Childs