reverse engineering things that predecessors left without any documentation and throwing them out the window because devops

Multi-master SaltStack setup is quite easy to build out. There is no need for VIPs or DNS CNAMEs (though they can be implemented) and all of the functionality is handled by Salt.  This greatly simplifies everything and you don’t have to rely on external tools.

To have working masters, you need to keep the a couple of directories in sync. You may use clustering filesystems or rsync to do that. In this example we will use rsync which is more than enough. With some extra ingenuity, you can even automate this sync process to happen automatically.

This howto describes how to do this on CentOS, but the setup should be the same on any other OS (such as FreeBSD).

Continue reading ›

To setup remote logging to a central Syslog server, you need to add the following line:

This will set up remote logging using UDP.  Note the single @ sign.  To set up TCP, use double @@ signs:

*.* stands for facility.severity.  Asterisks will pick up all facilities and severities so that means that all entries will be sent to the remote server.  :514 port portion is optional.  Syslog will use default port 514 but you may change that on the server.  If you’re not seeing any messages on the central log host get delivered, verify that ports are open on your firewalls and check if you’re getting any packets on the Syslog host using tcpdump:

Syslog service will open an initial connection using TCP and continue the session while sending packets with log entries.  Since TCP is connection-oriented (each packet is acknowledged) unlike UDP, every packet will be aknowledged.  UDP also lacks congestion control (useful when syslog client spams a ton of messages), may corrupt messages if there are issues on the line, and may deliver messages out of sequence.  Some of these cases are rare, but they may still happen.  It’s recommended to use TCP whenever possible.  If you know that your network equipment is reliable (i.e. no broadcast storms, etc) and you need every ounce of CPU processing power out of your systems, then UDP should be good enough.