Over the last year there has been a specific focus by all industry leaders to come up with technologies
to make a bunch of nodes appear as one big computer and deploy anything to your own private cloud; just
like Docker,Rocket,KVM but across any number of nodes. Kubernetes, Fleet, Apache Mesos are just few of
them which abstracting away discrete nodes and optimizing compute resources. Each implementation has its
own tweak, but they all follow the same basic setup: leaders, for coordination and scheduling;
some service discovery component; some underlying cluster tool (like Zookeeper); followers, for processing.
In this post we’ll use Ansible to install a multi-node Mesos cluster using packages from Mesosphere.
Mesos, as a cluster framework, allows you to run a variety of cluster-enabled software. You can also run
Jenkins, schedule tasks with Chronos, even run ElasticSearch and Cassandra without having to double to
specific servers. We’ll also set up Marathon for running services.
The playbook is on github, just follow the readme!. If you want to simply try out Mesos, Marathon, and
Docker mesosphere has an excellent tutorial to get you started on a single node. This tutorial outlines
the creation of a more complex multi-node setup.
The system is divided into two parts: a master, which handle scheduling and task distribution,
with a slave providing compute power. Mesos uses Zookeeper for cluster coordination and leader
election. A key component is service discovery: you don’t know which host or port will be assigned to a
task, which makes, say, accessing a website running on a slave difficult. The Marathon API allows you to
query task information, and we use this feature to configure HAProxy frontend/backend resources.
Our master run:
and our slave run:
The mesos-ansible playbook will use nodes in the mesos_masters for a variety of configuration options.
First, the /etc/zookeeper/conf/zoo.cfg will list all master nodes, with /etc/zookeeper/conf/myid being
set appropriately. It will also set up upstart scripts in /etc/init/mesos-master.conf,
/etc/init/mesos-slave.conf with default configuration files in /etc/defaults/mesos.conf
Marathon and HAProxy
If Mesos is the OS for the data center, Marathon is the init system. Marathon allows us to http post new tasks,
containing docker container configurations, which will run on Mesos slaves. With HAProxy we can use the masters
as a load balancing proxy server routing traffic from known hosts (the masters) to whatever node/port is running
the marathon task. HAProxy is configured via a cron job running a custom bash script. The script queries the
marathon API and will route to the appropriate backend by matching a host header prefix to the marathon job name.