Tag-Archive for » mcollective «

Tuesday, July 26th, 2011 | Author:

eth0 Mcollective has been able to fire up puppetd runs for a while now, via a standalone RPC call or through the puppet commander binary (check it out, spread your load). I wanted to be able to fire up my chef clients with mcollective, to use metadata to filter what should be impacted. So I wrote a little piece of ruby, mostly based on the puppet one, to achieve this. You can now do the following :

mco rpc chef runonce or  mco rpc chef status
The plugin is available on my github mcollective repository.

Category: Général  | Tags: ,  | Comments off
Tuesday, March 15th, 2011 | Author:

agent_smithIt started like a toy, to learn a little more about mcollective agents but I finally turned into something useful (at least for me). I pushed my agent “smith” on my github account. It allows you to install or remove agents within mcollective. I usually use my configuration management tool to deploy such pieces of software but it can be useful in some case to go without it.

The mandatory internet meme reference : yes xzibit approves.

Category: Général  | Tags: ,  | Comments off
Friday, March 11th, 2011 | Author:

eth0This post follows my previous one, dealing with the reuse of chef providers of chef in mcollective. In the comments Adam Jacob had an interesting word and when I wrote my second agent, to manage package I saw it would be a piece of cake to write a really generic agent, due to the nature of chef resource (and the way to invoke them)

So, this is a generic chef resource mcollective agent, with the associated example client code. It anyway deserves an little explanation; it is not mean to work with a command line invocation. Why ? Because I push quite “complex” data as the resourceactions parameter. The only way I found to make this work from command line is to use eval on the argument, which is no way acceptable. Anyway I hope some people will find this useful.

Category: BOFH Life, Code, SysAdmin, Tech  | Tags: , , ,  | Comments off
Tuesday, March 08th, 2011 | Author:

eth0It has been quite calm for a couple of months here. I have switched job, it explains why I had less time to post some things.I now work at fotolia, and I switched from puppet to chef (no troll intended, I still think puppet is a great tool, please read this).

However, a tool I still have is the awesome mcollective. Unfortunately, the most used agents (package, service) relay on puppet providers to do their actions. Fortunately, open source is here, so I wrote a (basic) service agent that uses chef providers to start/stop or restart an agent. It still needs some polish for the status part (ho the ugly hardcoded path) but I was quite excited to share this. Freshly pushed on github !

Thanks to Jordan Sissel for minstrel, an awesome debug tool, the opscode team for the help on the provider and R.I. Pienaar for mcollective (and the support).

Category: BOFH Life, Code, SysAdmin, Tech  | Tags: , , ,  | 3 Comments
Monday, September 27th, 2010 | Author:

eth0Basing more and more stuff on mcollective means relying more and more on one of its underlying components : the activeMQ middleware, and more precisely the stomp connector. I hit a weird bug a few days ago and realized that I was not functionnaly monitoring this part of the system. The port was bound and responded to connections, subscriptions were possible but messages didn’t pass through.So I wrote this little plugin that makes this possible : it creates a random string, sends it to a queue and then reads the queue to check if the result is the same.

This has been possible with the help of @ripienaar. Thanks for the explanation for the difference between topics & queues !

Category: Général  | Tags: , , ,  | Comments off
Tuesday, September 14th, 2010 | Author:

eth0I recently switched from puppet daemon to mcollective commander : it kicks the “stuck in outer space puppet daemon” feature out of the way and brings me nice features (as load control). To do so I deployed the puppetd agent over my boxes.

As most of the sysadmins say : “if it’s not monitored, it doesn’t exist” I had placed a script in cron based on the puppetlast script to report by mail once a day which hosts had not checked in during the last 30 minutes. This method had 2 serious flaws : it runs only once a day (I hate being flooded by mails) and test machines keep nagging you until you remove the yaml files on the puppetmaster. Talking with Volcane on #mcollective I discovered that the agent was able to report when the client last run, so I decided to use this to check my puppet runs freshness with a nagios plugin.

Good bye cron job, say hello nagios.

Category: Code, NetAdmin, Puppet, SysAdmin  | Tags: , ,  | Comments off
Wednesday, September 08th, 2010 | Author:

eth0 Most people that work with puppet use a VCS : subversion, git, CVS, mercurial… Pick yours. My company uses subversion and each cmmit to the repository needs to be pulled by the master. Since I have two masters, I also want them to be synchronized. Once again it’s mcollective that comes to the rescue. I wrote a very simple agent (a 5 minutes work, to be improved) that can update a specified path. Grab it here. Once it is deployed you can use a post commit hook that calls it.

Example of mine :

#!/usr/local/bin/ruby
 
require 'mcollective'
include MCollective::RPC
 
mc = rpcclient("svnagent")
mc.progress = false
mc.class_filter "puppet::master"
mc.update(:path => "/etc/puppet")

The agent will only be called on machines being puppet masters by using the class filter.

Category: Code, Puppet, SysAdmin  | Tags: , ,  | 2 Comments
Wednesday, August 18th, 2010 | Author:

eth0 In my last post I’ve been talking about mcollective (check the new website) and mongo and how awesome it is.

I’ve mentioned the docs available in the wiki but I’ve been too fast on this point; it’s not yet in the wiki (only some parts) so here is a little “guide” about setting the pieces work together. All comments are welcome.

  • Deploy meta.rb on all your  nodes. It will make the metadata available to other nodes.
  • Add the following lines to your server.cfg file :
registration = Meta
registerinterval = 300
  • Install a mongoDB server (debian ships it in squeeze)
  • Deploy the mongo registration agent on one node (don’t be like me, do not start deploying it on all nodes !). It will “suck” metadata from nodes, and insert it to the mongo database.
  • Add the following lines to your server.cfg on the host with the registration agent :
plugin.registration.mongohost = mongo.mycorp.net
plugin.registration.mongodb = puppet
plugin.registration.collection = nodes
  • Connect to your mongoDB and enjoy :
$ mongo mongo.mycorp.net/puppet
MongoDB shell version: 1.6.0
connecting to: mongo.mycorp.net/puppet
> db.nodes.find().count()
59
  • You’re done !
Category: BOFH Life, SysAdmin, Tech  | Tags: , , ,  | Comments off
Thursday, August 12th, 2010 | Author:

eth0I run a bunch of box under OpenBSD at $WORK and I wanted to be able to run mcollective on these too. Unfortunately, there were no package available for this OS. So I took time and with some help from landry@ I was able to build a port. It has been integrated into the github repo quickly. But to be able to use mcollective you also need to have the ruby stomp connector. As I don’t like using gems and prefer them packages, I also built a port for it. You can grab it in my github repository here.

R.I. Pienaar also added some cool mongo stuff to mcollective (see the project wiki for more details) and I built 2 more ports for the bson & mongo gems.

These packages have been tested under OpenBSD 4.7 (i386, but they are no_arch ones, ruby related), let me know if they work for you.

Category: BOFH Life, SysAdmin, Tech  | Tags: , , ,  | Comments off
Wednesday, June 23rd, 2010 | Author:

eth0If you read this blog you may know I daily use puppet at $WORK. Puppet is made to maintain configuration on machines, but not for one shot actions. For a couple of month I used to work with fabric for this but It had a few drawbacks, mainly because you need to maintain the list of hosts you want to act on and that it hates dead hosts, even if ghantoos dropped a link showing how to get rid of this. So I replaced it with mcollective : not exactly the same (it uses an agent instead of SSH) but it dynamicaly knows which hosts are up, allows to filter on facts (from facter, the puppet companion).

You could think that needing an agent is a drawback but it allows much complex actions, fine grained logic. Moreover it does not require much work to be installed if you already have puppet running. If you have a high number of machines you gain scalability with real parallel actions : it does not take longer to run on 5 machines than on 100.

One more point is the active development : R.I. Pienaar released 2 versions recently, improving access control, adding DDL to create interfaces easily. You can give your unprivileged staff some power through a web interface in less than 100 lines of code.

A tool for sysadmins that appreciate the devops spirit and want to do more in less time !

I’ll be publishing my agent and related tools on my github account.

Category: BOFH Life, SysAdmin, Tech  | Tags: , , ,  | Comments off