Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 9
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 2108 times and has 8 replies Next Thread
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Homefarm - A configuration and management system for BOINC

In the past 10 months I've gone from one spare machine running WCG, to my main workstation running WCG, to building machines explicitly for running WCG, to the realization that I'll soon have a small fleet of machines which do nothing but crunch WCG workunits. In other words, I'm on the verge of having a compute farm in my house.

Since I'm a devops engineer, this naturally meant there was only one thing to do: set up a configuration management and orchestration system for these machines. It's called Homefarm, and if this sounds interesting to you, you can check out the code and docs here on Github.

It assumes one Raspberry Pi to act as a control node, and any number of x86_64 machines running Alpine Linux as compute nodes. I chose Alpine because of its compactness and focus on security. Management is done via Ansible playbooks.

As of this writing, the version is 0.9.1. I don't plan to make any changes other than bugfixes before 1.0.0, and work on new capabilities will pick up after that.

If you have highly customized BOINC client configurations, then Homefarm won't deliver what you're looking for right now (managing cc_config.xml is on the roadmap for post-1.0.0).

It does handle some things nicely though, including per-node project configurations and automated config of boinctui on the control node (so that it always knows about your compute nodes and can connect to them).

I don't expect anyone to run out and reinstall all their machines to test this, but please do look things over and let me know what you think, especially if you feel like there's something I should be handling, or if anything in the documentation is unclear. Thanks!
----------------------------------------

[Sep 19, 2018 3:22:51 PM]   Link   Report threatening or abusive post: please login first  Go to top 
QuantumEthos
Senior Cruncher
Joined: Jul 2, 2011
Post Count: 336
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm is an excellent node config ....
Docker makes this even better as the alpine can be ran in a container ...
So you can use windows on your gaming pc too!

https://www.worldcommunitygrid.org/forums/wcg/viewthread_thread,40988
[Sep 20, 2018 1:39:09 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Former Member
Cruncher
Joined: May 22, 2018
Post Count: 0
Status: Offline
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Can I use Ubuntu with this? What is Ansible?
[Sep 20, 2018 2:44:18 PM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Can I use Ubuntu with this? What is Ansible?


Ansible is a configuration and orchestration system (now owned by Red Hat, I believe). The project's website is at https://www.ansible.com/ but it's kinda impenetrable due to all the marketing talk, IMO. If you're familiar with Puppet or Chef, it's a tool that does the same things they do.

You're welcome to take the work I've done, fork it, and make the modifications necessary to base things on Ubuntu rather than Alpine. But as-is, no, you can't just swap out Ubuntu for Alpine. The packages and package management tools are different.
----------------------------------------

[Sep 21, 2018 5:52:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm 0.11.0 has been released. The most important feature of this release is the addition of a whole-cluster update tool.

Other fixes from this release (and 0.10.0, released yesterday) include
* Fixes to init script (stop, restart)
* Fixes to boinctui config generation with multiple nodes
* Updating BOINC to 7.14.2

Code and docs are at https://github.com/firepear/homefarm
----------------------------------------

[Oct 13, 2018 7:52:43 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm 0.13.0 has been released.

The project has moved from ALpine Linux to Arch Linux as a base. This resolved issues with the kernel being too old to report temps from Ryzen CPUs, as well as packaging issues which made GPGPU projects very difficult.

This should be the last disruptive update before v1.0.0.

There's a guided installer script for compute nodes now, which takes away a little of the complexity of setting up a cluster. You'll still need to understand networking basics though.

More info on the github page , and there's an upgrade doc for people who were on the previous versions.
----------------------------------------

[Jan 7, 2019 8:13:03 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm 0.14.0 has been released.

This is a quality-of-life release. There are improvements and fixes to the installer, and the lm_sensors package is now auto-configured during install.

To upgrade your cluster from a 0.13.x release, just run `./bin/update` from the homefarm directory.
----------------------------------------

[Jan 14, 2019 1:13:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm v0.16.0 has been released.

This version is a release candidate for v1.0.0. There will be no new feature work between now and the release of 1.0, only bugfixes.

The big addition in this release is a local repository mirror. All compute nodes will now install and update their OS packages from a local Arch linux mirror on the control node.

More info can be found in the release notes .

If you've been curious about Homefarm, this is the time to try it! I've been dogfooding it since my initial announcment, and now it's ready for testing and use by other people.

As always, I'm happy to answer questions. Thanks!
----------------------------------------

[Feb 21, 2019 6:42:35 AM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Homefarm - A configuration and management system for BOINC

Homefarm v1.0.0 has been released.

A couple of bugs were found in the scripts to install and restore control nodes. A big overhaul of the cluster maintenance documentation was done as well.

So that's it for now: it's feature-complete and thoroughly dogfooded. If you've been wishing you could centrally manage your crunchboxes, this might be what you've been looking for. Give it a try :)
----------------------------------------

[Feb 24, 2019 2:00:39 AM]   Link   Report threatening or abusive post: please login first  Go to top 
[ Jump to Last Post ]
Post new Thread