Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go »
No member browsing this thread
Thread Status: Active
Total posts in this thread: 27
Posts: 27   Pages: 3   [ 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 8967 times and has 26 replies Next Thread
gs003
Cruncher
NL
Joined: Aug 11, 2021
Post Count: 11
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
raspberry pi cluster setup - calling on user dataman

This forum doesn't seem to allow direct messages to other users. Hopefully dataman can see this, or someone can reach out
I am interested in doing a raspberry pi cluster. in this comment dataman seems to already have done it.

dataman or anyone familiar with rpi cluster software setup, care to share instructions?
[Aug 14, 2021 8:13:31 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Acibant
Advanced Cruncher
USA
Joined: Apr 15, 2020
Post Count: 126
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

Ubuntu has a detailed tutorial on building a Kubernetes cluster. Note that I have not done this myself. I have 8 Odroid N2+ 4 GB single-board computers and they are each their individual nodes (not clustered).
----------------------------------------

[Aug 14, 2021 11:13:49 AM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 609
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

For the rest of us can you post your progress in this thread.
[Aug 14, 2021 3:49:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
gs003
Cruncher
NL
Joined: Aug 11, 2021
Post Count: 11
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

Yes i will post my updates, for now I will be trying the ubuntu instructions as mentioned by acibant, and also this one on opensource.com
[Aug 14, 2021 7:13:56 PM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 609
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

From what I can understand of datasman's comment, he has 16 in a Yahboom cluster which is just a brand name for a case or rack to hold multiple Pi4s (see the picture in the amazon link). The rest are standalone in their own case.

Acibant has 8 and each in it's own case. Now his link about "building a Kubernetes cluster" puts a different meaning to the term "cluster" and has nothing to do with a physical rack. It's all about Kubernetes nodes and joining multiple PIs to form a production-grade Kubernetes cluster. Multiple PIs acting as one fault tolerant cluster for apps or services like the cloud. The optional hardware suggests a cluster rack or cluster case. This is how we are using the term. Physical plastic hardware.

Your link is also about on building a Kubernetes cluster with the Raspberry Pi.

I doubt you need a Kubernetes cluster.

Look at this link and check "What you’ll need" and "optional" https://ubuntu.com/tutorials/how-to-kubernete...n-raspberry-pi#1-overview
They describe multiple standalone PIs each with a Ubuntu Server image on a microSD card (and more stuff). Later they link them up to form a "Kubernetes cluster".
20 PIs working standalone is probably what you want. I think that's all you need and you're in business. I make it sound trivial. It's lots of work.
----------------------------------------
[Edit 1 times, last edit by BobbyB at Aug 15, 2021 1:56:36 AM]
[Aug 15, 2021 1:55:29 AM]   Link   Report threatening or abusive post: please login first  Go to top 
gs003
Cruncher
NL
Joined: Aug 11, 2021
Post Count: 11
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

yes my question has always been about having multiple nodes acting as one . I've not done this before and my knowledge is limited. for this community though, is it better then to have multiple devices acting individually or as a cluster?
[Aug 15, 2021 9:53:27 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Acibant
Advanced Cruncher
USA
Joined: Apr 15, 2020
Post Count: 126
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

There's no need to cluster them. Each one can have its own work queue and do its own processing. This isn't a mission-critical situation where you need a fault-tolerant setup to keep a web application or similar going in the case of hardware failure.

As an aside, please make sure there is adequate cooling, either with ambient airflow across passive heatsinks added onto the chips or active cooling via a mini fan/heatsink combo. I liked the look of this case on Etsy that has built-in fans you can use across heatsinks on the individual Pis. If you don't already have power supplies then you can use a PoE (Power over Ethernet) switch with the PoE hat add-on to get an even cleaner setup using one cable for networking and power.
----------------------------------------

[Aug 16, 2021 12:29:35 AM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 609
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

From https://kubernetes.io/
Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications

Does "automating deployment, scaling" sound like what you want to do? You have 20 PIs. That's 80 cores crunching WUs. Management is an issue but as long as they are working there is nothing to manage really. Well... not really.
What each is doing. How to handle Ubuntu updates. Which Pi has what IP.
Can someone else shed some light on this using Kubernetes?

I see issues with 20 of anything which compute. None which can't be resolved.

Even in the link you listed each Pi is built standalone with a server image of Ubuntu. Then they link them up. Once that first server image is configured correctly and eats up WUs from WCG what do you need the K8 for? Want another Pi? Put it in that rack, make a clone of the OS, and plug it in. 4 more cores crunching.

I see no instructions on how to make a containerized application of the WCG software. All the case studies they show are all about offering services and apps to a client. With WCG you are the client.

I would start with building up the racks with your 20 PIs. In your link they give an example of building up a server image with all the parts needed like sshd. Then they show how to clone that microSD so you can insert one into each PIs. That is useful.
----------------------------------------
[Edit 1 times, last edit by BobbyB at Aug 16, 2021 12:40:00 AM]
[Aug 16, 2021 12:30:11 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Dataman
Ace Cruncher
Joined: Nov 16, 2004
Post Count: 4865
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

My apologies mates, I just noticed this thread. sad
I have 16 RPi's in a cluster and 5 stand-alone RPi's in various cases.
The 16 Pi cluster consists of:

3 x Yahboom Raspberry Pi Cluster Case ( https://www.amazon.com/gp/product/B07K72STFB/...o09_s00?ie=UTF8&psc=1 )
16 x Raspberry Pi
16 x Micro SSD with Raspberry OS
16 x 3 amp power supply w/switches
1 x Monitor
1 x Wireless keyboard and mouse
1 x HDMI cable (From KVM to monitor or KVM to VGA converter)
3 x HDMI to mini HDMI adapters (these are only needed for the Version 4's https://www.amazon.com/gp/product/B0111HRJT0/...o00_s00?ie=UTF8&psc=1 )
1 x HDMI to VGA converter (This is only needed if your monitor is VGA and the KVM is HDMI https://www.amazon.com/gp/product/B078NM7KYW/...o08_s00?ie=UTF8&psc=1 )
1 x 16 port HDMI KVM switch cables included. ( https://www.amazon.com/gp/product/B01H1Q2V2K/...o09_s01?ie=UTF8&psc=1 )
1 x 16 port data switch ( https://www.amazon.com/gp/product/B07GR9S6FN/...o09_s00?ie=UTF8&psc=1 )
16 x Cat 6 data patch cables (mine are 2 ft but size to your needs.)
1 x Cat6 cable (from your router to the data switch)
1 x rack (to keep things tidy)
4 x Belkin Power Strips (I attach these to the outside of the rack and chain them together so there is only 1 to plug into the wall outlet)

I think that is everything. There are ways to do it cheaper. My array has grown slowly over the years and if I did it today I could eliminate some stuff. Mine are hard wired to the Ethernet as some of the Pi's are older versions and kept losing WiFi connections.

Hope this helps and again sorry for being tardy.

cowboy
----------------------------------------


[Aug 16, 2021 5:11:43 PM]   Link   Report threatening or abusive post: please login first  Go to top 
BobbyB
Veteran Cruncher
Canada
Joined: Apr 25, 2020
Post Count: 609
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: raspberry pi cluster setup - calling on user dataman

Acibant: So far this is the best looking and functional rack I have seen.

Dataman: What was your decision to go with KVM instead headless? I also use a KVM for my machines.
[Aug 17, 2021 2:45:29 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 27   Pages: 3   [ 1 2 3 | Next Page ]
[ Jump to Last Post ]
Post new Thread