| Index | Recent Threads | Unanswered Threads | Who's Active | Guidelines | Search |
 |
World Community Grid Forums
Category: Active Research
Forum: OpenPandemics - COVID-19 Project
Thread: Optimize the use of available resources in WCG |
| No member browsing this thread |
|
Thread Status: Active Total posts in this thread: 12
|
|
| Author |
|
|
mgpointner
Advanced Cruncher Argentina Joined: Nov 16, 2009 Post Count: 55 Status: Offline Project Badges: 
|
Hello everyone, the following text was written in Spanish I hope the translation is good...
----------------------------------------I feel that there is a lack of prioritization in the use of resources available for WCG. On the one hand, a short time ago more support was requested from the MCM (Mapping Cancer Markers) subproject. This project currently only has CPU work (no GPU) and evenlimited to certain operating systems on certain CPU architectures, for example, they have no work for Linux on ARM. On the other hand, the OPN (OpenPandemics) subproject (compared to MCM) accepts work for GPU and also work for Linux on ARM, but recently from this project said that they restrict the workunits for GPUs because they do not have enough capacity to analyze the received data and for generate new workunits, therefore there is a large capacity available in unused GPUs, as was evident during the stress test. And this is where I see the lack of coordination between projects, it would be very good that from the OpenPandemics project, the maximum capacity would be generated GPU workunits and limit CPU workunits, inviting users with available capacity to process MCM CPU workunits even prioritizing work on CPU for platforms that MCM cannot take advantage of, for example Linux on ARM. My mother is suffering from cancer and it is very hard to see how the life of a loved one is fading, many of those who read this will have the great lucky not to be going through a similar situation but since this disease affects 1 in 4 people, the chances of suffering it in the future oneself or a loved one is very high. For this reason I would like to be able to give my full support to any project that improves the chances of coping with this damn disease. In Spanish: Hola a todos. Siento que esta faltando un ordenamiento de prioridades en el uso de los recursos disponibles para WCG (World Community Grid). Por un lado, hace poco tiempo se pidio mas apoyo desde el subproyecto MCM (Mapping Cancer Markers), este proyecto actualmente solo tiene trabajo para CPU (no GPU) e incluso limitado a determinados sistemas operativos sobre determinadas arquitecturas de CPU, por ejemplo, no tienen trabajo para Linux sobre ARM. Por otro lado, el subproyecto OPN (OpenPandemics), comparado con MCM, acepta trabajo para GPU y tambien trabajo para Linux sobre ARM. Recientemente desde este proyecto comentaron que restringen las workunits para GPU debido a que no tienen la suficiente capacidad para analizar los datos recibidos y generar nuevos workunits, por lo tanto queda una gran capacidad disponible en GPU sin uso, como quedo en evidencia durante el stress test. Y aqui es donde veo la falta de coordinacion entre proyectos, seria muy bueno que desde el proyecto OpenPandemics, se generara la maxima capacidad de workunits para GPU y limitar los workunits para CPU, invitando a los usuarios con capacidad disponible a procesar workunits de CPU para MCM incluso priorizando trabajo en CPU para plataformas que MCM no puede aprovechar, por ejemplo Linux en ARM. Mi madre esta sufriendo de cancer y es muy duro ver como se va apagando la vida de un ser querido, muchos de los que lean esto tendran la gran suerte de no estar pasando por una situacion similar pero desde que esta enfermedad afecta a 1 de cada 4 personas, las chances de sufrirlo en un futuro uno mismo o un ser querido es muy alta. Por esta razon me gustaria poder darle todo mi apoyo a todo proyecto que mejore las chances de sobrellevar esta maldita enfermedad.  |
||
|
|
Mike.Gibson
Ace Cruncher England Joined: Aug 23, 2007 Post Count: 12427 Status: Offline Project Badges: 
|
mgpointner
Maybe once Krembil are actually running WCG they will see the benefit of GPU, etc for MCM, but maybe their scientists wouldn't be able to cope. Some of us can not get get our GPUs to work on OPN so cutting down on CPU work and boosting GPU would leave us unable to do OPN. But then their scientists couldn't cope with the extra work. However, I do agree with your philosophy. Mike |
||
|
|
mgpointner
Advanced Cruncher Argentina Joined: Nov 16, 2009 Post Count: 55 Status: Offline Project Badges:
|
mgpointner Maybe once Krembil are actually running WCG they will see the benefit of GPU, etc for MCM, but maybe their scientists wouldn't be able to cope. Some of us can not get get our GPUs to work on OPN so cutting down on CPU work and boosting GPU would leave us unable to do OPN. But then their scientists couldn't cope with the extra work. However, I do agree with your philosophy. Mike And that's what I'm trying to say, your CPU cycles are more important in MCM than in OPN |
||
|
|
alanb1951
Veteran Cruncher Joined: Jan 20, 2006 Post Count: 977 Status: Offline Project Badges: 
|
mgpointner and Mike Gibson,
Good thoughts! Regarding making more platforms available for MCM1... The algorithms in MCM1 may not lend themselves to GPU use; I'd've thought that if it had looked to be worth the effort, the MCM team would already have gone there!!! However, making a version to run on ARM (RPi &c...) would seem to be a very good thing. I don't know what became of the last MCM Beta test (which seemed to be looking at using a slightly revised algorithm...) but one thing that stood out on my Raspberry Pi was that MCM1 tasks finished a lot quicker than OPN1 tasks, whilst on all my other machines, OPN1 tasks finish faster than MCM1 tasks! There's actually a good reason for that -- although OPN1 isn't the same sort of memory-hog that MIP1 was, it does hit on L3 cache quite a lot, albeit that the cache miss rate is not extremely high. MCM1, on the other hand, seems to be able to get quite a lot done within L1/L2 cache, not going out to L3 cache with anything like the same frequency. This matters for OPN1 on machines like the Raspberry Pi because they don't have L3 cache so all those L2 misses or refreshes go to main memory, with the predictable performance hit. I hope that Krembil will get the MCM people to make a generic ARM build; it probably won't matter whether they do a 64 bit version as well as a 32-bit version, though it would be nice if they did both. I have to trust that the mindset isn't "We've done it for Android..." I'll happily Beta-test anything they do build, and I'll do performance monitoring if they want it! Cheers - Al. P.S. Another science that might benefit from an ARM (non-Android) build is SCC1 if/when it resurfaces; it has a reasonable memory footprint and its L3 cache accesses are also low by comparison with OPN1. |
||
|
|
mgpointner
Advanced Cruncher Argentina Joined: Nov 16, 2009 Post Count: 55 Status: Offline Project Badges:
|
mgpointner and Mike Gibson, Good thoughts! Regarding making more platforms available for MCM1... The algorithms in MCM1 may not lend themselves to GPU use; I'd've thought that if it had looked to be worth the effort, the MCM team would already have gone there!!! However, making a version to run on ARM (RPi &c...) would seem to be a very good thing. I don't know what became of the last MCM Beta test (which seemed to be looking at using a slightly revised algorithm...) but one thing that stood out on my Raspberry Pi was that MCM1 tasks finished a lot quicker than OPN1 tasks, whilst on all my other machines, OPN1 tasks finish faster than MCM1 tasks! There's actually a good reason for that -- although OPN1 isn't the same sort of memory-hog that MIP1 was, it does hit on L3 cache quite a lot, albeit that the cache miss rate is not extremely high. MCM1, on the other hand, seems to be able to get quite a lot done within L1/L2 cache, not going out to L3 cache with anything like the same frequency. This matters for OPN1 on machines like the Raspberry Pi because they don't have L3 cache so all those L2 misses or refreshes go to main memory, with the predictable performance hit. I hope that Krembil will get the MCM people to make a generic ARM build; it probably won't matter whether they do a 64 bit version as well as a 32-bit version, though it would be nice if they did both. I have to trust that the mindset isn't "We've done it for Android..." I'll happily Beta-test anything they do build, and I'll do performance monitoring if they want it! Cheers - Al. P.S. Another science that might benefit from an ARM (non-Android) build is SCC1 if/when it resurfaces; it has a reasonable memory footprint and its L3 cache accesses are also low by comparison with OPN1. Another vote for MCM version for Linux on ARM. Despite having a Raspberry Pi 4 running Android with MCM, I couldn't find a way to make it work without being connected to a monitor via HDMI, if I disconnect it, Android shuts down. The power of Linux devices over ARM cannot be underestimated, Rosetta@home is getting 5,249 GigaFLOPS right now. |
||
|
|
mgpointner
Advanced Cruncher Argentina Joined: Nov 16, 2009 Post Count: 55 Status: Offline Project Badges:
|
Date----------Runtime----------Credit----------WorkUnits
----------------------------------------10/07/2021-----531:092:13:43:14-----1,514,942,211-----1,859,899-----Total WCG 10/07/2021-----377:253:22:51:21-----747,309,700-----1,206,484-----MCM 10/07/2021-----113:341:16:54:31-----693,253,954-----635,953-----OPN 10/07/2021-----0:148:00:32:22-----651,846-----222-----Help Stop TB 10/07/2021-----39:078:21:25:00-----73,726,711-----17,24-----Africa Rainfall It is interesting to see how the use of GPU influences the performance in the projects. OPN receives almost the same processing power as MCM (693 vs 747 million points) with 1/3 the runtime and 1/2 the workunits [Edit 2 times, last edit by mgpointner at Oct 8, 2021 2:45:02 AM] |
||
|
|
MJH333
Senior Cruncher England Joined: Apr 3, 2021 Post Count: 268 Status: Offline Project Badges:
|
Another vote for MCM version for Linux on ARM. Despite having a Raspberry Pi 4 running Android with MCM, I couldn't find a way to make it work without being connected to a monitor via HDMI, if I disconnect it, Android shuts down. mgpointner, I agree that it would be good to have an MCM version for Linux on ARM. I had the same problem you did when running MCM on Android on the Raspberry Pi 4. I have now solved this by connecting the Pi to something which calls itself an “HDMI Dummy Plug Headless Ghost Display Emulator”. This cost just a few pounds and apparently tricks the Pi into thinking that a display is attached. It seems to work, though I don’t understand how! By the way, I am grateful to you and others on the forum who have discussed running Android on the Pi. Your comments encouraged me to give it a try, and it was not as difficult as I had thought it would be! Cheers, Mark |
||
|
|
mgpointner
Advanced Cruncher Argentina Joined: Nov 16, 2009 Post Count: 55 Status: Offline Project Badges:
|
Hi Mark
----------------------------------------I had the same problem you did when running MCM on Android on the Raspberry Pi 4. I have now solved this by connecting the Pi to something which calls itself an “HDMI Dummy Plug Headless Ghost Display Emulator”. This cost just a few pounds and apparently tricks the Pi into thinking that a display is attached. It seems to work, though I don’t understand how! Can you post a link for the emulator? I want to make a cluster of Pis and I need it. Do you use EFMER Boinc Task for monitoring Boinc on the Pi? By the way, I am grateful to you and others on the forum who have discussed running Android on the Pi. Your comments encouraged me to give it a try, and it was not as difficult as I had thought it would be! Thanks |
||
|
|
MJH333
Senior Cruncher England Joined: Apr 3, 2021 Post Count: 268 Status: Offline Project Badges:
|
Can you post a link for the emulator? I want to make a cluster of Pis and I need it. Do you use EFMER Boinc Task for monitoring Boinc on the Pi? If you just search for the description above on a major online retailer's website, you will find the emulators. I would suggest you try one and check that it works for you before buying lots for your cluster! I use BoincTasks and VNC to keep an eye on my machines. Cheers, Mark |
||
|
|
Dayle Diamond
Senior Cruncher Joined: Jan 31, 2013 Post Count: 452 Status: Offline Project Badges: 
|
Mgpointner, you are correct.
I wish the very best for your mom. The feelings of donors who don't have eligible GPUs and want to participate in OPN should not overrule the needs of the world to have both projects completed as quickly and efficiently as possible. |
||
|
|
|