Index  | Recent Threads  | Unanswered Threads  | Who's Active  | Guidelines  | Search
 

Quick Go ยป
No member browsing this thread
Thread Status: Active
Total posts in this thread: 70
Posts: 70   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread
Author
Previous Thread This topic has been viewed 42485 times and has 69 replies Next Thread
julemand101
Cruncher
Denmark
Joined: Feb 28, 2020
Post Count: 7
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

That was also a theory I had but if you check the executable you can see it is statically linked:
[root@beta www.worldcommunitygrid.org]# file wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu
wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=39f7c6cb1605715b272571e2ffff7acf6b0af16c, stripped


And does therefore not have any external dependencies to any libraries (not even glibc):
[root@beta www.worldcommunitygrid.org]# ldd wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu
not a dynamic executable


The issue could then be the Linux kernel where I run 5.8.12. I am going to test with 5.4.68 which are the LTS kernel version available for my OS (Arch Linux).
[Sep 30, 2020 7:25:36 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7660
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

OK, I have completed the first 4 MIP units on a 2600K running Linux 4.4.0-21 generic x86 _64 kernel. Linux Mint 18 Sarah. all completed between 1.2 and 1.6 hours and all are valid. The other 4 jobs running were SCC. I will up the ante to 5 concurrent jobs and see hw that goes.
You may be right, it may be your updated kernel.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Sep 30, 2020 6:49:41 PM]   Link   Report threatening or abusive post: please login first  Go to top 
julemand101
Cruncher
Denmark
Joined: Feb 28, 2020
Post Count: 7
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

After running a day with Linux kernel 5.4.68-1-lts I can conclude that MIP1 are still failing. I will try to see if I can get access to an even older kernel but that is not something there is official support for in Arch Linux.

But I will not leave out the possibility that it is glibc since the stacktrace does contain a reference to that library (and libnss_files-2.32.so). So it could be that it is still using glibc even if it is statically linked.

Oct 01 00:07:13 beta.archlinux.dk systemd-coredump[16147]: Process 14664 (wcgrid_mip1_ros) of user 969 dumped core.

Stack trace of thread 14664:
#0 0x00000000047cc521 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x43cc521)
#1 0x00000000047bcb49 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x43bcb49)
#2 0x00000000047a815e n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x43a815e)
#3 0x00000000047b4936 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x43b4936)
#4 0x00000000047b1b27 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x43b1b27)
#5 0x00000000046c88c4 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x42c88c4)
#6 0x00000000046ddb60 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x42ddb60)
#7 0x00007f110c602e84 __nss_readline (/usr/lib/libc-2.32.so + 0x124e84)
#8 0x00007f110d57963d n/a (/usr/lib/libnss_files-2.32.so + 0x663d)
#9 0x00007f110d5799c4 _nss_files_getpwuid_r (/usr/lib/libnss_files-2.32.so + 0x69c4)
#10 0x0000000004808bbc n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x4408bbc)
#11 0x00000000048089ec n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x44089ec)
#12 0x00000000041d5234 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x3dd5234)
#13 0x0000000002d6bca8 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x296bca8)
#14 0x0000000002d7505d n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x297505d)
#15 0x0000000002d7d0f3 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x297d0f3)
#16 0x0000000002d7d35b n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x297d35b)
#17 0x0000000002d8c5e4 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x298c5e4)
#18 0x0000000002cd62f3 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x28d62f3)
#19 0x0000000002d5f48d n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x295f48d)
#20 0x00000000030e2439 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x2ce2439)
#21 0x00000000030e79bd n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x2ce79bd)
#22 0x0000000002d0d035 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x290d035)
#23 0x0000000002470ae3 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x2070ae3)
#24 0x000000000247106a n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x207106a)
#25 0x00000000010a1b94 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0xca1b94)
#26 0x0000000000faf2cc n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0xbaf2cc)
#27 0x0000000000fb17fc n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0xbb17fc)
#28 0x0000000000411ec4 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x11ec4)
#29 0x0000000004794bb4 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x4394bb4)
#30 0x0000000004794ce6 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x4394ce6)
#31 0x00000000009658d6 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x5658d6)

Stack trace of thread 14665:
#0 0x00000000046dd3d1 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x42dd3d1)
#1 0x000000000480b3b4 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x440b3b4)
#2 0x000000000469fa6f n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x429fa6f)
#3 0x000000000468c44d n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x428c44d)
#4 0x00000000046d6925 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x42d6925)
#5 0x000000000480ec89 n/a (/var/lib/boinc/projects/www.worldcommunitygrid.org/wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu + 0x440ec89)

----------------------------------------
[Edit 1 times, last edit by julemand101 at Oct 1, 2020 8:26:30 AM]
[Oct 1, 2020 8:24:47 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7660
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

I have now run through about 20 MIP units with nary a problem. I see you are partial to the Arch distro for your use, but it may pay to look at one of the many other distros if you can not find the kernel you are seeking. It would be interesting to see if any other Arch users are experiencing this difficulty. If no other users of your distro are experiencing the same problems, it might be hardware.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Oct 1, 2020 1:03:44 PM]   Link   Report threatening or abusive post: please login first  Go to top 
TeamAdmin
Cruncher
Joined: Aug 21, 2010
Post Count: 1
Status: Offline
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

Arch user here (Endeavour OS) with AMD Ryzen 1700.
All mip1 hare failing. I also noticed reference to /usr/lib/libc-2.32.so in the dumps.
[Oct 1, 2020 5:37:52 PM]   Link   Report threatening or abusive post: please login first  Go to top 
mdxi
Advanced Cruncher
Joined: Dec 6, 2017
Post Count: 109
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

Arch is a rolling release distro, so it's much closer to the leading edge of upstream development than a timed-release distro. But there's nothing exotic about it. We Arch users should probably be viewed as canaries because he issue currently affecting us is very likely going to become a problem for everyone else... eventually :)

I also thought BOINC apps were statically compiled, so I had also assumed it couldn't be a library issue. There is a way to find that out...
# ldd ./wcgrid_mip1_rosetta_7.16_x86_64-pc-linux-gnu 
not a dynamic executable

Yeah, that's static, so any references to libs should be to the compiled-in version? Unless my understanding of how static binaries is off (which is, of course, possible). But in any case, libc 2.32-4 is the current build on Arch, so that shouldn't be a mismatch in any situation?

So confusing.

Edit: I notice that MIP1 has declined from around 500k validated WUs/day a month ago, to 150K WUs/day for the past two weeks. I don't think that's related to this specific issue, but it's interesting.
----------------------------------------

----------------------------------------
[Edit 1 times, last edit by mdxi at Oct 5, 2020 6:05:55 AM]
[Oct 5, 2020 6:01:51 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7660
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

Edit: I notice that MIP1 has declined from around 500k validated WUs/day a month ago, to 150K WUs/day for the past two weeks. I don't think that's related to this specific issue, but it's interesting.

Probably more related to the increase in SCC.Since Linux is more efficient on the Vina projects than Windows, there may have been a rotation out of MIP1 to SCC by a significant number of Linux users.
The Arch distro problems with MIP1 may also be a contributing factor.
Update: I ran about 50 MIP units on Linux Mint in combination with other projects (SCC,ARP and OPN) and they all came back valid.
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Oct 5, 2020 11:52:42 AM]   Link   Report threatening or abusive post: please login first  Go to top 
buscher
Cruncher
Joined: Oct 3, 2011
Post Count: 5
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

I also have problems with MIP1, I did opt-out for a month or so, hoping that it would fix itself magically, but just tried again on 2 different PCs, but still, they all error out.

Kernel: 5.8.17 / 5.9.2
Glibc: 2.32

CPU: AMD Ryzen 7 2700X / Intel i7-10610U

EDIT: using Linux Gentoo btw :)
----------------------------------------
[Edit 1 times, last edit by buscher at Nov 1, 2020 9:46:26 AM]
[Nov 1, 2020 9:36:11 AM]   Link   Report threatening or abusive post: please login first  Go to top 
Sgt.Joe
Ace Cruncher
USA
Joined: Jul 4, 2006
Post Count: 7660
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

I also have problems with MIP1, I did opt-out for a month or so, hoping that it would fix itself magically, but just tried again on 2 different PCs, but still, they all error out.

Kernel: 5.8.17 / 5.9.2
Glibc: 2.32

CPU: AMD Ryzen 7 2700X / Intel i7-10610U

EDIT: using Linux Gentoo btw :)

Have you tried a different Linux distro ?
Cheers
----------------------------------------
Sgt. Joe
*Minnesota Crunchers*
[Nov 1, 2020 1:08:11 PM]   Link   Report threatening or abusive post: please login first  Go to top 
buscher
Cruncher
Joined: Oct 3, 2011
Post Count: 5
Status: Offline
Project Badges:
Reply to this Post  Reply with Quote 
Re: Lots of MIP1 WUs error out

No, I have not and frankly I will not, as all other projects are working fine and I have no other problems beside this one. (All other: the rest of WCG, Rosetta, GPUGRID, ...)
[Nov 1, 2020 1:51:50 PM]   Link   Report threatening or abusive post: please login first  Go to top 
Posts: 70   Pages: 7   [ Previous Page | 1 2 3 4 5 6 7 | Next Page ]
[ Jump to Last Post ]
Post new Thread