MLC@Home Cuda app is CPU limited

[biodoc]

I've noticed that the MLC cuda app is CPU limited so it may be a good project for some low powered cards.

Observations with a 2070S on a 3700X running universe on the CPUs:

1) 1 MLC cuda task per GPU with one free CPU thread (~65% shader utilization)
2) 2 MLC cuda tasks per GPU with 2 free CPU threads (~65% shader utilization)
3) 1 MLC cuda task per GPU with 2 free CPU threads (~70% shader utilization)
4) 1 MLC cuda task per GPU with universe suspended (~83% shader utilization)
5) 1 MLC cuda task per GPU with 8 free CPU threads (~80% shader utilization)( 80 watts draw on the GPU)

Here's the app_config.xml I used for test #2.

Code: Select all

<app_config>
	<app>
		<name>mlds-gpu</name>
		<gpu_versions>
			<gpu_usage>0.5</gpu_usage>
			<cpu_usage>0.01</cpu_usage>
		</gpu_versions>		
	</app>
</app_config>

[Fardringle]

I have the GTX 750ti in my daughters' computer (with an i7-4790) running MLC right now. It only gets about 8-10 hours of run time per day since they actually use that computer a lot for school and other stuff, and it's averaging a massive 46K per day over the 6 days I've had it attached to the project.

I'll have to remember to check the GPU utilization while it is running.

[10esseetony]

Thanks @biodoc

edit: and @StefanR5R for pointing me here

[Fardringle]

The 750Ti is running at 100% utilization with a single CPU core free on the i7-4790, so this is probably only an issue with fast GPUs.

[crashtech]

So something odd has popped up for me now that I have had time to look at what MLC@Home is doing. It looks like I can get better max GPU utilization in Windows(!) Linux seems to top out around 80% with no CPU going, but Windows can exceed 90%. Can anyone corroborate this?

[Fardringle]

I can't really test in Linux, but I can confirm that at least with my wimpy GPUs (GTX 750ti and GTX 1060) that GPU utilization is near 100% with a single task running and a single CPU core/thread free.

[Fardringle]

Since I have temporary use of a Quadro RTX 3000, I've been playing with it (stress testing ) and decided to let it have a go at the MLC GPU app for a bit.

Running a single GPU task with the CPU idle, Task Manager in Windows showed that it was using about 1.33 CPU threads and GPU-Z showed about 25-35% GPU utilization.

I added an app_config.xml file with <gpu_usage> and <cpu_usage> both set to .33 to see what would happen. BOINC did start running three GPU tasks simultaneously, and GPU-Z jumped up to 85-90% GPU utilization. However, it seems that the project is ignoring the <cpu_usage> tag, at least on this system, since each one of the three GPU threads was now using 1.33 CPU threads of its own (4 total threads in use).


That might explain biodoc's results that showed a significant improvement in GPU usage when the CPU is idle, if the app really wants to have that much CPU available for each running task.

After about 15 minutes of running three tasks simultaneously, GPU usage dropped to zero and % progress on the tasks also stopped, even though the Elapsed timer was still counting. I let it sit that way for a while with no change, then rebooted the computer but the tasks still wouldn't proceed any more.

Then, out of curiosity I changed the app_config file to
<gpu_usage>1</gpu_usage>
<cpu_usage>2</cpu_usage>
to see if it would run better with even more CPU dedicated to the single GPU task.

After I loaded the config files, a single MLC task started running normally again, with GPU-Z showing around 45-50% GPU load and Task Manager still showing 1.33 CPU threads active on the task. I'm not sure what the best settings will be and don't have time to do any more testing with this system right now, but I thought I'd share my initial findings to see if they make sense to (or help) anyone else..

[StefanR5R]

Originally, both <cpu_usage> and <gpu_usage> are doing the same: They inform boinc-client's task scheduler how the application behaves. But they do not control the application itself.

Only a few applications (vboxwrapper and some other multithreaded applications) have re-purposed <cpu_usage> and use it to modify application behavior. (Edit: Actually, they use app_version::avg_ncpus for this, but definitely not app::gpu_versions::cpu_usage. These applications don't have a GPU version after all.)

After about 15 minutes of running three tasks simultaneously, GPU usage dropped to zero and % progress on the tasks also stopped, even though the Elapsed timer was still counting. I let it sit that way for a while with no change, then rebooted the computer but the tasks still wouldn't proceed any more.
[...]
After I loaded the config files, a single MLC task started running normally again,

Sounds like a bug in the application, or in the CUDA libraries/driver stack.

[Fardringle]

Originally, both <cpu_usage> and <gpu_usage> are doing the same: They inform boinc-client's task scheduler how the application behaves. But they do not control the application itself.

Only a few applications (vboxwrapper and some other multithreaded applications) have re-purposed <cpu_usage> and use it to modify application behavior. (Edit: Actually, they use app_version::avg_ncpus for this, but definitely not app::gpu_versions::cpu_usage. These applications don't have a GPU version after all.)

Yeah. I just wanted to see what would happen with the changes.

After about 15 minutes of running three tasks simultaneously, GPU usage dropped to zero and % progress on the tasks also stopped, even though the Elapsed timer was still counting. I let it sit that way for a while with no change, then rebooted the computer but the tasks still wouldn't proceed any more.
[...]
After I loaded the config files, a single MLC task started running normally again,

Sounds like a bug in the application, or in the CUDA libraries/driver stack.

Could be a bug specifically between the app and the Quadro RTX drivers, too...