Are cloud applications blocking genuine HPC users from getting necessary resources?

[Over the past few weeks I have had several discussions with administrators from large HPC facilities in USA and Canada, who are part of the XSEDE and Compute Canada consortia respectively.
A common complaint I hear that is many of their high end resources are tied up supporting applications that are more ideally suited to operate on commercial clouds. Because many compute consortia like XSEDE and Compute Canada are largely supported through funding councils like NSF and CFI, researchers do not have to pay for access to these high end facilities. Virtually every researcher who makes an application through a peer review process is guaranteed a certain number of resources such as storage and/or computation on this infrastructure. But as a result many loosely coupled, highly parallelizable applications are consuming resources that would be much better suited for true HPC applications that are tightly coupled, compute and memory intensive. I have heard estimates of as much as 30-50% applications running on HPC facilities are in this loosely coupled category.

Many loose coupled applications could easily run on commercial clouds. Tightly coupled, compute intensive applications cannot run on clouds (at least not yet). Unfortunately researchers who have a loosely couple application don’t have an option of running them on commercial clouds because there is no funding program to support these initiatives. Their only option is to acquire their own computational resources (closet computing) or eat up precious, albeit free, resources available through compute consortia such as XSEDE and Compute Canada. True HPC researchers get short changed in the process. Compute consortia are reluctant to push certain applications to the cloud as this undermines their own justification to exist and possible future funding.

Anecdotal evidence is not data. I would be curious to know if anybody has statistics on the types of applications that run on their HPC facility? It would be interesting to get a true measure to see if applications suitable for the cloud are blocking genuine HPC users from getting the maximum benefit out of their machines. – BSA]