Beyond Single-Core R

Hardware:

• Node: A single motherboard, with possibly multiple sockets
• Processor/Socket: the silicon containing likely multiple cores
• Core: the unit of computation; often has hardware support for
• Pseudo-cores: can appear to the OS as multiple cores but share much functionality between other pseudo-cores on the same core

Processes and threads:

• Process: Data and code in memory
• One or more threads of execution within a process
• Threads in the same process can see most of the same memory
• Processes generally cannot peer into another processes memory

Interpreted languages: generally you can only directly work with processes

Can call libraries that invoke threads (BLAS/LAPACK)

One turns to parallel computing to solve one of three problems:

My program is too slow.

Perhaps using more cores — e.g., all cores on my desktop — will make things faster.

• Compute bound.
• Tools:
  • parallel/multicore
  • Rdsm
  • GPUs

My problem is too big.

Perhaps splitting the problem up onto multiple computers in a cluster will give it access to enough memory to run effectively.

• Memory bound
• Tools:
  • parallel/snow
  • pbdR

There are too many computations to do - one task runs in a reasonable amount of time, but I have to run thousands!

Perhaps splitting the problem up onto multiple computers in a cluster will give it access to enough memory to run.

• Tools:
  • gnu-parallel
  • parallel
  • job queues…