Editing Supercomputer (section)

===Software tools and message passing===
{{Main|Message passing in computer clusters}}
{{See also|Parallel computing|Parallel programming model}}
[[File:Wide-angle view of the ALMA correlator.jpg|thumb|Wide-angle view of the [[Atacama Large Millimeter Array|ALMA]] correlator<ref>{{cite news|title=Wide-angle view of the ALMA correlator|url=http://www.eso.org/public/images/eso1253a/|access-date=13 February 2013|newspaper=ESO Press Release}}</ref>]]

The parallel architectures of supercomputers often dictate the use of special programming techniques to exploit their speed. Software tools for distributed processing include standard [[Application programming interface|APIs]] such as [[Message Passing Interface|MPI]]<ref>{{cite book |first=Frank |last=Nielsen | title=Introduction to HPC with MPI for Data Science |  year=2016
| publisher=Springer |isbn=978-3-319-21903-5 |pages=185–221}}</ref> and [[Parallel Virtual Machine|PVM]], [[Virtual tape library|VTL]], and [[Open-source software|open source]] software such as [[Beowulf (computing)|Beowulf]].

In the most common scenario, environments such as [[Parallel Virtual Machine|PVM]] and [[Message Passing Interface|MPI]] for loosely connected clusters and [[OpenMP]] for tightly coordinated shared memory machines are used. Significant effort is required to optimize an algorithm for the interconnect characteristics of the machine it will be run on; the aim is to prevent any of the CPUs from wasting time waiting on data from other nodes. [[GPGPU]]s have hundreds of processor cores and are programmed using programming models such as [[CUDA]] or [[OpenCL]].

Moreover, it is quite difficult to debug and test parallel programs. [[Testing high-performance computing applications|Special techniques]] need to be used for testing and debugging such applications.