Editing Distributed computing (section)

==Parallel and distributed computing==
[[File:Distributed-parallel.svg|thumb|right|300px|(a), (b): a distributed system.<br/>(c): a parallel system.]]
Distributed systems are groups of networked computers which share a common goal for their work.
The terms "[[concurrent computing]]", "[[parallel computing]]", and "distributed computing" have much overlap, and no clear distinction exists between them.<ref>{{harvtxt|Ghosh|2007}}, p. 10. {{harvtxt|Keidar|2008}}.</ref> The same system may be characterized both as "parallel" and "distributed"; the processors in a typical distributed system run concurrently in parallel.<ref>{{harvtxt|Lynch|1996}}, p. xix, 1–2. {{harvtxt|Peleg|2000}}, p. 1.</ref> Parallel computing may be seen as a particularly tightly coupled form of distributed computing,<ref>{{harvtxt|Peleg|2000}}, p. 1.</ref> and distributed computing may be seen as a loosely coupled form of parallel computing.<ref name="harvtxt|Ghosh|2007"/> Nevertheless, it is possible to roughly classify concurrent systems as "parallel" or "distributed" using the following criteria:
* In parallel computing, all processors may have access to a [[shared memory]] to exchange information between processors.<ref>{{harvtxt|Papadimitriou|1994}}, Chapter 15. {{harvtxt|Keidar|2008}}.</ref>
* In distributed computing, each processor has its own private memory ([[distributed memory]]). Information is exchanged by passing messages between the processors.<ref>See references in [[#Introduction|Introduction]].</ref>
The figure on the right illustrates the difference between distributed and parallel systems. Figure (a) is a schematic view of a typical distributed system; the system is represented as a network topology in which each node is a computer and each line connecting the nodes is a communication link. Figure (b) shows the same distributed system in more detail: each computer has its own local memory, and information can be exchanged only by passing messages from one node to another by using the available communication links. Figure (c) shows a parallel system in which each processor has a direct access to a shared memory.

The situation is further complicated by the traditional uses of the terms parallel and distributed ''algorithm'' that do not quite match the above definitions of parallel and distributed ''systems'' (see [[#Theoretical foundations|below]] for more detailed discussion). Nevertheless, as a rule of thumb, high-performance parallel computation in a shared-memory multiprocessor uses parallel algorithms while the coordination of a large-scale distributed system uses distributed algorithms.<ref name="BetalebParallel16">{{cite web |url=http://www.comp.nus.edu.sg/~rahul/allfiles/cs6234-16-pds.pdf |archive-url=https://web.archive.org/web/20170326210614/http://www.comp.nus.edu.sg/~rahul/allfiles/cs6234-16-pds.pdf |archive-date=2017-03-26 |url-status=live |title=Parallel and Distributed Algorithms |author=Bentaleb, A. |author2=Yifan, L. |author3=Xin, J.|display-authors=et al |publisher=National University of Singapore |year=2016 |access-date=20 July 2018}}</ref>