System Models of Distributed Systems

A distributed system is a collection of autonomous computers that work together to provide a unified service. Each machine has its own processor, memory, and operating system. The computers do not share physical memory, and no central quartz clock synchronizes their operations.

To a user, the system appears as a single application. Behind the scenes, it coordinates across dozens or thousands of nodes. Common examples include cloud platforms, distributed databases, streaming networks, payment systems, and web search engines.

Because components execute concurrently and communicate over non-instantaneous networks, reasoning about their behavior is difficult. System models provide formal abstractions to analyze and verify these systems systematically.

Distributed applications contain many concurrent software components running on separate machines. Each machine has its own processor, memory, storage, operating system, and local clock.

First, communication takes time. A sender cannot know the exact microsecond a receiver gets a message. Second, different processes execute at varying speeds. Third, components fail independently: a server can crash or a switch can drop packets while the rest of the cluster stays healthy.

These factors make distributed systems difficult to understand. System models reduce complexity by defining consistent assumptions for design and analysis.

Abstraction represents a complex system by focusing only on the characteristics required for a specific purpose, filtering out irrelevant physical details.

Consider a cluster of hundreds of servers. When analyzing communication protocols, we do not need to know the specific processor model in every server. We need to know whether messages can be delayed or lost, and how long transmission can take.

Similarly, when studying fault tolerance, the internal code structure of a server is less important than its failure behavior: whether it halts completely upon crashing, whether it can recover from disk, and whether peers can detect the failure.

An architectural model describes the organization of a distributed system at a high level. It identifies the major components of the system, the services provided by those components, and the relationships between them.

For example, an online banking application contains user devices, web servers, application servers, authentication services, database servers, caching systems, and external payment gateways.

The architectural model does not describe the internal code implementation of each component. Instead, it explains how these components are arranged and how they cooperate to deliver the overall service.

A process is an executing instance of a program. In a centralized system, several processes can communicate using mechanisms provided by the operating system, including shared memory.

In a distributed system, processes located on different computers cannot directly access each other's local memory. Each process maintains its own private local state.

If one process requires information held by another, it must obtain that information through explicit network communication. For example, an application server requests customer records from a database process, which queries disk and replies.

Middleware is software that provides a common abstraction for communication and coordination between distributed applications. It sits between application programs and lower-level operating-system and network services.

Middleware provides facilities such as Remote Procedure Calls (RPC), Remote Method Invocation (RMI), message passing, naming, service discovery, authentication, serialization, and distributed transactions.

The primary purpose of middleware is to hide distribution complexities. For example, Java RMI allows an application to invoke methods on objects located in another Java Virtual Machine as if they were local references.

The application layer contains programs that implement the functionality required by users or other applications. Examples include online banking, e-commerce platforms, streaming services, cloud storage, and enterprise resource planning systems.

An application that appears to be a single program to a user may consist of many distributed components running across multiple servers.

For example, when a user logs into an online banking system, the request may pass through a frontend server, an authentication service, an application server, a database, and a security service before the final response is returned.

Every computer participating in a distributed system has its own operating system. The operating system manages local resources, including the processor, memory, storage devices, network interfaces, and running processes.

The operating system provides system calls and services that applications use to perform local operations, allocate memory, and manage sockets.

Although the computers cooperate as one distributed system, their operating systems remain independent. A process running on one computer cannot directly manipulate the memory or processor of another computer. Communication requires network mechanisms.

The request-response pattern is one of the most common forms of interaction in client-server systems.

The client initiates communication by sending a request that identifies the required operation and provides necessary parameters. The server receives the request, performs the processing, accesses resources if necessary, and generates a response.

For example, a web browser requests a webpage over HTTP. The server processes the request, retrieves content from the file system or database, and sends an HTTP response back to the browser.

Two-tier architecture divides a distributed application into two major tiers: a client tier and a server tier.

The client tier contains the user interface and application logic. The server tier provides data management services. A traditional desktop application communicating directly with a database server is a common example.

Two-tier architecture is straightforward for small systems. However, as client numbers grow, direct client-to-database connections create scalability, security, and maintenance difficulties.

In a peer-to-peer architecture, participating nodes can act both as service providers and service consumers. There is no requirement for a permanent central server.

A peer may request information from another peer while simultaneously providing information to other peers in the network. Processing, storage, and communication responsibilities are distributed among participating nodes.

However, decentralization introduces additional engineering challenges involving node discovery, coordination, security, consistency, and handling dynamic node departures.

A thin client is a client in which relatively little application processing is performed locally. Most of the computation and data processing are provided by remote servers.

Web applications accessed through a browser are a common example. The browser handles presentation and basic user input, while business logic and database queries execute on remote servers.

Thin clients simplify centralized management because application software is maintained primarily on servers. However, they depend heavily on continuous network connectivity and server availability.

A fat client, also called a thick client, performs a substantial amount of computational processing and business logic locally on the client machine.

Desktop applications that perform complex calculations or graphics rendering locally while communicating with a central server for shared records are common examples.

Fat clients reduce server processing demands and can operate during temporary network disconnections. However, distributing software updates, configuration changes, and security patches across many client machines is more difficult.

A proxy server is an intermediary between a client and another server. Instead of communicating directly with the destination server, the client sends its request to the proxy.

The proxy processes or forwards the request and returns the result to the client. Proxies provide caching, access control, security filtering, traffic management, and protocol translation.

Caching is particularly useful when multiple clients request identical resources. The proxy stores previously retrieved resources and serves subsequent requests from cache, reducing network traffic and improving response times.

Replication involves maintaining multiple copies or instances of a service or data resource across separate physical computers.

Replication increases system capacity because multiple servers can process requests concurrently. It also improves availability: if one server fails, another instance can continue providing the service without interrupting users.

However, replication introduces the challenge of maintaining consistency between copies, especially when replicated resources can be modified concurrently by multiple clients.

Mobile code refers to program code that can be transferred from one computer to another and executed in the receiving environment.

Traditional distributed computing moves data across the network to a program for processing. With mobile code, the program can be moved closer to where the data is stored.

For example, if a large dataset is stored on a remote server, transferring a small processing script to that system requires less network bandwidth than transferring the entire dataset across the wire. However, running external code requires strict security sandboxing.

A mobile agent is a program that can move between computers or execution environments while continuing to perform a task.

An agent executes part of its task on Host A, migrates its code and execution state across the network to Host B, continues processing there, and then moves to Host C.

Mobile agents differ from ordinary mobile code because they are designed as autonomous programs capable of making decisions and carrying out multi-step activities across hosts. However, they introduce significant security, compatibility, and fault-tolerance challenges.

Architectural models explain the structure and organization of a distributed system. Fundamental models describe assumptions about the behavior of that system.

Two essential fundamental models are the interaction model and the failure model. The interaction model considers how processes execute and communicate, including timing and event ordering. The failure model considers how components fail and what incorrect behavior can result.

These models are essential when designing distributed algorithms because an algorithm can only be considered correct relative to the assumptions made about its execution environment.

A synchronous distributed system is one in which known upper bounds can be placed on important aspects of computation and communication.

Specifically, the model assumes that a process takes no more than a specified maximum time to execute an operation, that a message takes no more than a specified time to travel between nodes, and that local clocks drift by at most a known rate.

These assumptions simplify algorithm design because the absence of a response within a known timeout provides definitive information that a node has failed. However, real-world wide-area networks rarely guarantee strict synchronous bounds.

An asynchronous distributed system does not provide known fixed upper bounds on process execution time or message transmission latency.

A process may execute slowly, or a message may be delayed in transit for an unpredictable duration without violating the model.

This creates a fundamental difficulty: a process that has not received a response cannot immediately determine whether the remote process has crashed or whether the message is simply delayed. Asynchronous models are important because they accurately reflect internet and cloud environments.

Failures are an unavoidable reality in distributed systems. A failure model defines the ways in which components can fail and the behavior that can be expected from a faulty component.

A process may stop completely because of a crash. A communication channel may lose a message. A process may respond too slowly. In severe cases, a faulty component may behave arbitrarily and provide conflicting data to different peers.

By defining an explicit failure model, system designers determine which failures must be tolerated and what recovery mechanisms are required to maintain correct operation.

A crash failure occurs when a process stops executing and does not perform any further operations.

For example, a server may lose power, experience an operating-system panic, or terminate due to a critical software exception.

The difficulty is that other processes may not immediately know that the failed process has crashed. A lack of response could indicate either a crashed process or a delayed message, requiring timeout mechanisms and failure detectors.

A timing failure occurs when an operation takes longer than the time allowed by the system's timing assumptions.

For example, a server may eventually compute the correct response, but the response arrives after the deadline required by the application.

Timing failures are particularly critical in real-time systems, such as industrial control and live multimedia streaming. In these environments, a response that arrives late is considered invalid, even if its data is accurate.

A Byzantine failure represents an arbitrary failure in which a component behaves incorrectly in ways not limited to simply stopping or losing messages.

A Byzantine-faulty process may send incorrect information, deliver conflicting messages to different peers (equivocation), or behave inconsistently over time.

Byzantine failures are challenging because other processes cannot assume that a responding node is behaving correctly. Tolerating these failures requires redundancy, agreement protocols, cryptographic authentication, and quorum majorities of at least 3f + 1 nodes to tolerate f faulty nodes.

Knowing the architecture of a distributed system does not tell us everything about its behavior.

For example, an architecture may contain three replicated servers, but this alone does not explain what happens if one server crashes.

Similarly, knowing that two processes communicate does not tell us whether messages can be delayed indefinitely or whether there is a guaranteed maximum transmission time. To reason about these situations, architecture must be combined with interaction and failure assumptions.

Distributed algorithms operate under assumptions about the environment in which they execute.

For example, a leader-election algorithm must make assumptions about communication and failures. A distributed consensus algorithm must consider the possibility that processes may fail or messages may be delayed.

If an algorithm assumes synchronous communication but executes where messages experience arbitrary delays, its correctness may not hold. System models define the conditions under which distributed algorithms operate correctly.