Computer System Architectures Review

Key Topics (Connections):

• System Software → General-Purpose OS: OS is part of system software; a general-purpose OS shows how the OS supports diverse applications.
• General-Purpose OS → Kernel: Kernel is the core of the OS; it manages hardware and resources for all applications.
• Kernel → Kernel Architectures: Kernel design choices (monolithic, microkernel, hybrid) influence performance, modularity, and scalability.
• Kernel Architectures → SMP: OS design must adapt to multiple CPUs sharing memory; kernel handles scheduling and memory management.
• SMP → Multicore Systems: Multicore introduces parallelism within a single CPU; the OS must manage cores, cache, and threads.
• Multicore → NUMA: NUMA introduces memory locality considerations; the OS must optimize memory placement and access.
• NUMA → Multicore Issues: Multicore performance challenges include thread scheduling, instruction-level parallelism, and efficient resource use.
• Multicore Issues → Amdahl’s Law: Explains limits of speedup from parallelism; theoretical basis for multicore/SMP performance expectations.
• Amdahl’s Law → Distributed Systems: Moving from single-machine parallelism to distributed systems introduces coordination over message passing; the OS relies on middleware.

Centralized System:

• Single computer (may have multiple processors)
• Shared memory for all processes
• Resources controlled centrally
• Examples:
  – A personal computer (PC)
  – A mainframe system
  – A symmetric multiprocessor (SMP) server — e.g., a multi-core CPU server
  – A database running on one large machine
  – An old-style time-sharing system (multiple terminals, one central computer)

Distributed System:

• Multiple independent computers
• Appear as a single coherent system to the user (Tanenbaum)
• Each computer has its own memory and CPU
• Coordination and communication are required among nodes
• Examples:
  – Google Search or Gmail (runs on many servers but looks like one system)
  – Cloud computing systems (AWS, Azure, GCP)
  – Blockchain networks
  – Hadoop or Spark clusters
  – Multiplayer online games with multiple servers
  – Microservices-based applications

Key Idea: Centralized = single control/memory; distributed = multiple independent nodes working together.

Centralized Architectures with Multiple Processors (Tightly Coupled)

• Shared Memory: All processors access the same physical memory.
• Communication & Synchronization: Done via shared variables in memory.
• Type: SMP (Shared Memory Multiprocessor) or Symmetric Multiprocessor — all processors are equal and share the same memory space.

SMP (Shared Memory Multiprocessor) or Symmetric Multiprocessor

A Symmetric Multiprocessor is a centralized architecture in which:

• Two or more identical processors share a common main memory and I/O subsystem.
• All processors are connected through a system bus or interconnection network.
• Each processor can perform any operation — they are “symmetric.” There is no master or slave.
• Processors share access to I/O devices.
• The system is controlled by a single integrated operating system that manages communication and synchronization among processors.
• Example: Modern multi-core CPUs such as Intel Core i7 or AMD Ryzen.

Each processor has its own local cache to speed up access to frequently used data.

L1 and L2 Caches

• L1 Cache:
  – The smallest and fastest memory inside each CPU core.
  – Stores the most frequently accessed instructions and data.
• L2 Cache:
  – Slightly larger and slower than L1.
  – Serves as a second-level buffer before accessing main memory.
  – Reduces the number of times the CPU has to access the slower main memory.

Together, these caches minimize delays by keeping data “close” to the processor.

System Bus

• The communication pathway connects all major components (processors, memory, and I/O subsystem).
• It allows data, instructions, and control signals to move between components.
• Because all processors share this same bus, it’s the main coordination point.

Think of it as a highway where all CPUs and memory communicate.

Main Memory (RAM)

• The shared memory is accessible by all processors.
• Each CPU can read/write to this memory using the system bus.
• This shared memory model allows easy data sharing between threads and processes.

I/O Subsystem

• Handles input/output operations like disk access, networking, keyboard, etc.
• The I/O adapters are interfaces (controllers) that connect peripheral devices to the system bus.
• All processors share access to these I/O devices.

How It Works

1. Each processor executes its own instructions concurrently.
2. When it needs data:
   – It first checks its L1/L2 caches.
   – If not found, it accesses main memory through the system bus.
3. Multiple processors communicate through shared memory (read/write to the same RAM).

Advantages of SMP (Centralized System)

• Parallel Processing: Multiple processors can work simultaneously to improve overall performance.
• Shared memory: Simplifies inter-process communication (no need for message passing).
• Scalability (to an extent): Adding more processors improves throughput.
• Flexibility: Any processor can handle any job.
• Resource Utilization: Processors share I/O and memory efficiently.

Drawbacks / Limitations of SMP (Centralized System)

• Scalability Issues:
  – Performance doesn’t scale linearly with the addition of processors.
  – The system bus and main memory become bottlenecks as more processors are added.
• Memory Contention: Multiple CPUs competing for access to the same memory cause delays.
• Cache Coherence Problem: Each processor’s cache can hold outdated copies of shared data, requiring complex consistency mechanisms.
• Latency Limits: Caching improves access speed only up to a point; beyond that, maintaining consistency adds overhead.
• Not Suitable for Large Systems: Shared-memory SMPs are impractical for large-scale multiprocessor systems (hundreds or thousands of CPUs).
• Cost and Complexity: Designing hardware that efficiently manages shared memory and interconnects is expensive.

SMP is ideal for small to medium-sized centralized systems (e.g., modern multi-core servers and desktops).
However, for large-scale processing, distributed systems or loosely coupled architectures are preferred due to better scalability and reduced memory contention.

UMA (Uniform Memory Access)

All processors share a single physical memory, and each processor takes the same amount of time to access any memory location.

Characteristics

• Single shared physical memory.
• Equal access time to all memory locations for every processor.
• Typically uses a single system bus or interconnection network.
• Cache coherence is required since all processors share the same memory.
• Example: Standard SMP systems (like multi-core CPUs in a PC or server).

Advantages

• Simple design and programming model.
• All memory is uniformly accessible. Easy for OS and developers.

Disadvantages

• Scalability bottleneck: As more processors are added, the shared bus and memory become congested.
• Memory contention increases (Multiple processors attempt to access the same memory simultaneously, resulting in delays).
• Performance degrades when too many processors compete for the same memory bandwidth.

NUMA (Non-Uniform Memory Access)

All processors can access the entire memory space, but access time depends on the memory’s location, i.e., local memory is faster than remote memory.

Characteristics

• Physical memory is divided among processors (nodes).
• Each node has its own local memory and sometimes a local cache.
• Processors can still access other nodes’ memory via interconnects, but remote access is slower.
• Maintains a single logical address space (so it still looks like shared memory to software).
• Designed to solve SMP scalability issues.

Advantages

• Better scalability: Adding more processors and local memories avoids the single-bus bottleneck.
• Reduced contention: Most memory accesses happen locally.
• Higher performance for workloads with good locality.

Disadvantages

• Non-uniform access time: Programmers and OS must optimize for memory locality.
• Complex memory management — must decide where to allocate memory for best performance.
• Cache coherence protocols become even more complicated.

In Relation to SMP
– All SMP systems are UMA systems by default.
– NUMA systems extend the SMP model to multiple nodes to overcome its scalability and memory bandwidth limits.

Multicore Computers

Similar to SMP, all processors share a single memory, but multiple cores are integrated on a single silicon die.

• Combines two or more complete processors (cores) on one chip.
• Faster and more power-efficient than SMP systems with processors on separate chips.
• Example: Intel’s 48-core “Single-Chip Cloud Computer” (SCC) was introduced in December 2009.

Multicore systems improve performance and energy efficiency by integrating multiple processors on a single chip while maintaining shared memory access.

Disadvantages of Multicore

Multicore systems introduce challenges similar to SMP but at a finer scale, requiring careful management of parallelism and scheduling.

• A high degree of parallelism is available even in small devices.
• Effective utilization requires consideration of multiple types of parallelism:
  – Instruction-Level Parallelism (ILP): Executing multiple instructions simultaneously within a core.
  – Multiprogramming on each core: OS must manage multiple programs or threads per core efficiently.
• Users must design programs to exploit parallelism (e.g., multithreading).
• OS must intelligently schedule related threads across cores to maximize performance and maintain consistency.

Maximizing multicore performance requires coordination between user-level parallel programming and OS-level thread scheduling, while efficiently exploiting instruction- and core-level parallelism.

SMP and Multicore differences

System Software

Software that manages hardware resources and provides services for application software.

Includes:

• Operating System (OS): Manages hardware, executes programs, handles I/O, memory, and process scheduling.
• Language Tools: Compilers, interpreters, and language runtime systems.
• Utilities: Software for maintenance, file management, and system monitoring.
• Middleware (Distributed Systems):
  – Runs on top of the OS.
  – Connects applications on different machines.
  – Examples: Communication packages, web services.

General Purpose Operating Systems

“General-purpose OS” is basically the type of operating system most people use on desktops, laptops, and servers.

A general-purpose operating system is designed to support a wide variety of applications and users, handling unpredictable workloads efficiently.

• Examples: Windows, Linux, macOS.
• It can run web browsers, games, programming tools, office apps, etc., all on the same machine.

General-purpose OS focuses on versatility and multitasking for many types of applications, while specialized OS focuses on efficiency and predictability for specific tasks.

Kernel

The core part of the operating system that remains in memory and manages system resources.
Types (Based on Structure and Design):

• Monolithic Kernel: Almost all OS functionality is included in a single large kernel program.
• Microkernel: Only minimal functionality (e.g., communication, basic scheduling) is in the kernel; other services run as user-space server processes.
• Hybrid Kernel: Combines features of monolithic and microkernels, keeping some services in the kernel for performance while others run in user space.

The kernel is the essential part of the OS that manages hardware and resources; design choices (monolithic, microkernel, hybrid) balance performance, modularity, and reliability.

OS = Kernel + Other components (utilities, system programs, user interfaces, etc.).

Kernel Architectures

Different designs of the kernel determine how OS functionality is organized and executed.

Types (Based on Architecture / Approach):

• Traditional Kernels:
  – Kernels following established designs, typically monolithic, with most OS services (scheduling, I/O, memory management, file systems) included inside the kernel.
  – Examples: UNIX/Linux, Windows, macOS
• Non-Traditional Kernels
  – Kernels that experiment with new approaches to improve modularity, performance, or flexibility, often minimizing services in the kernel or supporting virtualization.
  – Examples: Pure microkernels (Mach), extensible operating systems, and virtual machine monitors (hypervisors).

Kernel architecture affects system performance, modularity, and reliability.
Traditional kernels prioritize proven monolithic designs for stability and performance, while non-traditional kernels explore minimal, modular, or virtualized designs to innovate or improve efficiency.

Computer Architecture & the OS

As hardware evolves from single-processor to multiple-processor systems, OS complexity increases to manage resources and coordination effectively.

• Multiple Processor / Shared Memory Systems:
  – Resource sharing and coordination among processors adds OS complexity.
• Master-Slave Operating Systems:
  – One processor (master) controls the system; other processors (slaves) execute tasks under master supervision.
• SMP (Symmetric Multiprocessing) Operating Systems:
  – All processors are peers sharing memory and OS responsibilities equally.
  – Requires advanced scheduling and memory management for fairness and efficiency.
• Distributed Systems:
  – Each processor/machine runs a local OS.
  – Middleware coordinates communication and execution across machines.
  – Handles the complexity of geographically or logically separate resources.

OS design must evolve with hardware: moving from shared-memory multiprocessor systems to distributed systems introduces additional complexity, requiring local OS instances and middleware for coordination.

Amdahl’s Law

Predicts the maximum speedup of a program using multiple processors:

where:
– f = fraction of the code that can be parallelized
– N = number of processors

• Not all code can be parallelized; the sequential portion limits overall speedup.
• Applications that benefit from parallelization include games, databases, and multithreaded JVM programs.

The maximum performance gain from adding more processors is limited by the portion of code that must execute sequentially.

Example:
  - Program = 100 seconds on 1 processor
  - 70% of the code can be parallelized (f=0.7)
  - Number of processors = 4
Step 1: Sequential + parallel execution on 4 processors
  - Sequential: 1−f=0.3⇒0.3×100=30 seconds
  - Parallel: f/N=0.7/4⇒0.175×100=17.5 seconds
Step 2: Total time on 4 processors
    T(4)=30+17.5=47.5 seconds
Step 3: Speedup
Speedup= 100 / 47.5 ≈ 2.1​
Even with 4 processors, maximum speedup is just 2.1x because 30% of the 
code is sequential.

Adding more processors gives diminishing returns if a significant portion of the program is sequential; parallel speedup is fundamentally limited by the non-parallelizable fraction.

Distributed Systems

A system of multiple computers connected via a network, where memory is not shared and communication occurs through messages.

• A hypothetical distributed OS would treat all computers in the network like individual processors in an SMP system, enabling parallel programs with minimal modification.
• No general-purpose distributed OS currently exists.
• Middleware provides support for distributed applications, handling communication, coordination, and resource sharing.

Distributed systems rely on message passing and middleware to coordinate multiple machines, as memory is not shared; unlike SMP or multicore systems, a single OS cannot directly manage all resources across the network.

🔙 Back to all notes
𝕏 Let’s Connect

Backlinks:
Advanced Operating System