Computing

/flæʃ/

noun … “Non-volatile memory with electrical erase and write.”

Flash is a type of non-volatile memory that can be electrically erased and reprogrammed. Unlike traditional ROM, Flash supports multiple write and erase cycles, making it suitable for storage devices like SSDs, USB drives, and embedded systems. It combines the speed of semiconductor memory with persistent data retention, bridging the gap between volatile RAM and slower mechanical storage.

Key characteristics of Flash memory include:

• Non-volatility: retains data without power.
• Block-based erase: memory is erased in large blocks before being rewritten.
• Limited write endurance: each cell can endure a finite number of program/erase cycles, requiring wear-leveling strategies.
• Fast read operations: often much quicker than mechanical storage, though slower than SRAM or DRAM.
• Integration: used in SSDs, microcontrollers, smartphones, and other embedded devices.

Workflow example: Writing data to an SSD using Flash:

function write_flash(address, data) {
    if block at address not empty:
        erase_block(address)
    program_block(address, data)
}

Here, a block must be erased before new data is programmed, reflecting the block-oriented nature of Flash. Wear-leveling algorithms distribute writes to maximize lifespan.

Conceptually, Flash is like a reusable sticky note: you can write, erase, and rewrite information repeatedly, and it retains the latest note even when the power is off.

See ROM, Memory, SSD, Wear Leveling, EEPROM.

/ˈmɛməri ˈmænɪdʒmənt/

noun … “Organizing, allocating, and reclaiming memory.”

Memory Management is the process by which a computing system controls the allocation, usage, and reclamation of memory. It ensures that programs receive the memory they require while optimizing performance, preventing leaks, and avoiding conflicts. Effective memory management balances speed, space, and safety, and is implemented via operating system services, language runtimes, and hardware support.

Key characteristics of Memory Management include:

• Allocation strategies: memory can be allocated statically (compile-time) or dynamically (runtime), including stack and heap allocation.
• Deallocation: reclaimed memory can be managed manually (e.g., C/C++) or automatically via garbage collection (e.g., Java, Python).
• Segmentation and paging: modern systems divide memory into fixed or variable-size segments or pages for efficient access and protection.
• Protection and isolation: memory management enforces access controls, preventing unauthorized access between processes.
• Fragmentation handling: minimizing wasted space due to fragmented allocation, both internally (within blocks) and externally (between blocks).

Workflow example: In a typical program using dynamic memory:

function create_array(size) {
    array = malloc(size * sizeof(int))  -- Allocate heap memory
    for i in 0..(size-1):
        array[i] = i * 2
    return array
}

function cleanup(array) {
    free(array)  -- Reclaim memory
}

Here, memory is dynamically allocated for the array, used within the program, and then explicitly released to prevent leaks. In garbage-collected languages, the runtime automates reclamation based on reachability.

Conceptually, Memory Management is like a warehouse with limited space: items must be stored efficiently, retrieved quickly, and removed when no longer needed to keep operations smooth and prevent overcrowding.

See Memory, RAM, Heap, Stack, Garbage Collection.

/kæʃ/

noun … “Fast memory for frequently used data.”

Cache is a high-speed memory layer that stores copies of frequently accessed data to reduce access latency and improve overall system performance. It acts as an intermediary between slower main memory (e.g., RAM) or storage and the CPU, allowing repeated reads and writes to be served quickly. Caches are used in hardware (CPU caches, GPU caches), software (database query caching), and networking (CDN caches).

Key characteristics of Cache include:

• Speed: typically implemented with faster memory types (e.g., SRAM) to minimize latency.
• Hierarchy: CPU caches are often divided into levels—L1 (smallest, fastest), L2, and L3 (larger, slightly slower).
• Locality: cache efficiency relies on temporal and spatial locality, predicting which data will be reused.
• Coherency: ensures cached data is synchronized with main memory to prevent stale reads.
• Eviction policies: strategies like LRU (Least Recently Used) decide which entries are replaced when the cache is full.

Workflow example: When a CPU requests data:

function read_data(address) {
    if cache.contains(address):
        return cache.get(address)  -- Fast access
    else:
        data = RAM.read(address)
        cache.update(address, data)
        return data
}

Here, the cache checks for the requested data. If present, it returns the value quickly. If not, it retrieves data from slower RAM and updates the cache for future access.

Conceptually, Cache is like keeping frequently referenced documents on your desk instead of fetching them from a filing cabinet every time—you trade a small amount of space for significant speed and convenience.

See Memory, RAM, CPU, GPU, Cache Coherency.

/roʊm/

noun … “Non-volatile storage for permanent instructions.”

ROM (Read-Only Memory) is a type of non-volatile memory used to store data or program instructions that must persist even when the system is powered off. Unlike volatile memory such as RAM, contents of ROM are typically fixed at manufacturing or written once and rarely modified. ROM is commonly used to hold firmware, bootloaders, and essential system-level instructions required to start and initialize hardware.

Key characteristics of ROM include:

• Non-volatility: retains data permanently without power.
• Limited write capability: often written once (e.g., mask ROM) or modified through specialized processes (e.g., EEPROM, flash ROM).
• Bootstrapping: contains critical instructions that allow a system to initialize hardware and load an operating system.
• Reliability: less prone to accidental modification or corruption compared to volatile memory.
• Integration: frequently embedded in motherboards, embedded devices, and microcontrollers.

Workflow example: During system startup, the CPU reads instructions from ROM to initialize hardware components and configure memory before transferring control to the operating system loaded into RAM:

-- Simplified boot sequence
cpu.fetch("ROM:Bootloader")
bootloader.initialize_hardware()
bootloader.load_os("RAM")

Conceptually, ROM is like a sealed instruction manual permanently attached to a machine. No matter how many times the machine is powered off and on, the manual is always available to guide its startup and operation.

See Memory, RAM, Flash Memory, Firmware, CPU.

/ˈmɛməri/

noun … “Storage for data and instructions.”

Memory is the component or subsystem in a computing environment responsible for storing and retrieving data and program instructions. It encompasses volatile storage such as RAM, non-volatile storage like ROM, and other forms including cache, registers, and persistent memory. Effective memory management is critical for performance, multitasking, and ensuring data integrity across CPU operations.

Key characteristics of Memory include:

• Volatility: volatile memory loses data when power is removed (e.g., RAM), while non-volatile memory retains it (e.g., ROM, SSDs).
• Hierarchy: memory is structured in layers, including registers, caches, main memory, and secondary storage, balancing speed and capacity.
• Addressability: each memory location has a unique address used by the CPU to read or write data.
• Access time: memory types differ in latency and bandwidth, influencing overall system performance.
• Persistence and durability: persistent memory retains state across power cycles, supporting file systems and databases.

Workflow example: In a typical program, data is loaded from persistent storage into RAM for computation. The CPU accesses instructions and variables from memory, often leveraging cache to minimize latency:

int main() {
    int x = 42  -- Stored in RAM or CPU register
    int y = x * 2
    std::cout << "Result: " << y << std::endl
}

Here, Memory holds the variables x and y, while instructions execute from memory locations accessible to the CPU.

Conceptually, Memory is like a library where books (data) are stored and retrieved by readers (the CPU) when needed. Fast access to frequently used books improves efficiency, while less-used volumes may reside in the stacks.

See RAM, ROM, CPU, Cache, Memory Management.

/ˌrɛplɪˈkeɪʃən/

noun … “Copy data across nodes to ensure reliability.”

Replication is the process of creating and maintaining multiple copies of data across different nodes in a Distributed System. Its purpose is to enhance Availability, fault tolerance, and performance by allowing data to remain accessible even if some nodes fail. Replication is fundamental to distributed databases, file systems, and cloud storage platforms.

Key characteristics of Replication include:

• Redundancy: multiple copies of the same data exist on different nodes to prevent data loss.
• Consistency: replication strategies define how and when updates propagate, balancing between strong consistency and eventual consistency.
• Durability: replicated data ensures that committed writes are not lost even if a node crashes.
• Performance: replication can improve read throughput by allowing multiple nodes to serve requests concurrently.
• Coordination: algorithms like Paxos and Raft are often used to ensure replicated logs remain consistent across nodes.

Workflow example: In a replicated key-value store, a client writes a new value. The leader node appends the value to its log and replicates it to follower nodes. Once a majority acknowledges, the value is committed and applied locally and on followers. Reads can be served from any node, and failed nodes can catch up with the latest state once they recover.

-- Simplified replication example
nodes = ["Node1", "Node2", "Node3"]
value = 100
leader = "Node1"
leader_log.append(value)
for node in nodes {
    if node != leader:
        replicate(node, value)
}
commit(value)
-- All nodes now contain the value 100

Conceptually, Replication is like distributing multiple copies of a book to several libraries. Even if one library is closed or destroyed, readers can still access the same content elsewhere.

See Distributed Systems, Consensus, Raft, Paxos, Availability.

/ræft/

noun … “Simplified consensus algorithm for distributed systems.”

Raft is a fault-tolerant Consensus algorithm designed to manage a replicated log in a Distributed System. Raft ensures that multiple nodes agree on a sequence of state changes, providing strong consistency and simplifying the complexity associated with other consensus protocols like Paxos. It is widely used in distributed databases, configuration services, and fault-tolerant systems.

Key characteristics of Raft include:

• Leader-based approach: one node acts as a leader, coordinating log replication and client requests.
• Log replication: the leader appends commands to its log and ensures follower nodes replicate the same entries in order.
• Election and fault tolerance: if the leader fails, a new leader is elected among followers using randomized timers to avoid conflicts.
• Safety: all committed entries are guaranteed to be durable and consistent across all non-faulty nodes.
• Simplicity: Raft separates leader election, log replication, and safety mechanisms to make understanding and implementation more straightforward than Paxos.

Workflow example: In a distributed key-value store using Raft, a client submits a write operation. The current leader appends the operation to its log, then sends append entries requests to follower nodes. Once a majority of followers acknowledge the entry, it is considered committed, and the leader applies it to its local state machine. Followers apply the entry once committed. If the leader crashes, a new leader is elected and resumes log replication without violating consistency.

-- Simplified Raft log replication
leader = "Node1"
followers = ["Node2", "Node3"]
entry = "Set X = 42"
leader_log.append(entry)
for follower in followers {
    send_append_entries(follower, entry)
}
if majority_acknowledged(followers, entry):
    commit(entry)
}
-- All nodes eventually apply the committed entry

Conceptually, Raft is like a conductor leading an orchestra: the leader ensures all musicians follow the same sheet of music in sync. If the conductor is unavailable, the orchestra quickly elects a new conductor to continue performing without missing a beat.

See Consensus, Paxos, Distributed Systems, Replication, CAP Theorem.

/ˈpæk.sɒs/

noun … “Consensus algorithm for unreliable networks.”

Paxos is a fault-tolerant Consensus algorithm designed to achieve agreement among nodes in a Distributed System, even when some nodes fail or messages are delayed or lost. It ensures that a single value is chosen and consistently replicated across all non-faulty nodes, providing a foundation for reliable state machines, replicated databases, and coordination services.

Key characteristics of Paxos include:

• Roles: nodes operate as Proposers (suggest values), Acceptors (vote on values), and Learners (learn the agreed value).
• Quorum-based decisions: a value is chosen only when a majority of acceptors agree, ensuring safety despite node failures.
• Safety: at most one value can be chosen, preventing conflicting decisions.
• Liveness: the algorithm guarantees progress if a sufficient number of nodes are operational and communication is eventually reliable.
• Fault tolerance: works correctly even if some nodes crash or messages are delayed, provided a majority of acceptors remain reachable.

Workflow example: In a distributed key-value store, when a client proposes a new value for a key, Proposers send prepare requests to Acceptors. Acceptors respond with promises to reject lower-numbered proposals. Once a majority agree, the value is accepted and propagated to Learners, ensuring all non-faulty nodes converge on the same value.

-- Simplified Paxos prepare phase
proposers = ["P1", "P2"]
acceptors = ["A1", "A2", "A3"]
proposal_number = 1
for proposer in proposers {
    for acceptor in acceptors {
        send_prepare(acceptor, proposal_number)
    }
}
-- Acceptors respond with promise to ignore lower-numbered proposals

Conceptually, Paxos is like a committee that must choose a single candidate among several options. Even if some members are absent or communication is delayed, the committee follows a strict protocol to ensure only one candidate is elected, and all members eventually learn the result.

See Consensus, Raft, Distributed Systems, Replication, CAP Theorem.

/kənˈsɪstənsi/

noun … “All nodes see the same data at the same time.”

Consistency is the property of a Distributed System that ensures every read operation returns the most recent write for a given piece of data. In the context of the CAP Theorem, consistency guarantees that all nodes observe the same state even in the presence of concurrent updates or network failures. Strong consistency simplifies reasoning about system behavior, as clients can assume a single, globally agreed-upon value for each piece of data.

Key characteristics of Consistency include:

• Linearizability: operations appear instantaneous and in some global order.
• Atomicity of updates: writes are applied fully or not at all across all replicas.
• Deterministic reads: the system ensures that the same query issued at the same logical time returns identical results from any node.
• Tradeoff with availability: during network partitions, maintaining consistency may require rejecting or delaying operations to prevent divergent states.
• Coordination mechanisms: consensus algorithms, locks, or quorum-based protocols are commonly used to enforce consistency across nodes.

Workflow example: In a replicated database with three nodes, a client writes a value to Node1. Before returning success, the system ensures that at least a majority of nodes have applied the update. Subsequent reads from any node return the same value, guaranteeing consistency even if one node is temporarily unreachable.

-- Example: simplified quorum write
nodes = ["Node1", "Node2", "Node3"]
value_to_write = 42
quorum_size = 2
successful_writes = 0
for node in nodes {
    if write(node, value_to_write) > 0 {  -- write returns 1 if successful
        successful_writes += 1
    }
    if successful_writes >= quorum_size:
        break
}
print("Write committed with quorum")
-- Output: Write committed with quorum

Conceptually, Consistency is like multiple clocks in a networked building synchronized to show the same time. Even if one clock temporarily stops or drifts, the system ensures that all visible clocks agree once synchronization completes.

See Distributed Systems, CAP Theorem, Partition Tolerance, Availability, Consensus.

/əˌveɪləˈbɪləti/

noun … “System responds to requests, even under failure.”

Availability is the property of a Distributed System that ensures every request receives a response, regardless of individual node failures or network issues. In the context of the CAP Theorem, availability guarantees that the system continues to serve read or write operations even during network partitions, although the returned data may not reflect the latest global state. High availability is a cornerstone of fault-tolerant services, web applications, and cloud platforms.

Key characteristics of Availability include:

• Continuous responsiveness: the system aims to answer every request without indefinite delays.
• Redundancy: multiple nodes or replicas handle requests, so failures of individual nodes do not prevent service.
• Graceful degradation: the system may reduce functionality under heavy load or partial failure but remains operational.
• Tradeoff with consistency: during partitions, maintaining availability may require returning data that is temporarily inconsistent.
• Monitoring and recovery: automated health checks, failover, and load balancing ensure sustained availability in production.

Workflow example: In a replicated key-value store with three nodes, if one node fails, the remaining nodes continue accepting reads and writes. Clients may receive slightly outdated values, but service is uninterrupted. Load balancers and replication mechanisms route requests to available nodes, maintaining responsiveness while the failed node recovers.

-- Example: simplified availability check
nodes = ["Node1", "Node2", "Node3"]
failed_node = "Node2"
available_nodes = [n for n in nodes if n != failed_node]
for node in available_nodes {
    respond("Request handled by " + node)
}
-- Output:
-- Request handled by Node1
-- Request handled by Node3

Conceptually, Availability is like a 24/7 convenience store with multiple entrances: even if one entrance is blocked, customers can still access the store through other doors, keeping service continuous.

See Distributed Systems, CAP Theorem, Partition Tolerance, Consistency, Replication.