Question 7 of 10

Question 7 (10%)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

The most significant difference is in how each paradigm treats state and mutation. A procedural language expresses a computation as an ordered sequence of statements that repeatedly read and mutate shared program state (variables) — assignment, loops that update a running total or index, and subroutines that may modify data structures passed to them by reference. Correctness in this style depends on the exact order statements execute in, because a later statement's meaning depends on state changes made by earlier ones. A functional language, by contrast, expresses a computation as the evaluation of pure functions applied to (typically immutable) data: a pure function's output depends only on its inputs, has no observable side effects, and never mutates shared state. Instead of loops with mutable counters, functional style favours recursion and higher-order functions such as map, filter, and reduce that transform whole collections at once. Because a pure function has no side effects and no dependence on external mutable state, the order in which independent function applications occur (or whether they occur concurrently) cannot change the result — a property procedural code, with its shared mutable state, does not have in general.

This absence of shared mutable state is exactly what makes the functional paradigm significant for big-data frameworks such as Hadoop (and its MapReduce programming model). MapReduce asks the programmer to supply two functions: a map function applied independently to each record of a (potentially enormous, distributed) dataset, and a reduce function that combines the mapped results grouped by key. This is a direct application of the functional map/reduce higher-order functions to a cluster-scale dataset. Because a well-written map function is pure — it reads one input record and produces output based only on that record, touching no shared state — the framework is free to run thousands of map invocations in parallel, across many machines, in any order, with no risk of one invocation corrupting another's result through a shared variable, and no need for locks or other concurrency-control mechanisms that a procedural, mutable-state implementation of the same logic would require. Purity also makes fault tolerance straightforward: if a worker node fails partway through, Hadoop can simply re-run the same map task on another node, because a pure function produces the identical result given the identical input — there is no mutable state left in an inconsistent, half-updated condition to worry about, unlike a re-run of a procedural task that had already mutated shared state before failing. In short, the procedural paradigm's reliance on ordered mutation of shared state is precisely the property that makes naive parallelization dangerous (race conditions, non-reproducible results), while the functional paradigm's avoidance of that same property is precisely what lets a framework like Hadoop safely and automatically distribute, reorder, retry, and parallelize computation across a cluster without the programmer having to reason about concurrency at all.