Chapter 1: Building Abstractions with Procedures
2015-09-29 016 Preface of this chapter
QUOTE: The acts of the mind, where in it exerts its power over simple ideas, are chiefly these three ...... (John Locke, An Essay Concerning Human Understanding).
My Understanding: First, combination, make complexity from simplicity. Second, compare, from which we get relations. Third, abstraction, return to simplicity from complexity.
QUOTE: We are about to study the idea of a computational process. Computational processes are abstract beings that inhabit computers. As they evolve, processes manipulate other abstract things called data. The evolution of a process is directed by a pattern called a program.
My Understanding: There are three important basic concepts in this paragraph: process, data and program. There are all abstract things, do not exist in the real life.
QUOTE: Thus, like the sorcerer's apprentice, novice programmers must learn to understand and to anticipate the consequences of their conjuring. Even small errors (usually called bugs or glitched) in programs can have complex and unanticipated consequences.
My Understanding: Understanding and anticipating, these sound very easy, but it becomes really hard when the project grow and the deadline is coming. But there is no other way we can learn how to program besides these, so you can't be too patient to be a programer. Just think yourself as a sorcerer, with insight and wisdom.
QUOTE: Master software engineers have the ability to organize programs so that they can be reasonably sure that the resulting processes will perform the tasks intended. They can visualize the behavior of their systems in advance.
My Understanding: One of my biggest problem is the uncertainty I feel when I finish a program. I am afraid that it will crash in the real world environment. I think this is a sign that I am still a novice in this field and it's something that I need to overcome.
QUOTE: Well designed computational systems, like well-designed automobiles or nuclear reactors, are designed in modular manner, so that the parts can be constructed, replaced, and debugged separately.
My Understanding: Modular is always emphasized in different articles and books, which means it is really important and is one of the most powerful tools to control complexity. But using it in the wrong way, it can do more harm than good, I need to understand it better and better as I develop my own programs.
2015-09-30 018 Programming in Lisp
QUOTE: Lisp was invented in the late 1950s as a formalism for reasoning about the use of certain kinds of logical expressions, called recursion equations, as a model for computation.
My Understanding: List was invented for computation, not for computer, this is another proof that computer science is not about computer, is a general theory about computational procedure.
QUOTE: Lisp was designed to provide symbol-manipulating capabilities for attacking programming problems such as the symbolic differentiation and integration of algebraic expressions. It includes for this purpose new data objects known as atoms and lists, which mostly strikingly set it apart from all other languages of the period.
My Understanding: For now I can not understand what is atom and what is list, and why they are so important for Lisp. So let me just remember these two concepts and try to find the answer in the future.
QUOTE: Lisp was not the product of concerted design effort. Instead, it evolved informally in an experimental manner in response to users' needs and to pragmatic implementation considerations.
My Understanding: The develop of the language is a good metaphor to the develop of a man. At first, you don't know what you can achieve and you don't design your whole life. You just have a few simple principles and then try to be flexible enough to adapt to mutable environments. Like a seed, eventually grows into a big tree no matter what challenge it has to face.
QUOTE: Lisp has become a language of choice for operating-system shell languages and for extension language for editors and computer-aided design systems.
My Understanding: We know that Emacs Lisp, one of the Lisp dialects, is used for the famous editor Emacs. So maybe if I can use Lisp effectively, I will learn how to use and extend Emacs.
QUOTE: The most significant of these features is the fact that Lisp descriptions of processes, called procedures, can themselves be represented and manipulated as Lisp data. The importance of this is that there are powerful program-design techniques that rely on the ability to blur the traditional distinction between "passive" data and "active" processes.
QUOTE: The ability to represent procedures as data also makes Lisp an excellent language for writing programs that must manipulate other programs as data, such as the interpreters and compilers that support computer languages.
2015-10-08 021 The Elements of Programming
QUOTE: Every powerful language has three mechanisms for accomplishing this:
primitive expressions, which represent the simplest entities the language is concerned with.
means of conbimation, by which compound elements are built from simpler ones.
means of abstraction, by which compound elements can be named and manipulated as units.
My Understanding: On the trip of this holiday, I saw the video of SICP, and the teacher emphasized these three machanisms over and over again. So I think the importance of these statements can not be overestimated. Just like the spirit in fairy tales, if we can call the name, we can seize the power.
QUOTE: Any powerful programming language should be able to describe primitive data and primitive procedures and should have method for combining and abstracting proceddures and data.
My Understanding: If I need to learn another programming language, now I know how to start. This gives me an insight understanding about the general structure of a practical language.
QUOTE: Expressions such as these, formed by delimiting a list of expressions within parentheses in order to denote procedure application, are called combinations. The leftmost element in the list is called operator, and the other elements are called operands. The value of a combination is obtained by applying the procedure specified by the operator to the arguments that are the values of the operands.
QUOTE: First advantage of prefix notation is that it can accommodate procedures that may take an arbitrary number of arguments. The seconde one is that it extends in a straightforward way to allow combinations to be nested, that is, to have combinations whose elements are themselves combinations.
My Understanding: I think the capability to have combinations whose elements are themselves combinations is very important. This is fundamental for Lisp to achieve its complexity and from this, we can build wonderful things from the simplest elements.
QUOTE: Indeed, complex programs are constructed by building, step by step, computational objects of increasing complexity. The interpreter makes this step-by-step program construction particular convenient because name-object associations can be created incrementally in successive interactions. This feature encourage the incremental development and testing of programs and is largely responsible for the fact that a Lisp program usually consists of a large number of relatively simple procedures.
QUOTE: It should be clear that the possibility of associating values with symbols and later retrieving them means that the interpreter must maintain some sort of memory that keeps track of the name-object pairs. This memory is called the environment (more precisely the global environment, since we will see later that a computation may involve a number of different environments).
2015-10-09 023 Evaluating Combinations
QUOTE: To evaluate a combination, do the following:
Evaluate the subexpressions of the combination.
Apply the procedure that is the value of the leftmost subexpression(the operator) to the arguments that are the values of the other subexpressions(the operands).
QUOTE: The first step dictates that in order to accomplish the evaluation process for a combination we must first perform the evaluation process on each element of the combination. Thus, the evaluation rule is recursive in nature; that is, it includes, as one of its steps, the need to invoke the rule itself.
My Understanding: Notice how simple but perfound the rule is! And the recursion used here make an important role to empower our simple rule.
QUOTE: In general, we shall see that recursion is a very powerful technique for dealing with hierarchical, treelike objects. In fact, the "percolate values upward" form of the evaluation rule is an example of a general kind of process known as tree accumulation.
My Understanding: Remember two things here: first, when speaking of tree, thinking about recursion; second, see whether we can find a way to percolate values upward when dealing with a structure like a tree.
QUOTE: We take care of the primitive cases by stipulating that:
the values of numerals are the numbers that they name,
the values of built-in operators are the machine instruction sequences that carry out the corresponding operations, and
the values of other names are the objects associated with those names in the environment.
My Understanding: Another very simple but powerful rule, and this is a wonderful example about how to isolate complexities and decrease the burden of our brain. Of course the first and the second stipulation are not as straightforward as they look like, but when we start to see the whole big picture, we can ignore the twisted details underground.
QUOTE: The key point to notice is the role of the environment in determining the meaning of the symbols in expressions. The general notion of the environment as providing a context in which evaluation takes place will play an important role in our understanding of program execution.
My Understanding: From this paragraph, I can see that environment must be a very important notion in Lisp. It sounds very easy, but as the other "simple" stipulations and notions in this book, there must be some perfound content underneath the simple form.
QUOTE: The various kinds of expression (each with its associated evaluation rule) constitute the syntax of the programming language. In comparison with most other programming languages, Lisp has a very simple syntax; that is, the evaluation rule for expressions can be described by a simple general rule together with specialized rules for a small number of special forms.
My Understanding: One of the special forms we've just seen is DEFINE, so a statement of a definition is not a combination.
2015-10-20 026 Compound procedures
QUOTE: Now we will learn about procedure definitions, a much more powerful abstraction technique by which a compound operation can be given a name and then referred to as a unit.
My Understanding: The key point here is that, once we define a compound procedure, we can then immediately ignore the truth that it's made by ourselves, and treat it as a primitive procedure.
QUOTE: The general form of a procedure definition is:
(define (<name> <formal parameters>) <body>)
The <name> is a symbol to be associated with the procedure definition in the environment. The <formal parameters> are the names used within the body of the procedure to referd to the corresponding arguments of the procedure. The <body> is an expression that will yield the value of the procedure application when the formal parameters are replaced by the actual arguments to which the procedure is applied.
My Understanding: We have three important elements here: name, formal parameters and body. In math, when we define a function, of cource we have function name, function parameters and function body, so what is the difference between define a compound procedure in Lisp and define a function in math?
QUOTE: Compound procedures are used in exactly the same way as primitive procedures. Indeed, one could not tell by looking at the definition of sum_of_squares given above whether square was built into the interpreter, like + and *, or defined as a compound procedure.
The Substitution Model for Procedure Application
QUOTE: We can assume that the mechanism for applying primitive procedures to arguments is built into the interpreter. For compound procedures, the application process is as follows:
- To apply a compound procedure to arguments, evaluate the body of the procedure with each formal parameters replaced by the corresponding argument.
QUOTE: The process we have just described is called the substitution model for procedure application. It can be taken as a model that determine the "meaning" of procedure application, insofar as the procedures in this chapter are concerned.
QUOTE: In general, when modeling phenomena in science and engineering, we begin with simplified, incomplete models. As we examine things in greater detail, these simple models become inadequate and must be replaced by more refined models. The substitution model is no exception.
2015-10-21 027 Applicative order versus normal order
QUOTE: An alternative evaluation model would not evaluate the operands until their values were needed. Instead it would first substitute operand expressions for parameters until it obtained an expression involving only primitive operators, and would then perform the evaluation.
My Understanding: Notice this definition is still recursive, because the oprands might be combinations as well, so the evaluation of the oprands will obey the same order.
QUOTE: This alternative "fully expand and then reduce" evaluation method is known as normal-order evaluation, in contrast to the "evaluate the argument and then apply" method that the interpreter actually uses, which is called applicative-order evaluation.
My Understanding: These two kinds of evaluation both have their disadvantage. In regard of normal-order evaluation, it can easily do redundant calculation if the compound procedure use its parameters repeatedly. On the other side, speaking of applicative-order evaluation, if it involves some condition statements, then some operands may not need to be evaluated in the first place, but you have to do it any way.
QUOTE: Lisp uses applicative-order evaluation, partly because of the additional efficiency obtained from avoiding multiple evaluations of expressions, more significantly, because normal-order evaluation becomes much more complicated to deal with when we leave the realm of procedures that can be modeled by substitution. On the other hand, normal-order evaluation can be extremely valuable tool, and we will investigate some of its implication.
My Understanding: How to understand "the realm of procedures that can be modeled by substitution"? It is very difficult for me to imagine a procedure which can't be modeled by substitution. I have to leave the question here and come back later when I learn more.
2015-10-26 030 Conditional Expressions and Predicates
QUOTE: The general form of a conditional expression is:
(cond (<p1> <e1>)
(<p2> <e2>)
...
(<pn> <en>))
consisting of the symbol cond followed by parenthesized pairs of expressions (<p> <e>) called clauses. The first expression in each pair is a predicate -- that is, an expression whose value is interpreted as either true or false.
QUOTE: The general form of a if expression is:
(if <predicate> <consequent> <alternative>)
To evaluate an if expression, the interpreter starts by evaluating the <predicate> part of the expression. If the <predicate> evaluates to a true value, the interpreter then evaluates the <consequent> and return its value. Otherwise it evaluates the <alternative> and return its value.
QUOTE: In addition to primitive predicates such as <, =, and >, there are logical composition operations, which enable us to construct compound predicates. The three most frequently used are there:
(and <e1> ... <en>)(or <e1> ... <en>)(not <e>)
Notice that and and or are special forms, not procedures, because the subexpressions are not necessarily all evaluated. Not is an ordinary procedure.
Example: Square Roots by Newton's Method
QUOTE: Procedures, as introduced above, are much like ordinary mathematical functions. They specify a value that is determined by one or more parameters. But there is an important difference between mathematical functions and computer procedures. Procedures must be effective.
My Understanding: The difference between mathematical functions and computer procedures are subtle -- one is declarative and the other one is imperative. For mathematical functions, you need to describe "What is it?", but for computer procedures, you have to answer "How to do it?".
2015-10-30 033 Example: Square Root by Newton's Method
QUOTE: The contrast between function and procedure is a reflection of a general distinction between describing properties of things and describing how to do things, or, as it is sometimes referred to, the distinction between declarative knowledge and imperative knowledge.
My Understanding: Declarative description means "what is it?", meanwhile imperative description means "how to do it?" In mathematics, we first describe the definition of a new concept, then we use induction and deduction to decomposite the complex new ideal into a series of simple, well-known ideals, therefore we can get a method to calculate or solve new questions about the new concept.
QUOTE: The sqrt program also illustrates that the simple procedural language we have introduced so far is sufficient for writing any purely numerical program that one could write in, say, C or Pascal. This might seen surprising, since we have not included in our language any iterative(looping) constructs that direct the computer to do sth over and over again. Sqrt-iter, on the other hand, demonstrates how iteration can be accomplished using no special construct other than the ordinary ability to call a procedure.
My Understanding: Lisp has the ability to use procedure calling instead of looping, but Java can not do that. This maybe has something to do with the concept "tail recursion". And, I read something about the magic number "0x5f3759df"(Fast Inverse Square Root), but it's too complicated for me now.
QUOTE: In real computers, arithmetic operations are almost always performed with limited precision. This makes our test ineffective for very small numbers and inadequate for very large numbers. An alternative strategy for implementing good-enough? is to watch how guess changes from one iteration to the next and to stop when the change is a very small fraction of the guess.
2015-11-10 035 Procedures as Black-Box Abstractions
QUOTE: Notice that the definition of sqrt-iter is recursive; that is, the procedure is defined in terms of itself. The idea of being able to define a procedure in terms of itself may be disturbing; it may seem unclear how such a "circulate" definition could make sense at all, much less specify a well-designed process to be carried out by a computer.
QUOTE: The entire sqrt program can be viewed as a cluster of procedures that mirrors the decomposition of the problem into subproblems.
My Understanding: How to construct a program -- separate into different layers and modules -- is always based on the problem realm, that is, how to decomposite the problem into a series of simple subproblems and how to build them together.
QUOTE: It is crucial that each procedure accomplishes an identifiable task that can be used as a module in defining other procedures. For example, when we define the good-enough? procedure in terms of square, we are able to regard the square procedure as a "black box." We are not at that moment concerned with how the procedure computes its result, only with the fact that it computes the square. The details of how the square is computed can be suppressed, to be considered at a later time.
QUOTE: Indeed, as far as the good-enough? procedure is concerned, square is not quite a procedure but rather an abstraction of a procedure, a so-called procedural abstraction. At this level of abstraction, any procedure that computes the square is equally good.
QUOTE: So a procedure definition should be able to suppress details. The users of the procedure may not have written the procedure themselves, but may have obtained it from another programmer as a black box. A user should not need to know how the procedure is implemented in order to use it.
My Understanding: In Object Oriented Programming, this is called encapsulation, and is one of the three basic characteristics. As we see here, the capability to suppress details help to deal with complexity, which is the core of programming.
Local Names
QUOTE: This principle -- that the meaning of a procedure should be independent of the parameter names used by its author -- seems on the surface to be self-evident, but its consequences are profound. The simplest consequence is that the parameter names of a procedure must be local to the body of the procedure.
2015-11-11 037 Internal Definitions and Block Structure
QUOTE: A formal parameter of a procedure has a very special role in the procedure definition, in that it doesn't matter what name the formal parameter has. Such a name is called a bound variable, and we say that the procedure definition binds its formal parameters.
My Understanding: We have two important concepts here: bound variable and bind. These two concepts are interconnected. Therefore, when we say a variable is bound variable, it is meaningful if only we know which environment binds it, or in other words, which environment the variable is bound to.
QUOTE: The problem with this program is that the only procedure that is important to users of sqrt is sqrt. The other procedures (sqrt-iter, good-enough? and improve) only clutter up their minds. They may not define any other procedure called good-enough? as part of another program to work together with the square-root program, because sqrt needs it. The problem is especially severe in the construction of large systems by many separate programmers.
QUOTE: We would like to localize the subprocedures, hiding them inside sqrt, so that sqrt could coexist with other successive approximations, each having its own private good-enough? procedure. To make this possible, we allow a procedure to have internal definitions that are local to that procedure. Such nesting of definitions, called block structure, is basically the right solution to the simplest name-packaging problem.
QUOTE: We will use block structure extensively to help us break up large programs into tractable pieces. The idea of block structure originated with the programming language Algol 60. It appears in most advanced programming languages and is an important tool for helping to organize the construction of large programs.
My Understanding: I use block struture and take advantage of it all the time but never notice its existence. The concept is so natural and straight forward that you just take it for granted. This is a profound example for "the simplicity on the other side of complexity".
2015-11-13 042 Procedures and the Processes They Generate
QUOTE: Like the novice chess player, we don't yet know the common patterns of usage in the domain. We lack the knowledge of which moves are worth making (which procedures are worth defining). We lack the experience to predict the consequences of making a move (executing a procedure).
My Understanding: Using chess as a metaphor, two aspects of programming are emphasized: the first is the common patterns in programming, the second is how to predict the consequences and side-effects of a program.
QUOTE: To become experts, we must learn to visualize the processes generated by various types of procedures. Only after we have developed such a skill can we learn to reliably construct programs that exhibit the desired behaviour.
My Understanding: This has the similar meaning as I read before from code completing. An programming expert will not hack a procedure, prior to typing the firt line of codes, he has the whole picture about how the codes will be organized, the control structure of the processes and the side-effects of his decisions.
Linear Recursion and Iteration
QUOTE: The substitution model reveals a shape of expansion followed by contraction. The expansion occurs as the process builds up a chain of deferred operations. The contraction occurs as the operations are actually performed. This type of process, characterized by a chain of deferred operation, is called a recursive process.
QUOTE: In the computation of n!, the length of the chain of deferred multiplications, and hence the amount of information needed to keep track of it, grows linearly with n (is proportional to n), just like the number of steps. Such a process is called a linear recursive process.
2015-11-16 043 Procedures and the Processes They Generate
QUOTE: In general, an iterative process is one whose state can be summarized by a fixed number of state variables, together with a fixed rule that describes how the state variables should be updated as the process moves from state to state and an (optional) end test that specifies conditions under which the process should terminate.
My Understanding: From the definition of iterative process, I guess this concept has something to do with another concept "Finite State Machine". They have substantial similarities: a finite number of states and a finite number of rules to control how the states change.
QUOTE: In contrasting iteration and recursion, we must be careful not to confuse the notion of recursive process with the notion of recursive procedure. When we describe a procedure as recursive, we are referring to the syntactic fact that the procedure definition refers (either directly or indirectly) to the procedure itself. But when we describle a processes following a pattern that is, say, linearly recursive, we are speaking about how the process evolves, not about the syntax of how a procedure is written.
QUOTE: One reason that the distinction between process and procedure may be confusing is that most implementations of common languages (including Ada, Pascal, and C) are designed in such a way that the interpretation of any recursive procedure consumes an amount of memory that grows with the number of procedure calls, even when the process described is, in principle, iterative. As a consequence, these languages can describle iterative processes only by resorting to special-purpose "looping construct" such as do, repeat, until, for, and while.
QUOTE: The implementation of Scheme does not share this defect. It will execute an iterative process in constant space, even if the iterative process is described by recursive procedure. An implementation with this property is called tail-recursive. With a tail-recursive implementation, iteration can be expressed using the ordinary call mechanism, so that special iteration constructs are useful only as syntactic sugar.
My Understanding: These paragraghs refer to a very important notion: tail-recursive. In functional programming, this property is fundamental because we use recursive procedure all the time. But in Java, if you use too many recursive procedures, even if they obey the form of tail-recursive, you can easily get a "stack over flow" error.
2015-11-27 046 Tree Recursion
QUOTE: In general, the evolved process looks like a tree. Notice that the branches split into two at each level (except at the bottom); this reflect the fact that fib procedure calls itself twice each time it is invoked.
QUOTE: This procedure is instructive as a prototypical tree recursion, but it is a terrible way to compute Fibonacci numbers because it does so much redundant computation.
QUOTE: In general, the number of steps required by a tree recursive process will be proportional to the number of nodes in the tree, while the space required will be proportional to the maximum depth of the tree.
My Understanding: The relations between number of steps and number of nodes, space required and maximum depth of the tree are very interesting. But I don't think they are self-confident, even though in previous paragraph, the writer didn't try to prove it, I have to think more deeply.
QUOTE: We can compute Fibonacci numbers iteratively using the procedure:
(define (fib n)
(fib-iter 1 0 n)
(define (fib-iter a b count)
(if (= count 0)
b
(fib-iter (+ a b)
a
(- count 1))))
QUOTE: One should not conclude from this that tree recursive processes are useless. When we consider processes that operate on hierarchically structured data rather than numbers, we will find that tree recursion is a natural and powerful tool.
2015-12-02 049 Tree Recursion
QUOTE: The observation that a tree-recursive process may be highly inefficient but often easy to specify and understand has led people to propose that one could get the best of both worlds by designing a "smart compiler" that could transform tree recursive procedures into more efficient procedures that compute the same result.
My Understanding: A tree-recursive process is often redundant and slow, but on the other hand, is straightforward, easy to write and read. So it will be wonderful if we can take advantage of its understandability and get rid of its inefficience. And that would make programmers' life much easier.
Orders of Growth
QUOTE: Processes can differ considerably in the rates at which they consume computational resources. One convenient way to describe this differences is to use the notion of order of growth to obtain a gross measure of the resources required by a process as the inputs become larger.
My Understanding: The ability to calculate the order of growth and therefore estimate comsumed computational resources used by an algorithm is the precondition of code tuning. If not, we can not predict the improvement we expect, and code tuning turn into an arbitrary behavior.
QUOTE: R(n) might measure the number of internal storage registers used, the number of elementary machine operations performed, and so on. In computers that do only a fixed number of operations at a time, the time required will be proportional to the number of elementary operations performed.
QUOTE: We say the R(n) has the order of growth Θ(f(n)), written R(n) = Θ(f(n)), if there are positive constants k1 and k2 independent of n such that
k1 * f(n) <= R(n) <= k2 * f(n)
for any sufficiently large value of n. (In other words, for large n, the value R(n) is sandwiched between k1f(n) and k2f(n).)
QUOTE: Orders of growth provide only a crude description of the behavior of a process. On the other hand, order of growth provide a useful indication of how we may expect the behavior of the process to change as we change the size of the problem.
My Understanding: There are three kind of typical processes: linear process, doubling the size will roughly double the amount of resouces used; exponential process, each increment in problem size will multiply the resource utilization by a constant factor; logarithmic process, doubling the problem size increases the resource requirement by a constant amount.
2015-12-03 050 Exponentiation
QUOTE: We can express this method as a procedure:
(define (fast-expt b n)
(cond ((= n 0) 1)
((even? n) (square (fast-expt b (/ n 2))))
(else (* b (fast-expt b (-n 1))))))
where the predicate to test whether an integer is even is defined in terms of the primitive procedure remainder by
(define (even? n)
(= (remainder n 2) 0))
The process evolved by fast-expt grows logarithmically with n in both space and number of steps.
QUOTE: The difference between Θ(log n) growth and Θ(n) growth becomes striking as n becomes large. It is also possible to use the idea of successive squaring to devise an iterative algorithm that computes exponentials with a logarithmic number of steps, although, as it is often the case with iterative algorithms, this is not written down so straightforwardly as the recursive algorithm.
QUOTE: In general, the technique of defining an invariant quantity that remains unchanged from state to state is a powerful way to think about the design of iterative algorithms.
My Understanding: Depend on the hints above, I write the procedure to generate a iterative process of fast exponential calculation. For this algorithm, the order of growth is Θ(log n) in number of steps and Θ(1) in space.
(define (fast-expt b n)
(define (fast-expt-iter b n a)
(cond ((= n 0) a)
((even? n) (fast-expt-iter (square b)
(/ n 2)
a))
(else (fast-expt-iter (square b)
(/ (- n 1) 2)
(* a b)))))
(fast-expt-iter b n 1))
My Understanding: The evolvement of exponential function described above show the potential of algorithm optimizing, even though the mathematical definition is simple and straightforward. The initial recursive definition needs Θ(n) in both space and number of steps, then the iterative version needs Θ(1) in space and Θ(n) in number of steps; after that, we use successive squaring to reduce the consumption, and recursive fast exponetial funtion needs Θ(log n) in both space and number of steps, meanwhile the iterative version reduce the space used to Θ(1).
2015-12-04 052 Greatest Common Divisors
QUOTE: The greatest common divisor (GCD) of two integers a and b is defined to be the largest integer that divides both a and b with no remainder. When we investigate how to implement rational-number arithmetic, we will need to be able to compute GCDs in order to reduce rational numbers to lowest terms. One way to find the GCD of two integers is to factor them and search for common factors, but there is a famous algorithm that is much more efficient.
QUOTE: The idea of the algorithm is based on the observation that, if r is the remainder when a is devided by b, then the common divisors of a and b are precisely the same as the common divisors of b and r. Thus we can use the equation:
GCD(a, b) = GCD(b, r)
to successively reduce the problem of computing a GCD to the problem of computing the GCD of smaller and smaller pairs of integers. It is possible to show that starting with any two positive integers and performing repeated reductions will always eventually produce a pair where the second number is 0. Then the GCD is the other number in the pair.
QUOTE: It is easy to express Euclid's Algorithm as a procedure:
(define (GCD a b)
(if (= b 0)
a
(GCD b (remainder a b))))
This generates a iterative process, whose number of steps grows as the logarithm of the numbers involved.
QUOTE: The fact that the number of steps required by Euclid's Algorithm has logarithmic growth bears an interesting relation to the Fibonacci numbers:
Lame's Theorem: If Euclid's Algorithm requires k steps to compute the GCD of some pair, then the smaller number in the pair must be greater than or equal to the kth Fibonacci numbers.
2015-12-07 054 Example: Testing for Primality
QUOTE: The following program finds the smallest integral divisor (greater than 1) of a given number n. It does this in a straightforward way, by testing n for divisibility by successive integers starting with 2.
(define (smallest-divisor n)
(find-divisor n 2))
(define (find-divisor n test-divisor)
(cond ((> (square test-divisor) n) n)
((divides? test-divisor n) test-divisor)
(else (find-divisor n (+ 1 test-divisor)))))
(define (divides? a b)
(= (remainder a b) 0))
We can test whether a number is prime as follows: n is prime if and only if n is its own smallest divisor.
(define (prime? n)
(= n (smallest-divisor n)))
Fermat's Little Theorem: If n is a prime number and a is any positive integer less than n, then a raised to the nth power is congruent to a modulo n.
QUOTE: To implement the Fermat test, we need a procudure that computes the exponential of a number modulo another number:
(define (expmod base exp m)
(cond ((= exp 0) 1)
((even? exp)
(remainder (square (expmod base
(/ exp 2)
m))
m))
(else
(remainder (* base (expmod base
(- exp 1)
m))
m))))
This is very similar to the fast-expt procedure introduced before. It uses successive squaring, so that the number of steps grows logarithmically with the exponent.
Random returns a nonnegative integer less than its integer input. Hence, to obtain a random number between 1 and n-1, we call random with an input of n-1 and add 1 to the result:
(define (fermat-test n)
(define (try-it a)
(= (expmod a n n) a))
(try-it (+ 1 (random (- n 1)))))
The following procedure runs the test a given number of times, as specified by a parameter. Its value is true if the test succeeds every time, and false otherwise.
(define (fast-prime? n times)
(cond ((= times 0) true)
((fermat-test n) (fast-prime? n (- times 1)))
(else false)))
2015-12-08 055 Probabilistic Methods
QUOTE: The Fermat test differs in character from most familiar algorithms, in which one computes an answer that is guaranteed to be correct. Here, the answer obtained is probably correct.
My Understanding: I think the existence of probabilistic methods is very interesting. In mathematics, most of the time the preciseness is required. But in engineering, we need to balance the result we want and the resources we have, and a reliable probabilistic method is a good compromise.
QUOTE: The existence of tests for which one can prove that the chance of error becomes arbitrarily small has sparked interest in algorithm of this type, which have come to be known as probabilistic algorithms. There is a great deal of research activity in this area, and probabilistic algorithms have been fruitfully applied to many fields.
My Understanding: There are two key points which make probabilistic algorithms practical. The first one is that we can estimate the chance of error, the second one is that we have the capability to reduce the chance to any degree we need.
2015-12-25 061 Formulating Abstractions with Higher-Order Procedures
Quote: One of the things we should demand from a powerful programming language is the ability to build abstractions by assigning names to common patterns and then to work in terms of the abstraction directly. Procedures provide this ability.
My Understanding: Modern programming languages provide us a natural division of abstraction layers: the abstraction level of methods, the abstraction level of classes and the abstraction level of packages. However, this is not enough, we need to think about the form of our own program, and take the most advantage of building abstractions.
Quote: Often the same programming patterns will be used with a number of different procedures. To express such patterns as concepts, we will need to construct procedures that can accept procedures as argument or return procedures as values. Procedures that manipulate procedures are called higher-order procedures.
My Understanding: Higher-order procedures contain two aspects of content, first, using procedures as argument, second, using procedures as return values. A typical example of higher-order procedures is derivation, which accept a function as argument and return another function.
2015-12-28 062 Procedures as Arguments
Quote: These three procedures clearly share a common underlying pattern. They are for the most part identical, differing only in the name of the procedure, the function of a used to compute the term to be added, and the function provides the next value of a. We could generate each of the procedures by filling the slots in the same template:
(define (<name> a b)
(if (> a b)
0
(+ (<term> a)
(<name> (<next> a) b))))
Quote: The power of sigma notation is that it allows mathematicians to deal with the concept of summation itself rather than only with particular sums -- for example, to formulate general results about sums that are independent of the particular series being summed.
My Understanding: The similarity here between mathematics and computer programming is very interesting.
Quote: Similarly, as program designers, we would like our language to be powerful enough so that we can write a procedure that expresses the concept of summation itself rather than only procedures that compute particular sums. We can do so readily in our procedural language by taking the common template shown above and transforming the "slots" into formal parameters:
(define (sum term a next b)
(if (> a b)
0
(+ (term a)
(sum term (next a) next b))))
2016-01-04 064 Exercise 1.33
Exercise: You can obtain an even more general version of accumulate by introducing the notion of a filter on the terms to be combined. That is, combine only those terms derived from values in the range that satisfy a specified condition. The resulting filtered-accumulate abstractions takes the same arguments as accumulate, together with an additional predicate of one arguments that specifies the filter. Write filtered-accumulate as a procedure. Show how to express the following using filtered-accumulate:
the sum of the squares of the prime numbers in the interval a to b.
the product of all the positive integers less than n that are relatively prime to n (i.e., all positive integers i < n such that GCD(i,n) = 1).
My Answers:
(define (filtered-accumulate filter? combiner term start end null-value)
(cond ((> start end) null-value)
((filter? start)
(combiner (term start)
(filtered-accumulate filter?
combiner
term
(+ start 1)
end
null-value)))
(else (filtered-accumulate filter?
combiner
term
(+ start 1)
end
null-value))))
(define (filtered-accumulate-iter filter? combiner term start end null-value)
(define (iter start result)
(cond ((> start end) result)
((filter? start)
(iter (+ start 1) (combiner (term start) result)))
(else (iter (+ start 1) result))))
(iter start null-value))
(define (prime? n)
(define (smallest-divisor k)
(cond ((> (square k) n) n)
((= (remainder n k) 0) k)
(else (smallest-divisor (+ k 1)))))
(= (smallest-divisor 2) n))
(define (sum-of-square-of-primes a b)
(filtered-accumulate prime? + square a b 0))
(define (gcd a b)
(if (= b 0)
a
(gcd b
(remainder a b))))
(define (product-of-relatively-primes n)
(define (relatively-prime? k)
(= (gcd n k) 1))
(define (self k)
k)
(filtered-accumulate-iter relatively-prime? * self 1 (- n 1) 1))
2015-01-05 066 Constructing Procedures Using Lambda
Quote: In general, lambda is used to create procedures in the same way as define, except that no name is specified for the procedure:
(lambda (<formal-parameters>) <body>)
The resulting procedure is just as much a procedure as one that is created using define. The only difference is that it has not been associated with any name in the environment.
Quote: Like any expression that has a procedure as its value, a lambda expression can be used as the operator in a combination such as
((lambda (x y z) (+ x y (square z))) 1 2 3)
or, more generally, in any context where we would normally use a procedure name.
Using let to create local variables
Quote: The general form of a let expression is
(let ((<var1> <exp1>)
(<var2> <exp2>)
...
(<varn> <expn>))
<body>)
which can be thought of as saying
let <var1> have the value <exp1> and
<var2> have the value <exp2> and
...
<varn> have the value <expn>
in <body>
2016-01-07 067 Using let to create local variables
Quote: The way this happen is that the let expresssion is interpreted as an alternative syntax for
((lambda (<var1> <var2> ... <varn>)
<body>)
(<exp1>
<exp2>
...
<expn>)))
No new mechanism is required in the interpreter in order to provide local variables. A let expression is simply syntactic sugar for underlying lambda expression.
My Understanding: I think the fact that a let expression is just syntactic sugar for lambda expression is very interesting. In many other programming languages, without the ability to define and use local variable, we almost don't know how to write code. But by learning Lisp, we find that local variable is not essential, so this is a process to simplify our understanding of computer programming and gradually reach to its true nature.
Quote: We can see from this equivalence that a scope of a variable specified by a let expression is the body of let. This implies that:
Let allows one to bind variables as locally as possible to where they are to be used.
The variables' values are computed outside the
let. This matters when the expressions that provide the values for the local variables depend upon variables having the same names as the local variables themselves.
My Understanding: The second point is very hard to understand just by the sentence itself. In other words, the scope of a local variable is the body of let expression, not the definition part of the local variable itself.