|
|
CSCI 256
|
Numbered problems are from Cormen, Leiserson, and Rivest. You should do all of the problems, but only turn in those from the second section..
Compute the running time of the following program. Only count the execution time of SimpleStatement inside the innermost loop. You may assume it takes one unit of time to execute.
algorithm something(n)
for i = 1 to sqrt(n) do
for j = sqrt(n) downto 1 do
for k = 1 to j do
SimpleStatement;
Technically the sqrt(n) should be floor(sqrt(n)), but we can ignore that
for the purposes of this problem. See the analysis of Insertion-sort on
page 8 of the text for an example (except you need not use constants or
keep track of anything but the executions of SingleStatement -- even the
computations of sqrt(n) should be considered to be free).
Solution: The two outer loops are executed exactly sqrt(n) times. The inner loops is executed j times for j running from 1 to sqrt(n). If we ignore the outer loop entirely, the rest of the program takes 1 step for the last time the inner loop is executed, 2 steps for the next to last, ..., sqrt(n) times for the first time the inner loop is executed (in reality, probably use floor or ceiling of sqrt(n)). Now
1 + 2 + ... + m = m(m+1)/2.
Thus
1 + 2 + ... + sqrt(n) = sqrt(n)(sqrt(n) + 1)/2.
The outer loop repeats this sqrt(n). Thus the total time is
sqrt(n)*sqrt(n)*(sqrt(n)+1)/2 = (n sqrt(n) + n)/2 which is O(n
sqrt(n)).
Problem 2-3 on pg 38 of CLR. You need only consider the functions (sqrt(2))lg n, n2, n!, (3/2)n, lg2 n, lg(n!), n1/lg n, and 2lg n. Give reasons for the ordering.
Solution:
n1/lg n = 2 < lg2 n
< (sqrt(2))lg n = (2lg n)1/2 = n1/2 = sqrt(n)
< 2lg n = n
< lg n! ~ n lg n
< n2
< (3/2)n
< n!
Most of these should be pretty straightforward. Powers of logs are
always bounded above by polynomials (including sqrt(n)).
The approximation for
lg n! comes from Stirling's approximation or by noticing that lg n! =
lg n + lg (n-1) + ... lg 1 <= n lg n because there are n terms, each
of which is smaller than lg n. Can you see how to bound it below
by a constant times n lg n?.
Exponentials always bound polynomials. Comparison between (3/2)n and n! is best done by taking the log of both sides. This gives a comparison between n lg (3/2) and n lg n (see above), where the second is clearly larger
Problem 2-4 a, b, d, g on pg 39 in CLR.
(a) False: n in O(n2), but n2 not in O(n).
Solutions:
(b) False: Let f(n) = n, g(n) = n2. Then min(f(n), g(n)) =
n, but f(n) + g(n) = n + n2 not in Theta(n).
(d) False: 2n in O(n) but 22n not in O(2n).
(g)False: Let f(n) = 22n. Then f(n/2) = 2n, but f(n) not in Theta(2n)
Problem 4.1-3 on pg 57 of CLR.
Solution: Change the inductive hypothesis to T(n) <= n lg n + n. Then T(1) = 1 = 1 lg 1 + 1.
Assume true for all k < n: T(k) <= k lg k + k. Show for n.
T(n) = 2 T(n/2) + n <= 2((n/2) lg (n/2) + n/2) + n
= n lg (n/2) + n + n = n lg n - n lg 2 + 2n
= n lg n + n.
Problem 4-1 a, c, f, g on pg 72 of CLR.
Solutions: Use Master Theorem for a, c, and f:
(a) T(n) = 2 T(n/2) + n3. Then a = 2, b = 2, so
logba = lg 2 = 1. In this case n3 in
Omega(n1), so the solution is Theta(n3).
(c) T(n) = 16 T(n/4) + n2. Then a = 16, b = 4, and log416 = 2. Now n2 in Theta(n2), so T(n) in Theta(n2 lg n).
(f) T(n) = 2 T(n/4) + sqrt(n). Then a = 2, b = 4, and log42 = 1/2. Now sqrt(n) in Theta(n1/2), so T(n) in Theta(sqrt(n) lg n).
(g) T(n) = T(n-1) + n. By substitution, get
T(n) = T(n-1) + n
= T(n-2) = (n-1) + n
...
= T(0) + 1 + 2 + ... n = T(0) + n(n+1)/2.
Prove that if T(0) = 0 then T(n) = n(n+1)/2. Clearly true for n = 0.
Suppose it is true for n: T(n) = n(n+1)/2. Show it for n+1:
T(n+1) = T(n) + (n+1) = n(n+1)/2 + (n+1) = (n+1)(n/2 + 1)
= (n+1)(n+2)/2
Therefore T(n) in Theta(n2).
Problem 2.2-4 on pg 37 of CLR. Use Stirling's approximation (equation 2.11 on page 35).
Proof: Stirling's approximation says
n! = sqrt(2 pi n) (n/e)n (1 + Theta(1/n))
Thus
lg (n!) = (1/2) lg (2 pi n) + n lg(n/e) + lg (1 + Theta(1/n))
Of these, clearly the middle term is the largest, and n lg(n/e) = n lg
n - n lg e in Theta(n lg n). Therefore lg(n!) in Theta(n lg n).
A more careful proof would work as follows:
n! = sqrt(2 pi n) (n/e)n (1 + Theta(1/n))
means that there exist c, c' such that
n! < sqrt(2 pi n) (n/e)n (1 + c(1/n))
and
n! > sqrt(2 pi n) (n/e)n (1 + c'(1/n))
for sufficiently large n. Therefore
lg n! < lg (sqrt(2 pi n) (n/e)n (1 + c(1/n)))
= (1/2)(lg 2 + lg pi + lg n) + n(lg n - lg e) + lg(1+c(1/n))
Therefore
lim (n!/(n lg n)) < lim ((1/2)(lg 2 + lg pi + lg n) + n(lg n - lg e) + lg(1+c(1/n))) / (n lg n)
< lim ((1/2)(lg 2 + lg pi + lg n)) / (n lg n) + lim (n lg n)/(n lg n)
+ lim (n lg e) / (n lg n) + lim (lg(1+c(1/n))) / (n lg n)
= 0 + 1 + 0 + lim (lg(1+c(1/n))) / (n lg n)
but since c(1/n) < 1 for n > c,
lim ((lg(1+c(1/n))) / (n lg n)) < lim ((lg 2) / (n lg n)) = 0
Thus lim ((lg n!) / (n lg n)) <= 1, so lg n! in O(n lg n).
Now we need to show n lg n in O(lg n!). We are going to get an upper bound on the lim (n lg n) / (lg n!). The key to doing this is to shrink the denominator. From the lower bound above on n!, we get
lg n! > lg (sqrt(2 pi n) (n/e)n (1 + c'(1/n)))
= (1/2)(lg 2 + lg pi + lg n) + n(lg n - lg e) + lg(1+c'(1/n))
< n lg n - n lg e
where the last inequality holds because we just dropped positive terms. If
we can bound this term below by c n lg n, for some c > 0, then we would
have
lim ((n lg n) / (lg n!)) < lim ((n lg n) / (c n lg n)) = 1 / c
and be done!
We show that for sufficiently large n, (1/2) n lg n - n lg e > 0. We can factor out the n to get that we need
n((1/2) lg n - lg e) = n (lg (sqrt(n)/e)) > 0
which is true when sqrt(n) > e or, equivalently, n > e2. Thus,
for n > e2,
n lg n - n lg e = (1/2) n lg n + ((1/2) n lg n - n lg e) > (1/2) n lg n
Thus
lim (n lg n) / (lg n!) < lim (n lg n) / ((1/2) n lg n) = 2
and thus n lg n in O(lg n!)
To show n! in o(nn), notice that
n! = 1*2*...*(n/2)*(n/2 + 1)* ... * n.
All of the first n/2 numbers are less than or equal to
n/2, while the last n/2 are all less than or equal to n. Therefore
n! < (n/2)n/2*nn/2 = nn/2n/2.
Therefore
lim (n!/nn) <= lim((nn/2n/2)/nn) = lim(1/2n/2) = 0,
and we are done!
These two results seem contradictory: lg n! in Theta(n lg n), while n! in o(n lg n), but they are not. It is similar to the result that 3n in Theta(n), while 23n not in O(2n, and in fact, 2n in o(3n). The idea is when you use a function as an exponent, you exagerate formerly unimportant differences. Conversely, when you take the lg of a function you collapse formerly different functions (at least with respect to Theta!).
Solve the recurrence T(0) = 0, T(n) = 2*T(n-1) + 1 for n > 1. You may use substitution to guess the answer, but prove the answer is correct by induction.
Solution:
T(n) = 2*T(n-1) + 1
= 2*(2*T(n-2) + 1) + 1 = 22*T(n-2) + 2 + 1
= 2*(22*T(n-2) + 2 + 1) + 1 = 23*T(n-3) + 4 + 2 + 1
= 2*(23*T(n-3) + 4 + 2 + 1) + 1 = 24*T(n-4) + 8 + 4 + 2 + 1
...
= 2k*T(n-k) + 2k-1 + ... + 1
If n = k, then T(n) = 2n*T(0) + 2n-1 + ... + 1 =
2n - 1 because T(0) = 0.
Prove T(n) = 2n - 1
Base: T(0) = 0 = 20 - 1
Induction: Suppose T(n) = 2n - 1, show true for n+1.
T(n+1) = 2*T(n) + 1
= 2*(2n - 1) + 1 = 2n+1 - 1.
Therefore T(n) in Theta(2n).
Solve the recurrence T(1) = 1, T(n) = 2*T(n/4) + lg n for n > 1. Do not use the master method. Instead use substitution to come up with an exact answer. Finally express the answer in as simple big-Theta form as possible. You need not prove it by induction. Hint: Only worry about n of the form 4k.
Solution:
T(n) = 2 T(n/4) + lg n
= 2(2T(n/42) + lg(n/4)) + lg n
= 22T(n/42) + 2 lg(n/4) + lg n
= 22T(n/42) + 2 (lg(n)-lg 4) + lg n
= 22T(n/42) + (2 + 1)lg(n) - 2*lg 4
= 22(2T(n/43) + lg(n/42)) + (2+1)lg n - 2*lg 4
= 23T(n/43) + (22+2+1)lg n - (22*2*lg 4+2*lg 4)
= 24T(n/44) + (23+22+2+1)lg n - (23*3*lg 4+22*2*lg 4+2*lg 4)
...
= 2kT(n/4k) + (2k-1+...+2+1)lg n - 2(lg 4)(2k-2*(k-1)+2k-3*(k-2)+...+1)
We show in class (and we repeat it below) that
(2k-2*(k-1)+2k-3*(k-2)+...+1 = k*2k-1 - (2k - 1)
so we use it below.
Now if n = 4k then sqrt(n) = 2k and k =
lg(sqrt(n)). Then
T(n) = sqrt(n)*T(1) + (2k-1)lg n - 4(k*2k-1-(2k-1)))
= sqrt(n) + (sqrt(n)-1)lg n - 4(log(sqrt(n))*sqrt(n)/2-(sqrt(n)-1)))
= sqrt(n) + (sqrt(n)-1)lg n - 4((1/2)(log n)*sqrt(n)/2-(sqrt(n)-1))
= sqrt(n) + (sqrt(n)-1)lg n - (log n)*sqrt(n) + 4*sqrt(n) - 4
= sqrt(n) + (sqrt(n)-1)lg n - (log n)*sqrt(n) + 4*sqrt(n) - 4
= 5*sqrt(n) - lg n - 4
Thus T(n) in Theta(sqrt(n))
We need to go back and show the complex sum above reduces to the simpler formula shown above. Note
1 + x + x2 + xk-1 = (xk-1)/(x-1)
is the sum of a geometric series with ratio x. As discussed in section
3.1 of the text, we can differentiate both sides of a series. If we do
that we get
1 + 2x + 3x2 + ... + (k-1)xk-2
= [(x-1)k(xk-1) - (xk - 1)]/(x-1)2
If we substitute 2 for x we get the result claimed above.
Back to:
kim@cs.williams.edu