Master theorem

From Wikipedia, the free encyclopedia

In the analysis of algorithms, the master theorem, which is a specific case of the Akra-Bazzi theorem, provides a cookbook solution in asymptotic terms for recurrence relations of types that occur in practice. It was popularized by the canonical algorithms textbook Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein, which introduces and proves it in sections 4.3 and 4.4, respectively. Nevertheless, not all recurrence relations can be solved with the use of the master theorem.

[edit] Generic form

The master theorem concerns recurrence relations of the form:

$T(n) = a \; T\!\left(\frac{n}{b}\right) + f(n) \;\;\;\; \mbox{where} \;\; a \geq 1 \mbox{, } b > 1.$

In the application to the analysis of a recursive algorithm, the constants and function take on the following significance:

n is the size of the problem.
a is the number of subproblems in the recursion.
n/b is the size of each subproblem. (Here it is assumed that all subproblems are essentially the same size.)
f (n) is the cost of the work done outside the recursive calls, which includes the cost of dividing the problem and the cost of merging the solutions to the subproblems.

It is possible to determine an asymptotic tight bound in these three cases:

[edit] Case 1

[edit] Generic form

If it is true that $f(n) = \mathcal{O}\left( n^{\log_b \left( a \right) - \epsilon} \right)$ for some constant $ε > 0$

it follows that:

$T(n) = \Theta\left( n^{\log_b a} \right).$

[edit] Example

$T(n) = 8 T\left(\frac{n}{2}\right) + 1000n^2$

As one can see in the formula above, the variables get the following values:

$a = 8 \,$ , $b = 2 \,$ , $f(n) = 1000n^2 \,$ , $\log_b a = \log_2 8 = 3 \,$

Now we have to check that the following equation holds:

$f(n) = \mathcal{O}\left( n^{\log_b a - \epsilon} \right)$

If we insert the values from above, we get:

$1000n^2 = \mathcal{O}\left( n^{3 - \epsilon} \right)$

If we choose $ε$ = 1, we get:

$1000n^2 = \mathcal{O}\left( n^{3 - 1} \right) = \mathcal{O}\left( n^{2} \right)$

Since this equation holds, the first case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n) = \Theta\left( n^{\log_b a} \right).$

If we insert the values from above, we finally get:

$T(n) = \Theta\left( n^{3} \right).$

Thus the given recurrence relation T(n) was in Θ(n³).

(This result is confirmed by the exact solution of the recurrence relation, which is $T (n) = 1001 n 3 - 1000 n 2$ , assuming $T (1) = 1$ .)

[edit] Case 2

[edit] Generic form

If it is true that:

$\exists k\ge 0, f(n) = \Theta\left( n^{\log_b a} \log^{k} n \right)$

it follows that:

$T(n) = \Theta\left( n^{\log_b a} \log^{k+1} n \right).$

[edit] Example

$T(n) = 2 T\left(\frac{n}{2}\right) + 10n$

As we can see in the formula above the variables get the following values:

$a = 2 \,$ , $b = 2 \,$ , $k = 0 \,$ , $f(n) = 10n \,$ , $\log_b a = \log_2 2 = 1 \,$

Now we have to check that the following equation holds (in this case k=0):

$f(n) = \Theta\left( n^{\log_b a} \right)$

If we insert the values from above, we get:

$10n = \Theta\left( n^{1} \right) = \Theta\left( n \right)$

Since this equation holds, the second case of the master theorem applies to the given recurrence relation, thus resulting in the conclusion:

$T(n) = \Theta\left( n^{\log_b a} \log n\right).$

If we insert the values from above, we finally get:

$T(n) = \Theta\left( n \log n\right).$

Thus the given recurrence relation T(n) was in Θ(n log n).

(This result is confirmed by the exact solution of the recurrence relation, which is $T (n) = n + 10 n log 2 n$ , assuming $T (1) = 1$ .)

[edit] Case 3

[edit] Generic form

If it is true that:

$f(n) = \Omega\left( n^{\log_b a + \epsilon} \right)$ for some constant

ε > 0

and if it is also true that:

$a f\left( \frac{n}{b} \right) \le c f(n)$ for some constant

c < 1

and sufficiently large n

it follows that:

$T\left(n \right) = \Theta \left(f \left(n \right) \right).$

[edit] Example

$T(n) = 2 T\left(\frac{n}{2}\right) + n^2$

As we can see in the formula above the variables get the following values:

$a = 2 \,$ , $b = 2 \,$ , $f(n) = n^2 \,$ , $\log_b a = \log_2 2 = 1 \,$

Now we have to check that the following equation holds:

$f(n) = \Omega\left( n^{\log_b a + \epsilon} \right)$

If we insert the values from above, and choose $ε$ = 1, we get:

$n^2 = \Omega\left( n^{1 + 1} \right) = \Omega\left( n^2 \right)$

Since this equation holds, we have to check the second condition, namely if it is true that:

$a f\left( \frac{n}{b} \right) \le c f(n)$

If we insert once more the values from above, we get:

$2 \left( \frac{n}{2} \right)^2$ $\le c n^2 \Leftrightarrow \frac{1}{2} n^2 \le cn^2$

If we choose $c = \frac{1}{2}$ , it is true that:

$\frac{1}{2} n^2 \le \frac{1}{2} n^2$ $\forall n \ge 1$

So it follows:

$T \left(n \right) = \Theta \left(f \left(n \right) \right).$

If we insert once more the necessary values, we get:

$T \left(n \right) = \Theta \left(n^2 \right).$

Thus the given recurrence relation T(n) was in Θ(n²), that complies with the f (n) of the original formula.

(This result is confirmed by the exact solution of the recurrence relation, which is $T (n) = 2 n 2 - n$ , assuming $T (1) = 1$ .)

[edit] Inadmissible

The following equations cannot be solved using the master theorem:^[1]

$T(n) = 2^nT\left (\frac{n}{2}\right )+n^n$

a is not a constant

$T(n) = 2T\left (\frac{n}{2}\right )+\frac{n}{\log n}$

non-polynomial difference between f(n) and $n^{\log_b a}$

$T(n) = 0.5T\left (\frac{n}{2}\right )+\frac{1}{n}$

a<1 cannot have less than one sub problem

$T(n) = 64T\left (\frac{n}{8}\right )-n^2\log n$

f(n) is not positive

$T(n) = T\left (\frac{n}{2}\right )+n(2-\cos n)$

case 3 but regularity violation

[edit] See also

Big O notation

[edit] Notes

^ http://www.cag.lcs.mit.edu/~thies/6.046-web/master.pdf

[edit] References

Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0-262-03293-7. Sections 4.3 (The master method) and 4.4 (Proof of the master theorem), pp.73–90.

Michael T. Goodrich and Roberto Tamassia. Algorithm Design: Foundation, Analysis, and Internet Examples. Wiley, 2002. ISBN 0-471-38365-1. The master theorem (including the version of Case 2 included here, which is stronger than the one from CLRS) is on pp. 268–270.

[edit] External links

[0] ttp://www.cag.lcs.mit.edu/~thies/6.046-web/master.pdf

[1]

Master theorem

From Wikipedia, the free encyclopedia

Contents

[edit] Generic form

[edit] Case 1

[edit] Generic form

[edit] Example

[edit] Case 2

[edit] Generic form

[edit] Example

[edit] Case 3

[edit] Generic form

[edit] Example

[edit] Inadmissible

[edit] See also

[edit] Notes

[edit] References

[edit] External links

Views

Personal tools

Navigation

Search

Interaction

Toolbox

Languages