Home Randomized Algorithm II — Global Min Cut & Median
Post
Cancel

Randomized Algorithm II — Global Min Cut & Median

In this article, we discuss two typical and educational problems which can be well solved using randomized algorithm - global min cut problem and median problem.

Global Min Cut

Problem Background

In graph theory, a cut is a partition of the vertices of a graph into two disjoint subsets. The size of a cut is defind as the number of edges across two partitions. A cut is minimum if the size of the cut is not larger than the size of any other cut. The illustration below shows a minimum cut: the size of this cut is 2.

An example of global min cut

The gloabl min cut problem is how to find the minimum cut in a given graph. A closely related problem is the min s-t cut problem, where two extral vertices s and t are provided. In the min s-t cut problem, we require s and t are divided by the cut. In other words, s should be in one vertex partition, and t should be in another.

One can solve the min s-t cut problem using any polynomial-time algorithm for max s-t flow, and the current fastest algorithm runs in O(m1+o(1)) time. Hence, a naive strategy is to divide the global min cut problem to n1 min s-t cut problems: If the graph vertex set V={v1,...,vn}, then we solve min v1 - vi cut for each i=2,...,n and output the best cut among these n1 cuts.

Karger’s Algorithm

Here we only consider multigraphs without self-loops, i.e., there are possibly multiple edges between a pair of distinct vertices but no edges linking to itself. Following is the randomized algorithm to find the global min cut.

Karger's Algorithm

In each loop, this algorithm randomly choose an edge from the graph, and do the edge contraction in line 3. During the edge contraction operation (G/e), The edge e is removed and its two incident vertices, u and v, are merged into a new vertex, as illustrated below.

Edge contraction

Note that after each edge contraction, the new graph is still a multigraph without self-loops.

If you remain confusion about the edge contraction after view the above picture, we provide a formal definition here. We define G/e to be the graph resulted from contraction of e by:

  1. Replace u,v by a new vertex w;
  2. Replace every edge ux or vx where xV{u,v} by a new edge wx.

The algorithm ends when there are only two vertices left in the graph, and output the number of edges between them.

In fact, the edge contraction process is partitioning the original vertex set, and the edges between last two vertices represent the cut of this partition.

Here is a lemma describing the success possibility of this algorithm:

Pr(Algorithm1 outputs the global min cut)1Cn2

The proof of this lemma is at the end of this article.

As we did before, we can boost the success possibility by multiple runs. Here, if we run this algorithm for Cn2clogn times where c>0 is some constant, and output the best cut among them. Then we have:

Pr(none of these runs output the global min cut)(11Cn2)Cn2clogneclogn=1nc

The first inequality we used here is union bound, which says that for a set of events, the probability that at least one of the events happens is no greater than the sum of the probabilities of the individual events; the second inequality is (1+1k)ke, which is an extended form of Bernoulli’s inequality. These two inequalities are extremely useful in the possibility bounding of randomized algorithm.

So, the success possibility after multiple runs is at least 11/nc. This means that to guarantee a success probability at least 11/ploy(n), it suffices to run the algorithm for O(n2logn) times.

Finally, let’s look at the overall running time. If we use the adjacency matrix representation, every edge contraction takes O(n) time. Thus, each run of the algorithm takes O(n2) time. The overall running time to get success probability at least 11/ploy(n) is O(n4logn). Such running time is even slower than the naive solution we provided in the Introduction section. Next, we will discuss how to improve this randomized algorithm.

Karger-Stein Algorithm

First, let’s examine the possibility of global min cut surviving when only l vertices are left in the Karger’s Algorithm, meaning that all of the first selected nl edges do not belong to the min cut. Suppse ei represents the i-th selected edge (i0), then we have: (calculation is omitted)

Pr(global min cut survives down to l vertices)=Pr(e0,e1,...,enl1global min cut)Cl2Cn2

Looking close to this inequality, we will find the lower bound of surviving possibility is close to one in initial loops. For example, after the first loop, where l=n1, the possibility is at least 12/n. However, this value drops faster and faster when l decreases from n to 2. In other words, in the Karger’s Algorithm, initial edge contractions are likely correct, while later edge contractions are much less likely to be correct. Thus, instead of running the whole Algorithm 1 for multiple times, a better way is to run initial edge contractions fewer times while later contractions more times.

We can observe that when ln/2, the possibility is at least 1/2. Based on this, the algorithm can be modified into such resursive style:

Karger-Stein Algorithm

Defind P(n) to be the probability of success of Algorithm 2 on any n-vertex graph. We have the recursion:

1P(n)(112P(n2))2

Solving the recursion gives P(n)=Ω(1/logn).

The running time of Algorithm2 satisfies the recursion:

T(n)=2T(n2)+O(n2)

Therefore, T(n)=O(n2logn). To get success probability at least 11/ploy(n), we run Algorithm2 for O(log2n) times and the overall running time is O(n2log3n).

Median

In this section, let’s disucss a easier one - the median problem, which is described as this: Given an unsorted list S=[s1,...,sn] of integers, find the median of them. A straight-forward way to find the median is to sort them first, taking O(nlogn) time, and then output the value we want.

QuickSelect is a simple randomized algorithm with expected running time O(n). The approach is described as follow.

  1. Find a good “pivot” pS;
  2. Partition S into Sp={xS:xp} and S>p={xS:x>p};
  3. The median is in one of Sp and S>p, and recurse.

We say the pivot p is good if both Sp3n/4 and S>p3n/4. This means that a good pivot should be close to the median. Assume that we always get a good pivot, the running time of QuickSelect then satisfies the recursion:

T(n)=T(3n4)+O(n)

Solving the recursion gives T(n)=O(n).

Here we introduce a simple strategy to find good pivots - picking random elements from S. According to our definition about good pivot, p is good if it is in between the (n/4)th smallest element and the (3n/4)th smallest element. Hence, there are (at least) n/2 good pivots, indicating that with 1/2 possibility we can find a good pivot by randomly picking.

Knowing this, we can follow such a procedure to find a good pivot: Choose a random pivot and check if it is good; if not, repeat. The number of rounds needed to find a good one is O(1) in expectation (beacuse it is in geometry distribution). As a result, the overall running time of QuickSelect is in O(n) expectation.

Checking whether a pivot p is good is equivalent to checking whether Sp3n/4 and S>p3n/4, which takes O(n) time.

Conclusion

After going through these two basic problems, you may find that the design of randomized algorithm is not as complicated as normal algorithms. All the algorithms we discussed in this article are not longer than four lines! The intricate thing is to decide how many runs are neeeded to boost the success possibility to the 11/ploy(n) level.

“Success possibility at least 11/ploy(n)” is a de facto standard in the theory field.

Proof

This section is optional to read.

Given G=(V,E) as a multigraph without self-loops. Suppose a vertex bipartition (S,VS) is a global min cut of G, where SV. Let δ denote the edge set of global min cut of G. Namely,

δ=E(S,VS)={uvE:uS,vVS}

Let’s prove the lemma:

Pr(Algorithm1 outputs δ)1Cn2.

Denote the sequence of edges chosen and contracted by Algorithm 1 as e0,e1,...,en3; note that we need to contract n2 edges to get down to two vertices.

A key observation here is that Algorithm 1 outputs δ if and only if e0,e1,...,en3δ. This means that each edge chosen to be contracted is inside of S or VS. No chosen edge is across these two vertex sets. So, we have

Pr(Algorithm1 outputs δ)=Pr(e0,e1,...,en3δ)

By the chain rule, we have

Pr(e0,e1,...,en3δ)=Pr(e0δ)Pr(e1δ|e0δ)Pr(e2δ|e0,e1δ)Pr(en3δ|e0,e1,...,en4δ)

For two events A and B, the chain rule states that
Pr(AB)=Pr(B|A)Pr(A),
where Pr(B|A) denotes the conditional probability of B given A.

Let’s look at the first term:

Pr(e0δ)=1Pr(e0δ)=1km,

where k=|δ| and m=|E|.

Next, we need to introduce a fact about the relationship between k and m.

The average degree of a graph G=(V,E) is given by

d=1nvVdeg(v)=2mn.

The symbol deg(v) means the degree of the vertex v.

Since (v,V{v}) is a cut of size deg(v) for all vV, it holds

k=|δ|minvVdeg(v)d.

Combining above two formulas, we get

km2n.

So,

Pr(e0δ)=1km12n.

For the second term, suppose e0δ is given and let G=(V,E)=Ge0. Since e0δ, the contraction of e0 will not affect the size of global min cut! Namely, |δ| of Ge0 is still k. Considering |V|=n1 and letting m=|E|, we deduce

Pr(e1δ|e0δ)=1km12n1.

More generally, for each i=0,1,...,n3 we have

Pr(eiδ|e0,e1,...,ei1δ)12ni.

Finally, we get

Pr(Algorithm1 outputs δ)(12n)(12n1)(124)(123)=n2nn3n12413=2n(n1)=1Cn2,

as claimed.

This post is licensed under CC BY 4.0 by the author.

Randomized Algorithm I — Introduction

Randomized Algorithm III— Concentration Inequalities