Scribd
Upload
105 views

Applications of Suffix Trees

This document discusses applications of suffix trees for string matching problems. It covers exact string matching, exact set matching, finding substrings in a set of texts, finding the longest common substring of two strings, and detecting DNA contamination. Suffix trees allow these problems to be solved in linear time after an initial preprocessing step to build the suffix tree. The document also discusses techniques to reduce the space usage of suffix trees, such as converting them to directed acyclic graphs or using suffix arrays instead of the full suffix tree structure.

Uploaded by

Yogendra Uikey
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
105 views

Applications of Suffix Trees

This document discusses applications of suffix trees for string matching problems. It covers exact string matching, exact set matching, finding substrings in a set of texts, finding the longest common substring of two strings, and detecting DNA contamination. Suffix trees allow these problems to be solved in linear time after an initial preprocessing step to build the suffix tree. The document also discusses techniques to reduce the space usage of suffix trees, such as converting them to directed acyclic graphs or using suffix arrays instead of the full suffix tree structure.

Uploaded by

Yogendra Uikey
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 40

Applications of Suffix Trees

Charles Yan 2008

1. Exact String Matching


|P|=n, |T|=m

P and T are both known at the same time Boyer-Moore, or Suffix trees. O(n+m) T is known and kept fixed. P varies. Suffix trees, O(m) in preprocess, O(n+k) in searching P is known and kept fixed. T varies. Boyer-Moore, O(n) in preprocess, O(m) in searching

2. Exact Set Matching


|T|=m, P={p1, P2, , pi}, |pi|=n

Aho-Corasick O(m+n+k)

Suffix trees. O(m) in building suffix tree O(ni+ki) in searching for pi O(m+ni+ ki) for all P, i.e. O(m+n+k)

3. Substring problem for a set of texts


Motivation 1: T is a DNA database containing millions of DNA sequences that have been previously sequenced. Given a new DNA sequence, to determine whether it has been previously sequenced. (1) Concatenate all T together, then use Boyer-Moore O(m+n+k) for searching each P, m is huge! (2) Build a suffix tree for each Ti O(m) for total preprocessing, but O( i* n+k) for searching each P, i is in the order of 106!
4

Substring problem for a set of texts


Motivation 2: To identify the remains of military personnel For each soldier, a set of DNA sequences (T= {T1, T2, , Ti}) is kept when he/she joins the army. (The whole genome sequence is very difficult to obtain for technical reasons.) A DNA sequence (P) is extracted from the remains of personnel that have been killed. To determine whether the remains belong to soldier A, we just need to see whether P matches any sequence in the T of A.
5

3. Substring problem for a set of texts


Given T={T1, T2, , Ti}, |Ti|=m, |P|=n, set T is fixed, P varies. O(m) preprocessing time is allowed. For each coming P, to find all occurrences of P in all T with O(n+k) time For each given P, this a the reverse of exact set matching problem. (1) Concatenate all T together, then use Boyer-Moore O(m+n+k) for searching each P (2) Build a suffix tree for each Ti, O(m) for total preprocessing, but O( i* n+k) for searching each P (3) Build a suffix tree (generalized suffix tree) for the set T, the searching will take O(n+k) time but how to build the a generalized suffix tree in O(m)?
6

Generalized Suffix Trees


How to build the generlized suffix tree for a set T = {T1, T2, , Ti) in O(m)? (1) Append a marker to the end of each string and concatenated them together to build a new string S. (2) Build a suffix tree for S. a (3) But, suffixes span multiple Ti,
b $ e f
7

Generalized Suffix Trees


Minor subtleties (1) Each edge is associated with three indices (i,p,q), where indicates that the substring come from Ti. p and q are the begin and end positions. (2) Suffixes from two texts may be identical. Thus, each leaf is associated with labels indicating all of the strings and starting positions of the associated suffix.

Generalized Suffix Trees


T1: xabxa T2: babxba

Generalized Suffix Trees


How to build the suffix tree for a set T = {T1, T2, , Ti) in O(m)? (1) Build a suffix tree for T1$ (2) Start from the root of the tree search for T2. Assume that i characters in T2 are matched,
The suffix tree has implicitly encoded every suffix of T2[1,..i] The suffix tree contains Ii for T2 We can skip phase 1,..,i for T2

(3) Continue the Ukkonens algorithm on T2 in phase i+1


Walk up from the end of T2[1,..i], (4) Until all Ti are included in the suffix tree.

10

4. Longest Common Substring (LCS) of Two Strings


Given strings S1 and S2, find the LCS of them. Different from longest common subsequence problem. S1: xabxa S2: babxba LCS is abx

11

4. Longest Common Substring (LCS) of Two Strings

Build a generalized suffix tree for S1 and S2 If a leave is from S1, then mark all its ancestors with 1. If a leave is from S2, then mark all its ancestors with 2. The path-label of any node that is marked with both 1 and 2 is a common substring of S1 and S2. Find the node that is labeled with 1 and 2, and has the greatest string-depth (number of characters on the path to it).

12

Generalized Suffix Trees


T1: xabxa T2: babxba
1,2

1,2
1,2 1 2 2 2 1,2 2

1,2 1,2 1,2 2 1

1
1

13

4. Longest Common Substring (LCS) of Two Strings


(1)

(2)

(3) (4)

O(m) for building generalized suffix tree O(m) for calculating the string-depth of each node (e.g. Breadth first) O(m) for marking node with 1 or 2 (e.g. Depth first) O(m) finding the longest.

14

5. DNA Contamination Problem


DNA contamination: During laboratory processes, unwanted DNA inserted into the DNA of interest.

Contamination sources: Human, bacteria, DNA from Dinosaur bone: More similar to human DNA than to bird and crockodilian DNA
15

5. DNA Contamination Problem


S: DNA of interest P: DNA of possible contamination source If S and P share a common substring longer than l , then S has been contaminated by P. To find all common substrings of S and P that are longer than l .

In general, P is set of DNA that are potential contamination sources.

16

Generalized Suffix Trees


T1: xabxa T2: babxba
1,2

1,2
1,2 1 2 2 2 1,2 2

1,2 1,2 1,2 2 1

1
1

17

6. Common Substrings Of More Than Two Strings


Motivation

18

6.Common Substrings Of More Than Two Strings


Problem statement: Given K strings whose lengths sum to n, let l(i) be the length of the longest substring common to at least i strings, to compute a table of K-1 entries, where entry i give l(i) and one of the common substrings of that length (and that is shared by at least i strings) {sandollar, sandlot, handler, grand, pantry}
i 2 3 4 5 l(i) 4 3 3 2 substring sand and and an
19

6. Common Substrings Of More Than Two Strings


It can be solve in O(n) time. But, an easy algorithm that uses O(kn) time first.

Build a generalized suffix tree for the k strings giving each string a unique end marker. Each leaf belong to only one string For a node (v), let c(v) be the number of distinct string identifiers that appear at the subtree below it. V is a vector with V(i) denoting the length of the longest substring that occurs exactly in i strings (and a pointer to the node). From V(i) compute l (i), for i=k; i>1; i if (V(i)<V(i+1)), then l(i)= V(i+1) else l(i)= V(i)
20

6. Common Substrings Of More Than Two Strings


5

4 2 2

V l

21

6. Common Substrings Of More Than Two Strings


Calculating c(v) is the bottle neck. Cant just count the number of leaves below it.

For each node keep a C vector of k bits, with one bit correspond to one string. ith is set to 1 if a leave that belongs to ith string appear below the node The V vector of a parent is obtained by ORing the vectors of its children. n nodes. O(Kn) in calculating c(v).
22

Suffix Trees to DAGs


Space is a big problem for suffix trees. S: xyxaxaxa$ The subtree under p is isomorphic to that under q except for leaf labels

q p 8

7 5 3

23

Suffix trees to DAGs


Directed acyclic graph (DAG)

a
8

6 1

24

Suffix Trees to DAGs


S: xyxaxaxa$ P: xax

2 8

-1

8 6 4

7
1 5 3

25

Suffix Trees to DAGs


q p 2 8 6 3 4
a

-1

8 6 4

7
1 5

If the subtrees under p and q are isomorphic (except leaf lables) and stringdepth(p)> stringdepth(q), then Merge p into q, by adding a direct edge from parent(p) to q Associated the directed edge with d=stringdepth(q)- stringdepth(p)

When search for P in the S (text), let i be the leaf below the path labeled with P, if the directed edge is traversed then P occurs at i+d, otherwise P occurs at i.

26

Suffix Trees to DAGs


How to determine whether a subtree is isomorphic to another one? Theorem 7.7.1 In suffix tree T the subtree below a node p is isomorphic to the subtree below a node q if and only if there is a directed path of suffix links from one node to the other node and the numbers of leaves in the two subtrees are equal. A if and only if B BA AB

27

Ukkonent Algorithm
Suffix links Let xa denote an arbitrary string, where x denotes a single character and a denotes a (possible empty) substring. For an internal node v with path-label xa, if there is another node s(v) with path-label a, then a pointer from v to s(v) is called a suffix link, denoted as (v,s(v)).
v

The root has no suffix link from it. If a is empty, then the suffix link points to the root.

s(v)

28

Suffix Trees to DAGs


BA Only one suffix link For every path from p to a leaf in its subtree, there is an identical path from q to a leaf in its subtree.
b

x
a a q

p
b

i x i a b

i+1

29

Suffix Trees to DAGs


BA A path of suffix links For every path from p to a leaf in its subtree, there is an identical path from q to a leaf in its subtree.
x a a u p t1 b b t2
30

t3

Suffix Trees to DAGs


AB
a g q

Either a is a proper suffix of g or g is a proper suffix of a There is a directed path of suffix links from one node to the other.
b

p
b

i a b

i+1

31

Suffix Trees to DAGs


AB a Either a is a proper suffix of g or g is a proper suffix of a There is a directed path of suffix links from one node to the other. g u p t1 b t2 t3 l q

a l

b b
32

Suffix Trees to DAGs


Let Q be the set of all pairs (p,q) such that there is a suffix link from p to q. While there is a pair (p,q) in Q Merge p into q; Remove (p,q); The merge of the pairs can be done in arbitrary order. In practice, we can start merge in a top-down approach (depth-first).
33

Suffix Arraysmore space reduction


Given a m-character string T, a suffix array for, called Pos, is an array of integers in the range 1 to m, specifying the lexicographic order of the m suffixes of string T. Pos[i] lexically less than Pos[i+1] mississippi pos 11,8,5,2,1,10,9,7,4,6,3

34

Suffix tree to suffix array

In O(m) time Lexical depth-first search

35

Pattern searching using suffix arrays


Observation: If p occurs in T then all the locations of those occurrences will be grouped consecutively in Pos. P=issi T=mississipi

36

Pattern searching using suffix arrays


Basic idea: Binary search O(nlogm) (worst) O(n+logm) (expected)

37

A simple accelerant
L and R are left and right boundaries of the current search interval. Query will be made at M=(L+R)/2 of Pos. l: the length of the longest prefix of Pos(L) that match a prefix of P r: the length of the longest prefix of Pos(R) that match a prefix of P lmr=min{l,r} Compare P and Pos(M) starting from position lmr+1 of the two string. O(nlogm)
38

A super accelerant

Lcp (i,j): length of the longest prefix of Pos(i) and Pos(j) Use Lcp(L,M), Lcp(M,R)

Suppose l>r, If Lcp(L,M) >l, LM, and l, r unchanged If Lcp(L,M) <l, RM, r=Lcp(L,M) If Lcp(L,M)=l, comparison of P and Pos(M) starting at l+1.

O(n+logm)

39

To obtain Lcp (i,j)

Lcp (i,i+1) for i=1 to m-1

Lexical depth first search

For any i<j, Lcp(i,j) is the smallest value of Lcp(k,k+1), where, k=i to j-1

Lexical depth first search in a complete binary tree

40

You might also like