MANUAL FOR USING AND MANAGING

DATA EXPLOITATION SYSTEM

VIETSPIDER

Ha Noi, 2011
 Contents

1 OVERVIEW ABOUT VIETSPIDER.............................................................6

1.1 Introduction..............................................................................................................6
1.1.1 Preamble.............................................................................................................................6
1.1.2 Presentation form of this book............................................................................................6
1.1.3 Questions and feedback......................................................................................................6
1.1.4 What is Vietspider data exploitation system?....................................................................6
1.1.5 Advantages of Vietspider....................................................................................................7
1.1.6 Working flow chart of Vietspider......................................................................................7
1.2 How to start and login to the system.........................................................................8
1.3 How to logout the system ......................................................................................10
1.4 The main components.............................................................................................11
1.4.1 Titlte bar............................................................................................................................11
1.4.2 Left menu..........................................................................................................................11
1.4.3 Browsing content..............................................................................................................11
1.4.4 Tool...................................................................................................................................12

2 DATA EXPLOITATION...............................................................................14
2.1Exploitation of data on the user interface ................................................................17
2.1.1 The user interface..............................................................................................................17
2.1.2 Deleting dowloaded data of the day..................................................................................19
2.1.3 Adding flow data collection .............................................................................................19
2.1.4 View all downloaded data.................................................................................................20
2.1.5 Updating............................................................................................................................21
2.2 Exploitation of content on the browser interface..........................................................21
2.2.1 The user interface..............................................................................................................21
2.2.2 The way to login on browser............................................................................................22
2.2.3 Logout system...................................................................................................................23
2.2.4 Optional categories...........................................................................................................24
2.2.5 Creating filter....................................................................................................................25
2.2.6 Deleting optional categories and creating filter function..................................................26
2.2.7 Search function.................................................................................................................27

VietSpider Trang
 Contents

3 TOOLS MANAGEMENT..............................................................................28
3.1 Creating data flow...................................................................................................36
3.1.1 Interface............................................................................................................................36
3.1.2 The way to create categories.............................................................................................37
3.1.3 The way to delete category or data flow ..........................................................................38
3.1.4 View deleted catagories and data flows............................................................................39
3.1.5 How to recover deleted catagories or data flows..............................................................40
3.1.6 Browsing data into flow....................................................................................................41
3.1.7 Copy of data flow..............................................................................................................42
3.1.8 Pasting data flow...............................................................................................................43
3.1.9 Sending categories or data flows to another computer.....................................................44
3.1.10 Saving of template configuration ...................................................................................45
3.1.11 How to set scanning level...............................................................................................46
3.1.12 How to set the priority of scanning level........................................................................47
3.1.13 Atributes setting..............................................................................................................48
3.1.14 Updating..........................................................................................................................50
3.1.15 Saving the configuration for the flow with the same name............................................51
3.1.16 Way to create new data flow...........................................................................................51
3.2 Supervision management...........................................................................................58
3.2.1 Interface............................................................................................................................58
3.2.2 Monitoring the loading flow.............................................................................................60
3.2.3 Browsing log.....................................................................................................................61
3.3 Crawling...............................................................................................................................62
3.3.1 Interface............................................................................................................................62
3.3.2 How to get data.................................................................................................................63
3.3.3 How to stop crawling........................................................................................................63
3.3.4 Add more flow to the list of data stream running.............................................................63
3.3.5 Remove flow from the list of scanning stream.................................................................64
3.3.6 Right-click menu...............................................................................................................65
3.4 User........................................................................................................................ 66
3.4.1 Interface............................................................................................................................66
3.4.2 User account......................................................................................................................67

VietSpider Trang
 Contents

3.4.2.1 Interface of user account................................................................................................67

3.4.2.2 Information about user account.....................................................................................68
3.4.2.3 Editing user account ......................................................................................................68
3.4.2.4 Creating user account.....................................................................................................68
3.4.2.5 Deleting user account.....................................................................................................69
3.4.3 User group.........................................................................................................................69
3.4.3.1 Group.............................................................................................................................70
3.4.3.2 User in a group...............................................................................................................71
3.4.3.3 User group’s categories..................................................................................................74
3.5 Configuration..........................................................................................................76
3.5.1 Interface............................................................................................................................76
3.5.1.1 System configuration.....................................................................................................77
3.5.1.2 Proxy setting..................................................................................................................78
3.5.1.3 Crawler configuration....................................................................................................78
3.5.1.4 Data group configuration...............................................................................................80

VietSpider Trang
 Contents

VietSpider Trang
1

1.1

1.1.1




1.1.2

1.1.3

 nhudinhthuan@yahoo.com

1.1.4

o
o
o

1.1.5


 

1.1.6

1.2

 here

o
o



o
 o
o
o
o
o
o

1.3

1.
2.
3.
4.

5.
1.4

1.

1.5

1.5.1


1.5.2

1.5.3



1.5.4


1.5.5


1.

2.
3.

4.
5.
2


o





 


 

2.1

2.1.1


2.2

2.2.1



o http://host address : port/http://VanAnh:9245/
2.2.2







2.2.3




2.2.4





2.2.5


2.2.6





2.2.7



3

3.1.1




2877
3.1.2

2977
3.1.3

3077
 

3.1.4

3177
3.1.5

3277


3377
3.1.6

3.1.7

3477
3.1.8

3577
3677
 

3.1.9

3777


3877
 

3.1.10

3977
3.1.11

3.1.12

4077
3.1.13

3.1.14

4177


4277
4377
3.1.15

3.1.16

4477
3.1.17

4577


4677
 

3.1.18

4777


4877
3.1.19

1.
2.
o

3.

4977
 http://dantri.com.vn/c25/giaoduc-khuyenhoc.htm

http://dantri.com.vn/c36/thegioi.htm

 http://tintuconline.com.vn/vn/thegioi/index.html
 http://tintuconline.com.vn/vn/kinhte/index.html
 http://tintuconline.com.vn/vn/khoahoc/index.html

5077
4.

5177
 

5.

5277
5377





5477
 

6.


7.
8.
9.


5577
10.

11.


5677
 

4.1.1

5777
 4.1.2

4.1.3

5877


5977
6077
5

1.

6177
 

5.1

5.1.1




6277
5.1.2

6377
 

5.1.3




5.1.4




1.

6477
5.1.5




5.1.6




6577
 

5.2

5.2.1




6677
5.2.2

5.2.2.1

5.2.2.2

1.

6777
 5.2.2.3

5.2.2.4




5.2.2.5

6877
 1.

5.2.3

6977


7077
-

7177
 

5.3

5.3.1

7277
7377
 5.3.1.1





7477
 5.3.1.2










7577


5.3.1.3











7677
7777