BOT:
The term bot has come from the word robot .
Fundamentally, the bot is a software program which is used to
execute fully automated, well-defined, repetitive task over a
network.
The main reason behind the creation of bots is to execute
repetitive tasks at extremely high speed, impossible for an
actual human being to do the identical task.
Existing bots are created with good or malicious intentions.
According to Global Bot Traffic Report more than fifty
percent (50%) of the internet traffic is from the bots.
In literature, different categorizations are proposed by
different scholars.
I. Good Bot versus Bad Bots
II. Basic Bot versus Advanced Bots
III. Generalized Bot versus Specialized Bot
S.No Good Bot Bad Bot
1 It is an automated program for doing It is an automated program for doing destructive
constructive task. task.
2 The most widely used good bot is web The most widely used bad bot is web scrapper
crawler which is used to index web pages which is used to steal the contents of websites for
in search engines. illegal purposes.
3 Twenty eight percent (28%) of total web Nineteen percent (19%) of total web traffic is
traffic is from good bots. (web traffic from bad bots. (web traffic report 2016)
report 2016)
4 The good bots can be categorized as: The bad bots can be categorized as:
I. Web Crawler I. Web Scrapper
II. Social Media Bot II. Spammer
III. Chatbot III. Vulnerability Scanner
FB
Messenger
Chat bot
Rizwan Ur Rahman
Web search basics
Sponsored Links
CG Appliance Express
Discount Appliances (650) 756-3931
Same Day Certified Installation
User
www.cgappliance.com
San Francisco-Oakland-San Jose,
CA
Miele Vacuum Cleaners
Miele Vacuums- Complete Selection
Free Shipping!
www.vacuums.com
Miele Vacuum Cleaners
Miele-Free Air shipping!
All models. Helpful advice.
www.best-vacuum.com
Web Results 1 - 10 of about 7,310,000 for miele. (0.12 seconds)
Miele, Inc -- Anything else is a compromise
At the heart of your home, Appliances by Miele. ... USA. to miele.com. Residential Appliances.
Vacuum Cleaners. Dishwashers. Cooking Appliances. Steam Oven. Coffee System ...
www.miele.com/ - 20k - Cached - Similar pages
Web crawler Miele
Welcome to Miele, the home of the very best appliances and kitchens in the world.
www.miele.co.uk/ - 3k - Cached - Similar pages
Miele - Deutscher Hersteller von Einbaugeräten, Hausgeräten ... - [ Translate this
page ]
Das Portal zum Thema Essen & Geniessen online unter www.zu-tisch.de. Miele weltweit
...ein Leben lang. ... Wählen Sie die Miele Vertretung Ihres Landes.
www.miele.de/ - 10k - Cached - Similar pages
Herzlich willkommen bei Miele Österreich - [ Translate this page ]
Herzlich willkommen bei Miele Österreich Wenn Sie nicht automatisch
weitergeleitet werden, klicken Sie bitte hier! HAUSHALTSGERÄTE ...
www.miele.at/ - 3k - Cached - Similar pages
Search
Indexer
The Web
5 Indexes Ad indexes
Robot (spider) is a program that traverses the
hypertext sructure in the Web.
Web.
Collect information from visited pages
The Page(or pages) that the crawler starts with are
referred to as the seed URL
Used to construct indexes for search engines
Periodic Crawler – may visit a certain number of pages
and then stop, built an index, and replace the existing
index.
index.
It is activated periodically and serach portions of the
Web.
Web. ( Notices and Circular)
bots made for good. Ones that can help with your
marketing, increase your customer engagement and even
help boost your profits when you’re tight on time or
money.
promote a product 6
Web Content Outlier Mining
Extraction of Resources
Retrieve the desired web pages belonging to the
category of interest.
Use of web search engines or web crawlers.
Extract web page information from web which
contains Meta data field
Win Web Crawler 2.0
7
Web Scraping is the stealing of original and unique
contents from websites and publishing or pasting in another
website.
Content Scraping usually copy the complete content and
passed it to other website.
Contents can be breaking news, product reviews, product
prices, and Research Articles.
Price Scarping is the indexing of an entire e-commerce
website inventory and identifies what is in stock inventory,
price of the products and all the details related to the cost
of products.
Then, it creates an automated strategy to adjust their
pricing.
Form Spam is a method of submitting web forms
with unwanted information.
Unwanted information may be advertisements,
links to other pages, phishing URLs and abusive
texts.
Form spamming bots automatically find web
forms and post the advertisement links.
The link is then clicked by many through which
the spammer is able to generate traffic, ad
revenue or redirect them to phishing websites to
collect personal information.
The “general bots” are created for general
tasks.
For instance, the attacker can create a bot
which searches for website vulnerabilities
and loopholes.
As soon as it finds any vulnerability it attacks
the website by injecting malicious code.
On the other hand, “special bot” is created by
attacker to attack a particular website.
S.No Basic Bots Advanced Bots
1 Basic Bots are mostly written in simple Advanced Bots are written in fully functional
scripting languages. programming languages.
2 Common scripting languages used for Common languages used for advanced Bots are
basic Bots are Java Script, VB Script, and Ruby and Python
Perl.
3 They are standalone process They are embedded in real web browser.
4 They are lightweight processes easy to They are typically heavyweight sometimes custom
upload on web server. malwares are embedded.
5 They take few server resources to run. They take more server resources to run.
6 Typical used for parsing the HTML, Typically used for extracting information from web
storing cookies and executing Java Script sites and data uploading to web sites.
7 They are easy to detect. Requires, advanced techniques such as Machine
Learning and Artificial Intelligence to detect.
Credential Stuffing: The phrase “Credential Stuffing”
consists of two words credential which means claim and
proof.
The second part is stuffing; here, the meaning of Stuffing
is large scale stealing.
So, the Credential Stuffing is taking over large number of
credential that is, the user-name, and passwords.
False Accounts: In this attack, the bot creates fake or false
accounts at large scale in order to be able to make fake
followers and fake likes.
Account Lockout: Numerous web applications implement
security mechanism where if the user tries to log into their
account unsuccessfully four times, websites presume that that
account is under attack and they lock out the account for 12 or
24 hours.
An attacker attempt with different user names that are
associated with that websites without legitimate password they
could make multiple password attempts and lock out
thousands of users accounts.
Code obfuscation is the transformation of program into
another program that has an equivalent behavior and is
harder to understand and reverse engineer.
Captcha - A CAPTCHA test is designed to determine if an
online user is really a human and not a bot. CAPTCHA is an
acronym that stands for "Completely Automated Public
Turing test to tell Computers and Humans Apart.“
Honepots-A honeypot is a controlled and safe environment
for showing how attackers work and examining different
types of threats.
In the given example, the
size of the original HTML file
is 203 bytes on disk.
On the other hand the size
of minified HTML file is 103
bytes of data.
By applying first level
obfuscation all the
comments, whitespaces and
new line characters are
eliminated.
In this particular case the file
size is reduced by 50.73%.
19
Dead code injection is a natural way of adding
confusion to a source code, and thus increasing the
potency of obfuscation.
Being Dead code, the code isn't really executed, so
this has no impact on execution.
It is randomly injected.
20
Rizwan Ur Rahman
