记录黑客技术中优秀的内容,传播黑客文化,分享黑客技术精华

对于crawlergo|rad|burpsuite|awvs爬虫的对比

2021-01-25 13:35

前言

最近在写代码,涉及了web爬取链接的方面,在百度过程中了解到了这篇文章:superSpider,突然就好奇平时常见的爬虫 工具和扫描器里的爬虫模块能力如何,所以来测试下。

主要测试1个自己手写的瞎眼爬虫,还有crawlergo、rad、burpsuite pro v202012、awvs 2019

文章里推荐的http://demo.aisec.cn/demo/站点打不开了所以就使用awvs的了;

测试站点:http://testphp.vulnweb.com

一 手写的基准爬虫

只抓取a标签下的href和script标签下的src;

from urllib.parse import urlparse,urljoin
from bs4 import BeautifulSoup
import requests
import validators
from queue import Queue
import threading
requests.packages.urllib3.disable_warnings()


class jsfinder():
def __init__(self,url,cookie=""):
self.baseUrl = self.return_entire_url(url)
self.headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.108 Safari/537.36",
"cookie": cookie}
self.q = Queue()
self.crawed_list = set()
self.urlList = []
self.q.put(url)

self.spider_status = 1

def return_entire_url(self,url):
if url is not None:
if url.startswith('http') or urlparse(url).scheme:
return url.strip()
else:
if self.baseUrl == "":
self.baseUrl = "http://" + url
print(self.baseUrl)
return urljoin(self.baseUrl,url.strip())
else:
pass

def spider(self):
while(not self.q.empty() or self.spider_status):
url = self.q.get()
if url in self.crawed_list :
continue
print("requesting:",url)
try:
resp = requests.get(url=url, headers=self.headers, timeout=5, verify=False)
self.htmlParse(resp)
self.crawed_list.add(url)
except:
print("requests error:",url)

if self.spider_status == 1:
time.sleep(5)
self.spider_status = 0

print(self.q.qsize())

def htmlParse(self,response):
tempList = []
blacklist = ['#',None,'javascript:']

soup = BeautifulSoup(response.text.encode('utf-8'), 'html.parser')
for href in soup.find_all('a'):
#print(self.urlParse(href.get('href')))
tempList.append(href.get('href'))

for href in soup.find_all('script'):
#print(self.urlParse(href.get('src')))
tempList.append(href.get('src'))

tempList = list(set(tempList)-set(blacklist))
for i in tempList:
url = self.return_entire_url(i)
if validators.url(url):
print("get:",url)
#print(i,self.return_entire_url(i))
if url not in self.crawed_list :
self.urlList.append(url)
if urlparse(url).netloc in self.baseUrl:
self.q.put(url)

if __name__ == "__main__":
A = jsfinder("http://testphp.vulnweb.com")
t = threading.Thread(target=A.spider)
t.start()
t.join()
for i in list(set(A.urlList)):
print(i)

结果:
46个链接,夹杂着很多其他域名的链接,有很多带参数的链接

http://testphp.vulnweb.com/product.php?pic=3
http://testphp.vulnweb.com/cart.php
https://www.acunetix.com/blog/articles/prevent-sql-injection-vulnerabilities-in-php-applications/
http://testphp.vulnweb.com/hpp/
http://testphp.vulnweb.com/product.php?pic=7
http://testphp.vulnweb.com/guestbook.php
http://testphp.vulnweb.com/listproducts.php?cat=2
http://testphp.vulnweb.com/Details/network-attached-storage-dlink/1/
http://testphp.vulnweb.com/categories.php
http://testphp.vulnweb.com/artists.php
http://www.eclectasy.com/Fractal-Explorer/index.html
http://testphp.vulnweb.com/artists.php?artist=1
http://testphp.vulnweb.com/showimage.php?file=./pictures/5.jpg
http://testphp.vulnweb.com/showimage.php?file=./pictures/4.jpg
http://testphp.vulnweb.com/listproducts.php?artist=1
http://testphp.vulnweb.com/product.php?pic=1
http://testphp.vulnweb.com/showimage.php?file=./pictures/7.jpg
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/product.php?pic=5
http://testphp.vulnweb.com/listproducts.php?artist=3
http://www.acunetix.com
http://testphp.vulnweb.com/showimage.php?file=./pictures/2.jpg
http://testphp.vulnweb.com/Details/color-printer/3/
http://testphp.vulnweb.com/listproducts.php?artist=2
http://testphp.vulnweb.com/disclaimer.php
http://testphp.vulnweb.com/login.php
http://testphp.vulnweb.com/listproducts.php?cat=1
http://testphp.vulnweb.com/artists.php?artist=2
http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg
http://testphp.vulnweb.com/Details/web-camera-a4tech/2/
https://www.acunetix.com/vulnerability-scanner/php-security-scanner/
http://testphp.vulnweb.com/listproducts.php?cat=4
http://testphp.vulnweb.com/privacy.php
http://testphp.vulnweb.com/AJAX/index.php
http://testphp.vulnweb.com/listproducts.php?cat=3
https://www.acunetix.com/vulnerability-scanner/
http://testphp.vulnweb.com/signup.php
http://testphp.vulnweb.com/product.php?pic=2
http://testphp.vulnweb.com/showimage.php?file=./pictures/3.jpg
https://www.acunetix.com/
http://testphp.vulnweb.com/index.php
http://testphp.vulnweb.com?pp=12
http://testphp.vulnweb.com/Mod_Rewrite_Shop/
http://testphp.vulnweb.com/artists.php?artist=3
http://blog.mindedsecurity.com/2009/05/client-side-http-parameter-pollution.html
http://testphp.vulnweb.com/product.php?pic=4

二 crawlergo爬取

在官方示例代码上加了几行

#!/usr/bin/python3
# coding: utf-8

import simplejson
import subprocess


def main():
target = "http://testphp.vulnweb.com/"
cmd = ["/home/loser/MySimpleScanner-master-v2/tools/crawlergo", "-c", "/usr/bin/google-chrome", "-o", "json", target]
rsp = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output, error = rsp.communicate()
# "--[Mission Complete]--" 是任务结束的分隔字符串
result = simplejson.loads(output.decode().split("--[Mission Complete]--")[1])
req_list = result["req_list"]
for req in req_list:
print(req)
#print(req_list[0])


if __name__ == '__main__':
main()

结果:
48条

{'url': 'http://testphp.vulnweb.com/', 'method': 'GET', 'headers': {'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Target'}
{'url': 'https://testphp.vulnweb.com/', 'method': 'GET', 'headers': {'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Target'}
{'url': 'http://testphp.vulnweb.com/artists.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/index.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/categories.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/disclaimer.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/guestbook.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/AJAX/index.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/cart.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/login.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/userinfo.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/privacy.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/hpp/', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/Mod_Rewrite_Shop/', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/search.php?test=query', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'searchFor=Crawlergo', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/search.php?test=query', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'searchFor=Crawlergo&goButton=go', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/signup.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/login.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/userinfo.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/login.php', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'uname=crawlergo%40gmail.com&pass=Crawlergo6.', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/listproducts.php?cat=1', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/categories.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/artists.php?artist=1', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/artists.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/comment.php?aid=1', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/artists.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'OpenWindow'}
{'url': 'http://testphp.vulnweb.com/AJAX/artists.php', 'method': 'GET', 'headers': {'Accept': '*/*', 'Referer': 'http://testphp.vulnweb.com/AJAX/index.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/AJAX/categories.php', 'method': 'GET', 'headers': {'Accept': '*/*', 'Referer': 'http://testphp.vulnweb.com/AJAX/index.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/AJAX/titles.php', 'method': 'GET', 'headers': {'Accept': '*/*', 'Referer': 'http://testphp.vulnweb.com/AJAX/index.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/AJAX/showxml.php', 'method': 'POST', 'headers': {'Accept': '*/*', 'Referer': 'http://testphp.vulnweb.com/AJAX/index.php', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36', 'content-type': 'text/xml'}, 'data': '<xml><node name="nodename1">nodetext1</node><node name="nodename2">nodetext2</node></xml>', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/hpp/?pp=12', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/hpp/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/userinfo.php', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/search.php?test=query', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/listproducts.php?artist=1', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/artists.php?artist=1', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/secured/newuser.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/signup.php', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'uuname=carwalwelregrogo%40gmail.com&upass=Crawlergo6.&upass2=Crawlergo6.&urname=crawlergo%40gmail.com&ucc=Crawlergo&uemail=crawlergo%40gmail.com&uphone=18812345678&uaddress=Cr', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/secured/newuser.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/signup.php', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'uuname=carwalwelregrogo%40gmail.com&upass=Crawlergo6.&upass2=Crawlergo6.&urname=crawlergo%40gmail.com&ucc=Crawlergo&uemail=crawlergo%40gmail.com&uphone=18812345678&uaddress=Cr&signup=signup', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg&size=160', 'method': 'GET', 'headers': {'Accept': 'image/avif,image/webp,image/apng,image/*,*/*;q=0.8', 'Referer': 'http://testphp.vulnweb.com/listproducts.php?cat=1', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/product.php?pic=2', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/listproducts.php?cat=1', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/listproducts.php?cat=1', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/comment.php?pid=1', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/listproducts.php?cat=1', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'OpenWindow'}
{'url': 'http://testphp.vulnweb.com/userinfo.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/login.php', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'uname=crawlergo%40gmail.com&pass=Crawlergo6.', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/comment.php?aid=1', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'name=Ccrraawwlleerrggoo%40gmail.com%3Cyour+name+here%3E&comment=&phpaction=echo+%24_POST%5Bcomment%5D%3B', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/comment.php?aid=1', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'name=Ccrraawwlleerrggoo%40gmail.com%3Cyour+name+here%3E&comment=&Submit=Submit&phpaction=echo+%24_POST%5Bcomment%5D%3B', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/hpp/params.php?p=valid&pp=12', 'method': 'GET', 'headers': {'Referer': 'http://testphp.vulnweb.com/hpp/?pp=12', 'Spider-Name': 'crawlergo', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'DOM'}
{'url': 'http://testphp.vulnweb.com/hpp/params.php?', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Referer': 'http://testphp.vulnweb.com/hpp/?pp=12', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/hpp/params.php?aaaa%2F=Submit', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Referer': 'http://testphp.vulnweb.com/hpp/?pp=12', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/AJAX/showxml.php', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/secured/newuser.php', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/comment.php?aid=1', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'name=Ccrraawwlleerrggoo%40gmail.com%3Cyour+name+here%3E&comment=&phpaction=echo+%24_POST%5Bcomment%5D%3B', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/cart.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/product.php?pic=2', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'price=800&addcart=2', 'source': 'XHR'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'POST', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Content-Type': 'application/x-www-form-urlencoded', 'Origin': 'http://testphp.vulnweb.com', 'Referer': 'http://testphp.vulnweb.com/comment.php?aid=1', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': 'name=Ccrraawwlleerrggoo%40gmail.com%3Cyour+name+here%3E&comment=&Submit=Submit&phpaction=echo+%24_POST%5Bcomment%5D%3B', 'source': 'Navigation'}
{'url': 'http://testphp.vulnweb.com/comment.php', 'method': 'GET', 'headers': {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', 'Spider-Name': 'crawlergo', 'Upgrade-Insecure-Requests': '1', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.0 Safari/537.36'}, 'data': '', 'source': 'Navigation'}

清洗后:

http://testphp.vulnweb.com/
https://testphp.vulnweb.com/
http://testphp.vulnweb.com/artists.php
http://testphp.vulnweb.com/index.php
http://testphp.vulnweb.com/categories.php
http://testphp.vulnweb.com/disclaimer.php
http://testphp.vulnweb.com/guestbook.php
http://testphp.vulnweb.com/AJAX/index.php
http://testphp.vulnweb.com/cart.php
http://testphp.vulnweb.com/login.php
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/privacy.php
http://testphp.vulnweb.com/hpp/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/
http://testphp.vulnweb.com/search.php?test=query
http://testphp.vulnweb.com/search.php?test=query
http://testphp.vulnweb.com/signup.php
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/listproducts.php?cat=1
http://testphp.vulnweb.com/artists.php?artist=1
http://testphp.vulnweb.com/comment.php?aid=1
http://testphp.vulnweb.com/AJAX/artists.php
http://testphp.vulnweb.com/AJAX/categories.php
http://testphp.vulnweb.com/AJAX/titles.php
http://testphp.vulnweb.com/AJAX/showxml.php
http://testphp.vulnweb.com/hpp/?pp=12
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/search.php?test=query
http://testphp.vulnweb.com/listproducts.php?artist=1
http://testphp.vulnweb.com/secured/newuser.php
http://testphp.vulnweb.com/secured/newuser.php
http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg&size=160
http://testphp.vulnweb.com/product.php?pic=2
http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg
http://testphp.vulnweb.com/comment.php?pid=1
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/hpp/params.php?p=valid&pp=12
http://testphp.vulnweb.com/hpp/params.php?
http://testphp.vulnweb.com/hpp/params.php?aaaa%2F=Submit
http://testphp.vulnweb.com/AJAX/showxml.php
http://testphp.vulnweb.com/secured/newuser.php
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/cart.php
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/comment.php

3 rad 爬取

./rad_linux_amd64 --target http://testphp.vulnweb.com --text-output rad.log
结果:42条 , 由于存在get和post的区别,清洗后去重为39条

GET http://testphp.vulnweb.com/
GET http://testphp.vulnweb.com/index.php
GET http://testphp.vulnweb.com/artists.php
GET http://testphp.vulnweb.com/cart.php
GET http://testphp.vulnweb.com/guestbook.php
GET http://testphp.vulnweb.com/AJAX/index.php
GET http://testphp.vulnweb.com/images/
GET http://testphp.vulnweb.com/login.php
POST http://testphp.vulnweb.com/search.php?test=query
GET http://testphp.vulnweb.com/categories.php
GET http://testphp.vulnweb.com/disclaimer.php
GET http://testphp.vulnweb.com/userinfo.php
POST http://testphp.vulnweb.com/guestbook.php
POST http://testphp.vulnweb.com/userinfo.php
GET http://testphp.vulnweb.com/Flash/
GET http://testphp.vulnweb.com/AJAX/artists.php
GET http://testphp.vulnweb.com/privacy.php
GET http://testphp.vulnweb.com/AJAX/infoartist.php?id=1
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/
GET http://testphp.vulnweb.com/hpp/
GET http://testphp.vulnweb.com/artists.php?artist=1
GET http://testphp.vulnweb.com/comment.php?aid=1
GET http://testphp.vulnweb.com/signup.php
GET http://testphp.vulnweb.com/listproducts.php?cat=1
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/images/
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/
GET http://testphp.vulnweb.com/hpp/?pp=12
POST http://testphp.vulnweb.com/comment.php
POST http://testphp.vulnweb.com/secured/newuser.php
GET http://testphp.vulnweb.com/product.php?pic=1
GET http://testphp.vulnweb.com/listproducts.php?artist=1
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/RateProduct-1.html
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/
GET http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg
GET http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg&size=160
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-1/
POST http://testphp.vulnweb.com/cart.php
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-2/
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/RateProduct-2.html
GET http://testphp.vulnweb.com/hpp/params.php?p=valid&pp=12
GET http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/

4 burpsuite v202012

爬取较耗费时间,截图的时候是49个,但是随着时间增加数量还在上升,在后面回看的时候数量已经一百多了

image.png

http://testphp.vulnweb.comGET/burp.f5s@306052ce2005175HTMLHome of Acunetix Art1611359458449
http://testphp.vulnweb.comGET/AJAX/burp.f5s@cd689982004453HTMLajax test1611359674072
http://testphp.vulnweb.comGET/AJAX/index.phpburp.f5s@126828be2004453HTMLajax test1611359674872
http://testphp.vulnweb.comGET/Flash/burp.f5s@510aed85200514HTMLIndex of /Flash/1611359682400
http://testphp.vulnweb.comGET/Flash/add.flaburp.f5s@63ce2348200154877HTML1611359714830
http://testphp.vulnweb.comGET/Flash/add.swfburp.f5s@5becece020017674flash1611359684049
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/burp.f5s@81212fb2001191HTML1611359686649
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-1/burp.f5s@ef2a0b9200316HTML1611359784523
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/burp.f5s@1cb4164c200291HTML1611359788669
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/2.phpburp.f5s@200362d6200386script1611360605080
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/3.phpburp.f5s@389e39e7200386script1611360605176
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/BuyProduct-3/burp.f5s@23f2b125200291HTML1611360609454
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/cart/burp.f5s@1fc8c561200291HTML1611360609615
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/categories/burp.f5s@2466019c200291HTML1611360609749
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/categories/Mod_Rewrite_Shopburp.f5s@6d7e45f6200386script1611360666497
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/categories/indexburp.f5s@5bb3bae5200386script1611360665770
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/categories/logoburp.f5s@2099f3f200386script1611360665634
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-2/cgi-bin/burp.f5s@16f71403200291HTML1611360609615
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-3/burp.f5s@9b9a2de200308HTML1611359793221
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-3/RateProduct-1.aspburp.f5s@4f1b459e200386script1611360727449
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-3/params.phpburp.f5s@1a5db25200386script1611360725439
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-3/privacy.aspxburp.f5s@2fdc801e200386script1611360725841
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/BuyProduct-3/product.aspburp.f5s@6b377869200386script1611360727028
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/color-printer/3/burp.f5s@7e95f724200529HTML1611359733180
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/color-printer/3/1/burp.f5s@51c66720200535HTML1611360417812
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/color-printer/3/2/burp.f5s@1ad1d176200495HTML1611360417956
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/burp.f5s@4af51675200535HTML1611359721331
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/Details.phpburp.f5s@1b88f4d8200386script1611360185772
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/Flash.htmlburp.f5s@79957fee200386script1611360185898
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/disclaimer.htmlburp.f5s@6d5b4bcb200386script1611360185841
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/favicon.htmlburp.f5s@f7faeab200386script1611360185721
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/burp.f5s@538da5a8200495HTML1611359725032
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/Mod_Rewrite_Shop/burp.f5s@135ca38200386script1611360306031
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/logo/burp.f5s@3607ccc6200386script1611360304942
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/logo/BuyProduct-1.htmburp.f5s@447f265b200386script1611360785562
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/logo/BuyProduct-2.htmburp.f5s@7ae17b99200386script1611360786103
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/logo/BuyProduct-3.htmburp.f5s@55aa0af7200386script1611360784930
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/logo/artists.phpburp.f5s@5d438d78200386script1611360785810
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/network-attached-storage-dlink/burp.f5s@60333575200386script1611360306304
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/RateProduct-1.htmlburp.f5s@11ffb759200316HTML1611359785570
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/RateProduct-3.htmlburp.f5s@1487ea23200308HTML1611359795219
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/images/burp.f5s@55ee8d86200656HTMLIndex of /Mod_Rewrite_Shop/images/1611359714160
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/index.phpburp.f5s@2c8f82d32001191HTML1611360008044
http://testphp.vulnweb.comGET/admin/burp.f5s@40a6ad64200405HTMLIndex of /admin/1611359695435
http://testphp.vulnweb.comGET/admin/create.sqlburp.f5s@6b5b91a1200771script1611359768567
http://testphp.vulnweb.comGET/categories.phpburp.f5s@4af8b3f12006332HTMLpicture categories1611359533220
http://testphp.vulnweb.comGET/hpp/burp.f5s@1ab12967200419HTMLHTTP Parameter Pollution Example1611359684548
http://testphp.vulnweb.comGET/hpp/params.phpburp.f5s@6f896ad82002141611359777049
http://testphp.vulnweb.comGET/images/burp.f5s@58683811200520HTMLIndex of /images/1611359667907
http://testphp.vulnweb.comGET/secured/burp.f5s@57007fd62002141611359774940
http://testphp.vulnweb.comGET/secured/newuser.phpburp.f5s@44698e40200631HTMLadd new user1611359776066
http://testphp.vulnweb.comGET/AJAXburp.f5s@6012f3bf301371HTML301 Moved Permanently1611359538410
http://testphp.vulnweb.comGET/Flashburp.f5s@7923f71c301372HTML301 Moved Permanently1611359540411
http://testphp.vulnweb.comGET/Mod_Rewrite_Shopburp.f5s@2d09c921301383HTML301 Moved Permanently1611359667359
http://testphp.vulnweb.comGET/Mod_Rewrite_Shop/imagesburp.f5s@251a494e301390HTML301 Moved Permanently1611359707781
http://testphp.vulnweb.comGET/adminburp.f5s@52e2d959301372HTML301 Moved Permanently1611359667311
http://testphp.vulnweb.comGET/hppburp.f5s@341f4f0e301370HTML301 Moved Permanently1611359538318
http://testphp.vulnweb.comGET/imagesburp.f5s@57bcd86d301373HTML301 Moved Permanently1611359667272
http://testphp.vulnweb.comGET/artists.phpburp.f5s@209bbbed000
http://testphp.vulnweb.comGET/cart.phpburp.f5s@647786b6000
http://testphp.vulnweb.comGET/disclaimer.phpburp.f5s@2a5ec209000
http://testphp.vulnweb.comGET/guestbook.phpburp.f5s@1b90189f000
http://testphp.vulnweb.comGET/index.phpburp.f5s@66298cd3000
http://testphp.vulnweb.comGET/login.phpburp.f5s@3e33e496000
http://testphp.vulnweb.comGET/privacy.phpburp.f5s@622137d3000
http://testphp.vulnweb.comGET/userinfo.phpburp.f5s@79ee9fe8000

五 awvs

扫描相对burp很快,不知道是不是自家网站缘故,扫描结果数量405,但是很多都是Mod_Rewrite模块下的

http://testphp.vulnweb.com/
http://testphp.vulnweb.com/.idea/
http://testphp.vulnweb.com/.idea/.name
http://testphp.vulnweb.com/.idea/acuart.iml
http://testphp.vulnweb.com/.idea/encodings.xml
http://testphp.vulnweb.com/.idea/misc.xml
http://testphp.vulnweb.com/.idea/modules.xml
http://testphp.vulnweb.com/.idea/scopes/
http://testphp.vulnweb.com/.idea/scopes/scope_settings.xml
http://testphp.vulnweb.com/.idea/vcs.xml
http://testphp.vulnweb.com/.idea/workspace.xml
http://testphp.vulnweb.com/404.php
http://testphp.vulnweb.com/AJAX/
http://testphp.vulnweb.com/AJAX/artists.php
http://testphp.vulnweb.com/AJAX/categories.php
http://testphp.vulnweb.com/AJAX/htaccess.conf
http://testphp.vulnweb.com/AJAX/index.php
http://testphp.vulnweb.com/AJAX/infoartist.php
http://testphp.vulnweb.com/AJAX/infocateg.php
http://testphp.vulnweb.com/AJAX/infotitle.php
http://testphp.vulnweb.com/AJAX/showxml.php
http://testphp.vulnweb.com/AJAX/styles.css
http://testphp.vulnweb.com/AJAX/titles.php
http://testphp.vulnweb.com/CVS/
http://testphp.vulnweb.com/CVS/Entries
http://testphp.vulnweb.com/CVS/Entries.Log
http://testphp.vulnweb.com/CVS/Repository
http://testphp.vulnweb.com/CVS/Root
http://testphp.vulnweb.com/Connections/
http://testphp.vulnweb.com/Connections/DB_Connection.php
http://testphp.vulnweb.com/Flash/
http://testphp.vulnweb.com/Flash/add.fla
http://testphp.vulnweb.com/Flash/add.swf
http://testphp.vulnweb.com/Mod_Rewrite_Shop/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/details.php3/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-3/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/color-printer/3/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/.htaccess
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/images/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/rate.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/RateProduct-3.html
http://testphp.vulnweb.com/Mod_Rewrite_Shop/buy.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/details.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/images/
http://testphp.vulnweb.com/Mod_Rewrite_Shop/index.php
http://testphp.vulnweb.com/Mod_Rewrite_Shop/rate.php
http://testphp.vulnweb.com/Templates/
http://testphp.vulnweb.com/Templates/main_dynamic_template.dwt.php
http://testphp.vulnweb.com/_mmServerScripts/
http://testphp.vulnweb.com/_mmServerScripts/MMHTTPDB.php
http://testphp.vulnweb.com/_mmServerScripts/mysql.php
http://testphp.vulnweb.com/adm1nPan3l/
http://testphp.vulnweb.com/adm1nPan3l/index.php
http://testphp.vulnweb.com/admin/
http://testphp.vulnweb.com/admin/create.sql
http://testphp.vulnweb.com/artists.php
http://testphp.vulnweb.com/bxss/
http://testphp.vulnweb.com/bxss/adminPan3l/
http://testphp.vulnweb.com/bxss/adminPan3l/index.php
http://testphp.vulnweb.com/bxss/adminPan3l/style.css
http://testphp.vulnweb.com/bxss/cleanDatabase.php
http://testphp.vulnweb.com/bxss/database_connect.php
http://testphp.vulnweb.com/bxss/index.php
http://testphp.vulnweb.com/bxss/test.js
http://testphp.vulnweb.com/bxss/vuln.php
http://testphp.vulnweb.com/cart.php
http://testphp.vulnweb.com/categories.php
http://testphp.vulnweb.com/clearguestbook.php
http://testphp.vulnweb.com/clientaccesspolicy.xml
http://testphp.vulnweb.com/comment.php
http://testphp.vulnweb.com/crossdomain.xml
http://testphp.vulnweb.com/database_connect.php
http://testphp.vulnweb.com/disclaimer.php
http://testphp.vulnweb.com/guestbook.php
http://testphp.vulnweb.com/hpp/
http://testphp.vulnweb.com/hpp/index.php
http://testphp.vulnweb.com/hpp/params.php
http://testphp.vulnweb.com/hpp/test.php
http://testphp.vulnweb.com/images/
http://testphp.vulnweb.com/index.bak
http://testphp.vulnweb.com/index.php
http://testphp.vulnweb.com/listproducts.php
http://testphp.vulnweb.com/login.php
http://testphp.vulnweb.com/logout.php
http://testphp.vulnweb.com/medias/
http://testphp.vulnweb.com/medias/css/
http://testphp.vulnweb.com/medias/css/main.css
http://testphp.vulnweb.com/medias/img/
http://testphp.vulnweb.com/medias/js/
http://testphp.vulnweb.com/medias/js/common_functions.js
http://testphp.vulnweb.com/pictures/
http://testphp.vulnweb.com/pictures/1.jpg.tn
http://testphp.vulnweb.com/pictures/2.jpg.tn
http://testphp.vulnweb.com/pictures/3.jpg.tn
http://testphp.vulnweb.com/pictures/4.jpg.tn
http://testphp.vulnweb.com/pictures/5.jpg.tn
http://testphp.vulnweb.com/pictures/6.jpg.tn
http://testphp.vulnweb.com/pictures/7.jpg.tn
http://testphp.vulnweb.com/pictures/8.jpg.tn
http://testphp.vulnweb.com/pictures/WS_FTP.LOG
http://testphp.vulnweb.com/pictures/credentials.txt
http://testphp.vulnweb.com/pictures/ipaddresses.txt
http://testphp.vulnweb.com/pictures/path-disclosure-unix.html
http://testphp.vulnweb.com/pictures/path-disclosure-win.html
http://testphp.vulnweb.com/pictures/wp-config.bak
http://testphp.vulnweb.com/privacy.php
http://testphp.vulnweb.com/product.php
http://testphp.vulnweb.com/redir.php
http://testphp.vulnweb.com/search.php
http://testphp.vulnweb.com/secured/
http://testphp.vulnweb.com/secured/database_connect.php
http://testphp.vulnweb.com/secured/index.php
http://testphp.vulnweb.com/secured/newuser.php
http://testphp.vulnweb.com/secured/office.htm
http://testphp.vulnweb.com/secured/office_files/
http://testphp.vulnweb.com/secured/office_files/filelist.xml
http://testphp.vulnweb.com/secured/phpinfo.php
http://testphp.vulnweb.com/secured/style.css
http://testphp.vulnweb.com/sendcommand.php
http://testphp.vulnweb.com/showimage.php
http://testphp.vulnweb.com/signup.php
http://testphp.vulnweb.com/style.css
http://testphp.vulnweb.com/userinfo.php
http://testphp.vulnweb.com/wvstests/
http://testphp.vulnweb.com/wvstests/pmwiki_2_1_19/
http://testphp.vulnweb.com/wvstests/pmwiki_2_1_19/scripts/
http://testphp.vulnweb.com/wvstests/pmwiki_2_1_19/scripts/version.php

六 比较

由上所有数据可以看出,awvs和burpsuite爬取的数据量是属于最多的一层的,crawlergo和rad和我手写的爬虫好像是一层。。仅数据量来说。。

再看下数据质量,这里我主要看手写的爬虫(下面称为基准),和rad 、crawlergo

首先,我们先把基准数据和rad的数据比较
先取交集,
image.png

存在17个交集数据,各自减去交集后的数据,排序,对比
image.png
image.png

细心查看的话会发现中间那栏基准数据基本path基本都能在左边交集栏查看到,而右侧rad栏黄色部分基本都是左侧即基准数据里没有的。本人查看了前面的burpsuite和awvs报告,他们多出的部分基本都是http://testphp.vulnweb.com/Mod_Rewrite_Shop/目录下的,这边基准爬虫和rad里都存在了这个目录。

再看下基准与crawlergo比较
存在18个交集
image.png
image.png
依然是右侧有的左边基本都没有,但是左右那几个details目录右侧也没有发现

看下crawlergo与rad的比较
image.png
惊人的相似-_- ,

七 简单的总结

几款工具都扫描出了手写代码里没扫除的目录、路径,其实力都是蛮强的。

其中,crawlergo和rad扫描的数量和质量类似,burpsuite和awvs的扫描结果类似,都是数量最多,但是burpsuite的扫描速度个人感觉有点慢

当然,数量多少的问题是由于burosuite和awvs存在系列安全扫描及其他用途的缘由,对于纯粹只为获取url的话还是得靠crawlergo和rad方便些。

crawlergo和rad的区别存在crawlergo返回的数据中包含了所有headers,包括其自动填充的表单数据,rad返回的只有请求方法+url,如Get http://xxx

最后自己做了一张表,仅代表自己观点,也不一定准确
image.png


知识来源: https://www.freebuf.com/articles/web/261818.html

阅读:91297 | 评论:0 | 标签:无

想收藏或者和大家分享这篇好文章→复制链接地址

“对于crawlergo|rad|burpsuite|awvs爬虫的对比”共有0条留言

发表评论

姓名:

邮箱:

网址:

验证码:

黑帝公告 📢

永久免费持续更新精选优质黑客技术文章Hackdig,帮你成为掌握黑客技术的英雄

↓赞助商 🙇🧎

标签云 ☁