Hands-On Bug Hunting for Penetration Testers: A practical guide to help ethical hackers discover web application security flaws

Hands-On Bug Hunting for Penetration Testers

A practical guide to help ethical hackers discover web application security flaws

Joseph Marshall

BIRMINGHAM - MUMBAI

Hands-On Bug Hunting for Penetration Testers

Copyright © 2018 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Gebin George

Acquisition Editor: Shweta Pant

Content Development Editor: Sharon Raj

Technical Editor: Prashant Chaudhari

Copy Editor: Safis Editing

Project Coordinator: Drashti Panchal

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Graphics: Tom Scaria

Production Coordinator: Arvindkumar Gupta

First published: September 2018

Production reference: 1070918

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-78934-420-2

www.packtpub.com

I'd like to dedicate this book to my beautiful wife, for helping me see this project through.

I love you, Lizzie.

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

Packt.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at customercare@packtpub.com for more details.

At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. 

Contributors

About the author

Joseph Marshall is a web application developer and freelance writer with credits from The Atlantic, Kirkus Review, and the SXSW film blog. He also enjoys moonlighting as a freelance security researcher, working with third-party vulnerability marketplaces such as Bugcrowd and HackerOne. His background and education include expertise in development, nonfiction writing, linguistics, and instruction/teaching. He lives in Austin, TX.

About the reviewers

Sachin Wagh is a young information security researcher from India. His core area of expertise includes penetration testing, vulnerability analysis, and exploit development. He has found security vulnerabilities in Google, Tesla Motors, LastPass, Microsoft, F-Secure, and other companies. Due to the severity of many bugs discovered, he has received numerous awards for his findings. He has participated in several security conferences as a speaker, such as Hack In Paris, Infosecurity Europe, and HAKON.

I would specially like to thank Shweta Pant and Drashti Panchal for offering me this opportunity. I would also like to thank my family and close friends for supporting me.

Himanshu Sharma has already achieved fame for finding security loopholes and vulnerabilities in Apple, Google, Microsoft, Facebook, Adobe, Uber, and many more, with hall of fame listings as proof. He has helped celebrities such as Harbhajan Singh, and also assisted an international singer in tracking down his hacked account and recovering it. He was a speaker at the international conferences Botconf 2013 and CONFidence 2018. He has also spoken at IEEE conferences in California and Malaysia, as well as for TEDx. Currently, he is the cofounder of BugsBounty, a crowd-sourced security platform for ethical hackers and companies interested in cyber services. He has also authored a book titled Kali Linux - An Ethical Hacker's Cookbook.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Title Page

Copyright and Credits

Hands-On Bug Hunting for Penetration Testers

Dedication

Packt Upsell

Why subscribe?

Packt.com

Contributors

About the author

About the reviewers

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Conventions used

Get in touch

Reviews

Joining the Hunt

Technical Requirements

The Benefits of Bug Bounty Programs

What You Should Already Know – Pentesting Background

Setting Up Your Environment – Tools To Know

What You Will Learn – Next Steps

How (Not) To Use This Book – A Warning

Summary

Questions

Further Reading

Choosing Your Hunting Ground

Technical Requirements

An Overview of Bug Bounty Communities – Where to Start Your Search

Third-Party Marketplaces

Bugcrowd

HackerOne

Vulnerability Lab

BountyFactory

Synack

Company-Sponsored Initiatives

Google

Facebook

Amazon

GitHub

Microsoft

Finding Other Programs

Money Versus Swag Rewards

The Internet Bug Bounty Program

ZeroDisclo and Coordinated Vulnerability Disclosures

The Vulnerability of Web Applications – What You Should Target

Evaluating Rules of Engagement – How to Protect Yourself

Summary

Questions

Further Reading

Preparing for an Engagement

Technical Requirements

Tools

Using Burp

Attack Surface Reconnaisance – Strategies and the Value of Standardization

Sitemaps

Scanning and Target Reconaissance

Brute-forcing Web Content

Spidering and Other Data-Collection Techniques

Burp Spider

Striker

Scrapy and Custom Pipelines

Manual Walkthroughs

Source Code

Building a Process

Formatting the JS Report

Downloading the JavaScript

Putting It All Together

The Value Behind the Structure

Summary

Questions

Further Reading

Unsanitized Data – An XSS Case Study

Technical Requirements

A Quick Overview of XSS – The Many Varieties of XSS

Testing for XSS – Where to Find It, How to Verify It

Burp Suite and XSS Validator

Payload Sets

Payload Options

Payload Processing

XSS – An End-To-End Example

XSS in Google Gruyere

Gathering Report Information

Category

Timestamps

URL

Payload

Methodology

Instructions to Reproduce

Attack Scenario

Summary

Questions

Further Reading

SQL, Code Injection, and Scanners

Technical Requirements

SQLi and Other Code Injection Attacks – Accepting Unvalidated Data

A Simple SQLi Example

Testing for SQLi With Sqlmap – Where to Find It and How to Verify It

Trawling for Bugs – Using Google Dorks and Python for SQLi Discovery

Google Dorks for SQLi

Validating a Dork

Scanning for SQLi With Arachni

Going Beyond Defaults

Writing a Wrapper Script

NoSQL Injection – Injecting Malformed MongoDB Queries

SQLi – An End-to-End Example

Gathering Report Information

Category

Timestamps

URL

Payload

Methodology

Instructions to Reproduce

Attack Scenario

Final Report

Summary

Questions

Further Reading

CSRF and Insecure Session Authentication

Technical Requirements

Building and Using CSRF PoCs

Creating a CSRF PoC Code Snippet

Validating Your CSRF PoC

Creating Your CSRF PoC Programmatically

CSRF – An End-to-End Example

Gathering Report Information

Category

Timestamps

URL

Payload

Methodology

Instructions to Reproduce

Attack Scenario

Final Report

Summary

Questions

Further Reading

Detecting XML External Entities

Technical requirements

A simple XXE example

XML injection vectors

XML injection and XXE – stronger together

Testing for XXE – where to find it, and how to verify it

XXE – an end-to-end example

Gathering report information

Category

Timestamps

URL

Payload

Methodology

Instructions to reproduce

Attack scenario

Final report

Summary

Questions

Further reading

Access Control and Security Through Obscurity

Technical Requirements

Security by Obscurity – The Siren Song

Data Leaks – What Information Matters?

API Keys

Access Tokens

Passwords

Hostnames

Machine RSA/Encryption Keys

Account and Application Data

Low Value Data – What Doesn’t Matter

Generally Descriptive Error Messages

404 and Other Non-200 Error Codes

Username Enumeration

Browser Autocomplete or Save Password Functionality

Data Leak Vectors

Config Files

Public Code Repos

Client Source Code

Hidden Fields

Error Messages

Unmasking Hidden Content – How to Pull the Curtains Back

Preliminary Code Analysis

Using Burp to Uncover Hidden Fields

Data Leakage – An End-to-End Example

Gathering Report Information

Final Report

Summary

Questions

Further Reading

Framework and Application-Specific Vulnerabilities

Technical Requirements

Known Component Vulnerabilities and CVEs – A Quick Refresher

WordPress – Using WPScan

WPScan as a Dockerized CLI

Burp and WPScan

Ruby on Rails – Rubysec Tools and Tricks

Exploiting RESTful MVC Routing Patterns

Checking the Version for Particular Weaknesses

Testing Cookie Data and Authentication

Django – Strategies for the Python App

Checking for DEBUG = True

Probing the Admin Page

Summary

Questions

Further Reading

Formatting Your Report

Technical Requirements

Reproducing the Bug – How Your Submission Is Vetted

Critical Information – What Your Report Needs

Maximizing Your Award – The Features That Pay

Example Submission Reports – Where to Look

Hackerone Hacktivity

Vulnerability Lab Archive

GitHub

Summary

Questions

Further Reading

Other Tools

Technical Requirements

Evaluating New Tools – What to Look For

Paid Versus Free Editions – What Makes a Tool Worth It?

A Quick Overview of Other Options – Nikto, Kali, Burp Extensions, and More

Scanners

Nikto

Zed Attack Proxy 

w3af

nmap and python-nmap

Aircrack-ng

Wireshark

SpiderFoot

Resources

FuzzDB

Pentesting Cheatsheet

Exploit DB

Awesome Web Security

Kali Linux

Source Code Analysis (White Box) Tools

Pytaint

Bandit

Brakeman

Burp

Burp Extensions

JSON Beautifier

Retire.js

Python Scripter

Burp Notes

Burp REST API

SaaS-Specific Extensions

Using Burp Pro to Generate a CSRF PoC

Metasploit and Exploitation Frameworks

Summary

Questions

Further Reading

Other (Out of Scope) Vulnerabilities

Technical Requirements

DoS/DDoS – The Denial-of-Service Problem

Sandboxed and Self-XSS – Low-Threat XSS Varieties

Non-Critical Data Leaks – What Companies Don’t Care About

Emails

HTTP Request Banners

Known Public Files

Missing HttpOnly Cookie Flags

Other Common No-Payout Vulnerabilities

Weak or Easily Nypassed Captchas

The HTTP OPTIONS Method Enabled

BEAST (CVE-2011-3389) and Other SSL-Based Attacks

Brute Forcing Authentication Systems

CSRF Logout

Anonymous Form CSRF

Clickjacking and Clickjacking-Enabled Attacks

Physical Testing Findings

Outdated Browsers

Server Information

Rate-Limiting

Summary

Questions

Further Reading

Going Further

Blogs

The SANS Institute

Bugcrowd

Darknet

HighOn.Coffee

Zero Day Blog

SANS AppSec Blog

Courses

Penetration Testing With Kali Linux

The Infosec Institute Coursework

Udemy Penetration Testing Classes

Terminology

Attack Scenario

Attack Surface

Black Box Testing

Bugs

Bug Bounty Programs

CORS

Data Exfiltration

Data Sanitation

Data Leakage

Exploit

Fingerprinting

Fuzzing

Google Dorks

Known Component Vulnerabilities

OSINT

Passive Versus Active Scanning

Payload

Proof-of-Concept (PoC)

Rules of Engagement (RoE)

Red Team

Remote Code Execution (RCE)

Safe Harbor

Scope

Security Posture

Single-Origin Policy

Submission Report

Vulnerability

White Box Testing

Workflow

Zero-Day

Summary

Questions

Further Reading

Assessment

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 12

Chapter 13

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

This book is designed to give interested coders (part-time, professional, and otherwise) the skills they need to start participating in public bug bounty programs, covering both general pentesting subjects, such as scoping your testing sessions appropriately, and bounty-specific security topics, such as how to format your bug submission report to ensure the best chance of earning a reward.

As the need for security audits on the public web grows, crowdsourced solutions are becoming more popular. This book aims to give you everything you need to participate in those programs—walking you through important topics with a mix of theory and direct, hands-on examples.

Who this book is for

This book is written for developers, hobbyists, pentesters, and anyone with an interest (and maybe a little experience) in web application security and public bug bounty programs.

What this book covers

Chapter 1, Joining the Hunt, introduces the concept of bug bounties, their value to companies, and the most common types of programs. It also sets up expectations for what the reader should know going into the book.

Chapter 2, Choosing Your Hunting Ground, explains how to evaluate individual bug bounty programs and whether to participate in them. It explains factors such as payouts, community engagement, terms of engagements, and participating in company quality.

Chapter 3, Preparing for an Engagement, explains how to prepare for a pentesting engagement, from how to standardize the reconnaissance process, to understanding the application’s attack surface, to the importance of good note taking and, later, preparing submission reports.

Chapter 4, Unsanitized Data – An XSS Case Study, describes how and where to find XSS vulnerabilities - a variety of code injection that represents one of the most common web application vulnerabilities today.

Chapter 5, SQL, Code Injection and Scanners, describes the different varieties of code injection attacks and how to safely test for them, covering different types of injection, such as blind or error-based injection.

Chapter 6, CSRF and Insecure Session Authentication, discusses vulnerabilities related to insecure session authentication, focusing on CSRF and how to create a CSRF PoC to test for them.

Chapter 7, Detecting XML External Entities (XEE), focuses on XML External Entity vulnerability detection and related XML injection techniques that can work in conjunction with XXE.

Chapter 8, Access Control and Security Through Obscurity, goes over how to find hidden information/data leaks in web applications and discerning between what data is important (and will win you an award) and what’s not. It covers different types of sensitive data and gives you examples from the field.

Chapter 9, Framework and Application-Specific Vulnerabilities, covers approaching a pentesting engagement from the perspective of testing for application/framework-specific vulnerabilities, focusing on general Known Common Vulnerabilities and Exposures (CVEs), as well as methods for testing WordPress, Rails, and Django apps, including strategies, tools, tips, and tricks.

Chapter 10, Formatting Your Report, goes over how to compose a bug report to receive the maximum payout, drawing on examples and information from earlier vulnerability-specific chapters and providing examples (with commentary) on the finer considerations of your submission.

Chapter 11, Other Tools, goes over other tools not covered in the course of the vulnerability examples and how to vet new ones. It also explains how to evaluate free versus paid products and jumping off points for pentesting regimens that focus on bugs not detailed extensively in the work (for example, weak WAF rules/network gaps).

Chapter 12, Other (Out-of-Scope) Vulnerabilities, goes over other vulnerabilities not covered in the course of the book and why they don’t command payouts in most bug bounty programs.

Chapter 13, Going Further, explains where the reader can turn to for more information about participating in bug bounty programs - running through courses and resources for continuing to develop your security acumen. It also features a dictionary of pentesting/security terms to clearly