Recursive Fuzzing

Overview

Recursive fuzzing is an automated technique for exploring nested directory structures in web applications. It extends traditional directory fuzzing by dynamically discovering and probing directories as they are found.

How Recursive Fuzzing Works

1. Initial Fuzzing

Starts at the top-level directory (usually /).
Uses a wordlist to send requests and identifies valid directories via successful HTTP responses (e.g., 200 OK).

2. Directory Discovery and Expansion

Upon finding a valid directory (e.g., /admin), the fuzzer appends it to the base URL to create a new fuzzing branch (e.g., http://localhost/admin/).
A new round of fuzzing is initiated within this directory using the same wordlist.

3. Iterative Depth

This process repeats recursively, diving deeper into each newly discovered directory.
Continues until:
- A predefined depth is reached (e.g., 3 levels deep).
- No new directories are found.

Visual analogy:

Imagine the web root as a tree trunk:
- Discovered directories are branches.
- Files are the leaves.
- Recursive fuzzing explores each branch systematically until it reaches the leaves or a defined depth.

Benefits of Recursive Fuzzing

Efficiency: Automates discovery of nested directories.
Thoroughness: Ensures deeper levels of directory structure are not missed.
Reduced Manual Input: No need to manually re-enter each discovered directory.
Scalability: Essential for large and complex web applications.

Recursive Fuzzing with ffuf

Let's use ffuf to demonstrate recursive fuzzing:

m4cc18@htb[/htb]$ ffuf -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt -ic -v -u http://IP:PORT/FUZZ -e .html -recursion

Notice the addition of the -recursion flag. This tells ffuf to fuzz any directories it finds recursively.
- For example, if ffuf discovers an admin directory, it will automatically start a new fuzzing process on http://localhost/admin/FUZZ.
In fuzzing scenarios where wordlists contain comments (lines starting with #), the ffuf -ic option proves invaluable. By enabling this option, ffuf intelligently ignores commented lines during fuzzing, preventing them from being treated as valid inputs.

Output:


        /'___\  /'___\           /'___\       
       /\ \__/ /\ \__/  __  __  /\ \__/       
       \ \ ,__\\ \ ,__\/\ \/\ \ \ \ ,__\      
        \ \ \_/ \ \ \_/\ \ \_\ \ \ \ \_/      
         \ \_\   \ \_\  \ \____/  \ \_\       
          \/_/    \/_/   \/___/    \/_/       

       v2.1.0-dev
________________________________________________

 :: Method           : GET
 :: URL              : http://IP:PORT/FUZZ
 :: Wordlist         : FUZZ: /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt
 :: Extensions       : .html 
 :: Follow redirects : false
 :: Calibration      : false
 :: Timeout          : 10
 :: Threads          : 40
 :: Matcher          : Response status: 200-299,301,302,307,401,403,405,500
________________________________________________

[Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 0ms]
| URL | http://IP:PORT/level1
| --> | /level1/
    * FUZZ: level1

[INFO] Adding a new job to the queue: http://IP:PORT/level1/FUZZ

[INFO] Starting queued job on target: http://IP:PORT/level1/FUZZ

[Status: 200, Size: 96, Words: 6, Lines: 2, Duration: 0ms]
| URL | http://IP:PORT/level1/index.html
    * FUZZ: index.html

[Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 0ms]
| URL | http://IP:PORT/level1/level2
| --> | /level1/level2/
    * FUZZ: level2

[INFO] Adding a new job to the queue: http://IP:PORT/level1/level2/FUZZ

[Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 0ms]
| URL | http://IP:PORT/level1/level3
| --> | /level1/level3/
    * FUZZ: level3

[INFO] Adding a new job to the queue: http://IP:PORT/level1/level3/FUZZ

[INFO] Starting queued job on target: http://IP:PORT/level1/level2/FUZZ

[Status: 200, Size: 96, Words: 6, Lines: 2, Duration: 0ms]
| URL | http://IP:PORT/level1/level2/index.html
    * FUZZ: index.html

[INFO] Starting queued job on target: http://IP:PORT/level1/level3/FUZZ

[Status: 200, Size: 126, Words: 8, Lines: 2, Duration: 0ms]
| URL | http://IP:PORT/level1/level3/index.html
    * FUZZ: index.html

:: Progress: [441088/441088] :: Job [4/4] :: 100000 req/sec :: Duration: [0:00:06] :: Errors: 0 ::

The fuzzing commences at the web root (http://IP:PORT/FUZZ). Initially, ffuf identifies a directory named level1, indicated by a 301 (Moved Permanently) response. This signifies a redirection and prompts the tool to initiate a new fuzzing process within this directory, effectively branching out its search.
As ffuf recursively explores level1, it uncovers two additional directories: level2 and level3. Each is added to the fuzzing queue, expanding the search depth. Furthermore, an index.html file is discovered within level1.
The fuzzer systematically works through its queue, identifying index.html files in both level2 and level3. Notably, the index.html file within level3 stands out due to its larger file size than the others.
Further analysis reveals this file contains the flag HTB{r3curs1v3_fuzz1ng_w1ns}, signifying a successful exploration of the nested directory structure.

Be Responsible

While recursive fuzzing is a powerful technique, it can also be resource-intensive, especially on large web applications. Excessive requests can overwhelm the target server, potentially causing performance issues or triggering security mechanisms.

To mitigate these risks, ffuf provides options for fine-tuning the recursive fuzzing process:

-recursion-depth: This flag allows you to set a maximum depth for recursive exploration. For example, -recursion-depth 2 limits fuzzing to two levels deep (the starting directory and its immediate subdirectories).
-rate: You can control the rate at which ffuf sends requests per second, preventing the server from being overloaded.
-timeout: This option sets the timeout for individual requests, helping to prevent the fuzzer from hanging on unresponsive targets.

Fuzzing files recursively

This is how a fuzz command to recursively fuzz files would look like

m4cc18@htb[/htb]$ ffuf -w /usr/share/seclists/Discovery/Web-Content/directory-list-2.3-medium.txt -ic -u http://IP:PORT/FUZZ -e .html -recursion -recursion-depth 2 -rate 500

Exercise

TARGET: 94.237.48.51:36742

Challenge 1

Recursively fuzz the "recursive_fuzz" path on the target system (ie http://IP:PORT/recursive_fuzz/) to find the flag.

1. Try a recursive file fuzz

Using the above command as an example, lets try using ffuf to recursively fuzz files within recursive_fuzz/

┌──(macc㉿kaliLab)-[~]
└─$ ffuf -w ~/SecLists/Discovery/Web-Content/DirBuster-2007_directory-list-2.3-medium.txt -ic -u http://94.237.51.125:54884/recursive_fuzz/FUZZ -e .html -recursion -recursion-depth 5 -t 500 -timeout 5

Note here we are specifically looking for .html files.
-t increases worker threads
-timeout stops requests waiting forever (very useful when many requests stall).

Output:

...
level1                  [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 76ms]
[INFO] Adding a new job to the queue: http://94.237.48.51:36742/recursive_fuzz/level1/FUZZ
...

[INFO] Starting queued job on target: http://94.237.51.125:54884/recursive_fuzz/level1/FUZZ

                        [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 116ms]
index.html              [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 117ms]
...
level2                  [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 76ms]
[INFO] Adding a new job to the queue: http://94.237.48.51:36742/recursive_fuzz/level1/level2/FUZZ
...

[INFO] Starting queued job on target: http://94.237.51.125:54884/recursive_fuzz/level1/level2/FUZZ

                        [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 112ms]
index.html              [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 113ms]
...
level3                  [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 112ms]
:: Progress: [37026/441092] :: Job [3/3] :: 4370 req/sec :: Duration: [0:00:08] :: Errors: 0 [INFO] Adding a new job to the queue: http://94.237.51.125:54884/recursive_fuzz/level1/level2/level3/FUZZ
...

[INFO] Starting queued job on target: http://94.237.51.125:54884/recursive_fuzz/level1/level2/level3/FUZZ

index.html              [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 113ms]
                        [Status: 200, Size: 111, Words: 8, Lines: 2, Duration: 113ms]
...

threatcon_level2        [Status: 301, Size: 0, Words: 1, Lines: 1, Duration: 120ms]
:: Progress: [35682/441092] :: Job [4/4] :: 4393 req/sec :: Duration: [0:00:08] :: Errors: 0 [INFO] Adding a new job to the queue: http://94.237.51.125:54884/recursive_fuzz/level1/level2/level3/threatcon_level2/FUZZ
...

[INFO] Starting queued job on target: http://94.237.51.125:54884/recursive_fuzz/level1/level2/level3/threatcon_level2/FUZZ

                        [Status: 200, Size: 146, Words: 10, Lines: 2, Duration: 116ms]
index.html              [Status: 200, Size: 146, Words: 10, Lines: 2, Duration: 116ms]

First we hit a recursive_fuzz/level1 directory, it gets added to the queue.
Then we reach a recursive_fuzz/level2 directory, it gets added to the queue.
Then we hit a recursive_fuzz/level2/level3 directory, it gets added to the queue.
Then we hit a recursive_fuzz/level2/level3/threatcon_level2/ directory, it gets added to the queue.
- Finally under this directory we immediately find an index.html file. Since this is the end of the recursive fuzz, lets start by inspecting this last discovered file.

2. Curl the flag

To get the flag, lets try a curl command that sends a GET request to the newly found path to the index.html file:

┌──(macc㉿kaliLab)-[~]
└─$ curl http://94.237.51.125:54884/recursive_fuzz/level1/level2/level3/threatcon_level2/

Output:

<html><head><title>Level 4 Directory</title></head><body><h1>This is Recursive Fuzz Level 4 - HTB{d33p3r_d1rector1es_ar3_c00l}</h1></body></html>

There is our flag!

flag: HTB