Directories and Files enumeration is one of the first steps that an attacker performs during web application pentesting.

This step is necessary to identify potential hidden areas of a website that aren’t supposed to be accessible to public users. And sometimes, if the attacker is lucky, this step can provide that needed information that would make the exploitation of the website possible.

DIRB and Dirbuster are well-known examples of tools that can help in this web enumeration process. However, for this article, we will not be talking about any of these two, but instead, we will be covering Gobuster.

Gobuster is a simple, but powerful, tool to enumerate hidden web content. Despite being used often as a directory enumeration tool, it can also be used to detect subdomains, virtual hostnames, and public s3 buckets.

How to Install Gobuster

If you have Kali, then chances are you already have Gobuster installed.

If not, there are three ways you can install it.

1- if you are on Debian or any Debian-based Linux distribution, you can use the apt-get command to install it:

apt-get install gobuster

2 - If you have a go environment, then you can use the following command:

go install github.com/OJ/gobuster/v3@latest

3 - If none of the other methods work for you, then you can download Gobuster directly from its Github repository. A more detailed explanation about the installation is provided there.

Wordlists

Before we can proceed to use Gobuster, we first need to download a wordlist that Gobuster can use for brute-forcing.

You can pick one from the wordlists that are provided here, but it is better to download them all as they might come in handy in the future. One wordlist may not always prove useful, it is always good to have other ones available when needed.

Although these wordlists are taken from the DIRB repo, they can still be used with Gobuster.

Now that we have everything that we need, let’s go ahead and see how we can discover some hidden web content.

How to Use Gobuster

You can launch Gobuster directly from the command line interface. To do so, you have to run the command using the following syntax.

gobuster [Mode] [Options]

Modes

After typing the “gobuster” command, you will have to specify the mode, or what you want to use the command for. As I mentioned earlier, Gobuster can have many uses :

• dir: Enumerating URIs (directories/files).
• dns: Enumerating Subdomains.
• vhost: Enumerating Virtual Hosts.
• s3: Enumerating S3 Buckets.

Most often, you will use Gobuster to enumerate directories and files. In this case, you will be mostly using the dir mode.

gobuster dir [options]

Options

After typing the mode, you will have to specify the options. Gobuster has many options to use, you can read the help using “gobuster -h” to determine which ones might be useful to you depending on your target (You can also use “gobuster dir -h” to view options that are specific to the dir mode).

To make things simple and to the point, we’ll only address here three important options that you will often use, two of which are mandatory.

Target Specification

So the first option that we need to specify is ‘-u‘ followed by a target. This can either be a URI, an IP address, or a hostname.

This option is mandatory, which is no surprise here since you need at least to have a target to test.

Here are some examples of how to use this option.

gobuster dir -u http://www.targetwebsite.com/

gobuster dir -u http://localhost/

gobuster dir -u 192.168.1.140

Note that these examples won’t work since we still have one other mandatory option that we haven’t yet added.

Wordlist Specification

As mentioned earlier, Gobuster enumerates directories and files by performing dictionary attacks.

A dictionary attack consists of testing a list of words, (or a combination of words) in the hope that the correct word is contained within this list.

So, in order for Gobuster to perform a dictionary attack, we need to provide it with a wordlist. To do that, just type in the ‘-w‘ option, followed by the path to the wordlist file. We can use a file from the wordlists that we’ve downloaded earlier.

gobuster dir -u http://www.targetwebsite.com/ -w /usr/share/wordlists/big.txt

At last, we have a functioning command that we can run without errors (Provided of course that you change the URL with your target website, and the wordlist path with where you have saved the wordlist file in your local computer).

Enumerating Files

Before we wrap up, let’s add one final option that is not mandatory, but still very useful in many scenarios.

Using only the two flags we’ve seen so far will only allow us to enumerate directories. But what if we want to also find hidden files?

Fortunately, Gobuster allows you to do that by using the ‘-x‘ flag, followed by the file extensions you’d like to search for.

Here is an example that will make things clear.

gobuster dir -u http://www.targetwebsite.com/ -w /usr/share/wordlists/big.txt -x php,html,htm

In this command, we are searching for files that have php, html or htm extensions.

---

These are all the basics that you need to know to start using Gobuster for your pentesting projects, CTFs, or within your personal lab (Of course, I trust that you won’t be using it for any other suspicious activity).

Sometimes, the options we’ve seen here won’t be enough to provide the intended result. In that case, I invite you to view the Gobuster help “gobuster -h” to see what other options you can add.