Features

Modes of Operation

While tartufo started its life with one primary mode of operation, scanning the history of a git repository, it has grown other time to have a number of additional uses and modes of operation. These are all invoked via different sub-commands of tartufo.

Git Repository History Scan

This is the “classic” use case for tartufo: Scanning the history of a git repository. There are two ways to invoke this functionality, depending if you are scanning a repository which you already have cloned locally, or one on a remote system.

Scanning a Local Repository

$ tartufo scan-local-repo /path/to/my/repo

To use docker, mount the local clone to the /git folder in the docker image:

$ docker run --rm -v "/path/to/my/repo:/git" godaddy/tartufo scan-local-repo /git

Note

If you are using podman in place of docker, you will need to add the --privileged flag to the run command, in order to avoid a permission denied error.

Scanning a Remote Repository

$ tartufo scan-remote-repo https://github.com/godaddy/tartufo.git

To use docker:

$ docker run --rm godaddy/tartufo scan-remote-repo https://github.com/godaddy/tartufo.git

When used this way, tartufo will clone the repository to a temporary directory, scan the local clone, and then delete it.

Accessing Repositories via SSH from Docker

When accessing repositories via SSH, the docker runtime needs to have access to your SSH keys for authorization. To allow this, make sure ssh-agent is running on your host machine and has the key added. You can verify this by running ssh-add -L on your host machine. You then need to point Docker at that running SSH agent.

Using Docker for Linux, that will look something like this:

$ docker run --rm -v "/path/to/my/repo:/git" \
  -v $SSH_AUTH_SOCK:/agent -e SSH_AUTH_SOCK=/agent \
  godaddy/tartufo scan-local-repo /git

When using Docker Desktop for Mac, use /run/host-services/ssh-auth.sock as both source and target, then point the environment variable SSH_AUTH_SOCK to this same location:

$ docker run --rm -v "/path/to/my/repo:/git" \
  -v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock \
  -e SSH_AUTH_SOCK="/run/host-services/ssh-auth.sock" godaddy/tartufo

Pre-commit Hook

This mode of operation instructs tartufo to scan staged, uncommitted changes in a local repository. This is the flip-side of the primary mode of operation. Instead of checking for secrets you have already checked in, this helps prevent you from committing new secrets!

When running this sub-command, the caller’s current working directory is assumed to be somewhere within the local clone’s tree and the repository root is determined automatically.

Note

It is always possible, although not recommended, to bypass the pre-commit hook by using git commit --no-verify.

Manual Setup

To set up a pre-commit hook for tartufo by hand, you can place the following in a .git/hooks/pre-commit file inside your local repository clone:

Executing tartufo Directly
#!/bin/sh

# Redirect output to stderr.
exec 1>&2

# Check for suspicious content.
tartufo --regex --entropy pre-commit
Or, Using Docker
#!/bin/sh

# Redirect output to stderr.
exec 1>&2

# Check for suspicious content.
docker run -t --rm -v "$PWD:/git" godaddy/tartufo pre-commit

Git will execute tartufo before actually committing any of your changes. If any problems are detected, they are reported by tartufo, and git aborts the commit process. Only when tartufo returns a success status (indicating no potential secrets were discovered) will git commit the staged changes.

Using the “pre-commit” tool

If you want a slightly more automated approach which can be more easily shared to ensure a unified setup across all developer’s systems, you can use the wonderful pre-commit tool.

Add a .pre-commit-config.yaml file to your repository. You can use the following example to get you started:

- repo: https://github.com/godaddy/tartufo
  rev: main
  hooks:
  - id: tartufo

Warning

You probably don’t actually want to use the main rev. This is the active development branch for this project, and can not be guaranteed stable. Your best bet would be to choose the latest version, currently 2.3.1.

That’s it! Now your contributors only need to install pre-commit, and then run pre-commit install --install-hooks, and tartufo will automatically be run as a pre-commit hook.

Scan Types

tartufo offers multiple types of scans, each of which can be optionally enabled or disabled, while looking through its target for secrets.

Regex Checking

tartufo can scan for a pre-built list of known signatures for things such as SSH keys, EC2 credentials, etc. These scans are activated by use of the --regex flag on the command line. They will be reported with an issue type of Regular Expression Match, and the issue detail will be the name of the regular expression which was matched.

Customizing

Additional rules can be specified in a JSON file, pointed to on the command line with the --rules argument. The file should be in the following format:

{
    "RSA private key": "-----BEGIN EC PRIVATE KEY-----"
}

Things like subdomain enumeration, s3 bucket detection, and other useful regexes highly custom to the situation can be added.

If you would like to deactivate the default regex rules, using only your custom rule set, you can use the --no-default-regexes flag.

Feel free to also contribute high signal regexes upstream that you think will benefit the community. Things like Azure keys, Twilio keys, Google Compute keys, are welcome, provided a high signal regex can be constructed.

tartufo’s base rule set currently sources from the truffleHogRegexes package.

High Entropy Checking

tartufo calculates the Shannon entropy of each commit, finding strings which appear to be generated from a stochastic source. In short, it looks for pieces of data which look random, as these are likely to be things such as cryptographic keys. These scans are activated by usage of the --entropy command line flag.

Scan Limiting (Exclusions)

By its very nature, especially when it comes to high entropy scans, tartufo can encounter a number of false positives. Whether those are things like links to git commit hashes, tokens/passwords used for tests, or any other variety of thing, there needs to be a way to tell tartufo to ignore those things, and not report them out as issues. For this reason, we provide multiple methods for excluding these items.

Limiting by Signature

New in version 2.0.0.

Every time an issue is found during a scan, tartufo will generate a “signature” for that issue. This is a stable hash generated from the filename and the actual string that was identified as being an issue.

For example, you might see the following header in the output for an issue:

_images/issue-signature.png

Looking at this information, it’s clear that this issue was found in a test file, and it’s probably okay. Of course, you will want to look at the actual body of what was found and determine that for yourself. But let’s say that this really is okay, and we want tell tartufo to ignore this issue in future scans. To do this, you can either specify it on the command line…

> tartufo -e 2a3cb329b81351e357b09f1b97323ff726e72bd5ff8427c9295e6ef68226e1d1
# No output! Success!
>

Or you can add it to your config file, so that this exclusion is always remembered!

[tool.tartufo]
exclude-signatures = [
  "2a3cb329b81351e357b09f1b97323ff726e72bd5ff8427c9295e6ef68226e1d1",
]

Done! This particular issue will no longer show up in your scan results.

Limiting Scans by Path

With the --include-paths and --exclude-paths options, it is also possible to limit scanning to a subset of objects in the Git history by defining regular expressions (one per line) in a file to match the targeted object paths. To illustrate, see the example include and exclude files below:

include-patterns.txt
src/
# lines beginning with "#" are treated as comments and are ignored
gradle/
# regexes must match the entire path, but can use python's regex syntax for
# case-insensitive matching and other advanced options
(?i).*\.(properties|conf|ini|txt|y(a)?ml)$
(.*/)?id_[rd]sa$
exclude-patterns.txt
(.*/)?\.classpath$
.*\.jmx$
(.*/)?test/(.*/)?resources/

These filter files could then be applied by:

> tartufo --include-paths include-patterns.txt \
  --exclude-paths exclude-patterns.txt \
  scan-local-repo file://path/to/my/repo.git

With these filters, issues found in files in the root-level src directory would be reported, unless they had the .classpath or .jmx extension, or if they were found in the src/test/dev/resources/ directory, for example.

Additional usage information is provided when calling tartufo with the -h or --help options.

These features help cut down on noise, and makes the tool easier to shove into a devops pipeline.

Would you like to know more?