Hello,

I will do a quick and dirty post about what’s out there to find / prevent leaks of secrets in your git repositories.
I did not try all of these tools. For the search part, i’m mainly using a fork of Trufflehog with some added features (search in filenames, commits comments, also with custom regexes).

Objectives :

  • Look into the commits history for sensitive information publicly accessible by an attacker ;
  • Prevent secrets leaks ;
  • Monitoring and integrating these checks in the Continous Delivery process - aka DevSecOps

Find sensitive data :

TOOLS :
=> TRUFFLEHOG

  • Python
  • also works for local repo
  • strings with high entropy
  • custom regex rules
  • no search in commits comments or filenames

=> GITROB

  • Go
  • Web App
  • do not work for local repo AFAIK

=> REPO-SUPERVISOR

  • NodeJS
  • generates HTML reports

=> GIT-ALL-SECRETS

=> SECRETS ANALYZER

=> GITLEAKS

  • Go
  • regex and entropy

=> SSHGIT

  • Go
  • Web App
  • monitoring git repos - Be the first to catch the secret before it gets deleted from git history
  • shhgit will watch real-time stream and pull out any accidentally committed secrets

Prevent commits :

What is a Git Hook ? As described here:

Like many other Version Control Systems, Git has a way to fire off custom scripts when certain important actions occur. There are two groups of these hooks: client-side and server-side. Client-side hooks are triggered by operations such as committing and merging, while server-side hooks run on network operations such as receiving pushed commits. You can use these hooks for all sorts of reasons.

Security is a good one.

What kind of hooks can used to prevent leak at an early stage of the git workflow? :

pre-commit: Used to check if any of the files changed in the commit use prohibited patterns.

commit-msg: Used to determine if a commit message contains a prohibited patterns.

prepare-commit-msg: Used to determine if a merge commit will introduce a history that contains a prohibited pattern at any point. Please note that this hook is only invoked for non fast-forward merges.

CLIENT-SIDE HOOKS :

=> GITHOUND

  • set githound command into a pre-commit hook
  • regexes configured in .githound.yml

=> GIT-SECRETS

  • from AWS
  • you also can manually scan for secrets before making your repo public: git secrets --scan-history

GITHUB ACTIONS :

GitHub Actions enables you to create custom software development life cycle (SDLC) workflows directly in your GitHub repository.

THE GITHUB.COM SCANNING PROJECT :

Github has a project which aims at monitoring leaked third parties tokens from your repositories: https://help.github.com/en/github/administering-a-repository/about-secret-scanning.

Once identified, Github will warn you and will request the provider (from the following list) in an automated way to ask for the immediate revokation of your leaked tokens:

Adafruit
Alibaba Cloud
Amazon Web Services (AWS)
Atlassian
Azure
CloudBees CodeShip
Databricks
Datadog
Discord
Dropbox
Dynatrace
GitHub
GoCardless
Google Cloud
Hashicorp Terraform
Hubspot
Mailgun
npm
NuGet
Palantir
Postman
Proctorio
Pulumi
Samsara
Slack
Stripe
Tencent Cloud
Twilio

Finally what to conclude :

For each leaked secrets linked to an environment which could be targeted by an adversary :

More globally :

  • Warn the stakeholders involved in this data leak (users, providers, …) ;
  • Check log files to detect former fraudulent access ;
  • Prevent the versioning of sensitive data (via .gitignore and set hooks to monitor your commits) ;
  • Include the secret scanning process as part as your continuous delivery process (build factory). For example for Github CI see the Github actions Trufflehog and Gitleaks as aforementioned.

See you soon, next posts will be more internal pentesting / windows oriented,
Cheers.