What solutions to prevent git leaks ?

Hello,

I will do a quick and dirty post about what’s out there to find / prevent leaks of secrets in your git repositories.
I did not try all of these tools. For the search part, i’m mainly using a fork of Trufflehog with some added features (search in filenames, commits comments, also with custom regexes).

Objectives :

• Look into the commits history for sensitive information publicly accessible by an attacker ;
• Prevent secrets leaks ;
• Monitoring and integrating these checks in the Continous Delivery process - aka DevSecOps

Tools to look for sensitive data :

• Written in Python
• Also works for local repo
• Strings with high entropy
• Custom regex rules
• No search in commits comments or filenames

• Written in Go
• Renders results in a Web App
• Do not work for local repo AFAIK

• Written in NodeJS
• Generates HTML reports

• Written in Go
• Using / merging results of Trufflehog and Repo-Supervisor

• Written in Go
• regex and entropy

• Written in Go
• Renders results in a Web App
• Monitors GIT repositories - Be the first to catch the secret before it gets deleted from git history
• shhgit will watch real-time stream and pull out any accidentally committed secrets

Prevent commits :

What is a Git Hook ? As described here:

Like many other Version Control Systems, Git has a way to fire off custom scripts when certain important actions occur. There are two groups of these hooks: client-side and server-side. Client-side hooks are triggered by operations such as committing and merging, while server-side hooks run on network operations such as receiving pushed commits. You can use these hooks for all sorts of reasons.

Security is a good one.

What kind of hooks can used to prevent leak at an early stage of the git workflow? :

pre-commit: Used to check if any of the files changed in the commit use prohibited patterns.

commit-msg: Used to determine if a commit message contains a prohibited patterns.

prepare-commit-msg: Used to determine if a merge commit will introduce a history that contains a prohibited pattern at any point. Please note that this hook is only invoked for non fast-forward merges.

Client-side hooks :

• You have to set GitHound command into a pre-commit hook
• Regexes are configured in .githound.yml

• From AWS team
• You also can manually scan for secrets before making your repo public: git secrets --scan-history

GITHUB actions :

GitHub Actions enables you to create custom software development life cycle (SDLC) workflows directly in your GitHub repository.

• Gitleaks Github Action
• Trufflehog Github Action

GITHUB scanning project :

Github has a project which aims at monitoring leaked third parties tokens from your repositories: https://help.github.com/en/github/administering-a-repository/about-secret-scanning.

Once identified, Github will warn you and will request the provider (from the following list) in an automated way to ask for the immediate revokation of your leaked tokens:

Adafruit
Alibaba Cloud
Amazon Web Services (AWS)
Atlassian
Azure
Clojars
CloudBees CodeShip
Databricks
Datadog
Discord
Doppler
Dropbox
Dynatrace
Finicity
Frame.io
GitHub
GoCardless
Google Cloud
Hashicorp Terraform
Hubspot
Mailchimp
Mailgun
MessageBird
npm
NuGet
Palantir
Plivo
Postman
Proctorio
Pulumi
Samsara
Shopify
Slack
SSLMate
Stripe
Tencent Cloud
Twilio

Finally what to conclude :

For each leaked secrets linked to an environment which could be targeted by an adversary :

• Revoke / update the secret (password, any token / private key, …) as quick as possible ;
• Update the git commits history

More globally :

• Warn the stakeholders involved in this data leak (users, providers, …) ;
• Check log files to detect former fraudulent access ;
• Prevent the versioning of sensitive data (via .gitignore and set hooks to monitor your commits) ;
• Include the secret scanning process as part as your continuous delivery process (build factory). For example for Github CI see the Github actions Trufflehog and Gitleaks as aforementioned.

See you soon, next posts will be more internal pentesting / windows oriented,
Cheers.