What solutions to prevent git leaks ?

    Hello,

    I will do a quick and dirty post about what’s out there to find / prevent leaks of secrets in your git repositories.
    I did not try all of these tools. For the search part, i’m mainly using a fork of Trufflehog with some added features (search in filenames, commits comments, also with custom regexes).

    Objectives :

    • Look into the commits history for sensitive information publicly accessible by an attacker ;
    • Prevent secrets leaks ;
    • Monitoring and integrating these checks in the Continous Delivery process - aka DevSecOps

    Tools to look for sensitive data :

    TRUFFLEHOG
    • Written in Python
    • Also works for local repo
    • Strings with high entropy
    • Custom regex rules
    • No search in commits comments or filenames
    GITROB
    • Written in Go
    • Renders results in a Web App
    • Do not work for local repo AFAIK
    REPO-SUPERVISOR
    • Written in NodeJS
    • Generates HTML reports
    GIT-ALL-SECRETS
    GITLEAKS
    • Written in Go
    • regex and entropy
    SSHGIT
    • Written in Go
    • Renders results in a Web App
    • Monitors GIT repositories - Be the first to catch the secret before it gets deleted from git history
    • shhgit will watch real-time stream and pull out any accidentally committed secrets

    Prevent commits :

    What is a Git Hook ? As described here:

    Like many other Version Control Systems, Git has a way to fire off custom scripts when certain important actions occur. There are two groups of these hooks: client-side and server-side. Client-side hooks are triggered by operations such as committing and merging, while server-side hooks run on network operations such as receiving pushed commits. You can use these hooks for all sorts of reasons.

    Security is a good one.

    What kind of hooks can used to prevent leak at an early stage of the git workflow? :

    pre-commit: Used to check if any of the files changed in the commit use prohibited patterns.
    
    commit-msg: Used to determine if a commit message contains a prohibited patterns.
    
    prepare-commit-msg: Used to determine if a merge commit will introduce a history that contains a prohibited pattern at any point. Please note that this hook is only invoked for non fast-forward merges.
    

    Client-side hooks :

    GITHOUND
    • You have to set GitHound command into a pre-commit hook
    • Regexes are configured in .githound.yml
    GIT-SECRETS
    • From AWS team
    • You also can manually scan for secrets before making your repo public: git secrets --scan-history

    GITHUB actions :

    GitHub Actions enables you to create custom software development life cycle (SDLC) workflows directly in your GitHub repository.

    GITHUB scanning project :

    Github has a project which aims at monitoring leaked third parties tokens from your repositories: https://help.github.com/en/github/administering-a-repository/about-secret-scanning.

    Once identified, Github will warn you and will request the provider (from the following list) in an automated way to ask for the immediate revokation of your leaked tokens:

    Adafruit
    Alibaba Cloud
    Amazon Web Services (AWS)
    Atlassian
    Azure
    Clojars
    CloudBees CodeShip
    Databricks
    Datadog
    Discord
    Doppler
    Dropbox
    Dynatrace
    Finicity
    Frame.io
    GitHub
    GoCardless
    Google Cloud
    Hashicorp Terraform
    Hubspot
    Mailchimp
    Mailgun
    MessageBird
    npm
    NuGet
    Palantir
    Plivo
    Postman
    Proctorio
    Pulumi
    Samsara
    Shopify
    Slack
    SSLMate
    Stripe
    Tencent Cloud
    Twilio
    

    Finally what to conclude :

    For each leaked secrets linked to an environment which could be targeted by an adversary :

    More globally :

    • Warn the stakeholders involved in this data leak (users, providers, …) ;
    • Check log files to detect former fraudulent access ;
    • Prevent the versioning of sensitive data (via .gitignore and set hooks to monitor your commits) ;
    • Include the secret scanning process as part as your continuous delivery process (build factory). For example for Github CI see the Github actions Trufflehog and Gitleaks as aforementioned.

    See you soon, next posts will be more internal pentesting / windows oriented,
    Cheers.