Security & Governance339 projects

This repository is maintained by Omar Santos (@santosomar) and includes thousands of resources related to ethical hacking, bug bounties, digital forensics and incident response (DFIR), AI security, vulnerability research, exploit development, reverse engineering, and more. 🔥 Also check: https://hackertraining.org

Fully automatic censorship removal for language models

A game theoretic approach to explain the output of any machine learning model.

FHEVM, a full-stack framework for integrating Fully Homomorphic Encryption (FHE) with blockchain applications

🔒 A compiled checklist of 300+ tips for protecting digital security and privacy in 2026

SafeLine is a self-hosted WAF(Web Application Firewall) / reverse proxy to protect your web apps from attacks and exploits.

Identity-aware VPN and tunneled reverse proxy for remote access based on WireGuard®.

The LLM Evaluation Framework

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

Zama Bounty Program: Contribute to the FHE space and Zama's open source libraries and get rewarded 💰

Supercharge Your LLM Application Evaluations 🚀

A framework for few-shot evaluation of language models.

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, and security risks.

Force Remove Copilot, Recall and More in Windows 11

Lime: Explaining the predictions of any machine learning classifier

AI Observability & Evaluation

Perform data science on data that remains in someone else's server

An open-source framework for detecting, redacting, masking, and anonymizing sensitive data (PII) across text, images, and structured data. Supports NLP, pattern matching, and customizable pipelines.

Cybersecurity AI (CAI), the framework for AI Security

Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization

BertViz: Visualize Attention in Transformer Models

This is a multi-use bash script for Linux systems to audit wireless networks.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

🛡️ Windows Hello™ style facial authentication for Linux

😈Awful AI is a curated list to track current scary usages of AI - hoping to raise awareness

OpenShell is the safe, private runtime for autonomous AI agents.

WiFi security auditing tools suite

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Adding guardrails to large language models.

Fit interpretable models. Explain blackbox machine learning.

Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

An Industrial Grade Federated Learning Framework

Passbolt Community Edition (CE) API. The JSON API for the open source password manager for teams!

Democratizing Reinforcement Learning for LLMs

A huge blocklist of manually curated sites that contain AI generated imagery for uBlock Origin & uBlacklist.

Robust recipes to align language models with human and AI preferences

Fawkes, privacy preserving tool against facial recognition systems. More info at https://sandlab.cs.uchicago.edu/fawkes

SWE-bench: Can Language Models Resolve Real-world Github Issues?

Adversary simulation and Red teaming platform with AI

Agentic execution layer for modern cyber security, turning security intent into precise, governed, auditable action through AI agents, MCP-native tools, knowledge, approvals, and attack-chain context.

《可解释的机器学习--黑盒模型可解释性理解指南》，该书为《Interpretable Machine Learning》中文版

Open-Source Unified Vulnerability Management, DevSecOps & ASPM

Universal and Transferable Attacks on Aligned Language Models

AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10.

A collection of infrastructure and tools for research in neural network interpretability.

Align Anything: Training All-modality Model with Feedback

Autonomous Hacking Agent for Red Team

Local-first healthcare AI: clinical NER & HIPAA PII de-identification that runs 100% on-device. 1,000+ medical models, 12 languages, Apple MLX + Python, no cloud, no patient data leaving your network. Apache-2.0

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

CISO Assistant is a one-stop-shop GRC platform for Risk Management, AppSec, Compliance & Audit, TPRM, BIA, Privacy, and Reporting. It supports 150+ global frameworks with automatic control mapping, including ISO 27001, NIST CSF, SOC 2, CIS, PCI DSS, NIS2, DORA, GDPR, HIPAA, CMMC, and more.

Open Source Data Security Platform for Developers to Monitor and Detect PII, Anonymize Production Data and Sync it across environments.

The Python Risk Identification Tool for generative AI (PyRIT) is an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI systems.

A full-stack AI Red Teaming platform securing AI ecosystems via OpenClaw Security Scan, Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

A Web app to manage your Two-Factor Authentication (2FA) accounts and generate their security codes

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

A list of AI agents and robots to block.

autonomous red teaming platform; multi-agent offensive-security meta-harness

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

Open-source security automation platform for teams and AI agents

Automation for internal Windows Penetrationtest / AD-Security

The Learning Interpretability Tool: Interactively analyze ML models to understand their behavior in an extensible and framework agnostic interface.

A library for mechanistic interpretability of GPT-style language models

OML 1.0 via Fingerprinting: Open, Monetizable, and Loyal AI

TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/

Evaluation and Tracking for LLM Experiments and AI Agents

Leaderboard Comparing LLM Performance at Producing Hallucinations when Summarizing Short Documents

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

The Security Toolkit for LLM Interactions

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Neural network visualization toolkit for keras

A Python toolbox to create adversarial examples that fool neural networks in PyTorch, TensorFlow, and JAX

Sandbox any AI agent in seconds - zero setup, zero latency.

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

A unified evaluation framework for large language models

A library for debugging/inspecting machine learning classifiers and explaining their predictions

Security scanner for AI agents, MCP servers and agent skills.

A unified framework for privacy-preserving data analysis and machine learning

Algorithms for outlier, adversarial and drift detection

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

Inspect: A framework for large language model evaluations

Explain, analyze, and visualize NLP language models. Ecco creates interactive visualizations directly in Jupyter notebooks explaining the behavior of Transformer-based language models (like GPT2, BERT, RoBERTA, T5, and T0).

Source code about machine learning and security.

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Convert Apple NeuralHash model for CSAM Detection to ONNX.

Information Gathering Instagram.

LLM Prompt Injection Detector

Secrets of RLHF in Large Language Models Part I: PPO

Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model in just 2 lines of code.

Privacy-Preserving Computing Platform 由密码学专家团队打造的开源隐私计算平台，支持多方安全计算、联邦学习、隐私求交、匿踪查询等。

Human ChatGPT Comparison Corpus (HC3), Detectors, and more! 🔥

Semantica • Build AI systems that can explain, trace, and justify every decision. Knowledge graphs, context graphs, reasoning engines, provenance, and governance for production AI.

OWASP Top 10 for Large Language Model Apps (Part of the GenAI Security Project)

AttackGen is a cybersecurity incident response testing tool that leverages the power of large language models and the comprehensive MITRE ATT&CK framework. The tool generates tailored incident response scenarios based on user-selected threat actor groups and your organisation's details.

a security scanner for custom LLM applications

Chinese safety prompts for evaluating and improving the safety of LLMs. 中文安全prompts，用于评估和提升大模型的安全性。

Evaluate your LLM's response with Prometheus and GPT4 💯

Benchmarking Generalized Out-of-Distribution Detection

A framework for the evaluation of autoregressive code generation language models.

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

Reinforcement Learning via Self-Distillation (SDPO)

Framework-agnostic implementation for state-of-the-art saliency methods (XRAI, BlurIG, SmoothGrad, and more).

A security scanner for your LLM agentic workflows

:art: A CNN visualizer

Safety-first guardrails for AI-driven cloud and Kubernetes operations

🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀

The nnsight package enables interpreting and manipulating the internals of deep learned models.

Scan MCP servers for potential threats & security findings.

Offen Fair Web Analytics

OmniXAI: A Library for eXplainable AI

AI agent security scanner. Detect vulnerabilities in agent configurations, MCP servers, and tool permissions. Available as CLI, GitHub Action, ECC plugin, and GitHub App integration. 🛡️

AI-Native Risk Intelligence Systems, OpenDeRisk——Your application system risk intelligent manager provides 7* 24-hour comprehensive and in-depth protection.

FacTool: Factuality Detection in Generative AI

This is a repository that aims to provide updates on the status of jailbreaking the OpenAI GPT language model.

Open Security Controls Assessment Language (OSCAL)

Winner of Mozilla's $50,000 prize for AI

Safety guardrails for ai coding agents and human terminal commands

AI-first security scanner. NEW in v2026.7: Claude Code compromise detection — vet .claude/ hooks, permissions & skills before you clone — plus an always-on AI attack-signature scanner and native Rust & PHP rules. Also: medusa scan --git to vet any repo, medusa secrets scan for leaked API keys. 40,000+ patterns, zero setup.

[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Official repository for the paper "LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code"

An easy-to-use Python framework to generate adversarial jailbreak prompts.

Repository for the paper "Automated Hate Speech Detection and the Problem of Offensive Language", ICWSM 2017

A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.

The repository is for safe reinforcement learning baselines.

Diffusion attentive attribution maps for interpreting Stable Diffusion.

Metis is an open-source, AI-driven tool for deep security code review

BLEURT is a metric for Natural Language Generation based on transfer learning.

A filter list that hides website features & content that uses Generative AI.

AIRecon is an autonomous cybersecurity agent that combines a self-hosted Large Language Model (Ollama) with a Kali Linux Docker sandbox and a Textual TUI. It is designed to automate security assessments, penetration testing, and bug bounty reconnaissance — without any API keys or cloud dependency.

CodeGate: Security, Workspaces and Multiplexing for AI Agentic Frameworks

An Open-Source Package for Textual Adversarial Attack.

A deliberately vulnerable banking application designed for practicing Security Testing of Web App, APIs, AI integrated App and secure code reviews. Features common vulnerabilities found in real-world applications, making it an ideal platform for security professionals, developers, and enthusiasts to learn pentesting and secure coding practices.

Locating and editing factual associations in GPT (NeurIPS 2022)

BruteSploit is a collection of method for automated Generate, Bruteforce and Manipulation wordlist with interactive shell. That can be used during a penetration test to enumerate and maybe can be used in CTF for manipulation,combine,transform and permutation some words or file text :p

A Deep Graph-based Toolbox for Fraud Detection

A library for making RepE control vectors

OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

Make your GenAI Apps Safe & Secure :rocket: Test & harden your system prompt

Benchmark of federated learning. Dedicated to the community. 🤗

Raising the Cost of Malicious AI-Powered Image Editing

Examples of techniques for training interpretable ML models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.

Preimage attack against NeuralHash 💣

Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing

WinDBG Anti-RootKit Extension

Microsoft Security Copilot is a generative AI-powered security solution that helps increase the efficiency and capabilities of defenders to improve security outcomes at machine speed and scale, while remaining compliant to responsible AI principles

GenAI compliance benchmark is a evaluation benchmarks for generative AI in regulated industries.

一款帮助云租户发现和测试云上风险、增强云上防护能力的综合性开源工具

Tools for understanding how transformer predictions are built layer-by-layer

Real Intelligence Threat Analytics (RITA) is a framework for detecting command and control communication through network traffic analysis.

Failure archive for ChatGPT and similar models

Code for IDS-ML: intrusion detection system development using machine learning algorithms (Decision tree, random forest, extra trees, XGBoost, stacking, k-means, Bayesian optimization..)

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

Find zero-days while you sleep. DeepZero is an automated vulnerability research framework that parses, decompiles, and analyzes thousands of Windows kernel drivers for exploitable IOCTLs natively using AI agents.

VULNRΞPO - Free vulnerability report generator and repository, end-to-end encrypted! Templates of issues, CWE,CVE,MITRE ATT&CK,PCI DSS, import Nmap/Nessus/Burp/OpenVAS/Bugcrowd/Trivy, Jira export, TXT/JSON/MARKDOWN/HTML/DOCX, attachments, automatic changelog, stats, vulnerability management, bugbounty, local ai/llm, super fast pentest reporting!

Deliver safe & effective language models

Casbin AI & MCP security gateway for HTTP, online demo: https://door.caswaf.com

A Privacy-Preserving Framework Based on TensorFlow

A Model for Natural Language Attack on Text Classification and Inference

Official implementation of the paper "The Stable Signature Rooting Watermarks in Latent Diffusion Models"

Data-Driven Evaluation for LLM-Powered Applications

[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

The official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"

A robust, and flexible open source User & Entity Behavior Analytics (UEBA) framework used for Security Analytics. Developed with luv by Data Scientists & Security Analysts from the Cyber Security Industry. [BETA]

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusion

[Pattern Recognition 25] CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks

[CVPR'25 highlight] RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness

Machine learning driven web application firewall to detect malicious queries with high accuracy.

Govern & Secure your AI

A library that incorporates state-of-the-art explainers for text-based machine learning models and visualizes the result with a built-in dashboard.

Security and Privacy Risk Simulator for Machine Learning (arXiv:2312.17667)

A tool for evaluating LLMs

EVA is an AI-assisted penetration testing agent that enhances offensive security workflows by providing structured attack guidance, contextual analysis, and multi-backend AI integration.

Everything you want about DP-Based Federated Learning, including Papers and Code. (Mechanism: Laplace or Gaussian, Dataset: femnist, shakespeare, mnist, cifar-10 and fashion-mnist. )

Filter sensitive information from free text before sending it to external services or APIs, such as chatbots and LLMs.

🗣️ Tool to generate adversarial text examples and test machine learning models against them

Visualize the intermediate output of Mistral 7B

Simulate a federated setting and run differentially private federated learning.

Requests based captcha solver for Arkose Labs using speech recognition

Surveillance Detection Scout: Your Lookout on Autopilot

Backdoors Framework for Deep Learning and Federated Learning. A light-weight tool to conduct your research on backdoors.

Benchmark for automated failure attributions in agentic systems (🏆 ICML 2025 Spotlight)

Lyrie.ai — The world's first autonomous AI cybersecurity agent. Built by OTT Cybersecurity LLC.

AutoAudit—— the LLM for Cyber Security 网络安全大语言模型

Simulation framework for accelerating research in Private Federated Learning

AI engine for smart contract audit

A Python library for Interpretable Machine Learning in Text Classification using the SS3 model, with easy-to-use visualization tools for Explainable AI :octocat:

🤖 Admyral enables continuous control monitoring for any custom control

prompt attack-defense, prompt Injection, reverse engineering notes and examples | 提示词对抗、破解例子与笔记

Breaching privacy in federated learning scenarios for vision and text

MarkDiffusion: An Open-Source Toolkit for Generative Watermarking of Latent Diffusion Models

Autonomous white-hat security auditor for AI-driven code review, bug bounty research, exploit construction, and execution-grounded verification.