cado-security/cloudgrep

It currently supports searching log files, optionally compressed with gzip (.gz) or zip (.zip), in AWS S3. You can run this from your local laptop, or from an EC2 instance in the same region as the S3 bucket to avoid egress charges. You can authenticate in a number of ways.

Cloudgrep is a tool that allows users to search cloud storage, such as AWS S3 or Azure storage, for log files without the need to index them into a SIEM or Log Analysis tool. It is faster and cheaper than traditional methods, and searches files in parallel for speed. It is useful for debugging applications or investigating security incidents.

The tool is run from the command line, and users can specify the bucket, query, prefix, filename, start and end date, and file size. It can also be used to save the output to a file. An example of a simple command is python3 cloudgrep.py --bucket test-s3-access-logs --query 9RXXKPREHHTFQD77.

To deploy the tool, users can install it with pip3 install -r requirements.txt. It can be run from a local laptop or a virtual machine in the cloud. Authentication is different for each cloud provider. For AWS, users will need access to the S3 bucket, and for Azure, users will need to run az login.

Contributions to the project are welcome, and possible future work includes support for Google Cloud, zstd compression, log parsing and detection, and exporting parsed logs in a standard syslog format. If users have any questions or suggestions, they can open a GitHub issue. This is not an officially supported Cado Security product.

Cloudgrep is a useful tool for searching cloud storage quickly and efficiently. It is easy to deploy and use, and contributions are welcome.

📓 cado-security/cloudgrep

👉🏽 It currently supports searching log files, optionally compressed with gzip (.gz) or zip (.zip), in AWS S3. You can run this from your local laptop, or from an EC2 instance in the same region as the S3 bucket to avoid egress charges. You can authenticate in a number of ways. 👉🏽 Cloudgrep allows users to search cloud storage for log files without indexing. 👉🏽 Faster and cheaper than traditional methods. 👉🏽 Searches files in parallel for speed. 👉🏽 Useful for debugging applications or investigating security incidents. 👉🏽 Can be run from the command line with specified parameters. 👉🏽 Can save output to a file. 👉🏽 Simple command example: python3 cloudgrep.py --bucket test-s3-access-logs --query 9RXXKPREHHTFQD77. 👉🏽 Can be installed with pip3 install -r requirements.txt. 👉🏽 Can be run locally or on a virtual machine in the cloud. 👉🏽 Authentication is different for each cloud provider (AWS and Azure). 👉🏽 Contributions to the project are welcome. 👉🏽 Possible future work includes support for Google Cloud, zstd compression, log parsing, and detection. 👉🏽 Can export parsed logs in a standard syslog format. 👉🏽 Users can ask questions or make suggestions by opening a GitHub issue. 👉🏽 Cloudgrep is not an officially supported Cado Security product.

🔗 source link: https://github.com/cado-security/cloudgrep

🔗 summarized content: https://hut.threathunterz.com/battlefield-intel/tradecraft-tools/cado-securitycloudgrep

#Cloudgrep #Cloudstorage #Logfiles #Tool #Search