property | value |
tags | defensive-tradecraft,ioc-scanner,threat-detection |
url | |
original_word_count | 3046 |
Article Excerpt
During a forensic investigation, malicious activity may be discovered in logs. An incident responder may want to identify related malicious activity across a range of related log files.
Long Summary
This article explains how JsonHash, a novel fuzzy hashing algorithm, can be used to detect similarities in log files. JsonHash works by flattening the nested structures of the data record, tokenizing the values for each field, prepending the field name to the tokenized values, hashing the tokens for each field, placing the resultant hash for each token into a βbucketβ, counting the frequency of each bucket, scaling the counts, and outputting a hash digest string. To compare two JsonHash digests, a dissimilarity measure is used to approximate the number of tokens shared by both digests, as a ratio to the total number of tokens from both digests combined.
To illustrate how JsonHash can be used, a scenario is provided in which a threat actor has managed to exploit a vulnerability in an internet facing IIS server operated by an organisation named Contoso. This allows for remote code execution and remote file upload to be performed. Through forensic analysis, incident responders retrieve various Indicators of Compromise (IOCs) including the hash of the webshell, and the user agents and IP addresses observed interacting with it. Using JsonHash, Contosoβs IR team is able to analyse the IIS logs retrieved from the server where the webshell was found. The IIS logs are converted to CSV files and a JsonHash is generated for each log line. The resulting digests are then analysed to identify similarities.
The article also explains how to calculate the dissimilarity between two strings using the unicode codepoints from the strings. The process begins by assigning the unicode codepoints from the strings to two variables, x and y. Then, the mv-apply function is used to typecast the codepoints to integers and calculate the maximum and minimum values for each codepoint. The maximum and minimum values are then used to calculate the dissimilarity between the two strings. Finally, the project-away function is used to remove the xlen, x, and y variables from the equation.
The article concludes with a validation to assess how accurate JsonHash is at finding webshell activity logs. The experiment showed JsonHash was effective at finding Web Shell activity for the sample set tested. JsonHash is an algorithm under further development and future refinements will include improving accuracy by ignoring fields such as timestamps and UUIDs from the hash computation and conducting further experiments to identify the best
Short Summary
π JsonHash: Fuzzy hashing logs to find malicious activity
ππ½ JsonHash is a fuzzy hashing algorithm used to detect similarities in log files.
ππ½ During a forensic investigation, malicious activity may be discovered in logs. An incident responder may want to identify related malicious activity across a range of related log files with different formats.
ππ½ JsonHash flattens nested structures, tokenizes values, hashes tokens and places them in buckets. ππ½ The algorithm generates a hash digest string as output. ππ½ A dissimilarity measure is used to compare two JsonHash digests for similarities. ππ½ The algorithm is useful for the forensic investigation of cyberattacks. ππ½ It can identify Indicators of Compromise (IOCs) such as webshell activity. ππ½ Example for converting IIS logs to CSV files and generating a JsonHash for each log line. ππ½ The article explains how to calculate dissimilarity using unicode codepoints, amongst other things. ππ½ Example use case shows that JsonHash is effective at finding webshell activity logs. ππ½ Future refinements will improve accuracy and identify other uses for the algorithm.
π source link: https://techcommunity.microsoft.com/t5/microsoft-security-experts-blog/fuzzy-hashing-logs-to-find-malicious-activity/ba-p/3786669
π summary link: https://hut.threathunterz.com/battlefield-intel/articles-and-reports/fuzzy-hashing-logs-to-find-malicious-activity
#JsonHash #FuzzyHashing #LogSimilarities #IndicatorsOfCompromise #DissimilarityMeasure