Does Entropy Matter? A Pseudoscientific Study!
How useful is entropy for identifying packed samples?
Overview
Entroy is obviously useful and tells us information about the binary. However, can we use entropy alone to determine if a sample is packed or not. Let's define the parameters for our study.
- Without looking at the binary in IDA (or your RE tool of choice) can you use entropy to determine if the sample is packed?
- We are defining "packed" as a file that contains an encrypted, or compressed payload, where our analysis goals are to analyze the payload and not the packer.
- Are there specific data in the binary that can be tested for entopy that will give use a better answer than testing the full binary? For example, looking at the entropy of sections, or of resources.
References
- Understanding Shannon's Entropy metric for Information
- Using entropy to spot the malware hiding in plain sight
- merces/entropy (github)
- PowerShellArsenal/Misc/Get-Entropy.ps1
- Using Entropy Analysis to Find Encrypted and Packed Malware (bintropy)
- bintropy (github)
- Packer Detection for Multi-Layer Executables Using Entropy Analysis
- Generic unpacking using entropy analysis
Our Problem
Based on entropy can we make a decision about whether or not to open a sample in IDA by looking at entropy alone. If we cannot make this decision we will have to open the binary in IDA so why bother looking at entropy at all?
Our Study
For our study we are going to collect a set of known packed, and unpacked (payload) samples to use as our ground truth. We will then run differnt types of entropy calculation on the binaries and look for a common cutoff where we could made a decision that the samples are packed/unpacked. If the cutoff is such that we cannot classify all samples within an error margin of ERROR-RATE-TBD then we can conclude that entropy will not consistantly answer our problem statement.
Tools
We are going to use bintropy and the standard section entropy calculation from pefile as our two tools.
import pefile
import bintropy
def pe_test(file_path, all_sections=True):
#
# We will test the entropy of the non-executable sections
# and return the largest entropy value
#
pe = pefile.PE(file_path)
entropy_list = []
for s in pe.sections:
if all_sections:
entropy_list.append(s.get_entropy())
elif not s.IMAGE_SCN_CNT_CODE:
entropy_list.append(s.get_entropy())
if len(entropy_list) == 0:
return 0
return max(entropy_list)
def is_dotnet(file_path):
pe = pefile.PE(file_path)
isDotNet = pe.OPTIONAL_HEADER.DATA_DIRECTORY[14]
if isDotNet.VirtualAddress == 0 and isDotNet.Size == 0:
return False
else:
return True
def bintropy_test(file_path, get_average=True):
h_e, av_e = bintropy.bintropy(file_path, decide=False)
if get_average:
return av_e
else:
return h_e
UNPACKED_DIR = '/tmp/unpacked'
PACKED_DIR = '/tmp/packed'
file_path = '/tmp/packed/ff5ac0eb80d90c6a2a46a4133fc8d90cd165b8b2bac1cbaa8fadd35b186bd5c8.bin'
#
# threshold avg packed 6.677
# threshold highest packed 7.199
#
# test all sections
print("\ntesting all sections")
pe = pefile.PE(file_path)
for s in pe.sections:
print(f" is code: {s.IMAGE_SCN_CNT_CODE} -- {s.get_entropy()}")
# test the pe method only data
print("\ntest highest entropy from data sections")
print(pe_test(file_path, all_sections=False))
# test the pe method all sections
print("\ntest highest entropy from all sections")
print(pe_test(file_path, all_sections=True))
# test bintropy average
print("\ntest bintropy average")
print(bintropy_test(file_path, get_average=True))
# test bintropy average
print("\ntest bintropy highest")
print(bintropy_test(file_path, get_average=False))
import os
from rich.console import Console
from rich.table import Table
# assign directory
directory = PACKED_DIR
# iterate over files in
# that directory
table = Table(title="Packed Samples", expand=True)
table.add_column("file", justify="center", no_wrap=True, )
table.add_column(".NET", justify="center", no_wrap=True, )
table.add_column("pe data", justify="center", no_wrap=True)
table.add_column("pe all", justify="center", no_wrap=True)
table.add_column("pe all\npacked @ 7", justify="center", no_wrap=True)
table.add_column("bin ave", justify="center", no_wrap=True)
table.add_column("bin all", justify="center", no_wrap=True)
table.add_column("bin all\npacked @ 7", justify="center", no_wrap=True)
table.add_column("bin t/f", justify="center", no_wrap=True)
for filename in os.listdir(directory):
f = os.path.join(directory, filename)
# checking if it is a file
if os.path.isfile(f):
file_path = f
pe_data = pe_test(file_path, all_sections=False)
pe_all = pe_test(file_path, all_sections=True)
bin_ave = bintropy_test(file_path, get_average=True)
bin_all = bintropy_test(file_path, get_average=False)
bin_tf = bintropy.bintropy(file_path)
dotnet = is_dotnet(file_path)
table.add_row(filename[:5], str(dotnet),
str(pe_data)[:4],
str(pe_all)[:4],
str(True if pe_all > 7 else False),
str(bin_ave)[:4],
str(bin_all)[:4],
str(True if bin_all > 7 else False),
str(bin_tf))
console = Console()
console.print(table)
import os
from rich.console import Console
from rich.table import Table
# assign directory
directory = UNPACKED_DIR
# iterate over files in
# that directory
table = Table(title="Unpacked Samples", expand=True)
table.add_column("file", justify="center", no_wrap=True, )
table.add_column(".NET", justify="center", no_wrap=True, )
table.add_column("pe data", justify="center", no_wrap=True)
table.add_column("pe all", justify="center", no_wrap=True)
table.add_column("pe all\npacked @ 7", justify="center", no_wrap=True)
table.add_column("bin ave", justify="center", no_wrap=True)
table.add_column("bin all", justify="center", no_wrap=True)
table.add_column("bin all\npacked @ 7", justify="center", no_wrap=True)
table.add_column("bin t/f", justify="center", no_wrap=True)
for filename in os.listdir(directory):
f = os.path.join(directory, filename)
# checking if it is a file
if os.path.isfile(f):
file_path = f
pe_data = pe_test(file_path, all_sections=False)
pe_all = pe_test(file_path, all_sections=True)
bin_ave = bintropy_test(file_path, get_average=True)
bin_all = bintropy_test(file_path, get_average=False)
bin_tf = bintropy.bintropy(file_path)
dotnet = is_dotnet(file_path)
table.add_row(filename[:5], str(dotnet),
str(pe_data)[:4],
str(pe_all)[:4],
str(True if pe_all > 7 else False) ,
str(bin_ave)[:4],
str(bin_all)[:4],
str(True if bin_all > 7 else False) ,
str(bin_tf))
console = Console()
console.print(table)
Conclusions
Entropy May Not Be Useful for .NET
For .NET in atleast one sample 3664a0db89a9f1a8bf439d8117943d3e042abe488a761dc6c8e18b90d6081298 which was packed had an entropy in the 4 range but when unpacked 2333c19020f6e928198cea31c05dd685055991c921f3a1cd32ad9817b6c704e6 the entropy was in the 6 range. From this we can conclude that entropy may have no relation to the packed status of a .NET binary.
Bintropy Has A Poor Packed Detection Rate with Default Values
For detecting packed samples using the default threashold the failure rate was 16/26, however there were no unpacked false positives. One conclusion we can draw from this is that the tool can the relied on when it detects a packed sample (ie, the sample is probably packed) but it cannot be relied on for a decision, as it has a high false negative rate. This could be used as a filter, but not as a decision metric.
General Conclusions
The results from our small "pseudoscientific" study do not match the results from the two academic papers refernced in our overview (99% and 97% accuracy). In our study we had a high fidelity when detecting packed sample (ie. if it was detected as packed it was in fact packed) however, we also have very high false negative rates (ie. if it was detected as not-packed there was an over 50% chance that it was actually packed). From this we can draw two conclusions, obviously the first, this was not a scientific study (more data needed) and the second, we cannot use entropy alone for packer identification, it must be combined with other metrics.