Binary Similarity : Theory, Algorithms and Tool Evaluation

Binary Similarity : Theory, Algorithms and Tool Evaluation Liwei Ren, Ph.D, Trend Micro™ University of Houston-Downtown, Houston, Texas, October, 2015 Copyright 2011 Trend Micro Inc. 1 Agenda • What is binary similarity ? • Similarity Digesting: 3 Algorithms • A Mathematical Model • Tool Evaluation • A Novel Fuzzy Hashing • Summary and Further Research Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 2 What Is Binary Similarity? • Binary similarity or approximate matching. – What is binary similarity ? • 4 Use Cases specified by a NIST document: Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 3 What Is Binary Similarity? Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 4 Similarity Digesting : 3 Algorithms • Similarity digesting (aka, fuzzy hashing): – A class of hash techniques or tools that preserve similarity. – Typical steps for digest generation: – Detecting similarity with similarity digesting: • Three similarity digesting algorithms and tools: – ssdeep, sdhash & TLSH Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 5 Similarity Digesting : 3 Algorithms • ssdeep – Two steps for digesting: – Edit Distance: Levenshtein distance Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 6 Similarity Digesting : 3 Algorithms • Sdhash by Dr Vassil Roussev – Two steps for digesting: – Edit Distance: Hamming distance Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 7 Similarity Digesting : 3 Algorithms • TLSH – Two steps for digesting : – Edit Distance: A diff based evaluation function Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 8 A Mathematical Model • Summary of Three Similarity Digesting Schemes: – Using a first model to describe a binary string with selected features: • ssdeep model: a string is a sequence of chunks (split from the string). • sdhash model: a string is a bag of 64-byte blocks (selected with entropy values). • TLSH model: a string is a bag of triplets (selected from all 5-grams). – Using a second model to map the selected features into a digest which is able to preserve similarity to certain degree. • ssdeep model: a sequence of chunks is mapped into a 80-byte digest. • sdhash model: a bag of blocks is mapped into one or multiple 256-byte bloom filter bitmaps. • TLSH model: a bag of triplets is mapped into a 32-byte container. Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 9 A Mathematical Model • Three approaches for similarity evaluation: • 1st model plays critical role for similarity comparison. • Let focus on discussing various 1st models today. • Based on a unified format. • 2nd model saves space but further reduces accuracy. Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 10 A Mathematical Model • Unified format for 1st model: – A string is described as a collection of tokens (aka, features) organized by a data structure: • ssdeep: a sequence of chunks. • sdhash: a bag of 64-byte blocks with high entropy values. • TLSH: a bag of selected triplets. – Two types of data structures: sequence, bag. – Three types of tokens: chunks, blocks, triplets. • Analogical comparison: Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 11 A Mathematical Model • Four general types of tokens from binary strings: – k-grams where k is as small as , ,… – k-subsequences: any subsequence with length k. The triplet in TLSH is an example. – Chunks: whole string is split into non-overlapping chunks. – Blocks: selected substrings of fixed length. • Eight different models to describe a string for similarity. • Analogical thinking: – we define different distances to describe a metric space. Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 12 Tool Evaluation • Data Structure: – Bag: a bag ignores the order of tokens. It is good at handling content swapping. – Sequence: a sequence organizes tokens in an order. This is weak for handling content swapping. • Tokens: – k-grams: Due to the small k , , ,… , this fine granularity is good at handling fragmentation. – k-sequences: Due to the small k , , ,… , this fine granularity is good at handling fragmentation . – Chunks: This approach takes account of every byte in raw granularity. It should be OK at handling containment and cross sharing – Blocks: Depending on different selection functions, even though it does not take account of every byte, but it may present a string more efficiently and that is good for generating similarity digests. Due to the nature of fixed length blocks, it is good at handling containment and cross sharing. Copyright 2011 Trend Micro Inc. 13 Tool Evaluation Tool Model Minor Changes Containment Cross sharing Swap Fragmentation ssdeep M1.3 High Medium Medium Medium Low sdhash M2.4 High High High High Low TLSH M2.2 High Low Medium High High High High High High High Sdhash Hybrid + TLSH Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 14 Tool Evaluation Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 15 A Novel Fuzzy Hashing • We like to design a novel fuzzy hashing scheme based on the M2.4: – a string is presented by a bag of blocks. – Two steps: (1) Feature selection; (2) Digest generation. Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 16 A Novel Fuzzy Hashing • Continuing: Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 17 A Novel Fuzzy Hashing • This is TSFP – Trend String Fingerprint • Similarity measurement of TSFP: – Given two TSFP H and G where H = h1h2… hn and G= g1g2… gm . – Similarity is measured by function: • SIMH H,G = *|S ⋂T| / |S| + |T| – Where S = {h1, h2, … ,hn } and T = {g1, g2, … , gm } – SIMH G,H • Similarity measurement of two strings : – SIM(s,t) = SMTH(TSFP(s), TSFP(h)) Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 18 A Novel Fuzzy Hashing • Why do we need TSFP ? • We need to solve the following problems 1. 2. Similarity search problem: • B is a bag of binary strings {t1, t2 , …,tn} Given δ >0 and a binary string s, find t ϵ {t1, t2 , …,tn} such that SIM s, t δ. Similarity based clustering problem: • B is a bag of binary strings {{t1, t2 , …, tn }. Partition B into groups based on their binary similarity. • Why not {ssdeep, sdhash, TLSH} ? – NO… unless one applies Brute Force algorithm Copyright 2011 Trend Micro Inc. 19 A Novel Fuzzy Hashing • Similarity search problem: • B is a bag of binary strings {t1, t2 , …, tn }. Given δ >0 and a binary string s, find t ϵ {t1, t2 , …, tn} such that SIM s, t δ . • How does keyword based search engine work? – Extracting keywords from documents – Indexing keywords & documents – Searching via keywords. • Solution: – Given a string s, we get its fuzzy hash TSFP(s)= h1h2… hn . – Let S={h1, h2,…,hn}, each hj is a token of s that we treat it as a keyword. So we can create the indices TSFP-Index (B). – We can do two steps to solve the searching problems above. Copyright 2011 Trend Micro Inc. 20 A Novel Fuzzy Hashing • Similarity search problem: • B is a bag of binary strings {t1, t2 , …, tn }. Given δ >0 and a binary string s, find t ϵ {t1, t2 , …, tn} such that SIM s, t δ . • STEP 1: – Candidate selection • Let TSFP(s)= h1h2… hn to create the bag of tokens S={h1, h2,…, hn}. • Use this bag of tokens to search the indices TSFP-Index(B) so that we retrieve a list of candidates {s1, s2 , …, sm} ⊂ {t1, t2 , …, tn } ranked by number of common tokens. • STEP 2: – Brute force method at smaller scale • For each t ϵ {s1, s2 , …, sm}, if SIM( s, r) δ , t is what we are searching for. Copyright 2011 Trend Micro Inc. 21 Summary and Further Research • My practice of academic research in industry: Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 22 Summary and Further Research Framework of approximate matching, searching and clustering: Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 23 Q&A • Thank you for your interest. • Any questions? • My Information: – Email: liwei_ren@trendmicro.com – Academic Page: https://pitt.academia.edu/LiweiRen Classification 10/1/2015 Copyright 2011 Trend Micro Inc. 24