Secure image filtering

SECURE IMAGE FILTERING Nan Hu, Sen-ching S. Cheung Thinh Nguyen University of Kentucky Electrical & Computer Engineering Lexington, KY, 40506 nan.hu@uky.edu, cheung@engr.uky.edu Oregon State University EECS Department Corvallis, OR 97330 thinhq@eecs.oregonstate.edu ABSTRACT In today’s heterogeneous network environment, there is a growing demand for distrusted parties to jointly execute distributed algorithms on private data whose secrecy needed to be safeguarded. Protocols that support such kind of joint computation without complete sharing of information are called Secure Multiparty Computation (SMC) protocols. Applying SMC protocols in image processing is a challenging problem. Most of the existing SMC protocols are implemented based on cryptographic primitives like Oblivious Transfer that are too computational intensive for pixel-based operations. In this paper, we develop two efficient SMC protocols for distributed linear image filtering between two parties, one party with the original image and the other with the image filter. The first protocol is based on a combination of rank reduction and random permutation. The second one uses random perturbation with the help of a noncolluding third party. Experimental results show that both of them execute significantly faster than oblivious-transfer based techniques. Index Terms— Communication system security, Image Processing, Distributed Algorithms, Cryptography 1. INTRODUCTION The proliferation of imaging and storage devices and the ubiquitous presence of computer networks make sharing of visual data easier than ever. Such casual exchange of visual data, however, has increasingly raised questions on how sensitive visual information can be protected. Consider the scenario in which a user of a cellular-phone camera wants to send his/her pictures to an online photo-processing laboratory for image enhancement such as red-eye removal. The user would be concerned about the privacy of his/her pictures while the online store would need to protect the proprietary enhancement technologies against reverse-engineering. Consider another scenario that the police wants to search for possible suspects in a surveillance video owned by private company A, using a proprietary software from yet another private company B. The three parties involved (police, company A, company B) all have information they do not want to share with each other (criminal biometric database from the police, surveillance tape from company A and proprietary software from company B). To support such kinds of applications, one needs to establish a joint computation and communication platform at the pixel level that can guarantee the secrecy of private data and algorithms, and at the same time achieve a well-defined objective that benefits all parties involved. This is, however, not a new problem. Cryptographic protocols that support joint computation among multiple distrusted parties without complete sharing of information are called Secure Multiparty Computation (SMC) protocols. The general problem of SMC can be traced back to the classical paper by Yao [1]. The biggest challenge to any SMC technique is how to satisfy the security requirement. There are two types of security models - the ideal model and the real model [2]. In the ideal model, the computation is performed by a separate party trusted by all participants. This is perfectly secure as none of the participants gain information beyond the knowledge of their own data. However, the ideal model is not practical because all critical data reside at a single party and can easily be compromised. In the real model, the computation is performed via a SMC protocol agreed upon by all parties. The protocol is said to be secure in a semi-honest environment if all parties respect the protocol and are not able to derive more information than what can be deduced from the final results. Most of the SMC protocols, including those described in this paper, are developed under this model. Many SMC problems can be solved by using the Oblivious Transfer (OT) protocol in which one party (Alice) can select a particular item from a set owned by another party (Bob) without Bob knowing Alice’s selection or Alice knowing other items from Bob’s set [2]. The OT Transfer is typically implemented using different public-key encryption systems [3]. There has been little work in applying SMC to image processing problems. The only work known to us is by S. Avidan et al. on applying classical SMC protocols for two-party face detection [4]. In a typical classification task such as face detection, a significant portion of an image is transformed into feature vectors, which in most cases cannot be used to recover the original image. The manipulation of feature vectors is thus secure by definition and no special SMC protocols are required. As a result, the complex SMC protocols do not significantly affect the overall performance of the classification task. On the other hand, many common image processing applications require pixel-by-pixel processing. The sheer number of pixels in common images render most of the classical SMC techniques useless. Thus, it is imperative to develop fast computation techniques for these applications. Among all the image processing techniques, linear filtering is arguably the most basic and useful one. It is used in almost all image processing and computer vision applications such as enhancement, denoising, and feature extraction. Even though linear filtering by itself is inherently insecure as we will demonstrate in Section 2, we focus in this paper on “secure” and efficient linear filtering algorithms that can be used in conjunction with other types of non-linear processing. Our contributions are two novel secure linear filtering protocols that are significantly faster than those based on OT. These protocols are capable of processing mega-pixel images in less than one second using common desktops. The organization of the paper is as follows: our starting point in Section 2 is the problem definition and some notations of linear filter- ing. The classic two-party protocol based on OT is briefly reviewed in Section 3.1. In Section 3.2, we review the secure inner product protocol in [5] and introduce our two-party linear filtering protocol based on this inner product protocol. One drawback of this new protocol is that it cannot be used successively to implement multi-stage linear filtering. To solve this problem, we need the help from a third party. As we show in Section 3.3, the protocol guarantees that none of the parties have full knowledge of the data as long as no two parties collude to share data externally. The efficiency of the proposed techniques are demonstrated by the experimental results in Section 4. Finally, we conclude the paper and describe ongoing work in Section 5. 2. PROBLEM DEFINITION AND NOTATIONS Given an image {x(m, n) : 0 ≤ m ≤ M, 0 ≤ n ≤ N } and a filtering operation f (·) described by a set of parameters θ, we define the output y(m, n) of applying this filter to x as follows: y(m, n) = f (x; θ) (1) In the secure image filtering model, we have two users, Alice and Bob, who own the image data x and the filter parameter θ respectively. Our goal is to establish a computation protocol between Alice and Bob so that 1. Alice obtains f (x; θ) without any knowledge of θ, and 2. Bob does not know anything about x. For linear filtering, the filter parameters are specified as a filter mask h defined as {h(m, n) : − 2l ≤ m ≤ 2l and − 2l ≤ n ≤ 2l }. The linear filtering operation can then be written as y(m, n) = x ∗ h = l/2 X l/2 X h(i, j)x(m − i, n − j). (2) i=−l/2 j=−l/2 It is easy to see that Equation (2) is a scalar product between two (l + 1)2 dimensional vectors. All of our secure linear filtering protocols use the following conceptual model: Alice first forms a M N × (l + 1)2 matrix Xw whose rows are the signal data needed for the inner product operation. The total number of rows of Xw is the total number of pixels in the output image1 . If we denote the i-th row of Xw as Xw (i, :) and the output image as a vector y = [y1 · · · yM N ]T , the image filtering could be written as y = Xw h (3) A secure linear filtering protocol decomposes every yi = Xw (i, :)·h into yi = a + b such that Alice computes a without any knowledge of h and Bob computes b without any knowledge of Xw (i, :). If linear filtering is the end goal of the processing, Bob sends back his portion to Alice to compute the output image y. Using both the input x and y, Alice can estimate h using the least square estimate T ĥ = (Xw Xw )−1 Xw y. In other words, linear filtering is intrinsically insecure to Bob no matter how secure the protocols are. General non-linear filtering, on the other hand, is much harder to invert based on limited number of input and output pairs. If linear filtering is used as part of a secure non-linear image processing system, it is thus important that neither Alice nor Bob has the entire output of the linear filtering protocol. 1X w can be made to have M N rows with appropriate boundary handling. 3. SECURE LINEAR FILTERING In this section, we will develop two types of secure filtering protocol: 1) a two-party protocol based on rank reduction and random permutation and 2) a three-party protocol based on random perturbation of the data. Before introducing the new protocols, we first review the classic two-party protocol based on OT. A detailed explanation of OT could be found in [2], and an application of OT in face detection could be found in [4]. 3.1. Classical Two-party Solution Oblivious transfer allows Alice to select one element from the whole dataset Bob holds without revealing to Bob which element Alice has selected and without knowing any other element in the dataset rather than the one selected. Our current notion of oblivious transfer was originally suggested by Even, Goldreich and Lempel [6] as a generalization of Rabin’s ”oblivious transfer” [7]. We will denote OT1m to indicate the protocol that Alice choose one out of the m elements from Bob’s dataset. The state-of-the-art OT1m requires computing at least two long integer modular exponential, typically in the range of 512 bit to 1024 bit, per selection [3]. Secure scalar product can be implemented based on oblivious transfer as shown in [4]. Alice and Bob have x, h ∈ F n respectively, where F is a finite field (0-255 for grayscale image pixels). Let the totalPnumber of element in the finite field F be NF . To comn pute y = i=1 xi · hi , Bob computes for each hi a table of NF entries. The j-th entry of the table is j · hi − ri where ri , 1 ≤ i ≤ n are random numbers known only to Bob. Alice and Bob then engage in N rounds of OT1NF protocols in which Alice selects the j-th entry ofPthe table in the i-th round if xi = j. Finally, Alice computes Pn a= n i=1 ri . It is obvious that i=1 xi · hi − ri and Bob has b = y = a + b while Bob and Alice know nothing about each other’s vector. To prevent Alice from inferring Bob’s data based on successive execution of OT, Bob must use unpredictable encryption keys in each OT which also adds to the complexity of the algorithm. 3.2. Two-party Solution Our first protocol on linear filtering is based on the secure inner protocol proposed in [5]. We describe this protocol using the pseudocode InnerProductAlice and InnerProductBob listed below. Alice has a n-dimensional vector x and Bob has a n-dimensional vector y. They both know an invertible matrix P and itsn inverse P −1 . B∈ P is broken down into top and bottom halves T ∈ Rb 2 c×n and n −1 n×b n c 2 R(n−b 2 c)×n , while P into left and right halves L ∈ R and n R ∈ Rn×(n−b 2 c) . The inner product xT y can then be decomposed as follows: xT y = xT P −1 P y = xT LT y + xT RBy (4) Alice then sends xT R to Bob who computes xT RBy while Bob sends Alice T y so that she can compute xT LT y. The security of the protocols comes from the observation that xT R and T y project x and y into lower-rank subspaces, and thus the components of the original vectors inside the null spaces of the matrices are irrecoverably lost. In [5], the authors proposed a design of P based on decoding matrices used in error control coding so as to spread the projections of neighboring vectors as far as possible. In this paper, we adopt this method to generate an invertible matrix G and compute the actual P by multiplying G with a random orthogonal matrix. This approach allows us to generate different P if the protocols require multiple invertible matrices. Unlike the OT-based Algorithm 1 InnerProductAlice(x, P −1 ) n −1 Require: x ∈ R . P = (L R) is a n × n invertible matrix where n ≥ 2; L ∈ Rn×bn/2c and R ∈ Rn×(n−bn/2c) . 1: x1 ← LT x 2: x2 ← RT x 3: Transmit x2 to Bob. 4: Receive y1 from Bob. 5: return x1 T y1 Algorithm 2 InnerProductBob(y, P ) Require: y ∈ Rn . P T = (T T B T ) is a n × n invertible matrix where n ≥ 2; T ∈ Rbn/2c×n and B ∈ R(n−bn/2c)×n . 1: y1 ← T y 2: y2 ← By 3: Receive x2 from Alice. 4: Transmit y1 to Alice. 5: return x2 T y2 protocol which requires complex long-integer exponential and random key generation, this protocol requires only the highly optimized matrix multiplications. It may seem intuitive to implement secure linear filtering by applying the inner product algorithm on Xw row by row. However, it is not secure as adjacent rows in Xw overlap with each other. As a result, the redundancy in the rank-reduced data sent to Bob allows him to form a least-square estimation of the original image. This least square problem involves solving a least-square data matrix of 2 c × M N . To achieve secrecy, Alice needs to first size M N b (l+1) 2 randomly scramble the order of the rows in Xw before carrying out the inner product. Without the proper row order, Bob has no means of formulating the least-square estimation. The protocol is described in FilterAlice and FilterBob. Algorithm 3 FilterAlice(x, P −1 ) Require: x ∈ Rm . P −1 is a n × n invertible matrix as described in InnerProductAlice where n is the length of the linear filter. 1: Reformat x into Xw . 2: Scramble the row order of Xw . 3: for i ← 1 to m do 4: rA (i) ← InnerProductAlice(Xw (i, :), P −1 ) 5: end for 6: return rA Algorithm 4 FilterBob(h, m, P ) Require: h ∈ Rn . m is the number of inner product operations required. P T is a n × n invertible matrix as described in InnerProductBob. 1: for i ← 1 to m do 2: rB (i) ← InnerProductBob(h, P ) 3: end for 4: return rB At the end of FilterAlice and FilterBob, the filtered result y is decomposed into rA + rB where Alice has rA and Bob has rB , all in a scrambling order known only to Alice. As discussed earlier, rB cannot be sent back to Alice for unscrambling as this will allow Alice to estimate Bob’s filter. This creates a problem for multiple stages of linear filtering as Bob does not know the scrambling order of rB . Multiple stages of linear filtering are often used in image processing such as separable filtering (horizontal and vertical filtering) or wavelet transform (multiple stages of subband filtering). Thus, it is highly desirable to have a secure protocol that can support multiple stages of linear filtering. This is the subject of Section 3.3. 3.3. Three-party Solution In this section, we show how to implement the secure linear filtering with the help of a third party Clark, who we assume will not collude with either Bob or Alice. This protocol does not use any scrambling and thus allow multiple stages of linear filtering. On the other hand, its application is comparatively limited as a non-colluding third party may not be always present. The basic idea is that, instead of scrambling the rows of Xw , we randomly inject random noise into the rows of Xw and h for each inner product operation. The dependency between successive rows vanishes as random noise is used each time. The proposed protocol is shown in 3PartyInnerProductAlice, 3PartyInnerProductBob, and 3PartyInnerProductClark. The problem notation is the same as in Section 3.2 where Alice holds x and Bob holds y and they want to jointly compute xT y. Alice generates a random n-dimensional vector xa and computes xb = x − xa . Similarly, Bob generate a random n-dimensional vector ha and computes yb = y − ya . Then the inner product can be rewritten as xT y = xTa ya + xTa yb + xTb ya + xTb yb , (5) xa or xb alone provides no information about x. Neither does ya or yb alone about y. Unfortunately, it is impossible for Bob and Alice to compute all the four terms in (5) by just receiving one component of the vector from each other. For example, if Alice sends Bob xa , he can compute the first and the second terms rb = xTa ya + xTa yb . If Bob sends Alice ya , Alice can compute the third term but not the fourth. If Bob sends yb , Alice can compute the fourth but not the third. To solve this conundrum, we introduce a third party Clark. If Bob sends Alice ya and Alice computes ra = xTb ya , Alice and Bob can send Clark xb and yb so that Clark can compute the remaining term in (5), i.e. rc = xTb yb . Provided that no two parties collude with each other, the information Alice, Bob and Clark have are all random data which disclose no information about either x or h. Algorithm 5 3PartyInnerProductAlice(x) Require: x ∈ Rn . 1: Generate random vector xa . 2: xb ← (x − xa ) 3: Transmit xa to Bob. 4: Transmit xb to Clark. 5: Receive ya from Bob. 6: return ra = xb T ya To use this new protocol for linear filtering, we can simply modify FilterAlice and FilterBob by removing the scrambling and replacing InnerProductAlice with 3PartyInnerProductAlice and InnerProductBob with 3PartyInnerProductBob. At the end of the protocol, Alice, Bob and Clark will have rA , rB and rC respectively such that the output image y = rA + rB + rC . To perform a second stage linear filtering with, say Bob’s g, we can simply apply the distribution rule for convolution y ∗ g = rA ∗ g + rB ∗ g + rC ∗ g and compute each term using the secure linear filtering protocol between Bob and each of Alice and Clark. View publication stats Algorithm 6 3PartyInnerProductBob(y) Require: y ∈ Rn . 1: Generate random vector ya . 2: yb ← (y − ya ) 3: Receive xa from Alice. 4: Transmit ya to Alice. 5: Transmit yb to Clark. T 6: return rb = xT a ya + xa yb rithm described in Section 3.2 and 3.3 on a dual Wintel CPU (P43.4GHz) desktop with 1GB memory. The reason we did not test the classic protocol on real images is because it will take hours to do a linear filtering on a single image. The oblivious transfer based technique takes about 20 minutes to compute the inner product of two 20-dimensional vectors while our two-party protocol uses only 30 milliseconds and our three-party protocol uses 47 milliseconds. Despite our non-optimal implementation of the oblivious transfer protocol, its slow performance can be attributed to the handling of very long integers in the encryption/decryption process as well as the large amount of information exchanged between Alice and Bob. For linear filtering using a 7 × 7 Gaussian mask on the same computing platform, our two-party solution takes on average 0.7 seconds to denoise a 128 × 128 image and our three-party solution takes around 0.6 seconds. Algorithm 7 3PartyInnerProductClark() 1: Receive xb from Alice. 2: Receive yb from Bob. 3: return rc = xT b yb 4. EXPERIMENTS 5. CONCLUSION All the algorithms described in Section 3 are implemented and tested with MATLAB version 7.0. To ensure the accuracy of the design, all data exchange between Alice and Bob are conducted using actual TCP/IP connections. To encourage future development and independent verification, we have also made all our source code available at http://www.vis.uky.edu/∼nhu2/secureImage. Figure 1(a) shows the original 128 × 128 Lena image. Figure 1(b) shows the result of filtering Lena using a 7×7 Gaussian denoising filter. This image is the result of combining Bob’s and Alice’s partial copies. We use P ∈ R49×49 to completely hide the details of the filter and the output of FilterBob is shown in Figure 1(c). As shown, all the details from the original image are scrambled. This paper presents two different solutions for the secure linear filtering problem — a two-party solution based on rank reduction and random permutation, and a three-party solution based on random perturbation. A drawback of the two-party solution is that the scrambling prevents repeated use of the protocols. To avoid scrambling, our third-party solution relies on a third party to participate in independent random perturbation of each inner product computation. The comparison between our proposed protocols and the classic protocol based on OT shows that our protocols are much faster and more appropriate for pixel-wise computation. A more detailed evaluation of the security of these algorithms, especially under malicious attack, is among the most pressing issue we are currently addressing. Another important area is to assess the bandwidth requirement for these algorithms, which could quickly become a bottleneck as we move from image to video applications.. 6. REFERENCES [1] A. C. Yao, “How to generate and exchange secretes,” proceeding of 27th FOCS, pp. 162–167, 1986. [2] O. Goldreich, Foundations of Cryptography: Volume II Basic Applications, Cambridge, 2004. (a) (b) [3] M. Naor and B. Pinkas, “Efficient oblivious transfer protocols,” Proc. 12th Ann. Symp. Discrete Algorithms, pp. 448–457, 2001. [4] S. Avidan, “Blind vision,” ECCV2006, May 2006. [5] W. Du et al., “Privacy-preserving multivariate statistical analysis: Linear regression and classification,” proceeding of the 4th SIAM International Conference on Data Mining, pp. 222–233, 2004. [6] A. Lempel S. Even, O. Goldreich, “A randomized protocol for signing contracts,” Communications of the ACM 28, pp. 637– 647, 1985. (c) Fig. 1. (a) 128 × 128 Lena; (b) Lena smoothed by a 7 × 7 Gaussian filter using secure filtering with a disguise matrix M ∈ R49×49 ; (c) is the partial result held by Bob Our protocols are also computationally efficient. As a comparison, we have implemented a classic two-party protocol based on the description from [4], using our own 512-bit RSA public-key cryptosystem (PKCS). We then compare its performance with the algo- [7] M. O. Rabin, “How to exchange secrets by oblivious transfer,” Tech. Memo TR-81, Aiken Computation Laboratory, 1981.