Punnya Tuladhar, Grad. Student Dr. Vassil Roussev, Advisor Dept. of Computer Science Hash Based Similarity Identification 1400:27:10:2010 THESIS

19:33 HSBI 2 Hypothesis Manufacturer of source camera can be identified by examining quantization table and huffman table of the JPEG image. Does a camera use a specific q-table?

19:33 HSBI 3 Previous Work H. Farid, Dartmouth College, Computer Science Research: 2006 & 2008 Conclusion: “While the JPEG quantization table is clearly not unique, it (paired with the image resolution) is reasonably effective at narrowing the source of an image to a single camera make and model or to a small set of possible cameras.”

19:33 HSBI 4 Introduction JPEG: A compression method. JFIF: For exchange of jpeg encoded streams between incompatible systems. EXIF: Info. Of image/digicam; thumbnail Source: 109,774 images from http://domex.nps.edu/corp/files/govdocs1/ Tools & Env.: linux, jdk1.6, ghex2, md5sum, exiftool

19:33 HSBI 5 JPEG FFD8 FFD9 FFC4 FFDB FFE1 FFDA Image data Q-table FFDB + LL + XY + 64 bytes + XY + 64 bytes + ... Huffman table FFC4 + LL + XY + Li (1-16) + Vi given by Li (1-16) Exif data Marker FFXX + LL + DDDD ...

19:33 HSBI 6 JPEG Huffman table: Used in jpeg compression Determined by- Quality setting of camera Quantization table: 8*8 block Controls quality Determined by- Quality setting of camera Standards followed by camera or s/w

19:33 HSBI 7 Effects of Q-table 04 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F 1E

19:33 HSBI 8 Effects of Q-table 08 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F 1E

19:33 HSBI 9 Effects of Q-table 10 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F 1E

19:33 HSBI 10 Effects of Q-table 04 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F 1E

19:33 HSBI 11 Effects of Q-table 04 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F FF

19:33 HSBI 12 Effects of Q-table 04 03 04 04 04 03 05 04 04 04 05 05 05 06 07 0C 08 07 07 07 07 0F 0B 0B 09 0C 11 0F 12 12 11 0F 11 11 13 16 1C 17 13 14 1A 15 11 11 18 21 18 1A 1D 1D 1F 1F 1F 13 17 22 24 22 1E 24 1C 1E 1F 01

19:33 HSBI 13 Effects of H-Compression

19:33 HSBI 14 Effects of H-Compression 23:1

19:33 HSBI 15 Effects of H-Compression 144:1

19:33 HSBI 16 Research Workflow Hashing Exif Extraction Stats Calculation Extraction Of FFDB, FFC4

19:33 HSBI 17 Extraction of tables 1. Q-table: > Marker: FFDB|LL|XY|64bytes|XY|64bytes|... > Each extracted table of size 64 bytes. > ~204K tables. > FFDBExtractor.java 2. H-table: > Marker: FFC4|LL|XY|Li,1-16|Vi,1-16|XY|Li,1-16|Vi,1-16... > FFC4Extractor.java

19:33 HSBI 18 Hashing Md5sum: 32 bytes long hash value. ########### ffdb_hash ############ 0656ffc785f02947576c3b671c50db9b 000107.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000107.01.ff-db c30a1cdefc2d7ef4914b2bfdbf33743c 000108.00.ff-db 6307cac23d9b65d1c95cd695a22d1f26 000108.01.ff-db 0656ffc785f02947576c3b671c50db9b 000109.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000109.01.ff-db 0656ffc785f02947576c3b671c50db9b 000110.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000110.01.ff-db ########## ffdb_hash_formatted ######### 000107.jpg 00 0656ffc785f02947576c3b671c50db9b 000107.jpg 01 54e464a3c6056f015f7da1e86a0086f8 000108.jpg 00 c30a1cdefc2d7ef4914b2bfdbf33743c 000108.jpg 01 6307cac23d9b65d1c95cd695a22d1f26 000109.jpg 00 0656ffc785f02947576c3b671c50db9b 000109.jpg 01 54e464a3c6056f015f7da1e86a0086f8 000110.jpg 00 0656ffc785f02947576c3b671c50db9b 000110.jpg 01 54e464a3c6056f015f7da1e86a0086f8 Format.java

19:33 HSBI 19 Hashing ########## ffdb_hash_freq ########### 21784 197abd2b5741a96ff5b83121f760ed52 19561 5524f5067b8475f1e7a29fd90c1d98ef 12505 c714773c7a90b0f95a0a714f3e18729e 12331 8a7c44033e511bbf00563b8beea563a6 8717 784d68ba9112308689114a6816c628ce 5588 0e4362ba11351c4808b4e46aeaf94170 5541 3fee0917e01167d5795ed5e4f283e3ad 5539 2417671783bf9c3261707fb94f7b6cc3 ########### ffdb_hash ############ 0656ffc785f02947576c3b671c50db9b 000107.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000107.01.ff-db c30a1cdefc2d7ef4914b2bfdbf33743c 000108.00.ff-db 6307cac23d9b65d1c95cd695a22d1f26 000108.01.ff-db 0656ffc785f02947576c3b671c50db9b 000109.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000109.01.ff-db 0656ffc785f02947576c3b671c50db9b 000110.00.ff-db 54e464a3c6056f015f7da1e86a0086f8 000110.01.ff-db freq.sh

19:33 HSBI 20 Extraction of Exif > Exiftool > Format: filename,compressedbitsperpixel,quality,make,model,photoshopversion ################ data_set_final ######## 795043.jpg,0.056,,NIKON,d1x,CS2Mac 875003.jpg,0.3072,, ,,3.0 983694.jpg,0.4456,,KODAK,z740,CSWin 800534.jpg,3.24,, ,, 417505.jpg,1.6875,, ,,CSWin 542256.jpg,0.0702,60%, ,,

19:33 HSBI 21 Statistics ########## ffdb_hash_formatted ######### 000107.jpg 00 0656ffc785f02947576c3b671c50db9b ########## ffdb_hash_freq ########### 21784 197abd2b5741a96ff5b83121f760ed52 ################ data_set_final ######## 795043.jpg,0.056,,NIKON,d1x,CS2Mac ############# ffdb_data_freq ##################### 000ac84aa6fc3ae3c76872e55dc50c1d 164933.jpg 0.1 null null null null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 164935.jpg 0.1 null null null null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 173364.jpg 0.0 null null null null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 195389.jpg 0.3 null null null null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 251934.jpg 0.0 null olympus ud600 null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 253354.jpg 8.0 null sony dsc-h9 null 01 388 000ac84aa6fc3ae3c76872e55dc50c1d 255338.jpg 0.2 null panasonic dmc-lz2 null 01 388 join

19:33 HSBI 22 Statistics Distro. Of ~204K q-tables: 1 table/image => 9% 2 tables/image => 89% 3 tables/image => 2%

19:33 HSBI 23 Statistics Unique q-tables = 1,419 of 204K Distro. Of unique q-tables by make

19:33 HSBI 24 Statistics ########### q-table_stats_by_make ############# 21784 197abd2b5741a96ff5b83121f760ed52 sony 21784 197abd2b5741a96ff5b83121f760ed52 seiko 21784 197abd2b5741a96ff5b83121f760ed52 panasonic 21784 197abd2b5741a96ff5b83121f760ed52 olympus 21784 197abd2b5741a96ff5b83121f760ed52 null 21784 197abd2b5741a96ff5b83121f760ed52 nikon 21784 197abd2b5741a96ff5b83121f760ed52 minolta 21784 197abd2b5741a96ff5b83121f760ed52 kodak 21784 197abd2b5741a96ff5b83121f760ed52 hewlett-packard 21784 197abd2b5741a96ff5b83121f760ed52 fujifilm 21784 197abd2b5741a96ff5b83121f760ed52 casio 21784 197abd2b5741a96ff5b83121f760ed52 canon 19561 5524f5067b8475f1e7a29fd90c1d98ef sony 19561 5524f5067b8475f1e7a29fd90c1d98ef seiko 19561 5524f5067b8475f1e7a29fd90c1d98ef panasonic 19561 5524f5067b8475f1e7a29fd90c1d98ef olympus 19561 5524f5067b8475f1e7a29fd90c1d98ef null 19561 5524f5067b8475f1e7a29fd90c1d98ef nikon 19561 5524f5067b8475f1e7a29fd90c1d98ef minolta 19561 5524f5067b8475f1e7a29fd90c1d98ef kodak 19561 5524f5067b8475f1e7a29fd90c1d98ef hewlett-packard 19561 5524f5067b8475f1e7a29fd90c1d98ef fujifilm 19561 5524f5067b8475f1e7a29fd90c1d98ef casio

19:33 HSBI 25 Statistics ########### q-table_stats_by_model ############# 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-w7 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-t100 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-p92 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-p100 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-m1 21784 197abd2b5741a96ff5b83121f760ed52 sony dsc-h9 21784 197abd2b5741a96ff5b83121f760ed52 sony cybershot 21784 197abd2b5741a96ff5b83121f760ed52 sony cd-mavica 21784 197abd2b5741a96ff5b83121f760ed52 seiko photopc-850z 21784 197abd2b5741a96ff5b83121f760ed52 panasonic dmc-lc50 21784 197abd2b5741a96ff5b83121f760ed52 panasonic dmc-fz20 21784 197abd2b5741a96ff5b83121f760ed52 olympus x550 21784 197abd2b5741a96ff5b83121f760ed52 olympus u10d 21784 197abd2b5741a96ff5b83121f760ed52 olympus e-10 21784 197abd2b5741a96ff5b83121f760ed52 olympus c960z 21784 197abd2b5741a96ff5b83121f760ed52 olympus c900z 21784 197abd2b5741a96ff5b83121f760ed52 olympus c5060wz 21784 197abd2b5741a96ff5b83121f760ed52 olympus c5050z 21784 197abd2b5741a96ff5b83121f760ed52 olympus c3040z 21784 197abd2b5741a96ff5b83121f760ed52 null null 21784 197abd2b5741a96ff5b83121f760ed52 nikon e995 21784 197abd2b5741a96ff5b83121f760ed52 nikon e950 21784 197abd2b5741a96ff5b83121f760ed52 nikon e8700 21784 197abd2b5741a96ff5b83121f760ed52 nikon e5700 21784 197abd2b5741a96ff5b83121f760ed52 nikon e4600 21784 197abd2b5741a96ff5b83121f760ed52 nikon e3100 21784 197abd2b5741a96ff5b83121f760ed52 nikon e2000 21784 197abd2b5741a96ff5b83121f760ed52 nikon d50 21784 197abd2b5741a96ff5b83121f760ed52 nikon d200 21784 197abd2b5741a96ff5b83121f760ed52 nikon d1x 21784 197abd2b5741a96ff5b83121f760ed52 nikon d1h 21784 197abd2b5741a96ff5b83121f760ed52 nikon d100 21784 197abd2b5741a96ff5b83121f760ed52 nikon d1 21784 197abd2b5741a96ff5b83121f760ed52 minolta x

19:33 HSBI 26 Statistics ######### q-table_stats_by_ps_version ######### 21784 197abd2b5741a96ff5b83121f760ed52 null 21784 197abd2b5741a96ff5b83121f760ed52 doc 21784 197abd2b5741a96ff5b83121f760ed52 cswin 21784 197abd2b5741a96ff5b83121f760ed52 7.0 21784 197abd2b5741a96ff5b83121f760ed52 5.2 21784 197abd2b5741a96ff5b83121f760ed52 5.0 21784 197abd2b5741a96ff5b83121f760ed52 4.0 21784 197abd2b5741a96ff5b83121f760ed52 3.0 19561 5524f5067b8475f1e7a29fd90c1d98ef null 19561 5524f5067b8475f1e7a29fd90c1d98ef doc 19561 5524f5067b8475f1e7a29fd90c1d98ef cswin 19561 5524f5067b8475f1e7a29fd90c1d98ef 7.0 19561 5524f5067b8475f1e7a29fd90c1d98ef 5.2 19561 5524f5067b8475f1e7a29fd90c1d98ef 5.0 19561 5524f5067b8475f1e7a29fd90c1d98ef 4.0 19561 5524f5067b8475f1e7a29fd90c1d98ef 3.0

19:33 HSBI 27 Statistics No camera or editing s/w has its specific q-table.

19:33 HSBI 28 Statistics ######### h-table_stats_by_make ############## 80402 50a73d7013e9803e3b20888f8fcafb15 vivitar 80402 50a73d7013e9803e3b20888f8fcafb15 sound 80402 50a73d7013e9803e3b20888f8fcafb15 sony 80402 50a73d7013e9803e3b20888f8fcafb15 seiko 80402 50a73d7013e9803e3b20888f8fcafb15 samsung 80402 50a73d7013e9803e3b20888f8fcafb15 research 80402 50a73d7013e9803e3b20888f8fcafb15 pentax 80402 50a73d7013e9803e3b20888f8fcafb15 panasonic 80402 50a73d7013e9803e3b20888f8fcafb15 panaso 80402 50a73d7013e9803e3b20888f8fcafb15 olympus 80402 50a73d7013e9803e3b20888f8fcafb15 null 80402 50a73d7013e9803e3b20888f8fcafb15 nikon 80402 50a73d7013e9803e3b20888f8fcafb15 newsoft 80402 50a73d7013e9803e3b20888f8fcafb15 msm6100 80402 50a73d7013e9803e3b20888f8fcafb15 motorola 80402 50a73d7013e9803e3b20888f8fcafb15 minolta 80402 50a73d7013e9803e3b20888f8fcafb15 logitech 80402 50a73d7013e9803e3b20888f8fcafb15 kyocera 80402 50a73d7013e9803e3b20888f8fcafb15 konica 80402 50a73d7013e9803e3b20888f8fcafb15 kodak 80402 50a73d7013e9803e3b20888f8fcafb15 jvc 80402 50a73d7013e9803e3b20888f8fcafb15 hp 80402 50a73d7013e9803e3b20888f8fcafb15 hewlett-packard 80402 50a73d7013e9803e3b20888f8fcafb15 fujifilm 80402 50a73d7013e9803e3b20888f8fcafb15 fuji 80402 50a73d7013e9803e3b20888f8fcafb15 casio 80402 50a73d7013e9803e3b20888f8fcafb15 canon 80402 50a73d7013e9803e3b20888f8fcafb15 asahi 42616 1274c9b90c56e7f0e589412cbad79aa2 vivitar 42616 1274c9b90c56e7f0e589412cbad79aa2 sound 42616 1274c9b90c56e7f0e589412cbad79aa2 sony 42616 1274c9b90c56e7f0e589412cbad79aa2 seiko 42616 1274c9b90c56e7f0e589412cbad79aa2 samsung 42616 1274c9b90c56e7f0e589412cbad79aa2 research 42616 1274c9b90c56e7f0e589412cbad79aa2 pentax 42616 1274c9b90c56e7f0e589412cbad79aa2 panasonic 42616 1274c9b90c56e7f0e589412cbad79aa2 panaso 42616 1274c9b90c56e7f0e589412cbad79aa2 olympus 42616 1274c9b90c56e7f0e589412cbad79aa2 null

19:33 HSBI 29 Statistics ######### h-table_stats_by_model ############## 80402 50a73d7013e9803e3b20888f8fcafb15 canon ixy--50 80402 50a73d7013e9803e3b20888f8fcafb15 canon ixy--200a 80402 50a73d7013e9803e3b20888f8fcafb15 canon ixy 80402 50a73d7013e9803e3b20888f8fcafb15 canon ixus-400 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos--rebel-xt 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos--rebel 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-kiss 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-d60 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-5d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-40d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-350d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-30d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-20d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-1d-mark-ii 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-1d 80402 50a73d7013e9803e3b20888f8fcafb15 canon eos-10d 80402 50a73d7013e9803e3b20888f8fcafb15 asahi optio-330 42616 1274c9b90c56e7f0e589412cbad79aa2 vivitar vivicam-3715 42616 1274c9b90c56e7f0e589412cbad79aa2 sound null 42616 1274c9b90c56e7f0e589412cbad79aa2 sound . 42616 1274c9b90c56e7f0e589412cbad79aa2 sony picturegear 42616 1274c9b90c56e7f0e589412cbad79aa2 sony mvc-cd500 42616 1274c9b90c56e7f0e589412cbad79aa2 sony mavica 42616 1274c9b90c56e7f0e589412cbad79aa2 sony fdmavica 42616 1274c9b90c56e7f0e589412cbad79aa2 sony dslr-a100 42616 1274c9b90c56e7f0e589412cbad79aa2 sony dsc-w7 42616 1274c9b90c56e7f0e589412cbad79aa2 sony dsc-w1 42616 1274c9b90c56e7f0e589412cbad79aa2 sony dsc-v3 42616 1274c9b90c56e7f0e589412cbad79aa2 sony dsc-v1

19:33 HSBI 30 Statistics ####### h-table_stats_by_ps_version ######### 80402 50a73d7013e9803e3b20888f8fcafb15 null 80402 50a73d7013e9803e3b20888f8fcafb15 ele5.0 80402 50a73d7013e9803e3b20888f8fcafb15 ele2.0 80402 50a73d7013e9803e3b20888f8fcafb15 doc 80402 50a73d7013e9803e3b20888f8fcafb15 cswin 80402 50a73d7013e9803e3b20888f8fcafb15 csmac 80402 50a73d7013e9803e3b20888f8fcafb15 cs3mac 80402 50a73d7013e9803e3b20888f8fcafb15 cs2win 80402 50a73d7013e9803e3b20888f8fcafb15 cs2mac 80402 50a73d7013e9803e3b20888f8fcafb15 alb2.0 80402 50a73d7013e9803e3b20888f8fcafb15 7.0 80402 50a73d7013e9803e3b20888f8fcafb15 5.2 80402 50a73d7013e9803e3b20888f8fcafb15 5.0 80402 50a73d7013e9803e3b20888f8fcafb15 4.0 80402 50a73d7013e9803e3b20888f8fcafb15 3.0 42616 1274c9b90c56e7f0e589412cbad79aa2 null 42616 1274c9b90c56e7f0e589412cbad79aa2 ele5.0 42616 1274c9b90c56e7f0e589412cbad79aa2 ele2.0 42616 1274c9b90c56e7f0e589412cbad79aa2 doc 42616 1274c9b90c56e7f0e589412cbad79aa2 cswin 42616 1274c9b90c56e7f0e589412cbad79aa2 csmac 42616 1274c9b90c56e7f0e589412cbad79aa2 cs3mac 42616 1274c9b90c56e7f0e589412cbad79aa2 cs2win

19:33 HSBI 31 Statistics No camera or editing s/w has its specific h-table either !

19:33 HSBI 32 Summary Q-table or H-table to identify source camera not viable approach. - No co-relation between q/h-table & make or s/w. Q-table or H-table determined by: - quality factors Resolution, RGB, intensity etc. - standards followed by camera or s/w - compression type, ratio

19:33 HSBI 33 References [1] JPEG Quantization Ballistics Revisited, Dr. Vassil Roussev,NeFX 2010 [2] Digital Image Ballistics from JPEG Quantization: A Followup Study, TR2008-638, H. Farid, Dartmouth College, Computer Science [3] itu-t38.pdf [4] http://fileformat.info/format/jpeg/egff.htm [5] http://park2.wakwak.com/~tsuruzoh/Computer/Digicams/ exif-e.html [6] http://impulseadventure.com

19:33 HSBI 34 Thank You for your co-op.