Project Base Paper

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

WK,QWHUQDWLRQDO&RQIHUHQFHRQ&RPPXQLFDWLRQ6\VWHPV

1HWZRUNV &2061(76

Decentralised Image Sharing and Copyright


Protection using Blockchain and Perceptual Hashes
Rishabh Mehta Naman Kapoor, Soumya Sourav Rajeev Shorey
Indian Institute of Technology Delhi Delhi Technological University TCS Research & Innovation
rmehta.visitor@iitd.ac.in {nk2016ti, s.sourav15}@gmail.com rajeev.shorey@tcs.com

Abstract—In this paper, we aim to address the major prob- they are unable to identify manipulated versions of original
lem of correct image attribution for photos being uploaded images. This happens because cryptographic hashes are prone
on the stock photo websites and peer-to-peer image sharing to avalanche effect, a phenomenon where a small change in
marketplaces, ensuring that original photographers are rightfully
recognized and credited for their works. There are now millions input value leads to drastic change in output value. We address
of royalty-free images being shared every week on stock photo these problems in this paper by creating an efficient Ethereum
websites. There is a major discontent among photographers and based solution for image sharing and opt for perceptual hashes
graphics designers who share images on such websites regarding [6]. Perceptual hashes keep away from avalanche effect and
a widespread malpractice where others try to take credit for and are less susceptible to perceptual changes in images such as
benefit from original author’s work. This happens when someone
other than the original author uploads the same image as one rotation, cropping, re-sizing, conversion to gray scale, salt
already shared on the marketplace or uses image processing and pepper noise, exposure or saturation change etc. The
methods such as rotation, cropping, gray scale conversion etc and paper demonstrates how using a Blockchain network coupled
then uploads this tampered image to avoid copyright violation with use of perceptual hashes and appropriate smart contract
detection. To counter such problems, we propose a decentralised logic can provide better mechanism for copyright violation
peer-to-peer photo sharing marketplace built on top of Ethereum
test chain and demonstrate how it is fair, trustworthy and detection than current systems along with many benefits of
practical. Our decentralised application leverages perceptual Blockchain such as permanent record of trade, transparency,
hashes and robust smart contracts of Ethereum to automati- trust, immutability, high availability, information security and
cally detect and reject tampered images that are perceptually cost savings.
similar to images already present on the marketplace. Due
to inherent properties of Blockchain, our marketplace has no A. Motivation
central authority controlling it, no third party interference, no
single point of failure, zero-censorship and preserves online user The fundamental motivation behind proposing a decen-
privacy. To the best of our knowledge, this is the first work tralised image sharing marketplace is to correct image attribu-
that provides a practical solution for automatically detecting and tion and provide due credit to original image authors, which
rejecting perceptually similar images on a decentralised image is a major concern for photographers who share images on
sharing platform, thus protecting the copyrights of genuine image
authors. stock photo websites. We aim to put in place robust yet fast
Index Terms—Blockchain, Decentralised Image Sharing, Im- copyright violation detection mechanism. This when combined
age Copyright protection, Image Sharing Marketplace with benefits of Blockchain based architecture can put the
power back in the hands of image contributors. The high
I. I NTRODUCTION commission rates on stock photo websites, which may run
In the current era of internet, many photographers now earn up to 55%, is also a common concern in the photography
their income by selling their images online, often through community. We aim to get rid of these hefty commissions
stock photo agencies. The royalty-free and microstock busi- entirely and create an architecture where the user has to
ness models on such websites have led to intense compe- pay only for transaction charges related to processing of
tition among photographers where everyone is striving to transaction by the Blockchain network.
upload more images and get more downloads for their works.
More recently, this behaviour has led to a widespread image B. Novelty
attribution fraud where some non-authors to get credit for The novel contributions of the project include employing
other contributor’s images, in violation of copyright laws and a programmable Blockchain Ethereum [1], [4] to create a
affecting rightful image owner mentally and financially. They decentralised image sharing application, with the novel usage
accomplish this by accessing images of original author from of perceptual hashes and Ethereum’s robust smart contracts
stock photo websites and uploading thus obtained images [12], [15] for copyright violation detection. The photos on
as it is or after manipulating some of the image features the network are saved in a decentralised fashion through
on stock photo websites. While some stock photo websites InterPlanetary File System (IPFS). We also provide compu-
do check whether the new image being uploaded is already tational results showcasing the effectiveness as well as some
present on the marketplace by usage of cryptographic hashes, limitations of our approach.

k,(((
II. S YSTEM E NVIRONMENT AND A RCHITECTURE
We use the following tools and technologies to build our 1) Eth Store: In this architecture images and hashes are
system. Detailed information about each can be found on their stored on the Ethereum chain itself. Images and per-
respective websites and corresponding Github repositories. ceptual hashes are placed in Blockchain transactions.
• Metamask: Metamask is a plugin for browsers which The problem with this method is that as more and more
allows to make Ethereum transactions. It allows to run images get uploaded, the storage size of Blockchain will
Ethereum Dapps (Decentralised applications) without increase rapidly since all the content is stored on the
having to run the full Ethereum node, manage identities testchain. Whenever new nodes arrive, they’ll have to
and sign Blockchain transactions. download the entire chain containing all the images. This
• Ganache: Ganache is an Ethereum Blockchain emulator. solution is definitely not scalable, especially since image
Ganache is in-memory Ethereum node that allows to test sizes may run into many Megabytes.
all components locally. It also lets users deploy contracts, 2) IPFS Store: In this architecture the images and hashes
develop application and run tests. are stored with Interplanetary file system (IPFS), a
• Geth: Geth (Go-Ethereum) is a command line interface decentralised peer-to-peer hypermedia distribution pro-
which lets users run and operate full time Ethereum node. tocol. IPFS [5] is content addressable and uses cryp-
It is implemented in Go and allows to mine blocks to tographic hashes to give unique fingerprint to blocks
generate Ether, create smart contracts, send transactions within files, removes redundancy across the network
to transfer funds between Ethereum addresses, inspect and uses these unique hashes for information retrieval.
block history and many other functions. By storing images and perceptual hashes with the IPFS
• Truffle: Truffle is used to compile, test, build smart con- protocol, we make large savings in size of the Ethereum
tracts and provides a development framework to increase test chain. Data stored on IPFS can be encrypted with
speed in the development process. any standard encryption scheme and retrieved. The en-
The system architecture is shown in detail in Figure 1. cryption keys will only be shared with appropriate buyer
Various modules shown in the figure are described below: via smart contract. This approach is scalable and is the
one used in our implementation.
• Buyer module: It offers searching tools to find the class of
images buyer is looking for among all the images present A. Copyright infringement and computer vision
on the decentralised application. Perceptual hashes are multimedia fingerprinting algorithms
• Seller module: It provides simple way to upload the that take into account various features of the multimedia. It’s
image on the Blockchain. Seller uploads the image, along very essential for an online multimedia marketplace to ensure
with image name and description to help the searching that each work gets uploaded only once. Cryptographic hash
process. functions such as MD5 and SHA can do this task efficiently
• Wallet Module: This client side encrypted module allows by comparing the new image hash with hashes of all images
to create new Ethereum address or import existing ad- on Blockchain and check for a match. But we go a step further
dress, see wallet balance and transactions. Users can al- and also try to identify works that seem to be modifications
ternatively use Metamask to manage accounts and wallet. of the original works already present on the Blockchain and
As can be seen from Figure 1, Ethereum gate connects with are perceptually similar to these original works. For example
the web client and allows access to Ethereum ecosystem. Web if someone tries to upload slightly rotated picture, say 12°
client consists of buyer, seller and wallet modules. Transaction anticlockwise, then due to avalanche effect hashes such as
module is used to specify image selling price while image SHA will have totally different final hash. Instead if we use
upload module is used to upload new image. A local database perceptual hash, only some or none of the bits in final hash
stores this data and after the transaction is approved by will change. We can use this property of perceptual hashes to
Ethereum network, the data and image is transferred to IPFS find similarity between images. The way this is achieved is
after due encryption. We use public key cryptography for by calculating dissimilarity between original image and test
encryption. Since IPFS is content addressable, this encryption image. To quantify dissimilarity, find perceptual hash of both
step is crucial for information protection. Ethereum gate con- images and calculate hamming distance between thus obtained
sists of Geth, parity and JSON-RPC (Remote procedure call) hashes. Normalize this hamming distance between 0 and 1.
protocol. Ethereum gate provides access to main Ethereum This is the measure of dissimilarity between original and test
network which consists of all user nodes, miners, smart images and is named NHD. If this dissimilarity is within a
contracts and transaction blocks. Data flow among all modules certain threshold, test image will get rejected. But sometimes
is shown in figure. IPFS and Ethereum network for information two distinct and original images can also have near-similar
storage purposes are explained in next section. perceptual hashes, leading to small dissimilarity value. So the
critical task here is to find the threshold such that we can
III. M ETHODOLOGY identify between two distinct images and two perceptually
We present two different architectures which differ in the similar images as correctly as possible. We describe the
way they store data. method to obtain threshold in the result section.
Fig. 1. System architecture

There are many perceptual hashing algorithms, but we use four seen as penalty. Thus this financial penalty would discourage
most popular ones, namely phash (dct hash) ,difference hash, fraudulent behaviour.
average hash and wavelet hash. These algorithms are included
in ImageHash library in python where detailed documentation B. Smart contracts and Information flow
about these algorithms has been provided. So if someone
The information flow is shown in system architecture di-
uploads a tampered version of original image present on
agram in Figure 1. Smart contracts are at the heart of the
marketplace, smart contract will compute its perceptual hash,
Ethereum ecosystem. Smart contracts on Ethereum are little
compute hamming distance of this hash and hashes stored on
pieces of code that live on the Blockchain. The concept of
decentralised PERC HASH database and upon due comparing
smart contract was first introduced by Szabo [3]. They are
with threshold, will reject such image. Thus not only the
used to manage agreements between users. They can hold
fraudsters wouldn’t be able to upload such images, they will
Ether (ETH), release it to appropriate entity, hold information
also have paid transaction charges associated for processing
and provide utility to other smart contracts. Smart contracts
that transaction on the Ethereum network, which can be
are used to perform certain computations, store some variables
Fig. 2. Images from left to right: classic image barbara, barbara rot5, barbara rot25, barbara rot90, barbara exp-50, barbara sat+50.

TABLE I
ACCEPTANCE RESULTS ON CLASSIC BARBARA IMAGE AND ITS TRANSFORMATIONS .

Image WHASH (Hexadecimal) Hamming Distance Dissimilarity (NHD) Acceptance result


barbara.png a938f9fad3120d0d - - Accepted
barbara rot5.png 383afafaf3120d14 10 0.156 Rejected
barbara rot25.png 347476faf2da4608 24 0.375 Rejected
barbara rot90.png ab1c03f37cf038b8 30 0.469 Accepted
barbara exp-50.png a938f9fad3120d0d 0 0.000 Rejected
barbara sat+50.png e938f9fa93120d0d 2 0.031 Rejected

and trade digital goods while adhering to certain specified con- transformations of the same image.
ditions mentioned in the contract. Thus one can trade digital
assets over Ethereum network with self-defined conditions that We perform following transformations on each image
are specified in these contracts. Due to these qualities, they are from BSDS-500 dataset:
suitable for including image licensing and copyright protection • Rotation (in degrees): 5°, 10° ,15° ,20° ,25° (clockwise)
logic [12]. We use solidity [10] to write smart contracts, which • Gamma correction: 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5
is by far the most popular language for writing smart contracts.
• Salt and pepper noise: 0.05, 0.10, 0.15, 0.20, 0.25, 0.30,
We follow best practices for creating such contracts [16] and
0.35, 0.40, 0.45, 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80,
use lightweight algorithms such as wavelet hash (WHASH)
0.85, 0.90, 0.95, 1.00
for providing copyright protection, thus requiring less gas
amount [13]. All our transactions pass through smart contracts. • Crop (fixed aspect ratio mode): 5%, 10%, 15%, 20%,

We use smart contracts to calculate perceptual hashes, dis- 25%


tribute encryption and decryption keys to appropriate entities, • Resize (size reduction - fixed aspect ratio mode): 5%,
connect with IPFS, check for copyright violation and accept 10%, 15%, 20%, 25%
or reject images. After quantifying the image dissimilarity After these transformations, we end up with a total of
between images present on marketplace and newly uploaded 21500 images (500 original images and 42 transformations
image by calculating Normalized Hamming Distance (NHD), of each original image). To evaluate effectiveness of hashing
smart contract rejects images that have NHD below a certain algorithms, we create two cases:
threshold as can be seen from Table 2 and Figure 2. Here Positive Case: The images falling in this case are manipu-
WHASH is able to identify all transformations of classic lated versions of the original image present on the marketplace.
barbara image except 90° rotated version. For determining We expect the hashing algorithms to detect such fraudulent
acceptance or rejection of image, the threshold for WHASH is images and smart contract should reject them right away.
taken as 0.379 as calculated in result section. This shows both Negative Case: The images falling in this case are two
the level of effectiveness and reliability of perceptual hashes. distinct original images from the dataset. The newly uploaded
Perceptual hashes are able to detect tampering in the events distinct image should be allowed by the network to be added
where modifications made to images are not too extreme and on the marketplace for sale.
there’s still good level of structural similarity between original
image and its tampered version. Hamming Distance is the number of different bits between
two hashes. Therefore the hamming distance carries with it
IV. R ESULTS
an implicit sense of dissimilarity. We calculate the hamming
We test our model on a dataset of natural images and the distance between new image and all the previous images
transformations of these images for the purpose of evaluating uploaded on the Blockchain, normalize the distance value
effectiveness of perceptual hashes in detection of copyright and compare that to threshold in order to make a decision
violation. We use Berkeley Segmentation DataSet (BSDS) whether to accept or reject the new image. This way new
[7], which is the dataset of 500 natural images. The effective image is checked against all images that are being traded on
hashing algorithm for real life application would be the the marketplace. We calculate Normalized Hamming Distance
one that can distinguish between two authentic images and (NHD) as follows:
Fig. 3. The effectiveness of a particular perceptual hashes to classify positive and negative cases correctly. Here X-axis refers to NHD and Y-axis refers to
fraction of total images.

object detector on a particular dataset. Smaller value of IoU


HammingDistance is desired.
N HD =
HashBitsCount Threshold: Threshold value is the magnitude if not
exceeded by the Normalized Hamming Distance (NHD), then
All four hashing algorithms that we consider return 64 bits the image will get rejected. Therefore if NHD is greater than
hashes. threshold then dissimilarity is above the required value and
Following metrics are used to evaluate the quality of a image gets accepted on the marketplace. To select an optimal
perceptual hash function: threshold, focus on Figure 3. When the negative case bar first
crosses the positive bar, the corresponding scaled distance
Earth Movers Distance: Earth Mover’s Distance (EMD) is a value is taken as threshold.
measure of the distance between two probability distributions
over a region. EMD is based on minimum cost that must Figure 3 shows the graphical comparison of phash, dhash,
be paid to transform one distribution into another [9]. Larger average hash and wavelet hash respectively. Here the X-axis
value of EMD is desired. denotes hamming distance scaled between 0 to 1 while Y-
Intersection-Over-Union: Intersection over Union (IoU) is axis denotes the probability of image falling into positive or
an evaluation metric used to measure the accuracy of an negative case. We would prefer to use a perceptual hash-
TABLE II
C OMPARISON OF EFFECTIVENESS OF VARIOUS PERCEPTUAL HASHES CALCULATED ON BSDS-500 DATASET

Metric PHASH DHASH WHASH AVG HASH


EARTH MOVER DISTANCE 0.322 0.329 0.317 0.328
INTERSECTION OVER UNION 0.276 0.280 0.260 0.220
THRESHOLD VALUE 0.353 0.400 0.379 0.304

ing algorithm that gives minimum overlap between positive transformations, there are some cases for which hashing algo-
and negative case distributions. This can be calculated with rithms fails to identify fraudulent image as we saw with 90°
Intersection-Over-Union metric. We would prefer the algo- rotated barbara image in Table 1. But in general, perceptual
rithm that has highest Earth Mover Distance between positive hashes provide a great trade-off between accuracy and time
and negative distributions. complexity.
As can be seen from Table 2, all four hashing algorithms We intend to work on the following tasks in the future:
give comparable values on the metrics. Average hash and In order to close loopholes such as one explained with
Dhash have higher EMD values and average hash has the Figure 2 and Table 1, we need to study and employ more
least IoU value. The threshold for average hash comes out robust and efficient Computer Vision techniques than percep-
to be 0.304. This means if the new image’s average hash has tual hashes, if there are any.
more than 30.4% bits similar with average hash of an image Turning this Decentralised app (DApp) into full-fledged
present on marketplace, then new image will be approved by Distributed Autonomous Organization (DAO) which could
the network and added to marketplace. also provide claim assistance, decentralised voting on claims,
The comparison of average time for hashing is as follows: rewards management and remuneration for copyright breach.
• PHASH: 33.97 ms R EFERENCES
• DHASH: 29.93 ms
[1] G. Wood, ”Ethereum: A secure decentralised generalised transaction
• AVG HASH: 29.81 ms ledger”, Ethereum Project Yellow Paper, 2014.
[2] S. Nakamoto, ”Bitcoin: A peer-to-peer electronic cash system”, White
• WHASH: 60.99 ms paper, 2008.
[3] Nick Szabo, ”The idea of smart contracts,” 1997.
We’d like to point out that the the amount of time that it takes [4] Buterin V, ”A next-generation smart contract and decentralized applica-
to verify a new image is directly proportional to the number of tion platform”, White Paper, 2014.
images that have been uploaded on the marketplace. In other [5] Juan Benet, ”IPFS - Content Addressed, Versioned, P2P File System
(DRAFT 3)”. https://github.com/ipfs/papers, 2017.
words, time complexity of new image verification is linear. [6] C. Zaunerm, ”Implementation and benchmarking of perceptual image
hash functions”, PhD thesis, University of Applied Sciences Hagenberg,
V. C ONCLUSION Austria, 2010.
[7] P. Arbelaez, M. Maire, C. Fowlkes and J. Malik, ”Contour Detection
We conclude that Blockchain based solution for online stock and Hierarchical Image Segmentation”, IEEE TPAMI, Vol. 33, No. 5,
photo marketplace provides many benefits over traditional pp. 898-916, May 2011.
[8] V. Monga and B. L. Evans, ”Perceptual image hashing via feature
marketplaces such as high level of security, permanent record points: performance evaluation and tradeoffs”, Image Processing, IEEE
of trade, custom licenses, custom selling price for image, Transactions on, 15(11):34523465, 2006.
higher revenue for contributors and permits distributed storage [9] R Ubner Y., Tomasi C., Andguibas L. J., ”The earth movers distance as
a metric for image retrieval”, Int. J. Comput. Vision 40, 99121, 2000.
of images with the help of IPFS. Due to the absence of [10] Christian Reitwiessner and Gavin Wood, ”Solidity”, White Paper, 2015.
central authority, the marketplace is free from censorship [11] V. Buterin, ”On public and private blockchains”,
and interference. After thorough evaluation, we conclude that https://blog.ethereum.org/2015/08/07/on-public-and-private-
blockchains, 2015.
the perceptual hashes are effective and efficient copyright [12] V. Morabito, ”Smart contracts and licensing”, in Business Innovation
infringement detection tools. The Blockchain transactions only Through Blockchain, pp. 101-124, Springer, 2017.
contain IPFS hashes for retrieving images. By encrypting [13] T. Chen, X. Li, X. Luo, and X. Zhang, ”Under-optimized smart contracts
devour your money”, in 2017 IEEE 24th International Conference on
content beforehand in the smart contract with a choice of Software Analysis, Evolution and Reengineering (SANER), pp. 442-446,
an encryption scheme and then transferring data to IPFS, IEEE, 2017.
information security is enforced. The smart contract releases [14] A. Bogner, M. Chanson, and A. Meeuw, ”A decentralised sharing app
running a smart contract on the ethereum blockchain,” in Proceedings of
decryption keys only to appropriate buyer. Furthermore, our the 6th International Conference on the Internet of Things, pp. 177-178,
implementation is scalable. Thus the paper illustrates a novel ACM, 2016.
use-case of a proven technology to help fight a deep rooted [15] L. Luu, D.-H. Chu, H. Olickel, P. Saxena, and A. Hobor, ”Making
smart contracts smarter,” in Proceedings of the 2016 ACM SIGSAC
malicious practice in the stock-photo world. Conference on Computer and Communications Security, CCS ’16, pp.
254-269, ACM, 2016.
VI. L IMITATIONS AND F UTURE W ORK [16] K. Delmolino, M. Arnett, A. Kosba, A. Miller, and E. Shi, ”Step
by step towards creating a safe smart contract: Lessons and insights
Our system can detect fraudulent images that are per- from a cryptocurrency lab,” in International Conference on Financial
ceptually similar to original images to a reasonable extent. Cryptography and Data Security, pp. 79-94, Springer, 2016.
While perceptual hashes are very effective in detecting many

You might also like