Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Open access

Semantics and Non-fungible Tokens for Copyright Management on the Metaverse and Beyond

Published: 27 March 2024 Publication History

Abstract

Recent initiatives related to the Metaverse focus on better visualization, like augmented or virtual reality, but also persistent digital objects. To guarantee real ownership of these digital objects, open systems based on public blockchains and Non-Fungible Tokens (NFTs) are emerging together with a nascent decentralized and open creator economy. To manage this emerging economy in a more organized way, and fight the so common NFT plagiarism, we propose CopyrightLY, a decentralized application for authorship and copyright management. It provides means to claim content authorship, including supporting evidence. Content and metadata are stored in decentralized storage and registered on the blockchain. A token is used to curate these claims, and potential complaints, by staking it on them. Staking is incentivized by the fact that the token is minted using a bonding curve. The tokenomics include the resolution of complaints and enabling the monetization of curated claims. Monetization is achieved through licensing NFTs with metadata enhanced by semantic technologies. Semantic data makes explicit the reuse conditions transferred with the token while keeping the connection to the underlying copyright claims to improve the trustability of the NFTs. Moreover, the semantic metadata is flexible enough to enable licensing not just in the real world. Licenses can refer to reuses in specific locations in a metaverse, thus facilitating the emergence of creative economies in them.

1 Introduction

Creative industries have been dealing with profound changes since the introduction of digital technologies and communication networks, which facilitate content creation but also copying and uncontrolled distribution. Blockchain technologies reintroduce scarcity, so it is possible to manage unique assets and their ownership through Non-Fungible Tokens (NFTs) [7], but they do not guarantee authenticity.
Thus, serious problems with fake NFTs and plagiarism have emerged, where a new NFT collection of unique tokens points to copied pieces of content. Just like a copied painting with a forged certificate of authenticity, but much easier to fake. And the issue is even more acute in completely digital spaces like metaverses, where emerging creative economies might be hampered by the lack of some sort of authorship recognition.
Classical copyright law can help here, but it should be integrated and scaled to these emerging digital spaces while benefitting from the new opportunities offered by these technologies. Our proposal, presented in this article, is CopyrightLY, a decentralized application [5] that leverages blockchain [31] and semantic web [29] technologies to facilitate copyright management. A Proof of Concept (PoC) has been developed as part of the European Next Generation Internet ONTOCHAIN project,1 whose aim is at creating an ecosystem of blockchain-based knowledge management solutions.
In this context, CopyrightLY contributes a content ownership and a copyright management layer. One of its more direct usage scenarios, detailed in Section 1.1 is being integrated with existing social media platforms and allowing content creators to explore ways to exploit their novel media beyond those made possible by those platforms, especially through NFTs and even across metaverses.
CopyrightLY’s PoC provides a working blockchain-based copyright registry, where creators can make authorship claims linked to a hash of their content and a timestamp. They can then strengthen their claims by adding supporting evidence, like pictures of the creative process, witnesses recordings, or proofs of previous appearances of the creation on social media platforms. CopyrightLY also provides the means to curate authorship in a scalable way, through token-based incentives and the wisdom of the crowds. As a last resort, all accumulated evidence, both for authorship claims and complaints, can be used in the context of arbitration systems or even in court.
Additionally, CopyrightLY showcases the use of Semantic NFTs. And NFT is a unique and non-interchangeable token linked to metadata whose ownership is recorded in a blockchain. Blockchain transactions are used to transfer ownership and metadata usually points to the media the NFT represents. Semantic NFTs have their metadata enriched using semantics and formal knowledge representation languages based on ontologies. CopyrightLY uses them as the mechanism to license the reuse of the registered content. This way, a Semantic NFT is linked to an authorship claim and an unambiguous representation of the terms licensed to the NFT owner. Moreover, the semantic metadata can be used to automate copyright reasoning and assist users during content reuse. The ontologies being used are flexible enough to accommodate reasoning in different domains, including licensing in a metaverse.
The rest of the article is organized as follows. Section 2 presents the state-of-the-art regarding blockchain technologies for copyright management, including how it might be integrated into a metaverse architecture. Then, Section 3 presents the proposed approach focusing on its architecture. Highlights of CopyrightLY’s implementation are provided in Section 4, including token-based incentives to curate authorship claims and the use of Semantic NFTs to unambiguously capture the licensing terms. Finally, Section 5 presents the conclusions and Section 6 the future work.

1.1 Sample Usage Scenario

A typical CopyrightLY usage scenario is depicted in Figure 1 using a use case diagram. In this case, a YouTube video creator is willing to explore alternative ways to monetize her content. In addition to publishing the video on YouTube, she registers her authorship claim by uploading the video file to an immutable data store (off-chain) and registers its content hash using a blockchain transaction (on-chain). This produces a time-stamped and immutable authorship claim. The claim links the content through its hash (which can be used to retrieve the content), the blockchain identity used by the creator (linked to a private key used to sign the claim), and a timestamp.
Fig. 1.
Fig. 1. Typical usage scenario for a YouTube video creator using CopyrightLY as an alternative monetization mechanism through Non-fungible Tokens.
The creator can then strengthen her claim by linking it to her YouTube video, and the social media profile and reputation that she might have built there. As shown in Figure 1, ownership of the social media platform version is proven by including the authorship claim content hash into the YouTube video description. The presence is then checked through a blockchain oracle [19], a trusted service that brings data from other information systems to the blockchain. In this case, the oracle consumes the YouTube API, verifies that it includes the authorship claim hash, and registers on-chain the YouTube video as supporting evidence for the claim. The logic is similar to the one used to verify that someone has a particular e-mail, sending to it a link that, when followed, is used to prove control over that e-mail account.
To help curate these authorship claims and accompanying evidence, CopyrightLY includes mechanisms like the CLY token, complaints, and dispute resolution, as detailed in Section 4.1 and illustrated in Figure 3. Finally, if the authorship claim is not challenged, the content creator can start monetizing her claimed copyright through NFTs, which are linked to the authorship claim, YouTube evidence, and licensing terms. By following the ERC-721 standard,2 NFTs can be integrated into existing NFT marketplaces and become tradable right away, so they can be offered for a fixed price or through different kinds of auctions. They can be also reused in the context of blockchain-powered metaverses, for instance, displayed on a virtual gallery. And potential purchasers or users can trace the NFT or YouTube video back to the underlying copyright claim and reuse terms to verify them.

2 State-of-the-Art

Blockchain technologies are being applied to almost any conceivable domain, from logistics [9] to renewable energy production management [22]. In all cases, in addition to the technologies specific to the distributed ledger itself, developers face issues related to information management and integration similar to those faced by Web applications. In this regard, Semantic Web technologies are well-positioned to solve this kind of issue, and have been even used to model the key concepts in one of the main distributed ledgers, Ethereum’s EthOn [26].
However, attempts to apply blockchain technologies for completely decentralized copyright registration and monetization are quite recent and there is little literature regarding proposals similar to CopyrightLY’s. One similar proposal [20], which also aims at facilitating copyright protection of social media content, proposes a blockchain framework with smart contracts to protect social media contents using InterPlanetary File System (IPFS), a decentralized and content-addressable peer-to-peer hypermedia storage protocol [3]. Content uploaded to IPFS is securely stored using a secret sharing scheme. This is combined with a robust hash for images, a method of hashing images that is resistant to modification, rotation, and color alteration. The objective is to make it possible to detect near copies and block the registration of images that might be copies of previously registered ones. Unfortunately, no further information is provided about the robustness of this algorithm and its implications from a legal standpoint when infringements are not properly detected or uninfringing content is considered otherwise. Moreover, though the use of smart contracts is mentioned as the way to ask permission to reuse registered images, no further details are provided about these smart contracts or the way reuse terms are negotiated and then stored on-chain to provide trust in those agreements.
Another similar proposal [8] is a system to fight intended or accidental image copyright infringement on social media platforms, mainly when professional images are used to increase the impact of posts. Photographers can use it to register their photos and re-users can use it to check if the image they want to use is copyright protected or not. The main result of [8] is an algorithm that can extract a signature that is resistant to different levels of JPEG compression. The signature is stored on the blockchain along with the identification data of the copyright owner. It can be then used to detect copies when someone tries to register the same or a similar image, as determined by the algorithm. The algorithm can be also used by re-users to check if an image is already registered. In that case, the system allows the purchase of the right to use the photo. However, [8] does not provide mechanisms to deal with situations when the registration that is considered a copy is in fact the original one, giving rise to potential complaints. Moreover, the article focuses just on the detection mechanism and few details are provided about how registrations are stored on-chain, or how reuses are negotiated and managed using the smart contracts that are mentioned. Approaches based on watermarking algorithms that link the copyright information with the content file [25] experience the same kind of limitations, as there is no prior mechanism to ensure that the watermark is embedded by the actual creator or rights holder.
As will be detailed in the next section, the main innovation of CopyrightLY, and what distinguishes it most from blockchain-based copyright management proposals like the previous ones, is that it does not try to enforce what is an infringing authorship claim. As shown by previous experiences with Digital Rights Management (DRM) systems [21], a rigid and technologically enforced approach might put this kind of system outside copyright law. For instance, blocking fair use or categorizing as copyright infringement registration attempts that might be done by the original creator.
CopyrightLY also contributes beyond existing initiatives in the blockchain space, which just provide some of CopyrightLY’s features and do not support the whole copyright value chain in a decentralized manner. Regarding copyright registration, the most similar initiative, because it can be also used in case of legal litigation, is WIPO Proof.3 However, WIPO Proof by the World Intellectual Property Organisation just supports authorship claims, and not adding supporting evidence using Oracles or a decentralized curation mechanism. On the other hand, Snapmuse.io4 also helps YouTube creators explore new revenue streams using NFTs, but it is limited just to upfront funding based on future advertising revenue. Thus, it lacks all the licensing alternatives offered by CopyrightLY’s Semantic NFTs. Moreover, it also contributes beyond existing initiatives using NFTs to license content like Froopyland,5 which lacks a way of rooting their NFTs on authorship claims that can be verified and used in case of litigation.
Finally, in the yet scarce literature about the metaverse and blockchain technologies, we have been unable to find any previous work directly connected with copyright management. However, a proposal about metaverse architecture [10] is a valuable reference that helps us situate CopyrightLY’s contribution in the context of their proposed three main layers of a metaverse:
Infrastructure: this is the base layer on top of which the metaverse is deployed. It is based on a computation and communication infrastructure, including blockchain technologies, in our case, Ethereum, and storage, for CopyrightLY is decentralized storage provided by the IPFS.
Interaction: this layer deals with immersive user experiences, and interfaces to shape the metaverse and digital twins. CopyrightLY does not make any contribution in this regard and, when integrated into an existing metaverse, it just makes use of its interactive features. For instance, in the implementation example provided in Section 4.2, Copyright is integrated into the Voxels metaverse.6
Ecosystem: this is the metaverse layer where CopyrightLY focuses its contributions. The following aspects are highlighted for the three main components of the Ecosystem layer:
User Generated Content (UGC): CopyrightLY helps deal with UGC by managing its underlying copyright. UGC can be of any kind, from metaverse specific to social media like YouTube videos. In all cases, after their registration through curated authorship claims, they can be licensed and integrated as NFTs into marketplaces and metaverses for their reuse.
Economics: CopyrightLY’s NFT provides the ownership and trading mechanisms that enable open creator economies in the metaverse and beyond. Regarding the governance of the economics layer, CopyrightLY also features a dedicated token with economic value that drives the incentives for authorship claims curation, as detailed in Section 4.1.
Artificial Intelligence: CopyrightLY provides mechanisms to enrich NFTs with machine actionable copyright information. They are used to unambiguously state the reuse terms granted through token ownership, which can be fed into reasoners to assist the user during content reuse. For instance, as detailed in Section 4.2, the reasoner can determine if content can be displayed in a specific metaverse location based on the terms accompanying the corresponding NFT.

3 Approach

The most distinctive characteristic of the proposed approach is to not rely beforehand on automated mechanisms to determine infringement, like the algorithms just mentioned in the State-of-the-Art for robust hashing or fingerprinting. The content hashes used by CopyrightLY are just to permanently link the authorship claim registered on-chain with the content of the claimed creation stored on decentralized and immutable storage. This way it is possible to track the content linked to a claim and even retrieve it. Consequently, CopyrightLY just detects exact copies, because they generate the same content hash when making an authorship claim. Even in that case, as detailed in the following sections, the potentially original creator can proceed though in this case to file a complaint. Copyright is a social construct and, thus, it is difficult or maybe impossible to fully automate it and create an algorithm capable of discerning between originals and copies.
This approach was already introduced in [12], where we depicted just its preliminary architecture that is fully developed in this article in Section 3.1. We complement now the proposed approach with an incentives system to curate the pool of authorship claims and complaints made through CopyrightLY as detailed in Section 4.1. Additionally, it is important to note that all the accumulated claims, complaints, and evidence are registered on-chain and can be used in case of litigation and admissible as evidence in a judicial proceeding, together with expert assessment, like in the TikTok vs. Baidu case [24].
Consequently, the proposal is aligned with copyright law and courts, relying on them as last resort. However, to improve its scalability, it also integrates incentives and arbitration mechanisms following a blockchain-based decentralized approach. Grounding the system on copyright also helps deal with one of the main issues in NFTs state-of-the-art, issues derived from their disconnection from the underlying copyrights [28]. In addition to being grounded on copyright ownership, NFTs should also clearly state the licensing terms and conditions to become more than mere transfers of ownership on just a piece of metadata [23], as detailed in Section 4.2.
CopyrightLY is based on the combination of a set of Ethereum [1] blockchain smart contracts that manage authorship claims, complaints, evidence, and licensing terms. These smart contracts are connected with decentralized storage, where content and metadata are stored to avoid the prohibitive costs of storing them on-chain. Moreover, there are a set of blockchain oracles [19] that connect on-chain smart contracts to off-chain APIs of the main social media platforms, like Twitter, Facebook, or YouTube.
There is a web front-end that facilitates users’ interaction with on-chain logic and state. All blockchain read operations are carried out through an API that provides direct access to a representation of the on-chain state. This way, users can interact as they will do with any other web applications not based on blockchain technologies. For interactions modifying blockchain state, end-users use an Ethereum wallet, as a browser extension or on their mobile devices.
The wallet holds the public-private key pair used to sign on-chain transactions. The public key is pseudo-anonymous but, following a Self-Sovereign Identity approach [27], users can hold credentials in their wallets that can be presented to requesting parties when stronger identification is required. These credentials range from social media profiles, which can be openly disclosed and shown in the creator’s profile, to legal identity credentials that are kept private and selectively presented. For instance, in case of litigation to better support on-chain evidence in court.
As previously introduced, NFTs are used as the monetization mechanism through with reuse terms are licensed and traded. In addition to helping keep track of ownership, NFTs are enhanced with semantic metadata to capture the terms being transferred and the relationship between the token and the copyright claim. Semantic metadata is based on existing vocabularies like Schema.org [14] and the Copyright Ontology [13]. A rights module enhanced with semantic technologies and reasoning capabilities fetches data from NFTs and combines it with ownership information registered on-chain. It uses it to respond to rights-related issues, like checking if intended reuse is authorized by any owned licenses or finding available licensing NFTs that might provide it. The following subsection details the architecture developed to implement this approach.

3.1 Architecture Details

The central part of CopyrightLY’s architecture is on-chain and based on a set of smart contracts, as shown at the center of Figure 2. The smart contracts take care of registering Manifestations (i.e., authorship claims) and Complaints (to denounce authorship claims as potentially fraudulent). Authorship claims and complaints are supported with evidence. The simplest kind of evidence is based on files uploaded to decentralized storage, managed by the UploadEvidence smart contract that registers and links them to manifestations and the accounts triggering the transaction. All the previous smart contracts use decentralized storage based on IPFS to store manifestations, complaints, or evidence content (data, documents, or any kind of media can be uploaded).
Fig. 2.
Fig. 2. Proposed architecture for CopyrightLY, including a layer of smart contracts operating on-chain (center), a set of oracles bringing off-chain data on-chain (left), and a front-end and an API interfaces also managing user credentials (right).
Another way of supporting claims with evidence is through the oracles responsible for asserting on-chain the off-chain information about ownership of social media content (YouTube, Twitter, or Facebook). For each social media platform, the oracle verifies information about social media assets through the corresponding APIs, like YouTube’s API to verify that a predefined identifier has been added to a video description. This mechanism allows checking that an on-chain account has control over the corresponding off-chain social media assets. The oracle infrastructure is based on ChainLink,7 a decentralized network giving applications access to trusted off-chain computation and data.
Finally, there are a couple of smart contracts dealing with tokens. The License NFT contract implements an ERC-721 NFT, i.e., a kind of token that is unique and not interchangeable. It can be minted by a creator to monetize her copyright. The NFT metadata, in addition to the standard attributes like name or image, represents the reuse terms to be transferred to the future owner of the NFT, as further detailed in Section 4.2.
On the other hand, the CopyrightLY token (CLY) implemented by the CLYToken smart contract is an ERC-20 fungible token, interchangeable like money. This smart contract makes this token particular in the sense that it can just be minted, in exchange for Ethereum’s native cryptocurrency Ether, to be staked on an existing authorship claim or complaint. If unstaked, it is burned back into Ether. This token implements the incentive mechanism to curate authorship claims and complaints, as detailed in Section 4.1.
Three off-chain components are interacting with all this on-chain infrastructure. First of all, a Web front-end through which creators and reusers interact with on-chain logic and state. They do so using Ethereum wallets that link the accounts used to interact with the blockchain with verifiable credentials following a decentralized Self-Sovereign Identity approach [27]. This component stores the credentials issued to users’ on-chain identities in a private and self-sovereign way. Users can request that third parties issue these credentials after a verification process. In the case of CopyrightLY, the objective is to have both credentials about control of social media profiles and legal identities (required in case of litigation).
Additionally, there is a back-end that monitors blockchain events, keeps track of the on-chain state, and provides a better user experience because it does not require that users connect directly to the blockchain. This component is based on The Graph,8 an indexing protocol for querying networks like Ethereum. It makes a model of the on-chain state for a set of smart contracts by monitoring the events they generate, which is then available through a GraphQL [15] API.
The last off-chain component is the Rights Module, responsible for NFTs’ semantic data exploitation and providing reasoning mechanisms based, fundamentally, on the Copyright Ontology. License NFTs can be automatically traded and integrated into existing NFT marketplaces and metaverses, as shown in Figure 7, as they are based on the ERC-721 standard. The Rights Module monitors minting and ownership change events through the GraphQL API. Moreover, it retrieves NFTs’ semantic metadata from IPFS by following the token URI also defined by the NFT standard. All this information is stored in a semantic repository providing the reasoning and semantic query capabilities, available through a SPARQL [11] API. Through this API, reusers can, for instance, check if an intended reuse is allowed by the NFTs they own in the context of a specific metaverse location. Further details about the implementation of this feature are provided in Section 4.2.

4 Implementation Highlights

The previous architecture has been implemented as a PoC in the context of the ONTOCHAIN NGI H2020 project. It is possible to follow the development process, get access to the source code, and test the current version of the application from the project’s dashboard.9 The dashboard also includes the automated tests used to evaluate CopyrightLY compliance with the requirements specification. The following subsections highlight some of the most relevant parts of the project, including the incentive mechanisms to curate authorship claims and the implementation of licensing NFTs based on semantic technologies and ontologies, which enable copyright reasoning on top of NFTs ownership and tradeability mechanisms.

4.1 Incentives to Curate Authorship Claims

One of the biggest issues detected in the state-of-the-art proposals analyzed in Section 2 is that registering content on-chain does not provide any guarantee of its validity. It might be just the action of someone with access to a digital copy of a creation, but not necessarily the original creator. To deal with these false authorship claims, CopyrightLY leverages two incentive mechanisms. The first one is based on the fact that all claims and evidence are recorded on-chain and can be used in case of litigation. This is possible even in court, like in the TikTok vs. Baidu case [24], and disincentivizes false claims. However, it just works in the longer term and when some identifying information has been provided to strengthen the claim.
The second incentive mechanism aims at working on the shorter-term, scale thanks to the wisdom of the crowds and incentive stronger authorship claims and complaints, supported by evidence and some identifying information like social media accounts linked to the user identity. The approach is based on a Token Curated Registry [18], where a token with economic value, the CLY token, is used as an incentive mechanism [35] to curate the most reliable manifestations for authorship claims and complaints in a decentralized way.
The mechanism requires creators to stake an amount of the CLY token together with their authorship claims, as shown in the upper left part of Figure 3, which depicts CLY tokenomics. Following the “skin in the game” heuristic [30], creators risk losing their stake when other creators complain about existing claims, as shown in the upper right part of the same figure. Like original claimers, they can also provide supporting evidence to convince other users to support their position with additional stake. In case of conflicting claims, stakes cannot be removed till it is solved and the side with more stake wins the stake of the other side and shares it proportionally to their stake.
Fig. 3.
Fig. 3. Tokenomics of the CLY token to incentive the curation of authorship claims.
Additionally, filing a complaint on an existing authorship claim blocks the ability of its creator to continue monetizing it until the conflict is solved. Creators can start minting NFTs to license their copyright after staking some CLY and a time has passed without complaints, for instance, one week. On the other hand, to avoid spam complaints that look to block monetization opportunities, they become blocking just if the amount of CLY staken on the complaint exceeds that on the challenged manifestation.
To avoid players with big CLY token stakes dominating the game, an appeal mechanism based on an external arbitration court and decentralized justice [2] is also considered, as shown in the lower right part of Figure 3. In this case, any of the sides can appeal before the deadline to solve a complaint. The appealing side brings the case to the decentralized arbitration system Kleros using its dispute resolution integration plan.10
A court is opened on-chain with links to all evidence accumulated by both sides plus any identifying credentials that the parties might provide. Systems like Kleros are open to jurors that are invited to decide on cases. There is also a token, in the case of Kleros it is called Pinakion (PNK), which jurors have to stake and which is used to incentive participation and avoid attacks.11 In case of a complaint, the final outcome is that the winning side, that with the bigger CLY stake after the one-week deadline or the one winning the appeal, gets all the accumulated staked which is then split among the winners proportionally to their original stake.
Finally, to motivate participation in this staking game, the CLY token has an inflationary behavior [32] tied to its use. The token price, in Ether, follows a bonding curve [34] and grows with the token supply. The bonding curve is defined using the Bancor formula [16] depicted in Equation (1) and producing a price curve like that shown in Figure 4, which corresponds to an initially very conservative reserve ratio of \(\frac{1}{2}\) .
\begin{equation} CLY price = totalSupply \cdot {\left(1 + \frac{amount}{poolBalance}\right)}^{reserveRatio} - totalSupply. \end{equation}
(1)
Fig. 4.
Fig. 4. CLY token price at different supply values following the bonding curve in Equation (1) for a reserve ratio of \(\frac{1}{2}\) .
Consequently, if the CLY token succeeds and is used more, which requires minting it to increase the token supply, its price also grows. Early adopters can thus benefit from future players entering the game and unstake and burn their CLY for an amount of Ether bigger than what they originally invested. However, to avoid mere speculation, it is important to note that CLY can just be minted to be staked on a manifestation or complaint. It cannot be transferred, just unstaked and burnt in exchange for Ether at the price defined by the current total token supply. Therefore, stakers should take care of doing so on trustful claims that do not put them at risk, based on a reputation system that is built up in a decentralized way and for which the CLY staked is the main signal. In addition to that, each staker has access to all the information accumulated on-chain, including the authorship claim and the evidence accumulated, but also previous claims and complaints,… even the previous performance of the other stakers participating.

4.2 Semantic NFTs and Copyright Reasoning

The main contribution of CopyrightLY is that authorship claims can be monetized using Semantic NFTs, which solve the main issues identified in the state-of-the-art Section 2. First of all, NFTs are linked to manifestations and thus rooted in an authorship claim accompanied by evidence, a commitment in the form of staked CLY and potentially curated as previously detailed.
Additionally, NFTs’ metadata define the scope of the licensed rights, which are modelled using standard semantic technologies, like the Web Ontology Language (OWL) [17], and based on the Copyright Ontology [13]. They provide the necessary and unambiguous building blocks that also make the terms machine-actionable.
As shown in the architecture, Figure 2, the License NFT smart contract registers them on-chain and stores their metadata on decentralized storage based on IPFS. The licensing terms are serialized as semantic data based on the JSON-LD12 standard and stored on IPFS resulting in a content hash. This hash is used as the NFT token URI, used to point to the token metadata. The token URI is immutable so it can be verified that the terms have not changed after the token is minted.
The JSON-LD serialization is configured so the ERC-721 standard metadata fields (name, description, external_link, image, and animation_url) are available as plain JSON. This way, Semantic NFTs’ metadata is compatible with existing NFT marketplaces. For instance, Listing 1 shows the metadata for an NFT with the name “Reuse license for “Copyright Blockchain””, which corresponds to a piece of video art generated by an Artificial Intelligence, concretely VQGAN+CLIP [6]. Then, Figure 7 shows that Semantic NFT in an existing NFT marketplace, so ERC-721 interoperability with existing metaverses and marketplaces is retained by Semantic NFTs.
The metadata includes a description field that describes the licensing terms as plain text for user consumption. The rest of the metadata is tied to different vocabularies and ontologies to enable an unambiguous and rich representation of the terms, which makes them machine-actionable. First of all, the Schema.org vocabulary [14] is used for the NFT standard fields that have the same meaning in Schema.org, i.e., name, description, and image. The rest of the metadata is responsible for capturing the license terms that grant certain uses of the copyrighted work to the NFT owner.
For the copyright-related part, terms from the Copyright Ontology13 are used. They are tied to the cro prefix in the @context at the beginning of the JSON-LD serialization, Listing 1 line 4. Overall, the Copyright Ontology defines the following models:
Creation Model: the different shapes of copyright creations along their lifecycle, i.e., Manifestation, Performance, Recording…. In the case of Semantic NFTs, authorship claims are Manifestations, i.e., expression of the work into some kind of object (digital or not) that can be used to claim copyright.
Example: for the sample Semantic NFT in Listing 1, this corresponds to lines 30–33, where it is defined as the manifestation for which actions are granted by the NFT license.
Actions Model: captures the copyright actions moving creations along their lifecycle (Manifest, Perform, Communicate…) together with actions’ dimensions (who, what, when, where…) and other management actions like Agree or Disagree. The approach is similar to Schema.org Actions [4] and many of the Copyright Ontology terms are aligned or directly reuse Schema.org ones.
Example: the actions modeled in the Semantic NFT example are Agree and MakeAvailable, lines 11 and 23, respectively. Both are connected with who performs the action. In the first case, the creator granting the license is represented as a Decentralized Identifier (DID14) for the Ethereum account used by the creator as shown in line 19. In the second case, anyone owning the license NFT can make the associated content available. The NFT is also identified using a DID, shown in lines 10 and 27. The NFT metadata also details other conditions for the reuse, including a start and end time (lines 24 and 25), an instrument (line 36, which points to the Voxels metaverse Web application), or the authorized territories (lines 39 and 40, Voxels’ neighborhood “The Center” and the “Vibes” island).
Rights Model: this part of the Copyright Ontology represents the legal constructs regulating what actions are favored or restricted. Different legal systems can be represented, from generic ones geared towards worldwide harmonization like those proposed by the WIPO [33] to those in particular legal regimes. The Semantic NFT does not use any term from the Rights Model because what is licensed are specific actions and no rights are transferred.
Based on the semantic data in NFTs, and in the context of the Rights Module introduced as part of the architecture in Section 3.1, it is possible to make the licensing terms machine-actionable. The rights module fetches the semantic data from minted NFTs and monitors on-chain events to track ownership. All this information is combined into a semantic repository with reasoning and semantic query capabilities. Reasoning has been implemented using SPARQL [11] queries that check if the different dimensions of an intended reuse (who, what, when, where…) do fit within the licensing terms as illustrated in Figure 5.
Fig. 5.
Fig. 5. Illustration of how copyright reasoning can assist re-users to determine if an intended reuse is allowed by the NFT licenses they own. Licenses and intended uses are represented as puzzle pieces that, when they fit, are used to respond to copyright-related questions. The response might be yes if the pattern is an Agree on, or not authorized if there is no match or the pattern matched is a Disagree.
Matches can be direct, like that the content or data to be reused has a particular identifier (or content hash), but they can also be indirect and require some reasoning. For instance, if the territory where the intended action would take place is part or not of those where it is authorized. The SPARQL queries implementing copyright reasoning are hidden inside the Rights Module, which offers an API providing higher-level operations, like checking if an intended reuse action is allowed by the terms licensed to the user making the request.
Example: An example of a SPARQL query used by the Rights Module is presented in Listing 2. This query is used to check if a given user is authorized to reuse some specific content in a specific metaverse location, in this case, some specific coordinates of the Voxels metaverse. Based on the metadata for the NFT in Listing 1, and if the intended reuser is the owner of the NFT, it will respond affirmatively to the reuse request and include a link to the agreement granting it, i.e., the DID of the license NFT. In this particular case, the Voxels metaverse location where the reuse is intended is coordinates 6510W,100N,15 which correspond to Voxels’ parcel 496216 named “Generative Artworks Gallery”. Though the NFT metadata explicitly authorizes the reuse for Voxel’s The Center17 neighborhood and the Vibes island,18 the query implements all the reasoning about how territories are organized in Voxels metaverse. All that information has been captured using Voxels API and modeled as an ontology available online.19 When the license NFT and ownership information are combined with the ontology about Voxels’ territories and their “part of” relationships, the Rights Module is capable of determining that coordinates “6510W,100N”, i.e., (−65.1 1), are located on parcel 4962 which belongs to the Vibes island, so that is also an authorized location and the user can display the NFT there.

5 Conclusions

As shown in the previous sections, CopyrightLY contributes a decentralized authorship and rights management layer that has been evaluated in the context of the ONTOCHAIN NGI H2020 project ecosystem. This decentralized application leverages blockchain and semantic web technologies and implements services that allow claiming content authorship and providing supporting evidence.
To disincentivize false claims, a Token Curated Registry has been implemented based on the staking of the CLY token on authorship claims. Claims can be challenged through complaints, which also require staking CLY, and conflicts are resolved based on the amount of CLY that each party has been capable of attracting from anyone willing to participate in this crowdsources process. Participation is incentivized by binding CLY’s price in Ether, the main cryptocurrency of the Ethereum blockchain, to a bonding curve. Thus, early adopters can experience a capital gain when they burn CLY back into Ether.
An additional contribution is that CopyrightLY makes it also possible to monetize content using Semantic NFTs grounded on copyright claims and stating the licensing terms using semantic metadata based on the Copyright Ontology. This solves two of the main issues that the NFT market is experiencing, spam NFTs and lack of clarity regarding what is owned when an NFT is purchased.
License NFTs metadata captures the reuse terms in a machine-actionable way, so it is possible to use automated reasoning to assist users with intended reuses. Moreover, despite NFT metadata being enriched with semantics, they are still compatible with existing marketplaces and metaverses. This makes it possible, as shown through an example, to use reasoning to determine if a license NFT allows displaying the associated content at a particular metaverse location.
A prototype of the project is available online at20 and deployed on the Rinkeby Ethereum testnet. Figure 6 shows how an authorship claim looks in CopyrightLY, including a YouTube evidence and a licensing NFT. That NFT points to the terms and can be traded on NFT marketplaces, for instance, OpenSea’s testing marketplace,21 as shown in Figure 7.
Fig. 6.
Fig. 6. Authorship claim and supporting evidence linked to the Semantic NFT corresponding to the metadata in Listing 1.
Fig. 7.
Fig. 7. Semantic NFT traded on the OpenSea NFT marketplace, corresponding to the authorship claim shown in Figure 6.

6 Future Work

Though the tokenomics of the CLY token have been designed for both authorship claims and potential complaints, the current implementation only captures the authorship claims part. Future work is to complete the complaints’ part and implement the designed tokenomics mechanisms that settle the incentives to curate the set of authorship claims. This includes generating complaints when potentially infringing content is discovered and the incentives to do so. The tokenomics will be then evaluated using game theory analysis and simulation tools like cadCAD.22
To detect potentially infringing authorship claims, we anticipate using mechanisms beyond content hashes that detect just exact copies. However, as discussed in the state-of-the-art Section 2, mechanisms capable of detecting near-duplicates, like fingerprinting, should be used just as an aid for creators to find out false claims of their copyright. Then, they should go through the process of filing a complaint and providing evidence to attract the CLY stake to win the case. This might facilitate the curation process and, given the incentives, also attract other users to identify weak authorship claims and potential CLY stake rewards.
Additionally, though CopyrightLY licensing NFTs are tradeable on existing NFT marketplaces as it has been shown, existing marketplaces or metaverses miss the functionality to exploit the rich semantics captured by CopyrightLY NFTs metadata. Future work would be to develop NFT marketplaces and metaverse connectors capable of leveraging the copyright semantic metadata, for instance by reusing the Rights Module included in the project architecture.
Finally, work will continue by targeting potential users and other interested parties. For the moment, the following user profiles have been identified:
Social media creators that are willing to explore alternative monetization options, specially NFTs.
Independent creators, especially those in developing countries lack access to the media industry or markets, even those unbanked.
Media companies looking to license small creators to which current approaches do not scale, especially in developing countries.
Marketplaces and metaverses looking for more trustful NFTs, linked to copyright claims and clearly stating the licensing terms.
CopyrightLY’s user experience (UX) will be evaluated through the participation of potential users from previous profiles. Users will be requested to perform certain tasks under the supervision of UX experts that will evaluate the user experience. Moreover, post-test surveys will be used to evaluate user satisfaction and recommendations about future CopyrightLY improvements.

Footnotes

Supplementary Material

tomm-2022-0457-File003 (tomm-2022-0457-file003.mp4)
Supplementary video

Appendices

A Semantic NFT Example

Listing 1. JSON-LD serialization of the metadata of a Semantic NFT granting its owner permission to make available the content registered by an authorship claim.

B Copyright Reasoning Example

Listing 2. SPARQL query implementing the reasoning capabilities to respond to reuse requests based on the terms licensed by CopyrightLY’s NFTs.

References

[1]
Andreas M. Antonopoulos and Gavin Wood. 2018. Mastering Ethereum: Building Smart Contracts and DApps. O’Reilly Media, Sebastopol, CA.
[2]
Yann Aouidef, Federico Ast, and Bruno Deffains. 2021. Decentralized justice: A comparative analysis of blockchain online dispute resolution projects. Frontiers in Blockchain 4 (2021), 564551. DOI:
[3]
Juan Benet. 2014. IPFS - content addressed, versioned, P2P file system. arXiv:1407.3561. Retrieved February 13, 2023 from https://arxiv.org/abs/1407.3561.
[4]
Dan Brickley. 2014. Announcing Schema.org Actions. Retrieved February 13, 2023 from http://blog.schema.org/2014/04/announcing-schemaorg-actions.html.
[5]
Wei Cai, Zehua Wang, Jason B. Ernst, Zhen Hong, Chen Feng, and Victor C. M. Leung. 2018. Decentralized applications: The blockchain-empowered software system. IEEE Access 6 (2018), 53019–53033. DOI:
[6]
Eva Cetinic and James She. 2022. Understanding and creating art with AI: Review and outlook. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 2 (2022), 66:1–66:22. DOI:
[7]
Mitchell Clark. 2021. NFTs, explained. (Aug. 2021). Retrieved February 13, 2023 from https://www.theverge.com/22310188/nft-explainer-what-is-blockchain-crypto-art-faq.
[8]
Robert Alexandru Dobre, Radu Ovidiu Preda, Radu Alexandru Badea, Mihai Stanciu, and Alexandru Brumaru. 2020. Blockchain-based image copyright protection system using JPEG resistant digital signature. In Proceedings of the 2020 IEEE 26th International Symposium for Design and Technology in Electronic Packaging. IEEE, Pitesti, Romania, 206–210. DOI:
[9]
Mario Dobrovnik, David Herold, Elmar Fürst, and Sebastian Kummer. 2018. Blockchain for and in logistics: What to adopt and where to start. Logistics 2, 3 (2018), 18.
[10]
Haihan Duan, Jiaye Li, Sizheng Fan, Zhonghao Lin, Xiao Wu, and Wei Cai. 2021. Metaverse for social good: A university campus prototype. In Proceedings of the 29th ACM International Conference on Multimedia. Association for Computing Machinery, 153–161. DOI:
[11]
Bob DuCharme. 2013. Learning SPARQL: Querying and Updating with SPARQL 1.1 (2nd Ed.). O’Reilly Media, Sebastopol, CA.
[12]
Roberto García, Ana Cediel, Mercè Teixidó, and Rosa Gil. 2021. CopyrightLY: Blockchain and semantic web for decentralised copyright management. In Proceedings of the Economics of Grids, Clouds, Systems, and Services. Konstantinos Tserpes, Jörn Altmann, José Ángel Bañares, Orna Agmon Ben-Yehuda, Karim Djemame, Vlado Stankovski, and Bruno Tuffin (Eds.), Vol. 13072, Springer International Publishing, Cham, 199–206. DOI:
[13]
Roberto García and Rosa Gil. 2010. Content value chains modelling using a copyright ontology. Information Systems 35, 4 (2010), 483–495. DOI:
[14]
R. V. Guha, Dan Brickley, and Steve Macbeth. 2016. Schema.org: Evolution of structured data on the web. Communications of the ACM 59, 2 (2016), 44–51. DOI:
[15]
Olaf Hartig and Jorge Pérez. 2018. Semantics and complexity of GraphQL. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW’18. ACM, Lyon, France, 1155–1164. DOI:
[16]
Eyal Hertzog, Guy Benartzi, and Galia Benartzi. 2018. Bancor protocol. Continuous Liquidity for Cryptographic Tokens through their Smart Contracts. (2018), 22 pages. Retrieved February 13, 2023 from https://cryptopapers.info/assets/pdf/bancor.pdf.
[17]
Pascal Hitzler, Markus Krötzsch, Bijan Parsia, Peter F. Patel-Schneider, and Sebastian Rudolph. 2012. OWL 2 Web Ontology Language Primer. (2nd Ed.). Retrieved February 13, 2023 from http://www.w3.org/TR/owl2-primer/.
[18]
Jaspreet Kaur and B. Visveswaraiah. 2021. A brief survey of token curated registries. In Proceedings of the Emerging Technologies in Data Mining and Information Security. Aboul Ella Hassanien, Siddhartha Bhattacharyya, Satyajit Chakrabati, Abhishek Bhattacharya, and Soumi Dutta (Eds.), Vol. 1286, Springer Singapore, Singapore, 189–202. DOI:
[19]
Petar Kochovski, Sandi Gec, Vlado Stankovski, Marko Bajec, and Pavel D. Drobintsev. 2019. Trust management in a blockchain based fog computing platform with trustless smart oracles. Future Generation Computer Systems 101 (2019), 747–759. DOI:
[20]
M. Kripa, A. Nidhin Mahesh, R. Ramaguru, and P. P. Amritha. 2021. Blockchain framework for social media DRM based on secret sharing. In Proceedings of the Information and Communication Technology for Intelligent Systems. Tomonobu Senjyu, Parikshit N. Mahalle, Thinagaran Perumal, and Amit Joshi (Eds.), Vol. 195, Springer Singapore, Singapore, 451–458. DOI:
[21]
Lawrence Lessig. 1999. Code and other Laws of Cyberspace. Basic Books, New York.
[22]
Esther Mengelkamp, Benedikt Notheisen, Carolin Beer, David Dauer, and Christof Weinhardt. 2018. A blockchain-based smart grid: Towards sustainable local energy markets. Computer Science-Research and Development 33, 1–2 (2018), 207–214.
[23]
Peter Mezei, João Pedro Quintais, Alexandra Giannopoulou, and Balázs Bodó. 2021. The Rise of Non-Fungible Tokens (NFTs) and the Role of Copyright Law – Part II. (April 2021). Retrieved February 13, 2023 from http://copyrightblog.kluweriplaw.com/2021/04/22/the-rise-of-non-fungible-tokens-nfts-and-the-role-of-copyright-law-part-ii/.
[24]
Matej Michalko. 2020. How TikTok Used Blockchain to Defeat Copyright Infringement. (March 2020). Retrieved February 13, 2023 from https://www.ipwatchdog.com/2020/03/17/tiktok-used-blockchain-defeat-copyright-infringement/id=119878/.
[25]
Iynkaran Natgunanathan, Purathani Praitheeshan, Longxiang Gao, Yong Xiang, and Lei Pan. 2022. Blockchain-based audio watermarking technique for multimedia copyright protection in distribution networks. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 3 (2022), 86:1–86:23. DOI:
[26]
Johannes Pfeffer. 2018. EthOn-an Ethereum Ontology. Technical Report 0.2. Consensys. Retrieved from https://consensys.github.io/EthOn/EthOn_spec.html.
[27]
Alexander Preukschat and Drummond Reed. 2021. Self-sovereign Identity Decentralized Digital Identity and Verifiable Credentials. Manning Publications Co., Shelter Island, NY. Retrieved from https://www.manning.com/books/self-sovereign-identity.
[28]
Jonathan Schmalfeld. 2021. How copyright violations can crash your NFT party. (April 2021). Retrieved February 13, 2023 from https://fortune.com/2021/08/04/nfts-copyright-violations-penalties-non-fungible-tokens-collectibles-nfttorney-jonathan-schmalfeld/.
[29]
Nigel Shadbolt, Wendy Hall, and Tim Berners-Lee. 2006. The semantic web revisited. Intelligent Systems 21, 3 (2006), 96–101. DOI:
[30]
Nassim N. Taleb and Constantine Sandis. 2014. The skin in the game heuristic for protection against tail events. Review of Behavioral Economics 1, 1–2 (2014), 115–135. DOI:
[31]
M. Walport. 2016. Distributed Ledger Technology: Beyond Blockchain. Report 19. UK Government Office for Science. Retrieved from https://www.gov.uk/government/publications/distributed-ledger-technology-blackett-review.
[32]
Yi Lucy Wang and Bhaskar Krishnamachari. 2019. Enhancing engagement in token-curated registries via an inflationary mechanism. In Proceedings of the 2019 IEEE International Conference on Blockchain and Cryptocurrency. IEEE, Seoul, Korea (South), 188–191. DOI:
[33]
WIPO. 2004. WIPO Intellectual Property Handbook: Policy, Law, and Use. WIPO Publications. Retrieved from http://www.wipo.int/about-ip/en/iprm.
[34]
Michael Zargham, Jamsheed Shorish, and Krzysztof Paruch. 2020. From curved bonding to configuration spaces. In Proceedings of the 2020 IEEE International Conference on Blockchain and Cryptocurrency. IEEE, Toronto, ON, Canada, 1–3. DOI:
[35]
Zhili Zhou, Meimin Wang, Ching-Nung Yang, Zhangjie Fu, Xingming Sun, and Q. M. Jonathan Wu. 2021. Blockchain-based decentralized reputation system in E-commerce environment. Future Generation Computer Systems 124 (2021), 155–167. DOI:

Cited By

View all
  • (2024)Security and privacy of digital economic risk assessment system based on cloud computing and blockchainSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09586-828:3(2753-2768)Online publication date: 10-Jan-2024
  • (2023)Towards Reliable Utilization of AIGC: Blockchain-Empowered Ownership Verification MechanismIEEE Open Journal of the Computer Society10.1109/OJCS.2023.33158354(326-337)Online publication date: 2023
  • (2023)Semantic Literature Review on Non-Fungible Token: Expansion Area of Usage & Trends2023 International Conference on Information Management and Technology (ICIMTech)10.1109/ICIMTech59029.2023.10277808(351-356)Online publication date: 24-Aug-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 20, Issue 7
July 2024
973 pages
EISSN:1551-6865
DOI:10.1145/3613662
  • Editor:
  • Abdulmotaleb El Saddik
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2024
Online AM: 24 February 2023
Accepted: 21 February 2023
Revised: 13 February 2023
Received: 29 July 2022
Published in TOMM Volume 20, Issue 7

Check for updates

Author Tags

  1. Metaverse
  2. Non-Fungible Token
  3. copyright
  4. social media
  5. blockchain
  6. ontology

Qualifiers

  • Research-article

Funding Sources

  • European Union’s Horizon 2020 research and innovation programme
  • “ANGRU: Applying kNowledge Graphs to research data ReUsability”

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,373
  • Downloads (Last 6 weeks)117
Reflects downloads up to 02 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Security and privacy of digital economic risk assessment system based on cloud computing and blockchainSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09586-828:3(2753-2768)Online publication date: 10-Jan-2024
  • (2023)Towards Reliable Utilization of AIGC: Blockchain-Empowered Ownership Verification MechanismIEEE Open Journal of the Computer Society10.1109/OJCS.2023.33158354(326-337)Online publication date: 2023
  • (2023)Semantic Literature Review on Non-Fungible Token: Expansion Area of Usage & Trends2023 International Conference on Information Management and Technology (ICIMTech)10.1109/ICIMTech59029.2023.10277808(351-356)Online publication date: 24-Aug-2023
  • (2023)Toward Trustworthy Metaverse: Advancements and ChallengesIEEE Access10.1109/ACCESS.2023.332625811(118318-118347)Online publication date: 2023
  • (2022)A Survey of Mobile Edge Computing for the Metaverse: Architectures, Applications, and Challenges2022 IEEE 8th International Conference on Collaboration and Internet Computing (CIC)10.1109/CIC56439.2022.00011(1-9)Online publication date: Dec-2022

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media