DarkFi

Build Status Web - dark.fi Manifesto - unsystem Book - mdbook

Build

This project requires the Rust compiler to be installed. Please visit Rustup for instructions.

The following dependencies are also required:

DependencyDebian-based
gcc, gcc-c++, kernel headersbuild-essential
cmakecmake
jqjq
wgetwget
pkg-configpkg-config
clangclang
clang libslibclang-dev
llvm libsllvm-dev
udev libslibudev-dev
freetype2 libslibfreetype6-dev
expat xml liblibexpat1-dev

Users of Debian-based systems (e.g. Ubuntu) can simply run the following to install the required dependencies:

# apt-get update
# apt-get install -y build-essential cmake jq wget pkg-config \
    clang libclang-dev llvm-dev libudev-dev libfreetype6-dev \
    libexpat1-dev

Alternatively, users can chose one of the automated scripts under contrib folder by executing:

% bash contrib/*_setup.sh

The following setup script are provided:

  • mac_setup.sh: installation using brew (brew will be installed if not present).
  • void_setup.sh: Xbps dependencies for Void Linux.

To build the necessary binaries, we can just clone the repo, and use the provided Makefile to build the project. This will download the trusted setup params, and compile the source code.

% git clone https://github.com/darkrenaissance/darkfi
% cd darkfi/
% make

Development

If you want to hack on the source code, make sure to read some introductory advice in the DarkFi book.

Install

This will install the binaries on your system (/usr/local by default). The configuration files for the binaries are bundled with the binaries and contain sane defaults. You'll have to run each daemon once in order for them to spawn a config file, which you can then review.

# make install

Bash Completion

This will add the options auto completion of drk and darkfid.

% echo source (pwd)/contrib/auto-complete >> ~/.bashrc

Examples and usage

See the DarkFi book

Go Dark

Let's liberate people from the claws of big tech and create the democratic paradigm of technology.

Self-defense is integral to any organism's survival and growth.

Power to the minuteman.

Definition of Democratic Civilization

From 'The Sociology of Freedom: Manifesto of the Democratic Civilization, Volume 3' by Abdullah Ocalan.

Annotations are our own. The text is otherwise unchanged.

What is the subject of moral and political society?

The school of social science that postulates the examination of the existence and development of social nature on the basis of moral and political society could be defined as the democratic civilization system. The various schools of social science base their analyses on different units. Theology and religion prioritize society. For scientific socialism, it is class. The fundamental unit for liberalism is the individual. There are, of course, schools that prioritize power and the state and others that focus on civilization. All these unit-based approaches must be criticized, because, as I have frequently pointed out, they are not historical, and they fail to address the totality. A meaningful examination would have to focus on what is crucial from the point of view of society, both in terms of history and actuality. Otherwise, the result will only be one more discourse.

Identifying our fundamental unit as moral and political society is significant, because it also covers the dimensions of historicity and totality. Moral and political society is the most historical and holistic expression of society. Morals and politics themselves can be understood as history. A society that has a moral and political dimension is a society that is the closest to the totality of all its existence and development. A society can exist without the state, class, exploitation, the city, power, or the nation, but a society devoid of morals and politics is unthinkable. Societies may exist as colonies of other powers, particularly capital and state monopolies, and as sources of raw materials. In those cases, however, we are talking about the legacy of a society that has ceased to be.

Individualism is a state of war

There is nothing gained by labeling moral and political society—the natural state of society—as slave-owning, feudal, capitalist, or socialist. Using such labels to describe society masks reality and reduces society to its components (class, economy, and monopoly). The bottleneck encountered in discourses based on such concepts as regards the theory and practice of social development stems from errors and inadequacies inherent in them. If all of the analyses of society referred to with these labels that are closer to historical materialism have fallen into this situation, it is clear that discourses with much weaker scientific bases will be in a much worse situation. Religious discourses, meanwhile, focus heavily on the importance of morals but have long since turned politics over to the state. Bourgeois liberal approaches not only obscure the society with moral and political dimensions, but when the opportunity presents itself they do not hesitate to wage war on this society. Individualism is a state of war against society to the same degree as power and the state is. Liberalism essentially prepares society, which is weakened by being deprived of its morals and politics, for all kinds of attacks by individualism. Liberalism is the ideology and practice that is most anti-society.

The rise of scientific positivism

In Western sociology (there is still no science called Eastern sociology) concepts such as society and civilization system are quite problematic. We should not forget that the need for sociology stemmed from the need to find solutions to the huge problems of crises, contradictions, and conflicts and war caused by capital and power monopolies. Every branch of sociology developed its own thesis about how to maintain order and make life more livable. Despite all the sectarian, theological, and reformist interpretations of the teachings of Christianity, as social problems deepened, interpretations based on a scientific (positivist) point of view came to the fore. The philosophical revolution and the Enlightenment (seventeenth and eighteenth centuries) were essentially the result of this need. When the French Revolution complicated society’s problems rather than solving them, there was a marked increase in the tendency to develop sociology as an independent science. Utopian socialists (Henri de Saint-Simon, Charles Fourier, and Pierre-Joseph Proudhon), together with Auguste Comte and Émile Durkheim, represent the preliminary steps in this direction. All of them are children of the Enlightenment, with unlimited faith in science. They believed they could use science to re-create society as they wished. They were playing God. In Hegel’s words, God had descended to earth and, what’s more, in the form of the nation-state. What needed to be done was to plan and develop specific and sophisticated “social engineering” projects. There was no project or plan that could not be achieved by the nation-state if it so desired, as long as it embraced the “scientific positivism” and was accepted by the nation-state!

Capitalism as an iron cage

British social scientists (political economists) added economic solutions to French sociology, while German ideologists contributed philosophically. Adam Smith and Hegel in particular made major contributions. There was a wide variety of prescriptions from both the left and right to address the problems arising from the horrendous abuse of the society by the nineteenth-century industrial capitalism. Liberalism, the central ideology of the capitalist monopoly has a totally eclectic approach, taking advantage of any and all ideas, and is the most practical when it comes to creating almost patchwork-like systems. It was as if the right- and left- wing schematic sociologies were unaware of social nature, history, and the present while developing their projects in relation to the past (the quest for the “golden age” by the right) or the future (utopian society). Their systems would continually fragment when they encountered history or current life. The reality that had imprisoned them all was the “iron cage” that capitalist modernity had slowly cast and sealed them in, intellectually and in their practical way of life. However, Friedrich Nietzsche’s ideas of metaphysicians of positivism or castrated dwarfs of capitalist modernity bring us a lot closer to the social truth. Nietzsche leads the pack of rare philosophers who first drew attention to the risk of society being swallowed up by capitalist modernity. Although he is accused of serving fascism with his thoughts, his foretelling of the onset of fascism and world wars was quite enticing.

The increase in major crises and world wars, along with the division of the liberal center into right- and left-wing branches, was enough to bankrupt positivist sociology. In spite of its widespread criticism of metaphysics, social engineering has revealed its true identity with authoritarian and totalitarian fascism as metaphysics at its shallowest. The Frankfurt School is the official testimonial of this bankruptcy. The École Annales and the 1968 youth uprising led to various postmodernist sociological approaches, in particular Immanuel Wallerstein’s capitalist world-system analysis. Tendencies like ecology, feminism, relativism, the New Left, and world-system analysis launched a period during which the social sciences splintered. Obviously, financial capital gaining hegem- ony as the 1970s faded also played an important role. The upside of these developments was the collapse of the hegemony of Eurocentric thought. The downside, however, was the drawbacks of a highly fragmented social sciences.

The problems of Eurocentric sociology

Let’s summarize the criticism of Eurocentric sociology:

  1. Positivism, which criticized and denounced both religion and metaphysics, has not escaped being a kind of religion and metaphysics in its own right. This should not come as a surprise. Human culture requires metaphysics. The issue is to distinguish good from bad metaphysics.

  2. An understanding of society based on dichotomies like primitive vs. modern, capitalist vs. socialist, industrial vs. agrarian, progressive vs. reactionary, divided by class vs. classless, or with a state vs. stateless prevents the development of a definition that comes closer to the truth of social nature. Dichotomies of this sort distance us from social truth.

  3. To re-create society is to play the modern god. More precisely, each time society is recreated there is a tendency to form a new capital and power-state monopoly. Much like medieval theism was ideologically connected to absolute monarchies (sultanates and shāhanshāhs), modern social engineering—as re-creation— is essentially the divine disposition and ideology of the nation- state. Positivism in this regard is modern theism.

  4. Revolutions cannot be interpreted as the re-creation acts of society. When thusly understood they cannot escape positivist theism. Revolutions can only be defined as social revolutions to the extent that they free society from excessive burden of capital and power.

  5. The task of revolutionaries cannot be defined as creating any social model of their making but more correctly as playing a role in contributing to the development of moral and political society.

  6. Methods and paradigms to be applied to social nature should not be identical to those that relate to first nature. While the universalist approach to first nature provides results that come closer to the truth (I don’t believe there is an absolute truth), relativism in relation to social nature may get us closer to the truth. The universe can neither be explained by an infinite universalist linear discourse or by a concept of infinite similar circular cycles.

  7. A social regime of truth needs to be reorganized on the basis of these and many other criticisms. Obviously, I am not talking about a new divine creation, but I do believe that the greatest feature of the human mind is the power to search for and build truth.

A new social science

In light of these criticisms, I offer the following suggestions in relation to the social science system that I want to define:

A more humane social nature

  1. I would not present social nature as a rigid universalist truth with mythological, religious, metaphysical, and scientific (positivist) patterns. Understanding it to be the most flexible form of basic universal entities that encompass a wealth of diversities but are tied down to conditions of historical time and location more closely approaches the truth. Any analysis, social science, or attempt to make practical change without adequate knowledge of the qualities of social nature may well backfire. The monotheistic religions and positivism, which have appeared throughout the history of civilization claiming to have found the solution, were unable to prevent capital and power monopolies from gaining control. It is therefore their irrevocable task, if they are to contribute to moral and political society, to develop a more humane analysis based on a profound self-criticism.

  2. Moral and political society is the main element that gives social nature its historical and complete meaning and represents the unity in diversity that is basic to its existence. It is the definition of moral and political society that gives social nature its character, maintains its unity in diversity, and plays a decisive role in expressing its main totality and historicity. The descriptors commonly used to define society, such as primitive, modern, slave-owning, feudal, capitalist, socialist, industrial, agricultural, commercial, monetary, statist, national, hegemonic, and so on, do not reflect the decisive features of social nature. On the contrary, they conceal and fragment its meaning. This, in turn, provides a base for faulty theoretical and practical approaches and actions related to society.

Protecting the social fabric

  1. Statements about renewing and re-creating society are part of operations meant to constitute new capital and power monopolies in terms of their ideological content. The history of civilization, the history of such renewals, is the history of the cumulative accumulation of capital and power. Instead of divine creativity, the basic action the society needs most is to struggle against factors that prevent the development and functioning of moral and political social fabric. A society that operates its moral and political dimensions freely, is a society that will continue its development in the best way.

  2. Revolutions are forms of social action resorted to when society is sternly prevented from freely exercising and maintaining its moral and political function. Revolutions can and should be accepted as legitimate by society only when they do not seek to create new societies, nations, or states but to restore moral and political society its ability to function freely.

  3. Revolutionary heroism must find meaning through its contributions to moral and political society. Any action that does not have this meaning, regardless of its intent and duration, cannot be defined as revolutionary social heroism. What determines the role of individuals in society in a positive sense is their contribution to the development of moral and political society.

  4. No social science that hopes to develop these key features through profound research and examination should be based on a universalist linear progressive approach or on a singular infinite cyclical relativity. In the final instance, instead of these dogmatic approaches that serve to legitimize the cumulative accumulation of capital and power throughout the history of civilization, social sciences based on a non-destructive dialectic methodology that harmonizes analytical and emotional intelligence and overcomes the strict subject-object mold should be developed.

The framework of moral and political society

The paradigmatic and empirical framework of moral and political society, the main unit of the democratic civilization system, can be presented through such hypotheses. Let me present its main aspects:

  1. Moral and political society is the fundamental aspect of human society that must be continuously sought. Society is essentially moral and political.

  2. Moral and political society is located at the opposite end of the spectrum from the civilization systems that emerged from the triad of city, class, and state (which had previously been hierarchical structures).

  3. Moral and political society, as the history of social nature, develops in harmony with the democratic civilization system.

  4. Moral and political society is the freest society. A functioning moral and political fabric and organs is the most decisive dynamic not only for freeing society but to keep it free. No revolution or its heroines and heroes can free the society to the degree that the development of a healthy moral and political dimension will. Moreover, revolution and its heroines and heroes can only play a decisive role to the degree that they contribute to moral and political society.

  5. A moral and political society is a democratic society. Democracy is only meaningful on the basis of the existence of a moral and political society that is open and free. A democratic society where individuals and groups become subjects is the form of governance that best develops moral and political society. More precisely, we call a functioning political society a democracy. Politics and democracy are truly identical concepts. If freedom is the space within which politics expresses itself, then democracy is the way in which politics is exercised in this space. The triad of freedom, politics, and democracy cannot lack a moral basis. We could refer to morality as the institutionalized and traditional state of freedom, politics, and democracy.

  6. Moral and political societies are in a dialectical contradiction with the state, which is the official expression of all forms of capital, property, and power. The state constantly tries to substitute law for morality and bureaucracy for politics. The official state civilization develops on one side of this historically ongoing contradiction, with the unofficial democratic civilization system developing on the other side. Two distinct typologies of meaning emerge. Contradictions may either grow more violent and lead to war or there may be reconciliation, leading to peace.

  7. Peace is only possible if moral and political society forces and the state monopoly forces have the will to live side by side unarmed and with no killing. There have been instances when rather than society destroying the state or the state destroying society, a conditional peace called democratic reconciliation has been reached. History doesn’t take place either in the form of democratic civilization—as the expression of moral and political society—or totally in the form of civilization systems—as the expression of class and state society. History has unfolded as intense relationship rife with contradiction between the two, with successive periods of war and peace. It is quite utopian to think that this situation, with at least a five-thousand-year history, can be immediately resolved by emergency revolutions. At the same time, to embrace it as if it is fate and cannot be interfered with would also not be the correct moral and political approach. Knowing that struggles between systems will be protracted, it makes more sense and will prove more effective to adopt strategic and tactical approaches that expand the freedom and democracy sphere of moral and political society.

  8. Defining moral and political society in terms of communal, slave-owning, feudal, capitalist, and socialist attributes serves to obscure rather than elucidate matters. Clearly, in a moral and political society there is no room for slave-owning, feudal, or capitalist forces, but, in the context of a principled reconciliation, it is possible to take an aloof approach to these forces, within limits and in a controlled manner. What’s important is that moral and political society should neither destroy them nor be swallowed up by them; the superiority of moral and political society should make it possible to continuously limit the reach and power of the central civilization system. Communal and socialist systems can identify with moral and political society insofar as they themselves are democratic. This identification is, however, not possible, if they have a state.

  9. Moral and political society cannot seek to become a nation-state, establish an official religion, or construct a non-democratic regime. The right to determine the objectives and nature of society lies with the free will of all members of a moral and political society. Just as with current debates and decisions, strategic decisions are the purview of society’s moral and political will and expression. The essential thing is to have discussions and to become a decision-making power. A society who holds this power can determine its preferences in the soundest possible way. No individual or force has the authority to decide on behalf of moral and political society, and social engineering has no place in these societies.

Liberating democratic civilization from the State

When viewed in the light of the various broad definitions I have presented, it is obvious that the democratic civilization system—essentially the moral and political totality of social nature—has always existed and sustained itself as the flip side of the official history of civilization. Despite all the oppression and exploitation at the hands of the official world-system, the other face of society could not be destroyed. In fact, it is impossible to destroy it. Just as capitalism cannot sustain itself without noncapitalist society, civilization— the official world system— also cannot sustain itself without the democratic civilization system. More concretely the civilization with monopolies cannot sustain itself without the existence of a civilization without monopolies. The opposite is not true. Democratic civilization, representing the historical flow of the system of moral and political society, can sustain itself more comfortably and with fewer obstacles in the absence of the official civilization.

I define democratic civilization as a system of thought, the accumulation of thought, and the totality of moral rules and political organs. I am not only talking about a history of thought or the social reality within a given moral and political development. The discussion does, however, encompass both issues in an intertwined manner. I consider it important and necessary to explain the method in terms of democratic civilization’s history and elements, because this totality of alternate discourse and structures are prevented by the official civilization. I will address these issues in subsequent sections.

Notes for developers

Making life easy for others

Write useful commit messages.

If your commit is changing a specific module in the code and not touching other parts of the codebase (as should be the case 99% of the time), consider writing a useful commit message that also mentions which module was changed.

For example, a message like:

added foo

is not as clear as

crypto/keypair: Added foo method for Bar struct.

Also keep in mind that commit messages can be longer than a single line, so use it to your advantage to explain your commit and intentions.

cargo fmt pre-commit hook

To ensure every contributor uses the same code style, make sure you run cargo +nightly fmt before committing. You can force yourself to do this by creating a git pre-commit hook like the following:

#!/bin/sh
if ! cargo +nightly fmt -- --check >/dev/null; then
    echo "There are some code style issues. Run 'cargo +nightly fmt' to fix it."
    exit 1
fi

exit 0

Place this script in .git/hooks/pre-commit and make sure it's executable by running chmod +x .git/hooks/pre-commit.

Testing crate features

Our library heavily depends on cargo features. Currently there are more than 650 possible combinations of features to build the library. To ensure everything can always compile and works, we can use a helper for cargo called cargo hack.

The Makefile provided in the repository is already set up to use it, so it's enough to install cargo hack and run make check.

Architecture design

This section of the book shows the software architecture of DarkFi and the network implementations.

Overview

DarkFi is a layer one proof-of-stake blockchain that supports anonymous applications. It is currently under development. This overview will outline a few key terms that help explain DarkFi.

Cashier: The Cashier is the entry and exit point to the DarkFi network from other blockchains such as Ethereum, Bitcoin and Solana. It is essentially the bridge. Its role is to exchange cryptocurrency assets for anonymous darkened tokens that are pegged to the underlying currency, and visa versa. Currently, the role of the Cashier is trusted and centralized. As a next step, DarkFi plans to implement trust-minimized bridges and eventually fully trustless bridges.

Blockchain: Once new anonymous tokens (e.g. dETH) have been issued, the Cashier posts that data on the blockchain. This data is encrypted and the transaction link is broken.

The DarkFi blockchain is currently using a very simple consensus protocol called Streamlet. The blockchain is currently in devnet phase. This is a local testnet ran by the DarkFi community. Currently, the blockchain has no consensus token. DarkFi is working to upgrade to a privacy-enhanced proof-of-stake algorithm called Ouroborus Crypsinous.

Wallet: A wallet is a portal to the DarkFi network. It provides the user with the ability to send and receive anonymous darkened tokens. Each wallet is a full node and stores a copy of the blockchain. All contract execution is done locally on the DarkFi wallet.

P2P Network: The DarkFi ecosystem runs as a network of P2P nodes, where these nodes interact with each other over specific protocols (see node overview). Nodes communicate on a peer-to-peer network, which is also home to tools such as our P2P irc and P2P task manager tau.

ZK smart contracts: Anonymous applications on DarkFi run on proofs that enforce an order of operations. We call these zero-knowledge smart contracts. Anonymous transactions on DarkFi is possible due to the interplay of two contracts, mint and burn (see the sapling payment scheme). Using the same method, we can define advanced applications.

zkas: zkas is the compiler used to compile zk smart contracts in its respective assembly-like language. The "assembly" part was chosen as it's the bare primitives needed for zk proofs, so later on the language can be expanded with higher-level syntax. Zkas enables developers to compile and inspect contracts.

zkVM: DarkFi's zkVM executes the binaries produced by zkas. The zkVM aims to be a general-purpose zkSNARK virtual machine that empowers developers to quickly prototype and debug zk contracts. It uses a trustless zero-knowledge proof system called Halo 2 with no trusted setup.

Layers

  • system/ contains core functionality, types and utilities.
  • net/ is the p2p networking base layer.
  • service/ any services such as the cashier or gateways.
  • zk/ is the zk virtual machine and related zk code.
  • zkas/ is the zk assembly language and compiler.
  • crypto/ for the merkle tree classes and other crypto primitives.
  • node/ is a fully working darkfi node.
  • wallet/ key generation and management features such as derivation and storage.
  • tx/ for the transaction builder.
  • blockchain/ blockchain database functionality and state transition.
  • consensus/ implementation of the streamlet consensus algorithm.
  • rpc/ interface for remote procedure calls, includes adapter to interface with the node backend and the default jsonrpc transport mechanism.
  • util/ various utilities.

Anonymous assets

DarkFi network allows for the issuance and transfer of anonymous assets with an arbitrary number of parameters. These tokens are anonymous, relying on zero-knowledge proofs to ensure validity without revealing any other information.

New tokens are created and destroyed every time you send an anonymous transaction. To send a transaction on DarkFi, you must first issue a credential that commits to some value you have in your wallet. This is called the Mint phase. Once the credential is spent, it destroys itself: what is called the Burn.

Through this process, the link between inputs and outputs is broken.

Mint

During the Mint phase we create a new coin , which is bound to the public key . The coin is publicly revealed on the blockchain and added to the merkle tree, which is stored locally on the DarkFi wallet.

We do this using the following process:

Let be the coin's value. Generate random , and serial .

Create a commitment to these parameters in zero-knowledge:

Check that the value commitment is constructed correctly:

Reveal and . Add to the Merkle tree.

Burn

When we spend the coin, we must ensure that the value of the coin cannot be double spent. We call this the Burn phase. The process relies on a nullifier, which we create using the secret key for the public key . Nullifiers are unique per coin and prevent double spending. is the Merkle root. is the coin's value.

Generate a random number .

Check that the secret key corresponds to a public key:

Check that the public key corresponds to a coin which is in the merkle tree :

Check that the value commitment is constructed correctly:

Reveal , and . Check is a valid Merkle root. Check does not exist in the nullifier set.

The zero-knowledge proof confirms that binds to an unrevealed value , and that this coin is in the Merkle tree, without linking to . Once the nullifier is produced the coin becomes unspendable.

Adding values

Assets on DarkFi can have any number of values or attributes. This is achieved by creating a credential and hashing any number of values and checking that they are valid in zero-knowledge.

We check that the sum of the inputs equals the sum of the outputs. This means that:

And that is a valid point on the curve .

This proves that where is a secret blinding factor for the amounts.

Diagram

Dynamic Proof of Stake

Overview

DarkFi's current blockchain is based on Streamlet, a very simple consensus system based on voting between the participating parties. The blockchain is currently in the devnet phase and has no concept of a consensus token.

Darkfi is actively working to upgrade its consensus to Ouroboros Crypsinous, a privacy focused proof-of-stake algorithm. To accommodate this transition it has designed its data structures to be easy to upgrade.

Below is a specification of how DarkFi's current blockchain achieves consensus.

Blockchain

Blockchain is a series of epochs: it's a tree of chains, , , , , the chain of the max length in is the driving chain C.

Epoch

An epoch is a multiple of blocks. Some of those blocks might be empty due to the nature of the leader selection with VRF.

Genesis block

The first block in the epoch updates the stake for stakeholders, which influences the weighted random leader selection algorithm. For epoch j, the pair () is the genesis block's data for n stakeholders of the blockchain:

Note that new stakeholders need to wait for the next epoch to be added to the genesis block

Block

A block is the building block of the blockchain.

Block created for slot i by a stakeholder, and slot i leader :

Leader selection

At the onset of each slot each a stakeholder needs to verify if it's the weighted random leader for this slot.

check if VRF output is less than some threshold

This statement might hold true for zero or more stakeholders, thus we might end up with multiple leaders for a slot, and other times no leader. Also note that no one would know who the leader is, how many leaders are there for the slot, until you receive a signed block with a proof claiming to be a leader.

is random nonce generated from the blockchain, is block id

Note that , : the active slot coefficient is the probability that a party holding all the stake will be selected to be a leader. Stakeholder is selected as leader for slot j with probability , is stake.

The following are absolute stake aggregation dependent leader selection family of functions.

Linear family functions

In the previous leader selection function, it has the unique property of independent aggregation of the stakes, meaning the property of a leader winning leadership with stakes is independent of whether the stakeholder would act as a pool of stakes, or distributed stakes on competing coins. "one minus the probability" of winning leadership with aggregated stakes is , the joint "one minus probability" of all the stakes (each with probability winning aggregated winning the leadership thus:

A non-exponential linear leader selection can be:

Dependent aggregation

Linear leader selection has the dependent aggregation property, meaning it's favorable to compete in pools with sum of the stakes over aggregated stakes of distributed stakes:

let's assume the stakes are divided to stakes of value for , note that , thus competing with single coin of the sum of stakes held by the stakeholder is favorable.

Scalar linear aggregation dependent leader selection

A target function T with scalar coefficients can be formalized as let's assume , and then: then the lead statement is for example for a group order or l= 24 bits, and maximum value of , then lead statement:

Competing max value coins

For a stakeholder with absolute stake, it's advantageous for the stakeholder to distribute stakes on competing coins.

Inverse functions

Inverse lead selection functions doesn't require maximum stake, most suitable for absolute stake, it has the disadvantage that it's inflating with increasing rate as time goes on, but it can be function of the inverse of the slot to control the increasing frequency of winning leadership.

Leader selection without maximum stake upper limit

The inverse leader selection without maximum stake value can be and inversely proportional with probability of winning leadership, let it be called leadership coefficient.

Decaying linear leader selection

As the time goes one, and stakes increase, this means the combined stakes of all stakeholders increases the probability of winning leadership in next slots leading to more leaders at a single slot, to maintain, or to be more general to control this frequency of leaders per slot, c (the leadership coefficient) need to be function of the slot , i.e where is epoch size (number of slots in epoch).

Pairing leader selection independent aggregation function

The only family of functions that are isomorphic to summation on multiplication (having the independent aggregation property) is the exponential function, and since it's impossible to implement in plonk, a re-formalization of the lead statement using pairing that is isomorphic to summation on multiplication is an option.

Let's assume is isomorphic function between multiplication and addition, , thus: then the only family of functions satisfying this is the exponential function

no solution for the lead statement parameters, and constants defined over group of integers.

assume there is a solution for the lead statement parameters and constants defined over group of integers. for the statement , such that S where is the maximum stake value being , following from the previous proof that the family of function haveing independent aggregation property is the exponential function , and , the smallest value satisfying f is , then note that since thus , contradiction.

Leaky non-resettable beacon

Built on top of globally synchronized clock, that leaks the nonce of the next epoch a head of time (thus called leaky), non-resettable in the sense that the random nonce is deterministic at slot s, while assuring security against adversary controlling some stakeholders.

For an epoch j, the nonce is calculated by hash function H, as:

v is the concatenation of the value in all blocks from the beginning of epoch to the slot with timestamp up to , note that k is a persistence security parameter, R is the epoch length in terms of slots.

Protocol

Appendix

This section gives further details about the structures that will be used by the protocol. Since the Streamlet consensus protocol will be used at early stages of development, we created hybrid structures to enable seamless transition from Stremlet to Ouroboros Crypsinous, without the need of forking the blockchain.

Blockchain

FieldTypeDescription
blocksVec<Block>Series of blocks consisting the Blockchain
FieldTypeDescription
vu8Version
stblake3HashPrevious block hash
eu64Epoch
slu64Slot UID
timeTimestampBlock creation timestamp
rootMerkleRootRoot of the transaction hashes merkle tree

Block

FieldTypeDescription
magicu8Magic bytes
headerblake3HashHeader hash
txsVec<blake3Hash>Transaction hashes
metadataMetadataAdditional block information

BlockInfo

FieldTypeDescription
magicu8Magic bytes
headerHeaderHeader data
txsVec<Transaction>Transaction payload
metadataMetadataAdditional block information
smStreamletMetadataProposal information used by Streamlet consensus

Metadata

FieldTypeDescription
proofVRFOutputProof the stakeholder is the block owner
rSeedRandom seed for the VRF
sSignatureBlock owner signature

Streamlet Metadata

FieldTypeDescription
votesVec<Vote>Epoch votes for the block
notarizedboolBlock notarization flag
finalizedboolBlock finalization flag

DarkFi Node Architecture (DNA)

The DarkFi ecosystem runs as a network of P2P nodes, where these nodes interact with each other over specific programs (or layers). In this section, we'll explain how each of the layers fit together and when combined create a functioning network that becomes DarkFi.

The layers are organized as a bottom-up pyramid, much like the DarkFi logo:

We will start with the top-level daemon - validatord - which serves as the consensus and data storage layer, then we will explain darkfid and its communication with the layer above (validatord), and the layer below (drk).

An abstract view of the network looks like the following:

[drk] <--> [darkfid] <--> [validatord] <-+
                                         |
[drk] <--> [darkfid] <--> [validatord] <-+

validatord

validatord is the DarkFi consensus and data storage layer. Everyone that runs a validator participates in the network as a data archive, and is able to store incoming transactions, and relay them to other validators over the P2P network and protocol. Additionally, storing this data allows others to replicate it and participate in the same way.

Provided there is a locked stake on a running validator, the node can also participate in the Proof-of-Stake consensus, enabling the ability to vote on incoming transactions rather than just relaying (and validating) them.

In case the node is not participating in the consensus, it should still relay incoming transactions to other (ideally consensus-participating) validators in the network.

Inner workings

In its database, validatord stores transactions and blocks that have reached consensus. This is commonly known as a "blockchain". The blockchain is a shared state that is replicated between all validators in the network.

Additionally, validators keep a pool of incoming transactions and proposed blocks, which get validated and voted on by the consensus-participating validators.

The lifetime of an incoming transaction (and block) is as follows:

  1. Wait for a transaction
  2. Validate incoming transaction (and go back to 1. if invalid)
  3. Broadcast transaction to other validators in the network
  4. Other validators validate transaction (and go back to 1. if invalid)
  5. Leader validates the state transition and proposes a block
  6. Consensus-participating nodes validate the state transition and vote on the proposed block if the state transition is valid.
  7. If the block is confirmed, it is appended to the blockchain and is replicated between all validators in the network.

darkfid

TODO: Initial sync and retrieving wallet state?

darkfid is the client layer of DarkFi used for wallet management and transaction broadcasting. The wallet keeps a history of balances, coins, nullifiers, and merkle roots that are necessary in order to create new transactions.

By design, darkfid is a light client, since validatord stores all the blockchain data, and darkfid can simply query for anything it is interested in. This allows us to avoid data duplication and simply utilize our modular architecture. This also means that darkfid can easily be replaced with more specific tooling, if need be.

Inner workings

Using the P2P network and protocol, darkfid can subscribe to validatord in order to receive new nullifiers and merkle roots whenever a new block is confirmed. This allows darkfid to update its local state and enables it to create new valid transactions.

darkfid exposes a JSON-RPC endpoint for clients to interact with it. This allows a number of things, such as: listing balances, creating and submitting transactions, key management, and more.

When creating a new transactions, darkfid uses the local synced state in order to create new coins and combine them in a transaction. This transaction is then submitted to the above validator layer where the transaction will get validated and voted on in order to be included into a block.

drk

drk is a client tool that interacts with darkfid in a user-friendly way and provides a command-line interface to the DarkFi network and its functionality.

The interaction with darkfid is done over the JSON-RPC protocol and communicates with the endpoint exposed by darkfid.

Clients

This section gives information on DarkFi's clients, such as darkfid and cashierd. Currently this section offers documentation on the client's RPC API.

darkfid JSON-RPC API

blockchain methods

blockchain.get_slot

Queries the blockchain database for a block in the given slot. Returns a readable block upon success.
[src]

--> {"jsonrpc": "2.0", "method": "blockchain.get_slot", "params": [0], "id": 1}
<-- {"jsonrpc": "2.0", "result": {...}, "id": 1}

blockchain.merkle_roots

Queries the blockchain database for all available merkle roots.
[src]

--> {"jsonrpc": "2.0", "method": "blockchain.merkle_roots", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": [..., ..., ...], "id": 1}

tx methods

tx.transfer

Transfer a given amount of some token to the given address. Returns a transaction ID upon success.
[src]

--> {"jsonrpc": "2.0", "method": "tx.transfer", "params": ["darkfi" "gdrk", "1DarkFi...", 12.0], "id": 1}
<-- {"jsonrpc": "2.0", "result": "txID...", "id": 1}

wallet methods

wallet.keygen

Attempts to generate a new keypair and returns its address upon success.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.keygen", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": "1DarkFi...", "id": 1}

wallet.get_key

Fetches public keys by given indexes from the wallet and returns it in an encoded format. -1 is supported to fetch all available keys.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.get_key", "params": [1, 2], "id": 1}
<-- {"jsonrpc": "2.0", "result": ["foo", "bar"], "id": 1}

wallet.export_keypair

Exports the given keypair index. Returns the encoded secret key upon success.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.export_keypair", "params": [0], "id": 1}
<-- {"jsonrpc": "2.0", "result": "foobar", "id": 1}

wallet.import_keypair

Imports a given secret key into the wallet as a keypair. Returns the public counterpart as the result upon success.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.import_keypair", "params": ["foobar"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "pubfoobar", "id": 1}

wallet.set_default_address

Sets the default wallet address to the given index. Returns true upon success.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.set_default_address", "params": [2], "id": 1}
<-- {"jsonrpc": "2.0", "result": true, "id": 1}

wallet.get_balances

Queries the wallet for known balances. Returns a map of balances, indexed by network, and token ID.
[src]

--> {"jsonrpc": "2.0", "method": "wallet.get_balances", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": [{"btc": [100, "Bitcoin"]}, {...}], "id": 1}

misc methods

ping

Returns a pong to the ping request.
[src]

--> {"jsonrpc": "2.0", "method": "ping", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": "pong", "id": 1}

clock

Returns current system clock in Timestamp format.
[src]

--> {"jsonrpc": "2.0", "method": "clock", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": {...}, "id": 1}

cashierd JSON-RPC API

deposit

Executes a deposit request given network and token_id. Returns the address where the deposit shall be transferred to.
[src]

--> {"jsonrpc": "2.0", "method": "deposit", "params": ["network", "token", "publickey"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "Ht5G1RhkcKnpLVLMhqJc5aqZ4wYUEbxbtZwGCVbgU7DL", "id": 1}

withdraw

Executes a withdraw request given network, token_id, publickey and amount. publickey is supposed to correspond to network. Returns the transaction ID of the processed withdraw.
[src]

--> {"jsonrpc": "2.0", "method": "withdraw", "params": ["network", "token", "publickey", "amount"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "txID", "id": 1}

features

Returns supported cashier features, like network, listening ports, etc.
[src]

--> {"jsonrpc": "2.0", "method": "features", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": {"network": ["btc", "sol"]}, "id": 1}

zkas

zkas is a compiler for the Halo2 zkVM language used in DarkFi.

The current implementation found in the DarkFi repository inside src/zkas is the reference compiler and language implementation. It is a toolchain consisting of a lexer, parser, static and semantic analyzers, and a binary code compiler.

The main.rs file shows how this toolchain is put together to produce binary code from source code.

zkas bincode

The bincode design for zkas is the compiled code in the form of a binary blob, that can be read by a program and fed into the VM.

Our programs consist of three sections: constant, contract, and circuit. Our bincode represents the same. Additionally, there is an optional section called .debug which can hold debug info related to the binary.

We currently keep everything on the same stack, so we avoid having to deal with different types. Instead, we rely that the compiler does a proper parse and analysis of the source code, so we are sure that in the VM, when referenced, the types shall be correct.

The compiled binary blob has the following layout:

MAGIC_BYTES
BINARY_VERSION
.constant
CONSTANT_TYPE CONSTANT_NAME 
CONSTANT_TYPE CONSTANT_NAME 
...
.contract
WITNESS_TYPE
WITNESS_TYPE
...
.circuit
OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX
OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX
...
.debug
TBD

Integers in the binary are encoded using variable-integer encoding. See serial.rs for our Rust implementation.

MAGIC_BYTES

The magic bytes are the file signature consisting of four bytes used to identify the zkas binary code. They consist of:

0x0b 0xxx 0xb1 0x35

BINARY_VERSION

The binary code also contains the binary version to allow parsing potential different formats in the future.

0x01

.constant

The constants in the .constant section are declared with their type and name, so that the VM knows how to search for the builtin constant and add it to the stack.

.contract

The .contract section holds the circuit witness values in the form of WITNESS_TYPE. Their stack index is incremented for each witness as they're kept in order like in the source file. The witnesses that are of the same type as the circuit itself (typically Base) will be loaded into the circuit as private values using the Halo2 load_private API.

.circuit

The .circuit section holds the procedural logic of the ZK proof. In here we have statements with opcodes that are executed as understood by the VM. The statements are in the form of:

OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX

where:

ElementDescription
OPCODEThe opcode we wish to execute
ARG_NUMThe number of arguments given to this opcode
(Note the VM should be checking the correctness of this as well)
STACK_INDEXThe location of the argument on the stack.
(This is supposed to be repeated ARG_NUM times)

In case an opcode has a return value, the value shall be pushed to the stack and become available for later references.

.debug

TBD

Decoding the bincode

An example decoder implementation can be found in zkas' decoder.rs module.

Examples

This section holds practical and real-world examples of the use for zkas.

Sapling payment scheme

Sapling is a type of transaction which hides both the sender and receiver data, as well as the amount transacted. This means it allows a fully private transaction between two addresses.

Generally, the Sapling payment scheme consists of two ZK proofs - mint and burn. We use the mint proof to create a new coin , and we use the burn proof to spend a previously minted coin.

Mint proof

constant "Mint" {
	EcFixedPointShort VALUE_COMMIT_VALUE,
	EcFixedPoint VALUE_COMMIT_RANDOM,
}

contract "Mint" {
	Base pub_x,
	Base pub_y,
	Base value,
	Base token,
	Base serial,
	Base coin_blind,
	Scalar value_blind,
	Scalar token_blind,
}

circuit "Mint" {
	# Poseidon hash of the coin
	C = poseidon_hash(pub_x, pub_y, value, token, serial, coin_blind);
	constrain_instance(C);

	# Pedersen commitment for coin's value
	vcv = ec_mul_short(value, VALUE_COMMIT_VALUE);
	vcr = ec_mul(value_blind, VALUE_COMMIT_RANDOM);
	value_commit = ec_add(vcv, vcr);
	# Since the value commit is a curve point, we fetch its coordinates
	# and constrain them:
	value_commit_x = ec_get_x(value_commit);
	value_commit_y = ec_get_y(value_commit);
	constrain_instance(value_commit_x);
	constrain_instance(value_commit_y);

	# Pedersen commitment for coin's token ID
	tcv = ec_mul_short(token, VALUE_COMMIT_VALUE);
	tcr = ec_mul(token_blind, VALUE_COMMIT_RANDOM);
	token_commit = ec_add(tcv, tcr);
	# Since token_commit is also a curve point, we'll do the same
	# coordinate dance:
	token_commit_x = ec_get_x(token_commit);
	token_commit_y = ec_get_y(token_commit);
	constrain_instance(token_commit_x);
	constrain_instance(token_commit_y);

	# At this point we've enforced all of our public inputs.
}

As you can see, the Mint proof basically consists of three operations. First one is hashing the coin , and after that, we create Pedersen commitments1 for both the coin's value and the coin's token ID. On top of the zkas code, we've declared two constant values that we are going to use for multiplication in the commitments.

The constrain_instance call can take any of our assigned variables and enforce a public input. Public inputs are an array (or vector) of revealed values used by verifiers to verify a zero knowledge proof. In the above case of the Mint proof, since we have five calls to constrain_instance, we would also have an array of five elements that represent these public inputs. The array's order must match the order of the constrain_instance calls since they will be constrained by their index in the array (which is incremented for every call).

In other words, the vector of public inputs could look like this:

let public_inputs = vec![
    coin,
    *value_coords.x(),
    *value_coords.y(),
    *token_coords.x(),
    *token_coords.y(),
];

And then the Verifier uses these public inputs to verify a given zero knowledge proof.

Coin

During the Mint phase we create a new coin , which is bound to the public key . The coin is publicly revealed on the blockchain and added to the Merkle tree.

Let be the coin's value, be the token ID, be the unique serial number for the coin, and be a random blinding value. We create a commitment (hash) of these elements and produce the coin in zero-knowledge:

An interesting thing to keep in mind is that this commitment is extensible, so one could fit an arbitrary amount of different attributes inside it.

Value and token commitments

To have some value for our coin, we ensure it's greater than zero, and then we can create a Pedersen commitment where is the blinding factor for the commitment, and and are two predefined generators:

The token ID can be thought of as an attribute we append to our coin so we can have a differentiation of assets we are working with. In practice, this allows us to work with different tokens, using the same zero-knowledge proof circuit. For this token ID, we can also build a Pedersen commitment where is the token ID, is the blinding factor, and and are predefined generators:

Pseudo-code

Knowing this we can extend our pseudo-code and build the before-mentioned public inputs for the circuit:

#![allow(unused)]
fn main() {
    let bincode = include_bytes!("../proof/mint.zk.bin");
    let zkbin = ZkBinary::decode(bincode)?;

    // ======
    // Prover
    // ======

    // Witness values
    let value = 42;
    let token_id = pallas::Base::from(22);
    let value_blind = pallas::Scalar::random(&mut OsRng);
    let token_blind = pallas::Scalar::random(&mut OsRng);
    let serial = pallas::Base::random(&mut OsRng);
    let coin_blind = pallas::Base::random(&mut OsRng);
    let public_key = PublicKey::random(&mut OsRng);
    let coords = public_key.0.to_affine().coordinates().unwrap();

    let prover_witnesses = vec![
        Witness::Base(Value::known(*coords.x())),
        Witness::Base(Value::known(*coords.y())),
        Witness::Base(Value::known(pallas::Base::from(value))),
        Witness::Base(Value::known(token_id)),
        Witness::Base(Value::known(serial)),
        Witness::Base(Value::known(coin_blind)),
        Witness::Scalar(Value::known(value_blind)),
        Witness::Scalar(Value::known(token_blind)),
    ];

    // Create the public inputs
    let msgs = [*coords.x(), *coords.y(), pallas::Base::from(value), token_id, serial, coin_blind];
    let coin = poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<6>, 3, 2>::init()
        .hash(msgs);

    let value_commit = pedersen_commitment_u64(value, value_blind);
    let value_coords = value_commit.to_affine().coordinates().unwrap();

    let token_commit = pedersen_commitment_scalar(mod_r_p(token_id), token_blind);
    let token_coords = token_commit.to_affine().coordinates().unwrap();

    let public_inputs =
        vec![coin, *value_coords.x(), *value_coords.y(), *token_coords.x(), *token_coords.y()];

    // Create the circuit
    let circuit = ZkCircuit::new(prover_witnesses, zkbin.clone());

    let proving_key = ProvingKey::build(13, &circuit);
    let proof = Proof::create(&proving_key, &[circuit], &public_inputs, &mut OsRng)?;

    // ========
    // Verifier
    // ========

    // Construct empty witnesses
    let verifier_witnesses = empty_witnesses(&zkbin);

    // Create the circuit
    let circuit = ZkCircuit::new(verifier_witnesses, zkbin);

    let verifying_key = VerifyingKey::build(13, &circuit);
    proof.verify(&verifying_key, &public_inputs)?;
}

Burn

constant "Burn" {
	EcFixedPointShort VALUE_COMMIT_VALUE,
	EcFixedPoint VALUE_COMMIT_RANDOM,
	EcFixedPointBase NULLIFIER_K,
}

contract "Burn" {
	Base secret,
	Base serial,
	Base value,
	Base token,
	Base coin_blind,
	Scalar value_blind,
	Scalar token_blind,
	Uint32 leaf_pos,
	MerklePath path,
	Base signature_secret,
}

circuit "Burn" {
	# Poseidon hash of the nullifier
	nullifier = poseidon_hash(secret, serial);
	constrain_instance(nullifier);

	# Pedersen commitment for coin's value
	vcv = ec_mul_short(value, VALUE_COMMIT_VALUE);
	vcr = ec_mul(value_blind, VALUE_COMMIT_RANDOM);
	value_commit = ec_add(vcv, vcr);
	# Since value_commit is a curve point, we fetch its coordinates
	# and constrain them:
	value_commit_x = ec_get_x(value_commit);
	value_commit_y = ec_get_y(value_commit);
	constrain_instance(value_commit_x);
	constrain_instance(value_commit_y);

	# Pedersen commitment for coin's token ID
	tcv = ec_mul_short(token, VALUE_COMMIT_VALUE);
	tcr = ec_mul(token_blind, VALUE_COMMIT_RANDOM);
	token_commit = ec_add(tcv, tcr);
	# Since token_commit is also a curve point, we'll do the same
	# coordinate dance:
	token_commit_x = ec_get_x(token_commit);
	token_commit_y = ec_get_y(token_commit);
	constrain_instance(token_commit_x);
	constrain_instance(token_commit_y);

	# Coin hash
	pub = ec_mul_base(secret, NULLIFIER_K);
	pub_x = ec_get_x(pub);
	pub_y = ec_get_y(pub);
	C = poseidon_hash(pub_x, pub_y, value, token, serial, coin_blind);

	# Merkle root
	root = calculate_merkle_root(leaf_pos, path, C);
	constrain_instance(root);

	# Finally, we derive a public key for the signature and
	# constrain its coordinates:
	signature_public = ec_mul_base(signature_secret, NULLIFIER_K);
	signature_x = ec_get_x(signature_public);
	signature_y = ec_get_y(signature_public);
	constrain_instance(signature_x);
	constrain_instance(signature_y);

	# At this point we've enforced all of our public inputs.
}

The Burn proof consists of operations similar to the Mint proof, with the addition of a Merkle root2 calculation. In the same manner, we are doing a Poseidon hash instance, we're building Pedersen commitments for the value and token ID, and finally we're doing a public key derivation.

In this case, our vector of public inputs could look like:

let public_inputs = vec![
    nullifier,
    *value_coords.x(),
    *value_coords.y(),
    *token_coords.x(),
    *token_coords.y(),
    merkle_root,
    *sig_coords.x(),
    *sig_coords.y(),
];

Nullifier

When we spend the coin, we must ensure that the value of the coin cannot be double spent. We call this the Burn phase. The process relies on a nullifier , which we create using the secret key for the public key and a unique random serial . Nullifiers are unique per coin and prevent double spending:

Merkle root

We check that the merkle root corresponds to a coin which is in the Merkle tree

Value and token commitments

Just like we calculated these for the Mint proof, we do the same here:

Public key derivation

We check that the secret key corresponds to a public key . Usually, we do public key derivation my multiplying our secret key with a genera tor , which results in a public key:

Pseudo-code

Knowing this we can extend our pseudo-code and build the before-mentioned public inputs for the circuit:

#![allow(unused)]
fn main() {
    let bincode = include_bytes!("../proof/burn.zk.bin");
    let zkbin = ZkBinary::decode(bincode)?;

    // ======
    // Prover
    // ======

    // Witness values
    let value = 42;
    let token_id = pallas::Base::from(22);
    let value_blind = pallas::Scalar::random(&mut OsRng);
    let token_blind = pallas::Scalar::random(&mut OsRng);
    let serial = pallas::Base::random(&mut OsRng);
    let coin_blind = pallas::Base::random(&mut OsRng);
    let secret = SecretKey::random(&mut OsRng);
    let sig_secret = SecretKey::random(&mut OsRng);

    // Build the coin
    let coin2 = {
        let coords = PublicKey::from_secret(secret).0.to_affine().coordinates().unwrap();
        let messages =
            [*coords.x(), *coords.y(), pallas::Base::from(value), token_id, serial, coin_blind];

        poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<6>, 3, 2>::init()
            .hash(messages)
    };

    // Fill the merkle tree with some random coins that we want to witness,
    // and also add the above coin.
    let mut tree = BridgeTree::<MerkleNode, 32>::new(100);
    let coin0 = pallas::Base::random(&mut OsRng);
    let coin1 = pallas::Base::random(&mut OsRng);
    let coin3 = pallas::Base::random(&mut OsRng);

    tree.append(&MerkleNode(coin0));
    tree.witness();
    tree.append(&MerkleNode(coin1));
    tree.append(&MerkleNode(coin2));
    let leaf_pos = tree.witness().unwrap();
    tree.append(&MerkleNode(coin3));
    tree.witness();

    let root = tree.root(0).unwrap();
    let merkle_path = tree.authentication_path(leaf_pos, &root).unwrap();
    let leaf_pos: u64 = leaf_pos.into();

    let prover_witnesses = vec![
        Witness::Base(Value::known(secret.0)),
        Witness::Base(Value::known(serial)),
        Witness::Base(Value::known(pallas::Base::from(value))),
        Witness::Base(Value::known(token_id)),
        Witness::Base(Value::known(coin_blind)),
        Witness::Scalar(Value::known(value_blind)),
        Witness::Scalar(Value::known(token_blind)),
        Witness::Uint32(Value::known(leaf_pos.try_into().unwrap())),
        Witness::MerklePath(Value::known(merkle_path.try_into().unwrap())),
        Witness::Base(Value::known(sig_secret.0)),
    ];

    // Create the public inputs
    let nullifier = [secret.0, serial];
    let nullifier =
        poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<2>, 3, 2>::init()
            .hash(nullifier);

    let value_commit = pedersen_commitment_u64(value, value_blind);
    let value_coords = value_commit.to_affine().coordinates().unwrap();

    let token_commit = pedersen_commitment_scalar(mod_r_p(token_id), token_blind);
    let token_coords = token_commit.to_affine().coordinates().unwrap();

    let sig_pubkey = PublicKey::from_secret(sig_secret);
    let sig_coords = sig_pubkey.0.to_affine().coordinates().unwrap();

    let merkle_root = tree.root(0).unwrap();

    let public_inputs = vec![
        nullifier,
        *value_coords.x(),
        *value_coords.y(),
        *token_coords.x(),
        *token_coords.y(),
        merkle_root.0,
        *sig_coords.x(),
        *sig_coords.y(),
    ];

    // Create the circuit
    let circuit = ZkCircuit::new(prover_witnesses, zkbin.clone());

    let proving_key = ProvingKey::build(13, &circuit);
    let proof = Proof::create(&proving_key, &[circuit], &public_inputs, &mut OsRng)?;

    // ========
    // Verifier
    // ========

    // Construct empty witnesses
    let verifier_witnesses = empty_witnesses(&zkbin);

    // Create the circuit
    let circuit = ZkCircuit::new(verifier_witnesses, zkbin);

    let verifying_key = VerifyingKey::build(13, &circuit);
    proof.verify(&verifying_key, &public_inputs)?;
}
1

See section 3: The Commitment Scheme of Torben Pryds Pedersen's paper on Non-Interactive and Information-Theoretic Secure Verifiable Secret Sharing

Anonymous voting

Anonymous voting1 is a type of voting process where users can vote without revealing their identity, by proving they are accepted as valid voters.

The proof enables user privacy and allows for fully anonymous voting.

The starting point is a Merkle proof2, which efficiently proves that a voter's key belongs to a Merkle tree. However, using this proof alone would allow the organizer of a process to correlate each vote envelope with its voter's key on the database, so votes wouldn't be secret.

Vote proof

constant "Vote" {
	EcFixedPoint VALUE_COMMIT_VALUE,
	EcFixedPoint VALUE_COMMIT_RANDOM,
	EcFixedPoint NULLIFIER_K,
}

contract "Vote" {
	Base process_id_0,
	Base process_id_1,
	Base secret_key,
	Base vote,
	Scalar vote_blind,
	Uint32 leaf_pos,
	MerklePath path,
}

circuit "Vote" {
	# Nullifier hash
	process_id = poseidon_hash(process_id_0, process_id_1);
	nullifier = poseidon_hash(secret_key, process_id);
	constrain_instance(nullifier);

	# Public key derivation and hashing
	public_key = ec_mul_base(secret_key, NULLIFIER_K);
	public_x = ec_get_x(public_key);
	public_y = ec_get_y(public_key);
	pk_hash = poseidon_hash(public_x, public_y);

	# Merkle root
	root = calculate_merkle_root(leaf_pos, path, pk_hash);
	constrain_instance(root);

	# Pedersen commitment for vote
	vcv = ec_mul_short(vote, VALUE_COMMIT_VALUE);
	vcr = ec_mul(vote_blind, VALUE_COMMIT_RANDOM);
	vote_commit = ec_add(vcv, vcr);
	# Since vote_commit is a curve point, we fetch its coordinates
	# and constrain_them:
	vote_commit_x = ec_get_x(vote_commit);
	vote_commit_y = ec_get_y(vote_commit);
	constrain_instance(vote_commit_x);
	constrain_instance(vote_commit_y);
}

Our proof consists of four main operation. First we are hashing the nullifier using our secret key and the hashed process ID. Next, we derive our public key and hash it. Following, we take this hash and create a Merkle proof that it is indeed contained in the given Merkle tree. And finally, we create a Pedersen commitment3 for the vote choice itself.

Our vector of public inputs can look like this:

let public_inputs = vec![
    nullifier,
    merkle_root,
    *vote_coords.x(),
    *vote_coords.y(),
]

And then the Verifier uses these public inputs to verify the given zero-knowledge proof.

1

Specification taken from vocdoni franchise proof

3

See section 3: The Commitment Scheme of Torben Pryds Pedersen's paper on Non-Interactive and Information-Theoretic Secure Verifiable Secret Sharing

Miscellaneous tools

This section documents some miscellaneous tools provided in the DarkFi ecosystem.

vanityaddr

A tool for Vanity address generation for DarkFi keypairs. Given some prefix, the tool will bruteforce secret keys to find one which, when derived into an address, starts with a given prefix.

Usage

vanityaddr 0.3.0
Vanity address generation tool for DarkFi keypairs.

USAGE:
    vanityaddr [OPTIONS] <PREFIX>

ARGS:
    <PREFIX>    Prefix to search (must start with 1)

OPTIONS:
    -c                  Should the search be case-sensitive
    -h, --help          Print help information
    -t <THREADS>        Number of threads to use (defaults to number of available CPUs)
    -V, --version       Print version information

We can use the tool in our command line:

% vanityaddr 1Foo
[00:00:05] 53370 attempts

And the program will start crunching numbers. After a period of time, we will get JSON output containing an address, secret key, and the number of attempts it took to find the secret key.

{"address":"1FoomByzBBQywKaeBB5XPkAm5eCboh8K4CBhBe9uKbJm3kEiCS","attempts":78418,"secret":"0x16545da4a401adcd035ef51c8040acf5f4f1c66c0dd290bb5ec9e95991ae3615"}

P2P IRC

In DarkFi, we organize our communication using resilient and censorship-resistant infrastructure. For chatting, ircd is a peer-to-peer implementation of an IRC server in which any user can participate anonymously using any IRC frontend and by running the IRC daemon. ircd uses the DarkFi P2P engine to synchronize chats between hosts.

Installation

% git clone https://github.com/darkrenaissance/darkfi 
% cd darkfi
% make BINS=ircd
% sudo make install BINS=ircd

Usage (DarkFi Network)

Upon installing ircd as described above, the preconfigured defaults will allow you to connect to the network and start chatting with the rest of the DarkFi community.

First, try to start ircd from your command-line so it can spawn its configuration file in place. The preconfigured defaults will autojoin you to the #dev channel, where the community is most active and talks about DarkFi development.

% ircd

After running it for the first time, ircd will create a configuration file you can review and potentially edit. It might be useful if you want to add other channels you want to autojoin (like #philosophy and #memes), or if you want to set a shared secret for some channel in order for it to be encrypted between its participants.

When done, you can run ircd for the second time in order for it to connect to the network and start participating in the P2P protocol:

% ircd

Clients

Weechat

In this section, we'll briefly cover how to use the Weechat IRC client to connect and chat with ircd.

Normally, you should be able to install weechat using your distribution's package manager. If not, have a look at the weechat git repository for instructions on how to install it on your computer.

Once installed, we can configure a new server which will represent our ircd instance. First, start weechat, and in its window - run the following commands (there is an assumption that irc_listen in the ircd config file is set to 127.0.0.1:11066):

/server add darkfi localhost/11066 -autoconnect
/save
/quit

This will set up the server, save the settings, and exit weechat. You are now ready to begin using the chat. Simply start weechat and everything should work.

We can now proceed with installing the mallumo weechat script, which is used for E2E encryption in private messages on this IRC network.

E2E encryption with mallumo

mallumo is a Python plugin for Weechat that can be used to enable end-to-end encryption for private messages between you and other users of the DarkFi IRC network. The verbose installation and configuration instructions can be found in the mallumo git repository.

Briefly, make sure you install python3 and pynacl (can usually be done with your distribution's package manager or pip).

Then find where weechat has put its configuration files. It is usually ~/.weechat or ~/.local/share/weechat (here we will assume the latter). Go to the directory, clone the repo, and make a couple of symlinks:

% cd ~/.local/share/weechat
% mkdir -p src
% git clone https://github.com/darkrenaissance/mallumo src/mallumo
% cd python
% ln -s `realpath ../src/mallumo/mallumo` mallumo
% ln -s `realpath ../src/mallumo/mallumo/__init__.py` autoload/mallumo.py

Refer to darkrenaissance/mallumo and its README for usage instructions.

After this has been set up, the next time you start ircd and then weechat, you will be connected to the DarkFi IRC network and be able to chat with other participants.

Usage (Local Deployment)

These steps below are only for developers who wish to make a testing deployment. The previous sections are sufficient to join the chat.

Seed Node

First you must run a seed node. The seed node is a static host which nodes can connect to when they first connect to the network. The seed_session simply connects to a seed node and runs protocol_seed, which requests a list of addresses from the seed node and disconnects straight after receiving them.

The first time you run the program, a config file will be created in .config/darkfi. You must specify an inbound accept address in your config file to configure a seed node:

## P2P accept address
inbound="127.0.0.1:11001" 

Note that the above config doesn't specify an external address since the seed node shouldn't be advertised in the list of connectable nodes. The seed node does not participate as a normal node in the p2p network. It simply allows new nodes to discover other nodes in the network during the bootstrapping phase.

Inbound Node

This is a node accepting inbound connections on the network but which is not making any outbound connections.

The external address is important and must be correct.

To run an inbound node, your config file must contain the following info:

## P2P accept address
inbound="127.0.0.1:11002" 

## P2P external address
external_addr="127.0.0.1:11002"

## Seed nodes to connect to 
seeds=["127.0.0.1:11001"]

Outbound Node

This is a node which has 8 outbound connection slots and no inbound connections. This means the node has 8 slots which will actively search for unique nodes to connect to in the p2p network.

In your config file:

## Connection slots
outbound_connections=8

## Seed nodes to connect to 
seeds=["127.0.0.1:11001"]

Attaching the IRC Frontend

Assuming you have run the above 3 commands to create a small model testnet, and both inbound and outbound nodes above are connected, you can test them out using weechat.

To create separate weechat instances, use the --dir command:

weechat --dir /tmp/a/
weechat --dir /tmp/b/

Then in both clients, you must set the option to connect to temporary servers:

/set irc.look.temporary_servers on

Finally you can attach to the local IRCd instances:

/connect localhost/6667
/connect localhost/6668

And send messages to yourself.

Running a Fullnode

See the script script/run_node.sh for an example of how to deploy a full node which does seed session synchronization, and accepts both inbound and outbound connections.

Tau

Encrypted tasks management app using peer-to-peer network and raft consensus.
multiple users can collaborate by working on the same tasks, and all users will have synced task list.

Install

% git clone https://github.com/darkrenaissance/darkfi 
% cd darkfi
% make BINS="taud tau"
% sudo make install "BINS=taud tau"

Usage (Local Deployment)

Seed Node

First you must run a seed node. The seed node is a static host which nodes can connect to when they first connect to the network. The seed_session simply connects to a seed node and runs protocol_seed, which requests a list of addresses from the seed node and disconnects straight after receiving them.

in config file:

	## P2P accept address
	inbound="127.0.0.1:11001" 

Note that the above config doesn't specify an external address since the seed node shouldn't be advertised in the list of connectable nodes. The seed node does not participate as a normal node in the p2p network. It simply allows new nodes to discover other nodes in the network during the bootstrapping phase.

Inbound Node

This is a node accepting inbound connections on the network but which is not making any outbound connections.

The external address is important and must be correct.

in config file:
	
	## P2P accept address
	inbound="127.0.0.1:11002" 
	
	## P2P external address
	external_addr="127.0.0.1:11002"

	## Seed nodes to connect to 
	seeds=["127.0.0.1:11001"]

Outbound Node

This is a node which has 8 outbound connection slots and no inbound connections. This means the node has 8 slots which will actively search for unique nodes to connect to in the p2p network.

in config file:

	## Connection slots
	outbound_connections=8

	## Seed nodes to connect to 
	seeds=["127.0.0.1:11001"]

Also note that for the first time ever running seed node you must run it with --key-gen:

% taud --key-gen

This will generate a new secret key in /home/${USER}/.config/tau/secret_key that you can share with nodes you want them to get and decrypt your tasks, otherwise if you have already generated or got a copy from a peer place it in the same directory /home/${USER}/.config/tau/secret_key.

Usage (CLI)

% tau --help 
tau 0.3.0
Command-line client for taud

USAGE:
	tau [OPTIONS] <SUBCOMMAND>

OPTIONS:
	-e, --endpoint <ENDPOINT>    taud JSON-RPC endpoint [default: tcp://127.0.0.1:11055]
	-h, --help                   Print help information
	-v                           Increase verbosity (-vvv supported)
	-V, --version                Print version information

SUBCOMMANDS:
	add        Add a new task                                                    
	comment    Set or Get comment for a task
	help       Print this message or the help of the given subcommand(s)
	info       Get task info by ID
	state      Set or Get task state
	update     Update/Edit an existing task by ID
% tau help [SUBCOMMAND]

Example

% # add new task  
% tau add "new title"   
% tau add "new title" project:blockchain desc:"new description" rank:3 assign:dark
% 
% # lists tasks
% tau 		   		 
% tau open	# open tasks
% tau pause	# paused tasks
% tau 0522	# created at May 2022
% tau project:blockchain assign:dark
% tau rank:gt:n	# lists all tasks that have rank greater than n
% tau rank:ls:n	# lists all tasks that have rank lesser than n
% 
% # update task 
% tau update 3 project:network rank:20
% 
% # state 
% tau state 3		# get state
% tau state 3 pause	# set the state to pause 
% 
% # comments 
% tau comment 1			# list comments
% tau comment 3 "new comment"	# add new comment 

Dnetview

A simple tui to explore darkfi ircd network topology.

dnetview displays:

  1. all active nodes
  2. outgoing, incoming and manual sessions
  3. each associated connection and recent messages.

dnetview is based on the design-pattern Model, View, Controller. We create a logical separation between the underlying data structure or Model; the ui rendering aspect which is the View; and the Controller or game engine that makes everything run.

Install

% git clone https://github.com/darkrenaissance/darkfi 
% cd darkfi
% make BINS=dnetview

Usage

Run dnetview as follows:

dnetview -v

On first run, dnetview will create a config file in .config/darkfi. You must manually enter the RPC ports of the nodes you want to connect to and title them as you see fit.

Dnetview creates a logging file in /tmp/dnetview.log. To see json data and other debug info, tail the file like so:

tail -f /tmp/dnetview.log

Learn

This section contains learning resources related to DarkFi.

Research

DarkFi maintains a public resource of zero-knowledge and math research in the script/research directory of the repo.

It features simple sage implementations of zero-knowledge algorithms and math primitives, including but not limited to:

Zero-knowledge explainer

We start with this algorithm as an example:

def foo(w, a, b):
    if w:
        return a * b
    else:
        return a + b

ZK code consists of lines of constraints. It has no concept of branching conditionals or loops.

So our first task is to flatten (convert) the above code to a linear equation that can be evaluated in ZK.

Consider an interesting fact. For any value , then if and only if .

In our code above is a binary value. It's value is either or . We make use of this fact by the following:

  1. when
  2. when . If then the expression is .

So we can rewrite foo(w, a, b) as the mathematical function

We now can convert this expression to a constraint system.

ZK statements take the form of:

More succinctly as:

These statements are converted into polynomials of the form:

is the target polynomial and in our case will be . is the cofactor polynomial. The statement says that the polynomial has roots (is equal to zero) at the points when .

Earlier we wrote our mathematical statement which we will now convert to constraints.

Rearranging the equation, we note that:

Swapping and rearranging, our final statement becomes . Represented in ZK as:

The last line is a boolean constraint that is either or by enforcing that (re-arranged this is ).

LineL(x)R(x)O(x)
1
2
3

Because of how the polynomials are created during the setup phase, you must supply them with the correct variables that satisfy these constraints, so that (line 1), (line 2) and (line 3).

Each one of , and is supplied a list of (constant coefficient, variable value) pairs.

In bellman library, the constant is a fixed value of type Scalar. The variable is a type called Variable. These are the values fed into lc0 (the 'left' polynomial), lc1 (the 'right' polynomial), and lc2 (the 'out' polynomial).

In our example we had a function where for example . The verifier does not know the variables , and which are allocated by the prover as variables. However the verifier does know the coefficients (which are of the Scalar type) shown in the table above. In our example they only either or , but can also be other constant values.

#![allow(unused)]
fn main() {
pub struct LinearCombination<Scalar: PrimeField>(Vec<(Variable, Scalar)>);
}

It is important to note that each one of the left, right and out registers is simply a list of tuples of (constant coefficient, variable value).

When we wish to add a constant value, we use the variable called ~one (which is always the first automatically allocated variable in bellman at index 0). Therefore we end up adding our constant to the LinearCombination as (c, ~one).

Any other non-constant value, we wish to add to our constraint system must be allocated as a variable. Then the variable is added to the LinearCombination. So in our example, we will allocate , getting back Variable objects which we then add to the left lc, right lc or output lc.