DarkFi
Build
This project requires the Rust compiler to be installed. Please visit Rustup for instructions.
The following dependencies are also required:
Dependency | Debian-based |
---|---|
gcc, gcc-c++, kernel headers | build-essential |
cmake | cmake |
jq | jq |
wget | wget |
pkg-config | pkg-config |
clang | clang |
clang libs | libclang-dev |
llvm libs | llvm-dev |
udev libs | libudev-dev |
freetype2 libs | libfreetype6-dev |
expat xml lib | libexpat1-dev |
Users of Debian-based systems (e.g. Ubuntu) can simply run the following to install the required dependencies:
# apt-get update
# apt-get install -y build-essential cmake jq wget pkg-config \
clang libclang-dev llvm-dev libudev-dev libfreetype6-dev \
libexpat1-dev
Alternatively, users can chose one of the automated scripts
under contrib
folder by executing:
% bash contrib/*_setup.sh
The following setup script are provided:
- mac_setup.sh: installation using brew (brew will be installed if not present).
- void_setup.sh: Xbps dependencies for Void Linux.
To build the necessary binaries, we can just clone the repo, and use the provided Makefile to build the project. This will download the trusted setup params, and compile the source code.
% git clone https://github.com/darkrenaissance/darkfi
% cd darkfi/
% make
Development
If you want to hack on the source code, make sure to read some introductory advice in the DarkFi book.
Install
This will install the binaries on your system (/usr/local
by
default). The configuration files for the binaries are bundled with the
binaries and contain sane defaults. You'll have to run each daemon once
in order for them to spawn a config file, which you can then review.
# make install
Bash Completion
This will add the options auto completion of drk
and darkfid
.
% echo source (pwd)/contrib/auto-complete >> ~/.bashrc
Examples and usage
See the DarkFi book
Go Dark
Let's liberate people from the claws of big tech and create the democratic paradigm of technology.
Self-defense is integral to any organism's survival and growth.
Power to the minuteman.
Definition of Democratic Civilization
From 'The Sociology of Freedom: Manifesto of the Democratic Civilization, Volume 3' by Abdullah Ocalan.
Annotations are our own. The text is otherwise unchanged.
What is the subject of moral and political society?
The school of social science that postulates the examination of the existence and development of social nature on the basis of moral and political society could be defined as the democratic civilization system. The various schools of social science base their analyses on different units. Theology and religion prioritize society. For scientific socialism, it is class. The fundamental unit for liberalism is the individual. There are, of course, schools that prioritize power and the state and others that focus on civilization. All these unit-based approaches must be criticized, because, as I have frequently pointed out, they are not historical, and they fail to address the totality. A meaningful examination would have to focus on what is crucial from the point of view of society, both in terms of history and actuality. Otherwise, the result will only be one more discourse.
Identifying our fundamental unit as moral and political society is significant, because it also covers the dimensions of historicity and totality. Moral and political society is the most historical and holistic expression of society. Morals and politics themselves can be understood as history. A society that has a moral and political dimension is a society that is the closest to the totality of all its existence and development. A society can exist without the state, class, exploitation, the city, power, or the nation, but a society devoid of morals and politics is unthinkable. Societies may exist as colonies of other powers, particularly capital and state monopolies, and as sources of raw materials. In those cases, however, we are talking about the legacy of a society that has ceased to be.
Individualism is a state of war
There is nothing gained by labeling moral and political society—the natural state of society—as slave-owning, feudal, capitalist, or socialist. Using such labels to describe society masks reality and reduces society to its components (class, economy, and monopoly). The bottleneck encountered in discourses based on such concepts as regards the theory and practice of social development stems from errors and inadequacies inherent in them. If all of the analyses of society referred to with these labels that are closer to historical materialism have fallen into this situation, it is clear that discourses with much weaker scientific bases will be in a much worse situation. Religious discourses, meanwhile, focus heavily on the importance of morals but have long since turned politics over to the state. Bourgeois liberal approaches not only obscure the society with moral and political dimensions, but when the opportunity presents itself they do not hesitate to wage war on this society. Individualism is a state of war against society to the same degree as power and the state is. Liberalism essentially prepares society, which is weakened by being deprived of its morals and politics, for all kinds of attacks by individualism. Liberalism is the ideology and practice that is most anti-society.
The rise of scientific positivism
In Western sociology (there is still no science called Eastern sociology) concepts such as society and civilization system are quite problematic. We should not forget that the need for sociology stemmed from the need to find solutions to the huge problems of crises, contradictions, and conflicts and war caused by capital and power monopolies. Every branch of sociology developed its own thesis about how to maintain order and make life more livable. Despite all the sectarian, theological, and reformist interpretations of the teachings of Christianity, as social problems deepened, interpretations based on a scientific (positivist) point of view came to the fore. The philosophical revolution and the Enlightenment (seventeenth and eighteenth centuries) were essentially the result of this need. When the French Revolution complicated society’s problems rather than solving them, there was a marked increase in the tendency to develop sociology as an independent science. Utopian socialists (Henri de Saint-Simon, Charles Fourier, and Pierre-Joseph Proudhon), together with Auguste Comte and Émile Durkheim, represent the preliminary steps in this direction. All of them are children of the Enlightenment, with unlimited faith in science. They believed they could use science to re-create society as they wished. They were playing God. In Hegel’s words, God had descended to earth and, what’s more, in the form of the nation-state. What needed to be done was to plan and develop specific and sophisticated “social engineering” projects. There was no project or plan that could not be achieved by the nation-state if it so desired, as long as it embraced the “scientific positivism” and was accepted by the nation-state!
Capitalism as an iron cage
British social scientists (political economists) added economic solutions to French sociology, while German ideologists contributed philosophically. Adam Smith and Hegel in particular made major contributions. There was a wide variety of prescriptions from both the left and right to address the problems arising from the horrendous abuse of the society by the nineteenth-century industrial capitalism. Liberalism, the central ideology of the capitalist monopoly has a totally eclectic approach, taking advantage of any and all ideas, and is the most practical when it comes to creating almost patchwork-like systems. It was as if the right- and left- wing schematic sociologies were unaware of social nature, history, and the present while developing their projects in relation to the past (the quest for the “golden age” by the right) or the future (utopian society). Their systems would continually fragment when they encountered history or current life. The reality that had imprisoned them all was the “iron cage” that capitalist modernity had slowly cast and sealed them in, intellectually and in their practical way of life. However, Friedrich Nietzsche’s ideas of metaphysicians of positivism or castrated dwarfs of capitalist modernity bring us a lot closer to the social truth. Nietzsche leads the pack of rare philosophers who first drew attention to the risk of society being swallowed up by capitalist modernity. Although he is accused of serving fascism with his thoughts, his foretelling of the onset of fascism and world wars was quite enticing.
The increase in major crises and world wars, along with the division of the liberal center into right- and left-wing branches, was enough to bankrupt positivist sociology. In spite of its widespread criticism of metaphysics, social engineering has revealed its true identity with authoritarian and totalitarian fascism as metaphysics at its shallowest. The Frankfurt School is the official testimonial of this bankruptcy. The École Annales and the 1968 youth uprising led to various postmodernist sociological approaches, in particular Immanuel Wallerstein’s capitalist world-system analysis. Tendencies like ecology, feminism, relativism, the New Left, and world-system analysis launched a period during which the social sciences splintered. Obviously, financial capital gaining hegem- ony as the 1970s faded also played an important role. The upside of these developments was the collapse of the hegemony of Eurocentric thought. The downside, however, was the drawbacks of a highly fragmented social sciences.
The problems of Eurocentric sociology
Let’s summarize the criticism of Eurocentric sociology:
-
Positivism, which criticized and denounced both religion and metaphysics, has not escaped being a kind of religion and metaphysics in its own right. This should not come as a surprise. Human culture requires metaphysics. The issue is to distinguish good from bad metaphysics.
-
An understanding of society based on dichotomies like primitive vs. modern, capitalist vs. socialist, industrial vs. agrarian, progressive vs. reactionary, divided by class vs. classless, or with a state vs. stateless prevents the development of a definition that comes closer to the truth of social nature. Dichotomies of this sort distance us from social truth.
-
To re-create society is to play the modern god. More precisely, each time society is recreated there is a tendency to form a new capital and power-state monopoly. Much like medieval theism was ideologically connected to absolute monarchies (sultanates and shāhanshāhs), modern social engineering—as re-creation— is essentially the divine disposition and ideology of the nation- state. Positivism in this regard is modern theism.
-
Revolutions cannot be interpreted as the re-creation acts of society. When thusly understood they cannot escape positivist theism. Revolutions can only be defined as social revolutions to the extent that they free society from excessive burden of capital and power.
-
The task of revolutionaries cannot be defined as creating any social model of their making but more correctly as playing a role in contributing to the development of moral and political society.
-
Methods and paradigms to be applied to social nature should not be identical to those that relate to first nature. While the universalist approach to first nature provides results that come closer to the truth (I don’t believe there is an absolute truth), relativism in relation to social nature may get us closer to the truth. The universe can neither be explained by an infinite universalist linear discourse or by a concept of infinite similar circular cycles.
-
A social regime of truth needs to be reorganized on the basis of these and many other criticisms. Obviously, I am not talking about a new divine creation, but I do believe that the greatest feature of the human mind is the power to search for and build truth.
A new social science
In light of these criticisms, I offer the following suggestions in relation to the social science system that I want to define:
A more humane social nature
-
I would not present social nature as a rigid universalist truth with mythological, religious, metaphysical, and scientific (positivist) patterns. Understanding it to be the most flexible form of basic universal entities that encompass a wealth of diversities but are tied down to conditions of historical time and location more closely approaches the truth. Any analysis, social science, or attempt to make practical change without adequate knowledge of the qualities of social nature may well backfire. The monotheistic religions and positivism, which have appeared throughout the history of civilization claiming to have found the solution, were unable to prevent capital and power monopolies from gaining control. It is therefore their irrevocable task, if they are to contribute to moral and political society, to develop a more humane analysis based on a profound self-criticism.
-
Moral and political society is the main element that gives social nature its historical and complete meaning and represents the unity in diversity that is basic to its existence. It is the definition of moral and political society that gives social nature its character, maintains its unity in diversity, and plays a decisive role in expressing its main totality and historicity. The descriptors commonly used to define society, such as primitive, modern, slave-owning, feudal, capitalist, socialist, industrial, agricultural, commercial, monetary, statist, national, hegemonic, and so on, do not reflect the decisive features of social nature. On the contrary, they conceal and fragment its meaning. This, in turn, provides a base for faulty theoretical and practical approaches and actions related to society.
Protecting the social fabric
-
Statements about renewing and re-creating society are part of operations meant to constitute new capital and power monopolies in terms of their ideological content. The history of civilization, the history of such renewals, is the history of the cumulative accumulation of capital and power. Instead of divine creativity, the basic action the society needs most is to struggle against factors that prevent the development and functioning of moral and political social fabric. A society that operates its moral and political dimensions freely, is a society that will continue its development in the best way.
-
Revolutions are forms of social action resorted to when society is sternly prevented from freely exercising and maintaining its moral and political function. Revolutions can and should be accepted as legitimate by society only when they do not seek to create new societies, nations, or states but to restore moral and political society its ability to function freely.
-
Revolutionary heroism must find meaning through its contributions to moral and political society. Any action that does not have this meaning, regardless of its intent and duration, cannot be defined as revolutionary social heroism. What determines the role of individuals in society in a positive sense is their contribution to the development of moral and political society.
-
No social science that hopes to develop these key features through profound research and examination should be based on a universalist linear progressive approach or on a singular infinite cyclical relativity. In the final instance, instead of these dogmatic approaches that serve to legitimize the cumulative accumulation of capital and power throughout the history of civilization, social sciences based on a non-destructive dialectic methodology that harmonizes analytical and emotional intelligence and overcomes the strict subject-object mold should be developed.
The framework of moral and political society
The paradigmatic and empirical framework of moral and political society, the main unit of the democratic civilization system, can be presented through such hypotheses. Let me present its main aspects:
-
Moral and political society is the fundamental aspect of human society that must be continuously sought. Society is essentially moral and political.
-
Moral and political society is located at the opposite end of the spectrum from the civilization systems that emerged from the triad of city, class, and state (which had previously been hierarchical structures).
-
Moral and political society, as the history of social nature, develops in harmony with the democratic civilization system.
-
Moral and political society is the freest society. A functioning moral and political fabric and organs is the most decisive dynamic not only for freeing society but to keep it free. No revolution or its heroines and heroes can free the society to the degree that the development of a healthy moral and political dimension will. Moreover, revolution and its heroines and heroes can only play a decisive role to the degree that they contribute to moral and political society.
-
A moral and political society is a democratic society. Democracy is only meaningful on the basis of the existence of a moral and political society that is open and free. A democratic society where individuals and groups become subjects is the form of governance that best develops moral and political society. More precisely, we call a functioning political society a democracy. Politics and democracy are truly identical concepts. If freedom is the space within which politics expresses itself, then democracy is the way in which politics is exercised in this space. The triad of freedom, politics, and democracy cannot lack a moral basis. We could refer to morality as the institutionalized and traditional state of freedom, politics, and democracy.
-
Moral and political societies are in a dialectical contradiction with the state, which is the official expression of all forms of capital, property, and power. The state constantly tries to substitute law for morality and bureaucracy for politics. The official state civilization develops on one side of this historically ongoing contradiction, with the unofficial democratic civilization system developing on the other side. Two distinct typologies of meaning emerge. Contradictions may either grow more violent and lead to war or there may be reconciliation, leading to peace.
-
Peace is only possible if moral and political society forces and the state monopoly forces have the will to live side by side unarmed and with no killing. There have been instances when rather than society destroying the state or the state destroying society, a conditional peace called democratic reconciliation has been reached. History doesn’t take place either in the form of democratic civilization—as the expression of moral and political society—or totally in the form of civilization systems—as the expression of class and state society. History has unfolded as intense relationship rife with contradiction between the two, with successive periods of war and peace. It is quite utopian to think that this situation, with at least a five-thousand-year history, can be immediately resolved by emergency revolutions. At the same time, to embrace it as if it is fate and cannot be interfered with would also not be the correct moral and political approach. Knowing that struggles between systems will be protracted, it makes more sense and will prove more effective to adopt strategic and tactical approaches that expand the freedom and democracy sphere of moral and political society.
-
Defining moral and political society in terms of communal, slave-owning, feudal, capitalist, and socialist attributes serves to obscure rather than elucidate matters. Clearly, in a moral and political society there is no room for slave-owning, feudal, or capitalist forces, but, in the context of a principled reconciliation, it is possible to take an aloof approach to these forces, within limits and in a controlled manner. What’s important is that moral and political society should neither destroy them nor be swallowed up by them; the superiority of moral and political society should make it possible to continuously limit the reach and power of the central civilization system. Communal and socialist systems can identify with moral and political society insofar as they themselves are democratic. This identification is, however, not possible, if they have a state.
-
Moral and political society cannot seek to become a nation-state, establish an official religion, or construct a non-democratic regime. The right to determine the objectives and nature of society lies with the free will of all members of a moral and political society. Just as with current debates and decisions, strategic decisions are the purview of society’s moral and political will and expression. The essential thing is to have discussions and to become a decision-making power. A society who holds this power can determine its preferences in the soundest possible way. No individual or force has the authority to decide on behalf of moral and political society, and social engineering has no place in these societies.
Liberating democratic civilization from the State
When viewed in the light of the various broad definitions I have presented, it is obvious that the democratic civilization system—essentially the moral and political totality of social nature—has always existed and sustained itself as the flip side of the official history of civilization. Despite all the oppression and exploitation at the hands of the official world-system, the other face of society could not be destroyed. In fact, it is impossible to destroy it. Just as capitalism cannot sustain itself without noncapitalist society, civilization— the official world system— also cannot sustain itself without the democratic civilization system. More concretely the civilization with monopolies cannot sustain itself without the existence of a civilization without monopolies. The opposite is not true. Democratic civilization, representing the historical flow of the system of moral and political society, can sustain itself more comfortably and with fewer obstacles in the absence of the official civilization.
I define democratic civilization as a system of thought, the accumulation of thought, and the totality of moral rules and political organs. I am not only talking about a history of thought or the social reality within a given moral and political development. The discussion does, however, encompass both issues in an intertwined manner. I consider it important and necessary to explain the method in terms of democratic civilization’s history and elements, because this totality of alternate discourse and structures are prevented by the official civilization. I will address these issues in subsequent sections.
Notes for developers
Making life easy for others
Write useful commit messages.
If your commit is changing a specific module in the code and not touching other parts of the codebase (as should be the case 99% of the time), consider writing a useful commit message that also mentions which module was changed.
For example, a message like:
added foo
is not as clear as
crypto/keypair: Added foo method for Bar struct.
Also keep in mind that commit messages can be longer than a single line, so use it to your advantage to explain your commit and intentions.
cargo fmt pre-commit hook
To ensure every contributor uses the same code style, make sure
you run cargo +nightly fmt
before committing. You can force yourself
to do this by creating a git pre-commit
hook like the following:
#!/bin/sh
if ! cargo +nightly fmt -- --check >/dev/null; then
echo "There are some code style issues. Run 'cargo +nightly fmt' to fix it."
exit 1
fi
exit 0
Place this script in .git/hooks/pre-commit
and make sure it's
executable by running chmod +x .git/hooks/pre-commit
.
Testing crate features
Our library heavily depends on cargo features. Currently
there are more than 650 possible combinations of features to
build the library. To ensure everything can always compile
and works, we can use a helper for cargo
called
cargo hack
.
The Makefile
provided in the repository is already set up to use it,
so it's enough to install cargo hack
and run make check
.
Architecture design
This section of the book shows the software architecture of DarkFi and the network implementations.
Overview
DarkFi is a layer one proof-of-stake blockchain that supports anonymous applications. It is currently under development. This overview will outline a few key terms that help explain DarkFi.
Cashier: The Cashier is the entry and exit point to the DarkFi network from other blockchains such as Ethereum, Bitcoin and Solana. It is essentially the bridge. Its role is to exchange cryptocurrency assets for anonymous darkened tokens that are pegged to the underlying currency, and visa versa. Currently, the role of the Cashier is trusted and centralized. As a next step, DarkFi plans to implement trust-minimized bridges and eventually fully trustless bridges.
Blockchain: Once new anonymous tokens (e.g. dETH) have been issued, the Cashier posts that data on the blockchain. This data is encrypted and the transaction link is broken.
The DarkFi blockchain is currently using a very simple consensus protocol called Streamlet. The blockchain is currently in devnet phase. This is a local testnet ran by the DarkFi community. Currently, the blockchain has no consensus token. DarkFi is working to upgrade to a privacy-enhanced proof-of-stake algorithm called Ouroborus Crypsinous.
Wallet: A wallet is a portal to the DarkFi network. It provides the user with the ability to send and receive anonymous darkened tokens. Each wallet is a full node and stores a copy of the blockchain. All contract execution is done locally on the DarkFi wallet.
P2P Network: The DarkFi ecosystem runs as a network of P2P nodes, where these nodes interact with each other over specific protocols (see node overview). Nodes communicate on a peer-to-peer network, which is also home to tools such as our P2P irc and P2P task manager tau.
ZK smart contracts: Anonymous applications on DarkFi run on proofs that enforce an order of operations. We call these zero-knowledge smart contracts. Anonymous transactions on DarkFi is possible due to the interplay of two contracts, mint and burn (see the sapling payment scheme). Using the same method, we can define advanced applications.
zkas: zkas is the compiler used to compile zk smart contracts in its respective assembly-like language. The "assembly" part was chosen as it's the bare primitives needed for zk proofs, so later on the language can be expanded with higher-level syntax. Zkas enables developers to compile and inspect contracts.
zkVM: DarkFi's zkVM executes the binaries produced by zkas. The zkVM aims to be a general-purpose zkSNARK virtual machine that empowers developers to quickly prototype and debug zk contracts. It uses a trustless zero-knowledge proof system called Halo 2 with no trusted setup.
Layers
- system/ contains core functionality, types and utilities.
- net/ is the p2p networking base layer.
- service/ any services such as the cashier or gateways.
- zk/ is the zk virtual machine and related zk code.
- zkas/ is the zk assembly language and compiler.
- crypto/ for the merkle tree classes and other crypto primitives.
- node/ is a fully working darkfi node.
- wallet/ key generation and management features such as derivation and storage.
- tx/ for the transaction builder.
- blockchain/ blockchain database functionality and state transition.
- consensus/ implementation of the streamlet consensus algorithm.
- rpc/ interface for remote procedure calls, includes adapter to interface with the node backend and the default jsonrpc transport mechanism.
- util/ various utilities.
Anonymous assets
DarkFi network allows for the issuance and transfer of anonymous assets with an arbitrary number of parameters. These tokens are anonymous, relying on zero-knowledge proofs to ensure validity without revealing any other information.
New tokens are created and destroyed every time you send an anonymous transaction. To send a transaction on DarkFi, you must first issue a credential that commits to some value you have in your wallet. This is called the Mint phase. Once the credential is spent, it destroys itself: what is called the Burn.
Through this process, the link between inputs and outputs is broken.
Mint
During the Mint phase we create a new coin , which is bound to the public key . The coin is publicly revealed on the blockchain and added to the merkle tree, which is stored locally on the DarkFi wallet.
We do this using the following process:
Let be the coin's value. Generate random , and serial .
Create a commitment to these parameters in zero-knowledge:
Check that the value commitment is constructed correctly:
Reveal and . Add to the Merkle tree.
Burn
When we spend the coin, we must ensure that the value of the coin cannot be double spent. We call this the Burn phase. The process relies on a nullifier, which we create using the secret key for the public key . Nullifiers are unique per coin and prevent double spending. is the Merkle root. is the coin's value.
Generate a random number .
Check that the secret key corresponds to a public key:
Check that the public key corresponds to a coin which is in the merkle tree :
Check that the value commitment is constructed correctly:
Reveal , and . Check is a valid Merkle root. Check does not exist in the nullifier set.
The zero-knowledge proof confirms that binds to an unrevealed value , and that this coin is in the Merkle tree, without linking to . Once the nullifier is produced the coin becomes unspendable.
Adding values
Assets on DarkFi can have any number of values or attributes. This is achieved by creating a credential and hashing any number of values and checking that they are valid in zero-knowledge.
We check that the sum of the inputs equals the sum of the outputs. This means that:
And that is a valid point on the curve .
This proves that where is a secret blinding factor for the amounts.
Diagram
Dynamic Proof of Stake
Overview
DarkFi's current blockchain is based on Streamlet, a very simple consensus system based on voting between the participating parties. The blockchain is currently in the devnet phase and has no concept of a consensus token.
Darkfi is actively working to upgrade its consensus to Ouroboros Crypsinous, a privacy focused proof-of-stake algorithm. To accommodate this transition it has designed its data structures to be easy to upgrade.
Below is a specification of how DarkFi's current blockchain achieves consensus.
Blockchain
Blockchain is a series of epochs: it's a tree of chains, , , , , the chain of the max length in is the driving chain C.
Epoch
An epoch is a multiple of blocks. Some of those blocks might be empty due to the nature of the leader selection with VRF.
Genesis block
The first block in the epoch updates the stake for stakeholders, which influences the weighted random leader selection algorithm. For epoch j, the pair () is the genesis block's data for n stakeholders of the blockchain:
Note that new stakeholders need to wait for the next epoch to be added to the genesis block
Block
A block is the building block of the blockchain.
Block created for slot i by a stakeholder, and slot i leader :
Leader selection
At the onset of each slot each a stakeholder needs to verify if it's the weighted random leader for this slot.
This statement might hold true for zero or more stakeholders, thus we might end up with multiple leaders for a slot, and other times no leader. Also note that no one would know who the leader is, how many leaders are there for the slot, until you receive a signed block with a proof claiming to be a leader.
Note that , : the active slot coefficient is the probability that a party holding all the stake will be selected to be a leader. Stakeholder is selected as leader for slot j with probability , is stake.
The following are absolute stake aggregation dependent leader selection family of functions.
Linear family functions
In the previous leader selection function, it has the unique property of independent aggregation of the stakes, meaning the property of a leader winning leadership with stakes is independent of whether the stakeholder would act as a pool of stakes, or distributed stakes on competing coins. "one minus the probability" of winning leadership with aggregated stakes is , the joint "one minus probability" of all the stakes (each with probability winning aggregated winning the leadership thus:
A non-exponential linear leader selection can be:
Dependent aggregation
Linear leader selection has the dependent aggregation property, meaning it's favorable to compete in pools with sum of the stakes over aggregated stakes of distributed stakes:
let's assume the stakes are divided to stakes of value for , note that , thus competing with single coin of the sum of stakes held by the stakeholder is favorable.
Scalar linear aggregation dependent leader selection
A target function T with scalar coefficients can be formalized as let's assume , and then: then the lead statement is for example for a group order or l= 24 bits, and maximum value of , then lead statement:
Competing max value coins
For a stakeholder with absolute stake, it's advantageous for the stakeholder to distribute stakes on competing coins.
Inverse functions
Inverse lead selection functions doesn't require maximum stake, most suitable for absolute stake, it has the disadvantage that it's inflating with increasing rate as time goes on, but it can be function of the inverse of the slot to control the increasing frequency of winning leadership.
Leader selection without maximum stake upper limit
The inverse leader selection without maximum stake value can be and inversely proportional with probability of winning leadership, let it be called leadership coefficient.
Decaying linear leader selection
As the time goes one, and stakes increase, this means the combined stakes of all stakeholders increases the probability of winning leadership in next slots leading to more leaders at a single slot, to maintain, or to be more general to control this frequency of leaders per slot, c (the leadership coefficient) need to be function of the slot , i.e where is epoch size (number of slots in epoch).
Pairing leader selection independent aggregation function
The only family of functions that are isomorphic to summation on multiplication (having the independent aggregation property) is the exponential function, and since it's impossible to implement in plonk, a re-formalization of the lead statement using pairing that is isomorphic to summation on multiplication is an option.
Let's assume is isomorphic function between multiplication and addition, , thus: then the only family of functions satisfying this is the exponential function
no solution for the lead statement parameters, and constants defined over group of integers.
assume there is a solution for the lead statement parameters and constants defined over group of integers. for the statement , such that S where is the maximum stake value being , following from the previous proof that the family of function haveing independent aggregation property is the exponential function , and , the smallest value satisfying f is , then note that since thus , contradiction.
Leaky non-resettable beacon
Built on top of globally synchronized clock, that leaks the nonce of the next epoch a head of time (thus called leaky), non-resettable in the sense that the random nonce is deterministic at slot s, while assuring security against adversary controlling some stakeholders.
For an epoch j, the nonce is calculated by hash function H, as:
v is the concatenation of the value in all blocks from the beginning of epoch to the slot with timestamp up to , note that k is a persistence security parameter, R is the epoch length in terms of slots.
Protocol
Appendix
This section gives further details about the structures that will be used by the protocol. Since the Streamlet consensus protocol will be used at early stages of development, we created hybrid structures to enable seamless transition from Stremlet to Ouroboros Crypsinous, without the need of forking the blockchain.
Blockchain
Field | Type | Description |
---|---|---|
blocks | Vec<Block> | Series of blocks consisting the Blockchain |
Header
Field | Type | Description |
---|---|---|
v | u8 | Version |
st | blake3Hash | Previous block hash |
e | u64 | Epoch |
sl | u64 | Slot UID |
time | Timestamp | Block creation timestamp |
root | MerkleRoot | Root of the transaction hashes merkle tree |
Block
Field | Type | Description |
---|---|---|
magic | u8 | Magic bytes |
header | blake3Hash | Header hash |
txs | Vec<blake3Hash> | Transaction hashes |
metadata | Metadata | Additional block information |
BlockInfo
Field | Type | Description |
---|---|---|
magic | u8 | Magic bytes |
header | Header | Header data |
txs | Vec<Transaction> | Transaction payload |
metadata | Metadata | Additional block information |
sm | StreamletMetadata | Proposal information used by Streamlet consensus |
Metadata
Field | Type | Description |
---|---|---|
proof | VRFOutput | Proof the stakeholder is the block owner |
r | Seed | Random seed for the VRF |
s | Signature | Block owner signature |
Streamlet Metadata
Field | Type | Description |
---|---|---|
votes | Vec<Vote> | Epoch votes for the block |
notarized | bool | Block notarization flag |
finalized | bool | Block finalization flag |
DarkFi Node Architecture (DNA)
The DarkFi ecosystem runs as a network of P2P nodes, where these nodes interact with each other over specific programs (or layers). In this section, we'll explain how each of the layers fit together and when combined create a functioning network that becomes DarkFi.
The layers are organized as a bottom-up pyramid, much like the DarkFi logo:
We will start with the top-level daemon - validatord
- which
serves as the consensus and data storage layer, then we will explain
darkfid
and its communication with the layer above (validatord
),
and the layer below (drk
).
An abstract view of the network looks like the following:
[drk] <--> [darkfid] <--> [validatord] <-+
|
[drk] <--> [darkfid] <--> [validatord] <-+
validatord
validatord
is the DarkFi consensus and data storage layer. Everyone
that runs a validator participates in the network as a data archive,
and is able to store incoming transactions, and relay them to
other validators over the P2P network and protocol. Additionally,
storing this data allows others to replicate it and participate in
the same way.
Provided there is a locked stake on a running validator, the node can also participate in the Proof-of-Stake consensus, enabling the ability to vote on incoming transactions rather than just relaying (and validating) them.
In case the node is not participating in the consensus, it should still relay incoming transactions to other (ideally consensus-participating) validators in the network.
Inner workings
In its database, validatord
stores transactions and blocks that
have reached consensus. This is commonly known as a "blockchain".
The blockchain is a shared state that is replicated between all
validators in the network.
Additionally, validators keep a pool of incoming transactions and proposed blocks, which get validated and voted on by the consensus-participating validators.
The lifetime of an incoming transaction (and block) is as follows:
- Wait for a transaction
- Validate incoming transaction (and go back to 1. if invalid)
- Broadcast transaction to other validators in the network
- Other validators validate transaction (and go back to 1. if invalid)
- Leader validates the state transition and proposes a block
- Consensus-participating nodes validate the state transition and vote on the proposed block if the state transition is valid.
- If the block is confirmed, it is appended to the blockchain and is replicated between all validators in the network.
darkfid
TODO: Initial sync and retrieving wallet state?
darkfid
is the client layer of DarkFi used for wallet management
and transaction broadcasting. The wallet keeps a history of balances,
coins, nullifiers, and merkle roots that are necessary in order
to create new transactions.
By design, darkfid
is a light client, since validatord
stores all
the blockchain data, and darkfid
can simply query for anything it
is interested in. This allows us to avoid data duplication and simply
utilize our modular architecture. This also means that darkfid
can
easily be replaced with more specific tooling, if need be.
Inner workings
Using the P2P network and protocol, darkfid
can subscribe to
validatord
in order to receive new nullifiers and merkle roots
whenever a new block is confirmed. This allows darkfid
to update
its local state and enables it to create new valid transactions.
darkfid
exposes a JSON-RPC endpoint for clients to interact with it.
This allows a number of things, such as: listing balances, creating
and submitting transactions, key management, and more.
When creating a new transactions, darkfid
uses the local synced state
in order to create new coins and combine them in a transaction. This
transaction is then submitted to the above validator layer where the
transaction will get validated and voted on in order to be included
into a block.
drk
drk
is a client tool that interacts with darkfid
in a user-friendly
way and provides a command-line interface to the DarkFi network and
its functionality.
The interaction with darkfid
is done over the JSON-RPC protocol
and communicates with the endpoint exposed by darkfid
.
Clients
This section gives information on DarkFi's clients, such as darkfid and cashierd. Currently this section offers documentation on the client's RPC API.
darkfid JSON-RPC API
blockchain methods
blockchain.get_slot
Queries the blockchain database for a block in the given slot.
Returns a readable block upon success.
[src]
--> {"jsonrpc": "2.0", "method": "blockchain.get_slot", "params": [0], "id": 1}
<-- {"jsonrpc": "2.0", "result": {...}, "id": 1}
blockchain.merkle_roots
Queries the blockchain database for all available merkle roots.
[src]
--> {"jsonrpc": "2.0", "method": "blockchain.merkle_roots", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": [..., ..., ...], "id": 1}
tx methods
tx.transfer
Transfer a given amount of some token to the given address.
Returns a transaction ID upon success.
[src]
--> {"jsonrpc": "2.0", "method": "tx.transfer", "params": ["darkfi" "gdrk", "1DarkFi...", 12.0], "id": 1}
<-- {"jsonrpc": "2.0", "result": "txID...", "id": 1}
wallet methods
wallet.keygen
wallet.get_key
wallet.export_keypair
wallet.import_keypair
wallet.set_default_address
wallet.get_balances
wallet.keygen
Attempts to generate a new keypair and returns its address upon success.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.keygen", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": "1DarkFi...", "id": 1}
wallet.get_key
Fetches public keys by given indexes from the wallet and returns it in an
encoded format. -1
is supported to fetch all available keys.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.get_key", "params": [1, 2], "id": 1}
<-- {"jsonrpc": "2.0", "result": ["foo", "bar"], "id": 1}
wallet.export_keypair
Exports the given keypair index.
Returns the encoded secret key upon success.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.export_keypair", "params": [0], "id": 1}
<-- {"jsonrpc": "2.0", "result": "foobar", "id": 1}
wallet.import_keypair
Imports a given secret key into the wallet as a keypair.
Returns the public counterpart as the result upon success.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.import_keypair", "params": ["foobar"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "pubfoobar", "id": 1}
wallet.set_default_address
Sets the default wallet address to the given index.
Returns true
upon success.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.set_default_address", "params": [2], "id": 1}
<-- {"jsonrpc": "2.0", "result": true, "id": 1}
wallet.get_balances
Queries the wallet for known balances.
Returns a map of balances, indexed by network
, and token ID.
[src]
--> {"jsonrpc": "2.0", "method": "wallet.get_balances", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": [{"btc": [100, "Bitcoin"]}, {...}], "id": 1}
misc methods
ping
Returns a pong
to the ping
request.
[src]
--> {"jsonrpc": "2.0", "method": "ping", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": "pong", "id": 1}
clock
Returns current system clock in Timestamp
format.
[src]
--> {"jsonrpc": "2.0", "method": "clock", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": {...}, "id": 1}
cashierd JSON-RPC API
deposit
Executes a deposit request given network
and token_id
.
Returns the address where the deposit shall be transferred to.
[src]
--> {"jsonrpc": "2.0", "method": "deposit", "params": ["network", "token", "publickey"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "Ht5G1RhkcKnpLVLMhqJc5aqZ4wYUEbxbtZwGCVbgU7DL", "id": 1}
withdraw
Executes a withdraw request given network
, token_id
, publickey
and amount
. publickey
is supposed to correspond to network
.
Returns the transaction ID of the processed withdraw.
[src]
--> {"jsonrpc": "2.0", "method": "withdraw", "params": ["network", "token", "publickey", "amount"], "id": 1}
<-- {"jsonrpc": "2.0", "result": "txID", "id": 1}
features
Returns supported cashier features, like network, listening ports, etc.
[src]
--> {"jsonrpc": "2.0", "method": "features", "params": [], "id": 1}
<-- {"jsonrpc": "2.0", "result": {"network": ["btc", "sol"]}, "id": 1}
zkas
zkas is a compiler for the Halo2 zkVM language used in DarkFi.
The current implementation found in the DarkFi repository inside
src/zkas
is the reference compiler and language implementation. It is a
toolchain consisting of a lexer, parser, static and semantic analyzers,
and a binary code compiler.
The
main.rs
file shows how this toolchain is put together to produce binary code
from source code.
zkas bincode
The bincode design for zkas is the compiled code in the form of a binary blob, that can be read by a program and fed into the VM.
Our programs consist of three sections: constant
, contract
, and
circuit
. Our bincode represents the same. Additionally, there is
an optional section called .debug
which can hold debug info related
to the binary.
We currently keep everything on the same stack, so we avoid having to deal with different types. Instead, we rely that the compiler does a proper parse and analysis of the source code, so we are sure that in the VM, when referenced, the types shall be correct.
The compiled binary blob has the following layout:
MAGIC_BYTES
BINARY_VERSION
.constant
CONSTANT_TYPE CONSTANT_NAME
CONSTANT_TYPE CONSTANT_NAME
...
.contract
WITNESS_TYPE
WITNESS_TYPE
...
.circuit
OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX
OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX
...
.debug
TBD
Integers in the binary are encoded using variable-integer encoding.
See serial.rs
for our Rust implementation.
MAGIC_BYTES
The magic bytes are the file signature consisting of four bytes used to identify the zkas binary code. They consist of:
0x0b
0xxx
0xb1
0x35
BINARY_VERSION
The binary code also contains the binary version to allow parsing potential different formats in the future.
0x01
.constant
The constants in the .constant
section are declared with their type
and name, so that the VM knows how to search for the builtin constant
and add it to the stack.
.contract
The .contract
section holds the circuit witness values in the form
of WITNESS_TYPE
. Their stack index is incremented for each witness
as they're kept in order like in the source file. The witnesses
that are of the same type as the circuit itself (typically Base
)
will be loaded into the circuit as private values using the Halo2
load_private
API.
.circuit
The .circuit
section holds the procedural logic of the ZK proof.
In here we have statements with opcodes that are executed as
understood by the VM. The statements are in the form of:
OPCODE ARG_NUM STACK_INDEX ... STACK_INDEX
where:
Element | Description |
---|---|
OPCODE | The opcode we wish to execute |
ARG_NUM | The number of arguments given to this opcode |
(Note the VM should be checking the correctness of this as well) | |
STACK_INDEX | The location of the argument on the stack. |
(This is supposed to be repeated ARG_NUM times) |
In case an opcode has a return value, the value shall be pushed to the stack and become available for later references.
.debug
TBD
Decoding the bincode
An example decoder implementation can be found in zkas'
decoder.rs
module.
Examples
This section holds practical and real-world examples of the use for zkas.
Sapling payment scheme
Sapling is a type of transaction which hides both the sender and receiver data, as well as the amount transacted. This means it allows a fully private transaction between two addresses.
Generally, the Sapling payment scheme consists of two ZK proofs - mint and burn. We use the mint proof to create a new coin , and we use the burn proof to spend a previously minted coin.
Mint proof
constant "Mint" {
EcFixedPointShort VALUE_COMMIT_VALUE,
EcFixedPoint VALUE_COMMIT_RANDOM,
}
contract "Mint" {
Base pub_x,
Base pub_y,
Base value,
Base token,
Base serial,
Base coin_blind,
Scalar value_blind,
Scalar token_blind,
}
circuit "Mint" {
# Poseidon hash of the coin
C = poseidon_hash(pub_x, pub_y, value, token, serial, coin_blind);
constrain_instance(C);
# Pedersen commitment for coin's value
vcv = ec_mul_short(value, VALUE_COMMIT_VALUE);
vcr = ec_mul(value_blind, VALUE_COMMIT_RANDOM);
value_commit = ec_add(vcv, vcr);
# Since the value commit is a curve point, we fetch its coordinates
# and constrain them:
value_commit_x = ec_get_x(value_commit);
value_commit_y = ec_get_y(value_commit);
constrain_instance(value_commit_x);
constrain_instance(value_commit_y);
# Pedersen commitment for coin's token ID
tcv = ec_mul_short(token, VALUE_COMMIT_VALUE);
tcr = ec_mul(token_blind, VALUE_COMMIT_RANDOM);
token_commit = ec_add(tcv, tcr);
# Since token_commit is also a curve point, we'll do the same
# coordinate dance:
token_commit_x = ec_get_x(token_commit);
token_commit_y = ec_get_y(token_commit);
constrain_instance(token_commit_x);
constrain_instance(token_commit_y);
# At this point we've enforced all of our public inputs.
}
As you can see, the Mint
proof basically consists of three
operations. First one is hashing the coin , and after that,
we create Pedersen commitments1 for both the coin's value
and the coin's token ID. On top of the zkas code, we've declared
two constant values that we are going to use for multiplication in
the commitments.
The constrain_instance
call can take any of our assigned variables
and enforce a public input. Public inputs are an array (or vector)
of revealed values used by verifiers to verify a zero knowledge
proof. In the above case of the Mint proof, since we have five calls to
constrain_instance
, we would also have an array of five elements that
represent these public inputs. The array's order must match the
order of the constrain_instance
calls since they will be constrained
by their index in the array (which is incremented for every call).
In other words, the vector of public inputs could look like this:
let public_inputs = vec![
coin,
*value_coords.x(),
*value_coords.y(),
*token_coords.x(),
*token_coords.y(),
];
And then the Verifier uses these public inputs to verify a given zero knowledge proof.
Coin
During the Mint phase we create a new coin , which is bound to the public key . The coin is publicly revealed on the blockchain and added to the Merkle tree.
Let be the coin's value, be the token ID, be the unique serial number for the coin, and be a random blinding value. We create a commitment (hash) of these elements and produce the coin in zero-knowledge:
An interesting thing to keep in mind is that this commitment is extensible, so one could fit an arbitrary amount of different attributes inside it.
Value and token commitments
To have some value for our coin, we ensure it's greater than zero, and then we can create a Pedersen commitment where is the blinding factor for the commitment, and and are two predefined generators:
The token ID can be thought of as an attribute we append to our coin so we can have a differentiation of assets we are working with. In practice, this allows us to work with different tokens, using the same zero-knowledge proof circuit. For this token ID, we can also build a Pedersen commitment where is the token ID, is the blinding factor, and and are predefined generators:
Pseudo-code
Knowing this we can extend our pseudo-code and build the before-mentioned public inputs for the circuit:
#![allow(unused)] fn main() { let bincode = include_bytes!("../proof/mint.zk.bin"); let zkbin = ZkBinary::decode(bincode)?; // ====== // Prover // ====== // Witness values let value = 42; let token_id = pallas::Base::from(22); let value_blind = pallas::Scalar::random(&mut OsRng); let token_blind = pallas::Scalar::random(&mut OsRng); let serial = pallas::Base::random(&mut OsRng); let coin_blind = pallas::Base::random(&mut OsRng); let public_key = PublicKey::random(&mut OsRng); let coords = public_key.0.to_affine().coordinates().unwrap(); let prover_witnesses = vec![ Witness::Base(Value::known(*coords.x())), Witness::Base(Value::known(*coords.y())), Witness::Base(Value::known(pallas::Base::from(value))), Witness::Base(Value::known(token_id)), Witness::Base(Value::known(serial)), Witness::Base(Value::known(coin_blind)), Witness::Scalar(Value::known(value_blind)), Witness::Scalar(Value::known(token_blind)), ]; // Create the public inputs let msgs = [*coords.x(), *coords.y(), pallas::Base::from(value), token_id, serial, coin_blind]; let coin = poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<6>, 3, 2>::init() .hash(msgs); let value_commit = pedersen_commitment_u64(value, value_blind); let value_coords = value_commit.to_affine().coordinates().unwrap(); let token_commit = pedersen_commitment_scalar(mod_r_p(token_id), token_blind); let token_coords = token_commit.to_affine().coordinates().unwrap(); let public_inputs = vec![coin, *value_coords.x(), *value_coords.y(), *token_coords.x(), *token_coords.y()]; // Create the circuit let circuit = ZkCircuit::new(prover_witnesses, zkbin.clone()); let proving_key = ProvingKey::build(13, &circuit); let proof = Proof::create(&proving_key, &[circuit], &public_inputs, &mut OsRng)?; // ======== // Verifier // ======== // Construct empty witnesses let verifier_witnesses = empty_witnesses(&zkbin); // Create the circuit let circuit = ZkCircuit::new(verifier_witnesses, zkbin); let verifying_key = VerifyingKey::build(13, &circuit); proof.verify(&verifying_key, &public_inputs)?; }
Burn
constant "Burn" {
EcFixedPointShort VALUE_COMMIT_VALUE,
EcFixedPoint VALUE_COMMIT_RANDOM,
EcFixedPointBase NULLIFIER_K,
}
contract "Burn" {
Base secret,
Base serial,
Base value,
Base token,
Base coin_blind,
Scalar value_blind,
Scalar token_blind,
Uint32 leaf_pos,
MerklePath path,
Base signature_secret,
}
circuit "Burn" {
# Poseidon hash of the nullifier
nullifier = poseidon_hash(secret, serial);
constrain_instance(nullifier);
# Pedersen commitment for coin's value
vcv = ec_mul_short(value, VALUE_COMMIT_VALUE);
vcr = ec_mul(value_blind, VALUE_COMMIT_RANDOM);
value_commit = ec_add(vcv, vcr);
# Since value_commit is a curve point, we fetch its coordinates
# and constrain them:
value_commit_x = ec_get_x(value_commit);
value_commit_y = ec_get_y(value_commit);
constrain_instance(value_commit_x);
constrain_instance(value_commit_y);
# Pedersen commitment for coin's token ID
tcv = ec_mul_short(token, VALUE_COMMIT_VALUE);
tcr = ec_mul(token_blind, VALUE_COMMIT_RANDOM);
token_commit = ec_add(tcv, tcr);
# Since token_commit is also a curve point, we'll do the same
# coordinate dance:
token_commit_x = ec_get_x(token_commit);
token_commit_y = ec_get_y(token_commit);
constrain_instance(token_commit_x);
constrain_instance(token_commit_y);
# Coin hash
pub = ec_mul_base(secret, NULLIFIER_K);
pub_x = ec_get_x(pub);
pub_y = ec_get_y(pub);
C = poseidon_hash(pub_x, pub_y, value, token, serial, coin_blind);
# Merkle root
root = calculate_merkle_root(leaf_pos, path, C);
constrain_instance(root);
# Finally, we derive a public key for the signature and
# constrain its coordinates:
signature_public = ec_mul_base(signature_secret, NULLIFIER_K);
signature_x = ec_get_x(signature_public);
signature_y = ec_get_y(signature_public);
constrain_instance(signature_x);
constrain_instance(signature_y);
# At this point we've enforced all of our public inputs.
}
The Burn
proof consists of operations similar to the Mint
proof,
with the addition of a Merkle root2 calculation. In the same
manner, we are doing a Poseidon hash instance, we're building Pedersen
commitments for the value and token ID, and finally we're doing a
public key derivation.
In this case, our vector of public inputs could look like:
let public_inputs = vec![
nullifier,
*value_coords.x(),
*value_coords.y(),
*token_coords.x(),
*token_coords.y(),
merkle_root,
*sig_coords.x(),
*sig_coords.y(),
];
Nullifier
When we spend the coin, we must ensure that the value of the coin cannot be double spent. We call this the Burn phase. The process relies on a nullifier , which we create using the secret key for the public key and a unique random serial . Nullifiers are unique per coin and prevent double spending:
Merkle root
We check that the merkle root corresponds to a coin which is in the Merkle tree
Value and token commitments
Just like we calculated these for the Mint
proof, we do the same
here:
Public key derivation
We check that the secret key corresponds to a public key . Usually, we do public key derivation my multiplying our secret key with a genera tor , which results in a public key:
Pseudo-code
Knowing this we can extend our pseudo-code and build the before-mentioned public inputs for the circuit:
#![allow(unused)] fn main() { let bincode = include_bytes!("../proof/burn.zk.bin"); let zkbin = ZkBinary::decode(bincode)?; // ====== // Prover // ====== // Witness values let value = 42; let token_id = pallas::Base::from(22); let value_blind = pallas::Scalar::random(&mut OsRng); let token_blind = pallas::Scalar::random(&mut OsRng); let serial = pallas::Base::random(&mut OsRng); let coin_blind = pallas::Base::random(&mut OsRng); let secret = SecretKey::random(&mut OsRng); let sig_secret = SecretKey::random(&mut OsRng); // Build the coin let coin2 = { let coords = PublicKey::from_secret(secret).0.to_affine().coordinates().unwrap(); let messages = [*coords.x(), *coords.y(), pallas::Base::from(value), token_id, serial, coin_blind]; poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<6>, 3, 2>::init() .hash(messages) }; // Fill the merkle tree with some random coins that we want to witness, // and also add the above coin. let mut tree = BridgeTree::<MerkleNode, 32>::new(100); let coin0 = pallas::Base::random(&mut OsRng); let coin1 = pallas::Base::random(&mut OsRng); let coin3 = pallas::Base::random(&mut OsRng); tree.append(&MerkleNode(coin0)); tree.witness(); tree.append(&MerkleNode(coin1)); tree.append(&MerkleNode(coin2)); let leaf_pos = tree.witness().unwrap(); tree.append(&MerkleNode(coin3)); tree.witness(); let root = tree.root(0).unwrap(); let merkle_path = tree.authentication_path(leaf_pos, &root).unwrap(); let leaf_pos: u64 = leaf_pos.into(); let prover_witnesses = vec![ Witness::Base(Value::known(secret.0)), Witness::Base(Value::known(serial)), Witness::Base(Value::known(pallas::Base::from(value))), Witness::Base(Value::known(token_id)), Witness::Base(Value::known(coin_blind)), Witness::Scalar(Value::known(value_blind)), Witness::Scalar(Value::known(token_blind)), Witness::Uint32(Value::known(leaf_pos.try_into().unwrap())), Witness::MerklePath(Value::known(merkle_path.try_into().unwrap())), Witness::Base(Value::known(sig_secret.0)), ]; // Create the public inputs let nullifier = [secret.0, serial]; let nullifier = poseidon::Hash::<_, poseidon::P128Pow5T3, poseidon::ConstantLength<2>, 3, 2>::init() .hash(nullifier); let value_commit = pedersen_commitment_u64(value, value_blind); let value_coords = value_commit.to_affine().coordinates().unwrap(); let token_commit = pedersen_commitment_scalar(mod_r_p(token_id), token_blind); let token_coords = token_commit.to_affine().coordinates().unwrap(); let sig_pubkey = PublicKey::from_secret(sig_secret); let sig_coords = sig_pubkey.0.to_affine().coordinates().unwrap(); let merkle_root = tree.root(0).unwrap(); let public_inputs = vec![ nullifier, *value_coords.x(), *value_coords.y(), *token_coords.x(), *token_coords.y(), merkle_root.0, *sig_coords.x(), *sig_coords.y(), ]; // Create the circuit let circuit = ZkCircuit::new(prover_witnesses, zkbin.clone()); let proving_key = ProvingKey::build(13, &circuit); let proof = Proof::create(&proving_key, &[circuit], &public_inputs, &mut OsRng)?; // ======== // Verifier // ======== // Construct empty witnesses let verifier_witnesses = empty_witnesses(&zkbin); // Create the circuit let circuit = ZkCircuit::new(verifier_witnesses, zkbin); let verifying_key = VerifyingKey::build(13, &circuit); proof.verify(&verifying_key, &public_inputs)?; }
See section 3: The Commitment Scheme of Torben Pryds Pedersen's paper on Non-Interactive and Information-Theoretic Secure Verifiable Secret Sharing
Anonymous voting
Anonymous voting1 is a type of voting process where users can vote without revealing their identity, by proving they are accepted as valid voters.
The proof enables user privacy and allows for fully anonymous voting.
The starting point is a Merkle proof2, which efficiently proves that a voter's key belongs to a Merkle tree. However, using this proof alone would allow the organizer of a process to correlate each vote envelope with its voter's key on the database, so votes wouldn't be secret.
Vote proof
constant "Vote" {
EcFixedPoint VALUE_COMMIT_VALUE,
EcFixedPoint VALUE_COMMIT_RANDOM,
EcFixedPoint NULLIFIER_K,
}
contract "Vote" {
Base process_id_0,
Base process_id_1,
Base secret_key,
Base vote,
Scalar vote_blind,
Uint32 leaf_pos,
MerklePath path,
}
circuit "Vote" {
# Nullifier hash
process_id = poseidon_hash(process_id_0, process_id_1);
nullifier = poseidon_hash(secret_key, process_id);
constrain_instance(nullifier);
# Public key derivation and hashing
public_key = ec_mul_base(secret_key, NULLIFIER_K);
public_x = ec_get_x(public_key);
public_y = ec_get_y(public_key);
pk_hash = poseidon_hash(public_x, public_y);
# Merkle root
root = calculate_merkle_root(leaf_pos, path, pk_hash);
constrain_instance(root);
# Pedersen commitment for vote
vcv = ec_mul_short(vote, VALUE_COMMIT_VALUE);
vcr = ec_mul(vote_blind, VALUE_COMMIT_RANDOM);
vote_commit = ec_add(vcv, vcr);
# Since vote_commit is a curve point, we fetch its coordinates
# and constrain_them:
vote_commit_x = ec_get_x(vote_commit);
vote_commit_y = ec_get_y(vote_commit);
constrain_instance(vote_commit_x);
constrain_instance(vote_commit_y);
}
Our proof consists of four main operation. First we are hashing the nullifier using our secret key and the hashed process ID. Next, we derive our public key and hash it. Following, we take this hash and create a Merkle proof that it is indeed contained in the given Merkle tree. And finally, we create a Pedersen commitment3 for the vote choice itself.
Our vector of public inputs can look like this:
let public_inputs = vec![
nullifier,
merkle_root,
*vote_coords.x(),
*vote_coords.y(),
]
And then the Verifier uses these public inputs to verify the given zero-knowledge proof.
Specification taken from vocdoni franchise proof
See section 3: The Commitment Scheme of Torben Pryds Pedersen's paper on Non-Interactive and Information-Theoretic Secure Verifiable Secret Sharing
Miscellaneous tools
This section documents some miscellaneous tools provided in the DarkFi ecosystem.
vanityaddr
A tool for Vanity address generation for DarkFi keypairs. Given some prefix, the tool will bruteforce secret keys to find one which, when derived into an address, starts with a given prefix.
Usage
vanityaddr 0.3.0
Vanity address generation tool for DarkFi keypairs.
USAGE:
vanityaddr [OPTIONS] <PREFIX>
ARGS:
<PREFIX> Prefix to search (must start with 1)
OPTIONS:
-c Should the search be case-sensitive
-h, --help Print help information
-t <THREADS> Number of threads to use (defaults to number of available CPUs)
-V, --version Print version information
We can use the tool in our command line:
% vanityaddr 1Foo
[00:00:05] 53370 attempts
And the program will start crunching numbers. After a period of time, we will get JSON output containing an address, secret key, and the number of attempts it took to find the secret key.
{"address":"1FoomByzBBQywKaeBB5XPkAm5eCboh8K4CBhBe9uKbJm3kEiCS","attempts":78418,"secret":"0x16545da4a401adcd035ef51c8040acf5f4f1c66c0dd290bb5ec9e95991ae3615"}
P2P IRC
In DarkFi, we organize our communication using resilient and
censorship-resistant infrastructure. For chatting, ircd
is a
peer-to-peer implementation of an IRC server in which any user can
participate anonymously using any IRC frontend and by running the
IRC daemon. ircd
uses the DarkFi P2P engine to synchronize chats
between hosts.
Installation
% git clone https://github.com/darkrenaissance/darkfi
% cd darkfi
% make BINS=ircd
% sudo make install BINS=ircd
Usage (DarkFi Network)
Upon installing ircd
as described above, the preconfigured defaults
will allow you to connect to the network and start chatting with the
rest of the DarkFi community.
First, try to start ircd
from your command-line so it can spawn its
configuration file in place. The preconfigured defaults will autojoin
you to the #dev
channel, where the community is most active and
talks about DarkFi development.
% ircd
After running it for the first time, ircd
will create a configuration
file you can review and potentially edit. It might be useful if you
want to add other channels you want to autojoin (like #philosophy
and #memes
), or if you want to set a shared secret for some channel
in order for it to be encrypted between its participants.
When done, you can run ircd
for the second time in order for it to
connect to the network and start participating in the P2P protocol:
% ircd
Clients
Weechat
In this section, we'll briefly cover how to use the Weechat IRC
client to connect and chat with
ircd
.
Normally, you should be able to install weechat using your distribution's package manager. If not, have a look at the weechat git repository for instructions on how to install it on your computer.
Once installed, we can configure a new server which will represent our
ircd
instance. First, start weechat, and in its window - run the
following commands (there is an assumption that irc_listen
in the
ircd
config file is set to 127.0.0.1:11066
):
/server add darkfi localhost/11066 -autoconnect
/save
/quit
This will set up the server, save the settings, and exit weechat. You are now ready to begin using the chat. Simply start weechat and everything should work.
We can now proceed with installing the mallumo
weechat script, which
is used for E2E encryption in private messages on this IRC network.
E2E encryption with mallumo
mallumo
is a Python plugin for Weechat that can be used to
enable end-to-end encryption for private messages between you and
other users of the DarkFi IRC network. The verbose installation
and configuration instructions can be found in the mallumo git
repository.
Briefly, make sure you install python3 and
pynacl
(can usually be done with
your distribution's package manager or pip
).
Then find where weechat has put its configuration files. It is usually
~/.weechat
or ~/.local/share/weechat
(here we will assume the
latter). Go to the directory, clone the repo, and make a couple of
symlinks:
% cd ~/.local/share/weechat
% mkdir -p src
% git clone https://github.com/darkrenaissance/mallumo src/mallumo
% cd python
% ln -s `realpath ../src/mallumo/mallumo` mallumo
% ln -s `realpath ../src/mallumo/mallumo/__init__.py` autoload/mallumo.py
Refer to darkrenaissance/mallumo and its README for usage instructions.
After this has been set up, the next time you start ircd
and then
weechat
, you will be connected to the DarkFi IRC network and be
able to chat with other participants.
Usage (Local Deployment)
These steps below are only for developers who wish to make a testing deployment. The previous sections are sufficient to join the chat.
Seed Node
First you must run a seed node. The seed node is a static host which
nodes can connect to when they first connect to the network. The
seed_session
simply connects to a seed node and runs protocol_seed
,
which requests a list of addresses from the seed node and disconnects
straight after receiving them.
The first time you run the program, a config file will be created in .config/darkfi. You must specify an inbound accept address in your config file to configure a seed node:
## P2P accept address
inbound="127.0.0.1:11001"
Note that the above config doesn't specify an external address since the seed node shouldn't be advertised in the list of connectable nodes. The seed node does not participate as a normal node in the p2p network. It simply allows new nodes to discover other nodes in the network during the bootstrapping phase.
Inbound Node
This is a node accepting inbound connections on the network but which is not making any outbound connections.
The external address is important and must be correct.
To run an inbound node, your config file must contain the following info:
## P2P accept address
inbound="127.0.0.1:11002"
## P2P external address
external_addr="127.0.0.1:11002"
## Seed nodes to connect to
seeds=["127.0.0.1:11001"]
Outbound Node
This is a node which has 8 outbound connection slots and no inbound connections. This means the node has 8 slots which will actively search for unique nodes to connect to in the p2p network.
In your config file:
## Connection slots
outbound_connections=8
## Seed nodes to connect to
seeds=["127.0.0.1:11001"]
Attaching the IRC Frontend
Assuming you have run the above 3 commands to create a small model testnet, and both inbound and outbound nodes above are connected, you can test them out using weechat.
To create separate weechat instances, use the --dir
command:
weechat --dir /tmp/a/
weechat --dir /tmp/b/
Then in both clients, you must set the option to connect to temporary servers:
/set irc.look.temporary_servers on
Finally you can attach to the local IRCd instances:
/connect localhost/6667
/connect localhost/6668
And send messages to yourself.
Running a Fullnode
See the script script/run_node.sh
for an example of how to deploy
a full node which does seed session synchronization, and accepts both
inbound and outbound connections.
Tau
Encrypted tasks management app using peer-to-peer network and raft consensus.
multiple users can collaborate by working on the same tasks, and all users will have synced task list.
Install
% git clone https://github.com/darkrenaissance/darkfi
% cd darkfi
% make BINS="taud tau"
% sudo make install "BINS=taud tau"
Usage (Local Deployment)
Seed Node
First you must run a seed node. The seed node is a static host which nodes can
connect to when they first connect to the network. The seed_session
simply
connects to a seed node and runs protocol_seed
, which requests a list of
addresses from the seed node and disconnects straight after receiving them.
in config file:
## P2P accept address
inbound="127.0.0.1:11001"
Note that the above config doesn't specify an external address since the seed node shouldn't be advertised in the list of connectable nodes. The seed node does not participate as a normal node in the p2p network. It simply allows new nodes to discover other nodes in the network during the bootstrapping phase.
Inbound Node
This is a node accepting inbound connections on the network but which is not making any outbound connections.
The external address is important and must be correct.
in config file:
## P2P accept address
inbound="127.0.0.1:11002"
## P2P external address
external_addr="127.0.0.1:11002"
## Seed nodes to connect to
seeds=["127.0.0.1:11001"]
Outbound Node
This is a node which has 8 outbound connection slots and no inbound connections. This means the node has 8 slots which will actively search for unique nodes to connect to in the p2p network.
in config file:
## Connection slots
outbound_connections=8
## Seed nodes to connect to
seeds=["127.0.0.1:11001"]
Also note that for the first time ever running seed node you must run it with
--key-gen
:
% taud --key-gen
This will generate a new secret key in /home/${USER}/.config/tau/secret_key
that
you can share with nodes you want them to get and decrypt your tasks, otherwise if you
have already generated or got a copy from a peer place it in the same directory
/home/${USER}/.config/tau/secret_key
.
Usage (CLI)
% tau --help
tau 0.3.0
Command-line client for taud
USAGE:
tau [OPTIONS] <SUBCOMMAND>
OPTIONS:
-e, --endpoint <ENDPOINT> taud JSON-RPC endpoint [default: tcp://127.0.0.1:11055]
-h, --help Print help information
-v Increase verbosity (-vvv supported)
-V, --version Print version information
SUBCOMMANDS:
add Add a new task
comment Set or Get comment for a task
help Print this message or the help of the given subcommand(s)
info Get task info by ID
state Set or Get task state
update Update/Edit an existing task by ID
% tau help [SUBCOMMAND]
Example
% # add new task
% tau add "new title"
% tau add "new title" project:blockchain desc:"new description" rank:3 assign:dark
%
% # lists tasks
% tau
% tau open # open tasks
% tau pause # paused tasks
% tau 0522 # created at May 2022
% tau project:blockchain assign:dark
% tau rank:gt:n # lists all tasks that have rank greater than n
% tau rank:ls:n # lists all tasks that have rank lesser than n
%
% # update task
% tau update 3 project:network rank:20
%
% # state
% tau state 3 # get state
% tau state 3 pause # set the state to pause
%
% # comments
% tau comment 1 # list comments
% tau comment 3 "new comment" # add new comment
Dnetview
A simple tui to explore darkfi ircd network topology.
dnetview displays:
- all active nodes
- outgoing, incoming and manual sessions
- each associated connection and recent messages.
dnetview is based on the design-pattern Model, View, Controller. We create a logical separation between the underlying data structure or Model; the ui rendering aspect which is the View; and the Controller or game engine that makes everything run.
Install
% git clone https://github.com/darkrenaissance/darkfi
% cd darkfi
% make BINS=dnetview
Usage
Run dnetview as follows:
dnetview -v
On first run, dnetview will create a config file in .config/darkfi. You must manually enter the RPC ports of the nodes you want to connect to and title them as you see fit.
Dnetview creates a logging file in /tmp/dnetview.log. To see json data and other debug info, tail the file like so:
tail -f /tmp/dnetview.log
Learn
This section contains learning resources related to DarkFi.
Research
DarkFi maintains a public resource of zero-knowledge and math research in the script/research directory of the repo.
It features simple sage implementations of zero-knowledge algorithms and math primitives, including but not limited to:
Zero-knowledge explainer
We start with this algorithm as an example:
def foo(w, a, b):
if w:
return a * b
else:
return a + b
ZK code consists of lines of constraints. It has no concept of branching conditionals or loops.
So our first task is to flatten (convert) the above code to a linear equation that can be evaluated in ZK.
Consider an interesting fact. For any value , then if and only if .
In our code above is a binary value. It's value is either or . We make use of this fact by the following:
- when
- when . If then the expression is .
So we can rewrite foo(w, a, b)
as the mathematical function
We now can convert this expression to a constraint system.
ZK statements take the form of:
More succinctly as:
These statements are converted into polynomials of the form:
is the target polynomial and in our case will be . is the cofactor polynomial. The statement says that the polynomial has roots (is equal to zero) at the points when .
Earlier we wrote our mathematical statement which we will now convert to constraints.
Rearranging the equation, we note that:
Swapping and rearranging, our final statement becomes . Represented in ZK as:
The last line is a boolean constraint that is either or by enforcing that (re-arranged this is ).
Line | L(x) | R(x) | O(x) |
---|---|---|---|
1 | |||
2 | |||
3 |
Because of how the polynomials are created during the setup phase, you must supply them with the correct variables that satisfy these constraints, so that (line 1), (line 2) and (line 3).
Each one of , and is supplied a list of (constant coefficient, variable value) pairs.
In bellman library, the constant is a fixed value of type Scalar
.
The variable is a type called Variable
. These are the values fed
into lc0
(the 'left' polynomial), lc1
(the 'right' polynomial),
and lc2
(the 'out' polynomial).
In our example we had a function where for example .
The verifier does not know the variables ,
and which are allocated by the prover as variables. However
the verifier does know the coefficients (which are of the Scalar
type) shown in the table above. In our example they only either
or , but can also be other constant values.
#![allow(unused)] fn main() { pub struct LinearCombination<Scalar: PrimeField>(Vec<(Variable, Scalar)>); }
It is important to note that each one of the left, right and out registers is simply a list of tuples of (constant coefficient, variable value).
When we wish to add a constant value, we use the variable called
~one
(which is always the first automatically allocated variable in
bellman at index 0). Therefore we end up adding our constant to
the LinearCombination
as (c, ~one)
.
Any other non-constant value, we wish to add to our constraint system
must be allocated as a variable. Then the variable is added to the
LinearCombination
. So in our example, we will allocate
, getting back Variable
objects which we then add to
the left lc, right lc or output lc.