These data structures are constructed of mostly User Elements defined by the Federated servers.
A Directory Block consists of a header and a body. The body is a series of pairs of ChainIDs and Entry Block Merkle Roots. The Body and BodyMR are derived from iterating the process lists generated by the Federated servers over the last 10 minutes.
Note about the Timestamp: This timestamp differs from Bitcoin, as it signifies the opening rather than the closing of a block. All the lower level blocks have minute markers in them, signifying how many full minutes after opening that an Entry was seen. If block creation starts in the middle of a 10 minute window, it will still show being opened a the beginning of the window, but all the Entries will have minute markers indicating they were seen at the correct time.
This is a special block which accompanies this Directory Block. It contains the signatures and organizational data needed to validate previous and future Directory Blocks. It has a LookupHash, which is a SHA256 of the entire block. The LookupHash is included in the directory block body paired with the Admin ChainID.
There can be several distinct items in the expansion area. They are identified by a type byte. The length of the serialized data is also specified. This allows older versions of software to skip data types they don't understand. The length descriptor is a varInt_F.
Here is a serialization example:
... | Header Expansion Size varInt_F | BH type byte | Length of BH | 32 bytes of BH | Future type byte | length of Future object | Future object (8 bytes for example) | ...
... 2C 01 20 c76c73de94826eefa4e485469922ee3d98f8f3aec8ddb45b101a2012772a9cad 02 08 0123456789ABCDEF ...
Administrative Identifier (AdminID) bytes are single bytes which specify how to interpret the following data. It specifies the type, and the type determines how to interpret subsequent bytes.
The coinbase descriptor is an entry in the Admin block which specifies a number of factoid output addresses and amounts. These outputs are used to deterministically generate a coinbase transaction 1000 blocks (about 7 days) later. It is included in each block height that is divisible by 25. There can only be one per Admin block.
The Coinbase Descriptor cannot be larger than 10233 bytes (1 AdminID byte + 2 varint length bytes + (10 KiB max FCT tx - 10 header bytes of coinbase))
The Coinbase Descriptor Cancel (CDC) is an entry in the Admin block which reverses an output in a Coinbase Descriptor. When a coinbase is made in the future, any outputs that have a CDC pointed at them will be missing. The output will not appear in the coinbase, and there will be fewer outputs in the coinbase than in the Coinbase Descriptor. The Factoids which aren't created will be retained as potential Factoids in the Grant Pool. This allows the grant process to correct an accident that was prevented by the CDC.
The Coinbase Descriptor cannot be larger than 8 bytes (on a 100 year timeline) (1 AdminID byte + 1 varint length byte + 4 bytes for height + 2 bytes for max index)
An Entry Credit (EC) Block is a datastructure which packages Chain Commits, Entry Commits, and EC balance increases over a 10 minute period. The Entries are ordered in the Entry Block in the order that they were received by each Federated server. All the Federated servers contribute to the building of the EC Block.
The Entry Credit Block consists of a header and a body. The body is composed of primarily Commits and balance increases with minute markers and server markers distributed throughout the body.
Entry Credit Identifier (ECID) bytes are single bytes which specify how to interpret the following data. It specifies the type, and the type determines how to interpret subsequent bytes.
Factoid Block is a datastructure which packages Factoid transactions over a 10 minute period. The Factoid transactions are ordered in the Block in the order that they were received by the Federated server.
The Factoid Block consists of a header and a body. The body is composed of serialized Factoid transactions with minute markers distributed throughout the body. The minute markers consist of a single byte 0x00. There are 10 of them, each being placed at a minute boundary, and the 10th marker being the last item in the block. Factoid transactions begin with a version number above zero, which is how they can be differentiated from transactions. The minute markers are included in both the body and ledger Merkle roots.
An Entry Block is a datastructure which packages references to Entries all sharing a ChainID over a 10 minute period. The Entries are ordered in the Entry Block in the order that they were received by the Federated server. The Entry Blocks form a blockchain for a specific ChainID.
The Entry Block consists of a header and a body. The body is composed of primarily Entry Hashes with 10 one minute markers distributed throughout the body.
Time delimiters are 32 byte big endian objects between 1 and 10 (inclusive). They are inserted in into the Entry Block when a new Federated server yields control of the Chain and an Entry has been acknowledged prior to the handoff. They are not needed if there is not an Entry to include that minute. Note, there can be duplicate Entries included in an Entry Block. If an Entry is paid for twice, it is included twice. The times are organized when the Federated server saw and acknowledged the Entry.
Body With 1 Entry at 0:10 into block
Body With 1 Entry at 1:10 into block
Body With 1 Entry at 5:50 into block
Body With 1 Entry at 9:50 into block
Body With 2 Entries: 2:20 and 5:30 into block
Body With 3 Entries: 2:20, 2:40 and 5:30 into block
Body With 8 Entries: 0:05, 3:40, 3:59, 4:00, 4:01, 4:30, 5:30, 9:59 into block with a duplicate
These are some custom datastructures for Factom
A Key Merkle Root is a datastructure which allows fast validation of a header and also allows Merkle proofs to be built to the body data elements.
First a Merkle tree is constructed of all the body elements. It is called the BodyMR. This is very similar to how all Bitcoin transactions can be proven with a Merkle root in the header.
The BodyMR is included in the header, among other things. The serialized header is then hashed. The hashed header is combined with the BodyMR and hashed. This creates the KeyMR. With only the KeyMR, when a header is produced by a peer, the header can be validated with 2 hashes.
Note: the Factoid's LedgerKeyMR is constructed differently than the other KeyMRs. It concatenates the LedgerMR first, then the header hash. This is the opposite of other KeyMRs, which put the header hash first. The rationale for this is to have different hashes for the Factoid's PrevKeyMR and PrevLedgerKeyMR when there is only a coinbase, which increases the security level to that of other objects in this case.
Integers can be serialized in blocks in a compact form to save space when stored indefinitely, like in a blockchain. Factom's varInt_F is modeled after the Protobuff's variable length integer, but is big-endian compared to Protobuff's little endian, and operates on the byte level rather then 4 bytes.
Values 127 and below can be represented in a single byte. Only positive integers are supported. The highest number is 2^64, which takes 10 bytes.
Larger numbers are represented by a sequence of bytes. The sequence length is indicated by the Most Significant Bit of each byte. If the MSB is set to one, then the next byte is considered part of the number. The number itself is held in the lower 7 bits. The LSB of the number is held in the LSB of the last byte. Bits higher than the 7th are held in earlier bytes.
This is the algorithm to create the stream: Convert the value to big endian. Count the number of bits between the LSB and the most significant 1 bit, inclusive. Divide this number by 7 and take the cieling of the remainder. This is the byte count M. Create a byte sequence with M bytes. Take the least significant 7 bits of the number and place them in the Mth byte. Set the highest bit of the Mth byte to zero. Take the bits 13 through 7 and add them to the byte M-1. Set the highest bit of byte M-1 to one. Continue until all the M bytes have been filled with with data.
Here are some examples: