about summary refs log tree commit diff
path: root/tvix/docs/src/nix-daemon/serialization.md

UInt64

Little endian byte order

Bytes

  • len :: UInt64
  • len bytes of content
  • padding with zeros to ensure 64 bit alignment of content + padding

Int serializers

Int

UInt64 cast to C unsigned int with upper bounds checking.

Int64

UInt64 cast to C int64_t with upper bounds checking. This means that negative numbers can be written but not read. Since this is only used for cpuSystem and cpuUser it is fine that negative numbers aren't supported.

UInt8

UInt64 cast to C uint8_t with upper bounds checking.

Size

UInt64 cast to C size_t with upper bounds checking.

Time

UInt64 cast to C time_t with upper bounds checking. This means that negative numbers can be written but not read.

Bool

Sent as an Int where 0 is false and everything else is true.

Bool64

Sent as an UInt64 where 0 is false and everything else is true.

FileIngestionMethod

An UInt8 enum with the following possible values:

Name Int
Flat 0
NixArchive 1

BuildMode

An Int enum with the following possible values:

Name Int
Normal 0
Repair 1
Check 2

Verbosity

An Int enum with the following possible values:

Name Int
Error 0
Warn 1
Notice 2
Info 3
Talkative 4
Chatty 5
Debug 6
Vomit 7

GCAction

An Int enum with the following possible values:

Name Int
ReturnLive 0
ReturnDead 1
DeleteDead 2
DeleteSpecific 3

BuildStatus

An Int enum with the following possible values:

Name Int
Built 0
Substituted 1
AlreadyValid 2
PermanentFailure 3
InputRejected 4
OutputRejected 5
TransientFailure 6
CachedFailure 7
TimedOut 8
MiscFailure 9
DependencyFailed 10
LogLimitExceeded 11
NotDeterministic 12
ResolvesToAlreadyValid 13
NoSubstituters 14

ActivityType

An Int enum with the following possible values:

Name Int
Unknown 0
CopyPath 100
FileTransfer 101
Realise 102
CopyPaths 103
Builds 104
Build 105
OptimiseStore 106
VerifyPaths 107
Substitute 108
QueryPathInfo 109
PostBuildHook 110
BuildWaiting 111
FetchTree 112

ResultType

An Int enum with the following possible values:

Name Int
FileLinked 100
BuildLogLine 101
UntrustedPath 102
CorruptedPath 103
SetPhase 104
Progress 105
SetExpected 106
PostBuildLogLine 107
FetchStatus 108

FieldType

An Int enum with the following possible values:

Name Int
Int 0
String 1

OptTrusted

An UInt8 optional enum with the following possible values:

Name Int
None 0
Some(Trusted) 1
Some(NotTrusted) 2

Bytes serializers

String

Simply a Bytes that has some UTF-8 string like semantics sometimes.

StorePath

String representation of a full store path including the store directory.

BaseStorePath

String representation of the basename of a store path. That is the store path without the /nix/store prefix.

StorePathName

String representation of the name part of a base store path. This is the part of the store path after the nixbase32 hash and '-'

It must have the following format:

  • Deny ".", "..", or those strings followed by '-'
  • Otherwise check that each character is 0-9, a-z, A-Z or one of +-._?=

StorePathHash

String representation of the hash part of a base store path. This is the part of the store path at the beginning and before the '-' and is in nixbase32 format.

OutputName

String representation of the name of a derivation output. This is usually combined with the name in the derivation to form the store path name for the store path with this output.

Since output name is usually combined to form a store path name its format must follow the same rules as StorePathName:

  • Deny ".", "..", or those strings followed by '-'
  • Otherwise check that each character is 0-9, a-z, A-Z or one of +-._?=

OptStorePath

Optional store path.

If no store path this is serialized as the empty string otherwise it is the same as StorePath.

Path

String representation of an absolute path.

NARHash

String base16-encoded NAR SHA256 hash without algorithm prefix.

Signature

String with a signature for the given store path or realisation. This should be in the format name:base 64 encoded signature but this is not enforced in the protocol.

HashAlgorithm

String with one of the following values:

  • md5
  • sha1
  • sha256
  • sha512

HashDigest

String with a hash digest in any encoding

OptHashDigest

Optional version of HashDigest where empty string means no value.

ContentAddressMethodWithAlgo

String with one of the following formats:

OptContentAddressMethodWithAlgo

Optional version of ContentAddressMethodWithAlgo where empty string means no value.

ContentAddress

String with the format:

OptContentAddress

Optional version of ContentAddress where empty string means no content address.

DerivedPath

If protocol is 1.30 or newer

output-names = OutputName, { "," OutputName }
output-spec = "*" | output-names
derived-path = StorePath, [ "!", output-spec ]

If protocol is older than 1.30

StorePath, [ "!", OutputName, { "," OutputName } ]

DrvOutput

String with format:

Realisation

A JSON object sent as a String.

The JSON object has the following keys:

Key Value
id DrvOutput
outPath StorePath
signatures Array of Signature
dependentRealisations Object where key is DrvOutput and value is StorePath

Complex serializers

List of x

A list is encoded as a Size length n followed by n encodings of x

Map of x to y

A map is encoded as a Size length n followed by n encodings of pairs of x and y

Set of x

A set is encoded as a Size length n followed by n encodings of x

BuildResult

Protocol 1.29 or newer

  • timesBuilt :: Int (defaults to 0)
  • isNonDeterministic :: Bool64 (defaults to false)
  • startTime :: Time (defaults to 0)
  • stopTime :: Time (defaults to 0)

Protocol 1.37 or newer

Protocol 1.28 or newer

builtOutputs :: Map of DrvOutput to Realisation

KeyedBuildResult

OptMicroseconds

Optional microseconds.

If tag is 1

SubstitutablePathInfo

UnkeyedValidPathInfo

If protocol version is 1.16 or above

ValidPathInfo

DerivationOutput

BasicDerivation

TraceLine

  • havePos :: Size (hardcoded to 0)
  • hint :: String (If logger is JSON, invalid UTF-8 is replaced with U+FFFD)

Error

  • type :: String (hardcoded to Error)
  • level :: Verbosity
  • name :: String (removed and hardcoded to Error)
  • msg :: String (If logger is JSON, invalid UTF-8 is replaced with U+FFFD)
  • havePos :: Size (hardcoded to 0)
  • traces :: List of TraceLine

Field

If type is Int

If type is String

Framed

At protocol 1.23 AddToStoreNar introduced a framed streaming for sending the NAR dump and later versions of the protocol also used this framing for other operations.

At its core the framed streaming is just a series of UInt64 size followed by size bytes. The stream is terminated when size is zero.

Unlike Bytes, frames are NOT padded.

This method of sending data has the advantage of not having to parse the data to find where it ends. Older versions of the protocol would potentially parse the NAR twice.

AddMultipleToStore format

Paths must be topologically sorted.

Repeated expected times

Export path format

If hasSignature is 1

  • signature :: String (ignored)

Import paths format

While hasNext is 1