about summary refs log tree commit diff
path: root/tvix/docs/src/nix-daemon/serialization.md
blob: a2694a4dea30a8af7042b7ecb65d0da3fc51523d (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305

### UInt64
Little endian byte order

### Bytes

- len :: [UInt64](#uint64)
- len bytes of content
- padding with zeros to ensure 64 bit alignment of content with padding


## Int serializers

### Int
[UInt64](#uint64) cast to C `unsigned int` with upper bounds checking.

### Int64
[UInt64](#uint64) cast to C `int64_t` with upper bounds checking.

### UInt8
[UInt64](#uint64) cast to C `uint8_t` with upper bounds checking.

### Size
[UInt64](#uint64) cast to C `size_t` with upper bounds checking.

### Time
[UInt64](#uint64) cast to C `time_t` with upper bounds checking.
s
### Bool
Sent as an [Int](#int) where 0 is false and everything else is true.

### Bool64
Sent as an [UInt64](#uint64) where 0 is false and everything else is true.

### FileIngestionMethod
An [UInt8](#uint8) enum with the following possible values:

| Name      | Int |
| --------- | --- |
| Flat      |  0  |
| Recursive |  1  |

### BuildMode
An [Int](#int) enum with the following possible values:

| Name   | Int |
| ------ | --- |
| Normal |  0  |
| Repair |  1  |
| Check  |  2  |

### Verbosity
An [Int](#int) enum with the following possible values:

| Name      | Int |
| --------- | --- |
| Error     |  0  |
| Warn      |  1  |
| Notice    |  2  |
| Info      |  3  |
| Talkative |  4  |
| Chatty    |  5  |
| Debug     |  6  |
| Vomit     |  7  |

### GCAction
An [Int](#int) enum with the following possible values:

| Name           | Int |
| -------------- | --- |
| ReturnLive     |  0  |
| ReturnDead     |  1  |
| DeleteDead     |  2  |
| DeleteSpecific |  3  |

### BuildStatus
An [Int](#int) enum with the following possible values:

| Name                   | Int |
| ---------------------- | --- |
| Built                  |  0  |
| Substituted            |  1  |
| AlreadyValid           |  2  |
| PermanentFailure       |  3  |
| InputRejected          |  4  |
| OutputRejected         |  5  |
| TransientFailure       |  6  |
| CachedFailure          |  7  |
| TimedOut               |  8  |
| MiscFailure            |  9  |
| DependencyFailed       | 10  |
| LogLimitExceeded       | 11  |
| NotDeterministic       | 12  |
| ResolvesToAlreadyValid | 13  |
| NoSubstituters         | 14  |

### ActivityType
An [Int](#int) enum with the following possible values:

| Name          | Int |
| ------------- | --- |
| Unknown       |   0 |
| CopyPath      | 100 |
| FileTransfer  | 101 |
| Realise       | 102 |
| CopyPaths     | 103 |
| Builds        | 104 |
| Build         | 105 |
| OptimiseStore | 106 |
| VerifyPaths   | 107 |
| Substitute    | 108 |
| QueryPathInfo | 109 |
| PostBuildHook | 110 |
| BuildWaiting  | 111 |
| FetchTree     | 112 |

### ResultType
An [Int](#int) enum with the following possible values:

| Name             | Int |
| ---------------- | --- |
| FileLinked       | 100 |
| BuildLogLine     | 101 |
| UntrustedPath    | 102 |
| CorruptedPath    | 103 |
| SetPhase         | 104 |
| Progress         | 105 |
| SetExpected      | 106 |
| PostBuildLogLine | 107 |
| FetchStatus      | 108 |

### FieldType
An [Int](#int) enum with the following possible values:

| Name   | Int |
| ------ | --- |
| Int    |  0  |
| String |  1  |


## Bytes serializers

### String
Simply a [Bytes](#bytes) that has some UTF-8 string like semantics sometimes.

### StorePath
String representation of a full store path.

### BaseStorePath
String representation of the basename of a store path. That is the store path
without the /nix/store prefix.

### OptStorePath
Optional store path.

If no store path this is serialized as the empty string otherwise it is the same as
[StorePath](#storepath).

### ContentAddressMethodWithAlgo
One of the following strings:
- text:`hash algorithm`
- fixed:r:`hash algorithm`
- fixed:`hash algorithm`

### DerivedPath
#### If protocol is 1.30 or newer
        return DerivedPath::parseLegacy(store, s);
#### If protocol is older than 1.30
        return parsePathWithOutputs(store, s).toDerivedPath();

### ContentAddress
String with the format:
- [ContentAddressMethodWithAlgo](#contentaddressmethodwithalgo):`hash`

### OptContentAddress
Optional version of [ContentAddress](#contentaddress) where empty string means
no content address.

### DrvOutput
String with format:
- `hash with any prefix`!`output name`

### Realisation
A JSON object sent as a string.

The JSON object has the following keys:
| Key                   | Value                   |
| --------------------- | ----------------------- |
| id                    | [DrvOutput](#drvoutput) |
| outPath               | [StorePath](#storepath) |
| signatures            | Array of String         |
| dependentRealisations | Object where key is [DrvOutput](#drvoutput) and value is [StorePath](#storepath) |


## Complex serializers

### List of x
A list is encoded as a [Size](#size) length n followed by n encodings of x

### Map of x to y
A map is encoded as a [Size](#size) length n followed by n encodings of pairs of x and y


### BuildResult
- status :: [BuildStatus](#buildstatus)
- errorMsg :: [String](#string)

#### Protocol 1.29 or newer
- timesBuilt :: [Int](#int)
- isNonDeterministic :: [Bool64](#bool64)
- startTime :: [Time](#time)
- stopTime :: [Time](#time)

#### Protocol 1.37 or newer
- cpuUser :: [OptMicroseconds](#optmicroseconds)
- cpuSystem :: [OptMicroseconds](#optmicroseconds)

#### Protocol 1.28 or newer
builtOutputs ::  [Map](#map-of-x-to-y) of [DrvOutput](#drvoutput) to [Realisation](#realisations)

### KeyedBuildResult
- path :: [DerivedPath](#derivedpath)
- result :: [BuildResult](#buildresult)

### OptMicroseconds
Optional microseconds.

- tag :: [UInt8](#uint8)

#### If tag is 1
- seconds :: [Int64](#int64)


### SubstitutablePathInfo
- storePath :: [StorePath](#storepath)
- deriver :: [OptStorePath](#optstorepath)
- references :: [List](#list-of-x) of [StorePath](#storepath)
- downloadSize :: [UInt64](#uint64)
- narSize :: [UInt64](#uint64)


### UnkeyedValidPathInfo
- deriver :: [OptStorePath](#optstorepath)
- narHash :: [String](#string) SHA256 NAR hash base16 encoded
- references :: [List](#list-of-x) of [StorePath](#storepath)
- registrationTime :: [Time](#time)
- narSize :: [UInt64](#uint64)

#### If protocol version is 1.16 or above
- ultimate :: [Bool64](#bool64)
- signatures :: [List](#list-of-x) of [String](#string)
- ca :: [OptContentAddress](#optcontentaddress)


### ValidPathInfo
- path :: [StorePath](#storepath)
- info :: [UnkeyedValidPathInfo](#unkeyedvalidpathinfo)

### DerivationOutput
- path :: [String](#string)
- hashAlgo :: [String](#string)
- hash :: [String](#string)

### BasicDerivation
- outputs :: [Map](#map-of-x-to-y) of [String](#string) to [DerivationOutput](#derivationoutput)
- inputSrcs :: [List](#list-of-x) of [StorePath](#storepath)
- platform :: [String](#string)
- builder :: [String](#string)
- args :: [List](#list-of-x) of [String](#string)
- env :: [Map](#map-of-x-to-y) of [String](#string) to [String](#string)

### TraceLine
- havePos :: [Size](#size) (hardcoded to 0)
- hint :: [String](#string)

### Error
- type :: [String](#string) (hardcoded to `Error`)
- level :: [Verbosity](#verbosity)
- name :: [String](#string) (removed and hardcoded to `Error`)
- msg :: [String](#string)
- havePos :: [Size](#size) (hardcoded to 0)
- traces :: [List](#list-of-x) of [TraceLine](#traceline)

## Field
- type :: [FieldType](#fieldtype)

### If type is Int
- value :: [UInt64](#uint64)

### If type is String
- value :: [String](#string)


## Framed

At protocol 1.23 [AddToStoreNar](./operations.md#addtostorenar) introduced a
framed streaming for sending the NAR dump and later versions of the protocol
also used this framing for other operations.

At its core the framed streaming is just a series of [Bytes](#bytes) of
varying length and terminated by an empty [Bytes](#bytes).

This method of sending data has the advantage of not having to parse the data
to find where it ends. Older versions of the protocol would potentially parse
the NAR twice.