about summary refs log tree commit diff
path: root/third_party/git/Documentation/technical/partial-clone.txt
diff options
context:
space:
mode:
authorVincent Ambo <tazjin@google.com>2020-05-25T23·06+0100
committerVincent Ambo <tazjin@google.com>2020-05-25T23·06+0100
commit93ba78d6f4632ef1c5228965e3edc8c0faf88c1e (patch)
tree85730c182a9f5f492ade8e8ccdb1c2356f9900bd /third_party/git/Documentation/technical/partial-clone.txt
parent6f8fbf4aa4b1654ab27d4829e114538761817de0 (diff)
revert(3p/git): Revert merge of git upstream at v2.26.2 r/852
This causes cgit to serve error pages, which is undesirable.

This reverts commit 5229c9b232de5bfa959ad6ebbb4c8192ac513352, reversing
changes made to f2b211131f2347342dde63975b09cf603149f1a3.
Diffstat (limited to 'third_party/git/Documentation/technical/partial-clone.txt')
-rw-r--r--third_party/git/Documentation/technical/partial-clone.txt129
1 files changed, 39 insertions, 90 deletions
diff --git a/third_party/git/Documentation/technical/partial-clone.txt b/third_party/git/Documentation/technical/partial-clone.txt
index b9e17e7a2878..896c7b387886 100644
--- a/third_party/git/Documentation/technical/partial-clone.txt
+++ b/third_party/git/Documentation/technical/partial-clone.txt
@@ -30,20 +30,12 @@ advance* during clone and fetch operations and thereby reduce download
 times and disk usage.  Missing objects can later be "demand fetched"
 if/when needed.
 
-A remote that can later provide the missing objects is called a
-promisor remote, as it promises to send the objects when
-requested. Initially Git supported only one promisor remote, the origin
-remote from which the user cloned and that was configured in the
-"extensions.partialClone" config option. Later support for more than
-one promisor remote has been implemented.
-
 Use of partial clone requires that the user be online and the origin
-remote or other promisor remotes be available for on-demand fetching
-of missing objects.  This may or may not be problematic for the user.
-For example, if the user can stay within the pre-selected subset of
-the source tree, they may not encounter any missing objects.
-Alternatively, the user could try to pre-fetch various objects if they
-know that they are going offline.
+remote be available for on-demand fetching of missing objects.  This may
+or may not be problematic for the user.  For example, if the user can
+stay within the pre-selected subset of the source tree, they may not
+encounter any missing objects.  Alternatively, the user could try to
+pre-fetch various objects if they know that they are going offline.
 
 
 Non-Goals
@@ -108,18 +100,18 @@ or commits that reference missing trees.
 Handling Missing Objects
 ------------------------
 
-- An object may be missing due to a partial clone or fetch, or missing
-  due to repository corruption.  To differentiate these cases, the
-  local repository specially indicates such filtered packfiles
-  obtained from promisor remotes as "promisor packfiles".
+- An object may be missing due to a partial clone or fetch, or missing due
+  to repository corruption.  To differentiate these cases, the local
+  repository specially indicates such filtered packfiles obtained from the
+  promisor remote as "promisor packfiles".
 +
 These promisor packfiles consist of a "<name>.promisor" file with
 arbitrary contents (like the "<name>.keep" files), in addition to
 their "<name>.pack" and "<name>.idx" files.
 
 - The local repository considers a "promisor object" to be an object that
-  it knows (to the best of its ability) that promisor remotes have promised
-  that they have, either because the local repository has that object in one of
+  it knows (to the best of its ability) that the promisor remote has promised
+  that it has, either because the local repository has that object in one of
   its promisor packfiles, or because another promisor object refers to it.
 +
 When Git encounters a missing object, Git can see if it is a promisor object
@@ -131,12 +123,12 @@ expensive-to-modify list of missing objects.[a]
 - Since almost all Git code currently expects any referenced object to be
   present locally and because we do not want to force every command to do
   a dry-run first, a fallback mechanism is added to allow Git to attempt
-  to dynamically fetch missing objects from promisor remotes.
+  to dynamically fetch missing objects from the promisor remote.
 +
 When the normal object lookup fails to find an object, Git invokes
-promisor_remote_get_direct() to try to get the object from a promisor
-remote and then retry the object lookup.  This allows objects to be
-"faulted in" without complicated prediction algorithms.
+fetch-object to try to get the object from the server and then retry
+the object lookup.  This allows objects to be "faulted in" without
+complicated prediction algorithms.
 +
 For efficiency reasons, no check as to whether the missing object is
 actually a promisor object is performed.
@@ -165,7 +157,8 @@ and prefetch those objects in bulk.
 +
 We are not happy with this global variable and would like to remove it,
 but that requires significant refactoring of the object code to pass an
-additional flag.
+additional flag.  We hope that concurrent efforts to add an ODB API can
+encompass this.
 
 
 Fetching Missing Objects
@@ -189,63 +182,21 @@ has been updated to not use any object flags when the corresponding argument
   though they are not necessary.
 
 
-Using many promisor remotes
----------------------------
-
-Many promisor remotes can be configured and used.
-
-This allows for example a user to have multiple geographically-close
-cache servers for fetching missing blobs while continuing to do
-filtered `git-fetch` commands from the central server.
-
-When fetching objects, promisor remotes are tried one after the other
-until all the objects have been fetched.
-
-Remotes that are considered "promisor" remotes are those specified by
-the following configuration variables:
-
-- `extensions.partialClone = <name>`
-
-- `remote.<name>.promisor = true`
-
-- `remote.<name>.partialCloneFilter = ...`
-
-Only one promisor remote can be configured using the
-`extensions.partialClone` config variable. This promisor remote will
-be the last one tried when fetching objects.
-
-We decided to make it the last one we try, because it is likely that
-someone using many promisor remotes is doing so because the other
-promisor remotes are better for some reason (maybe they are closer or
-faster for some kind of objects) than the origin, and the origin is
-likely to be the remote specified by extensions.partialClone.
-
-This justification is not very strong, but one choice had to be made,
-and anyway the long term plan should be to make the order somehow
-fully configurable.
-
-For now though the other promisor remotes will be tried in the order
-they appear in the config file.
-
 Current Limitations
 -------------------
 
-- It is not possible to specify the order in which the promisor
-  remotes are tried in other ways than the order in which they appear
-  in the config file.
+- The remote used for a partial clone (or the first partial fetch
+  following a regular clone) is marked as the "promisor remote".
 +
-It is also not possible to specify an order to be used when fetching
-from one remote and a different order when fetching from another
-remote.
-
-- It is not possible to push only specific objects to a promisor
-  remote.
+We are currently limited to a single promisor remote and only that
+remote may be used for subsequent partial fetches.
 +
-It is not possible to push at the same time to multiple promisor
-remote in a specific order.
+We accept this limitation because we believe initial users of this
+feature will be using it on repositories with a strong single central
+server.
 
-- Dynamic object fetching will only ask promisor remotes for missing
-  objects.  We assume that promisor remotes have a complete view of the
+- Dynamic object fetching will only ask the promisor remote for missing
+  objects.  We assume that the promisor remote has a complete view of the
   repository and can satisfy all such requests.
 
 - Repack essentially treats promisor and non-promisor packfiles as 2
@@ -267,17 +218,15 @@ remote in a specific order.
 Future Work
 -----------
 
-- Improve the way to specify the order in which promisor remotes are
-  tried.
+- Allow more than one promisor remote and define a strategy for fetching
+  missing objects from specific promisor remotes or of iterating over the
+  set of promisor remotes until a missing object is found.
 +
-For example this could allow to specify explicitly something like:
-"When fetching from this remote, I want to use these promisor remotes
-in this order, though, when pushing or fetching to that remote, I want
-to use those promisor remotes in that order."
-
-- Allow pushing to promisor remotes.
+A user might want to have multiple geographically-close cache servers
+for fetching missing blobs while continuing to do filtered `git-fetch`
+commands from the central server, for example.
 +
-The user might want to work in a triangular work flow with multiple
+Or the user might want to work in a triangular work flow with multiple
 promisor remotes that each have an incomplete view of the repository.
 
 - Allow repack to work on promisor packfiles (while keeping them distinct
@@ -350,26 +299,26 @@ Related Links
 [0] https://crbug.com/git/2
     Bug#2: Partial Clone
 
-[1] https://lore.kernel.org/git/20170113155253.1644-1-benpeart@microsoft.com/ +
+[1] https://public-inbox.org/git/20170113155253.1644-1-benpeart@microsoft.com/ +
     Subject: [RFC] Add support for downloading blobs on demand +
     Date: Fri, 13 Jan 2017 10:52:53 -0500
 
-[2] https://lore.kernel.org/git/cover.1506714999.git.jonathantanmy@google.com/ +
+[2] https://public-inbox.org/git/cover.1506714999.git.jonathantanmy@google.com/ +
     Subject: [PATCH 00/18] Partial clone (from clone to lazy fetch in 18 patches) +
     Date: Fri, 29 Sep 2017 13:11:36 -0700
 
-[3] https://lore.kernel.org/git/20170426221346.25337-1-jonathantanmy@google.com/ +
+[3] https://public-inbox.org/git/20170426221346.25337-1-jonathantanmy@google.com/ +
     Subject: Proposal for missing blob support in Git repos +
     Date: Wed, 26 Apr 2017 15:13:46 -0700
 
-[4] https://lore.kernel.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/ +
+[4] https://public-inbox.org/git/1488999039-37631-1-git-send-email-git@jeffhostetler.com/ +
     Subject: [PATCH 00/10] RFC Partial Clone and Fetch +
     Date: Wed,  8 Mar 2017 18:50:29 +0000
 
-[5] https://lore.kernel.org/git/20170505152802.6724-1-benpeart@microsoft.com/ +
+[5] https://public-inbox.org/git/20170505152802.6724-1-benpeart@microsoft.com/ +
     Subject: [PATCH v7 00/10] refactor the filter process code into a reusable module +
     Date: Fri,  5 May 2017 11:27:52 -0400
 
-[6] https://lore.kernel.org/git/20170714132651.170708-1-benpeart@microsoft.com/ +
+[6] https://public-inbox.org/git/20170714132651.170708-1-benpeart@microsoft.com/ +
     Subject: [RFC/PATCH v2 0/1] Add support for downloading blobs on demand +
     Date: Fri, 14 Jul 2017 09:26:50 -0400