HTTP-stream image transport: treat the image as a streamed file rather than an NFS/SMB mount

[mastacontrola]

What

A transport that pulls image data over HTTP as a stream, written against the current code as a fully owned implementation — not a shim bolted onto the mount-based path.

Today writeImage() and its callers assume the image is a file on a mounted filesystem (NFS, SMB, etc.): the storage node is mounted, and the image is cat'd from a path. HTTP transport is not "the same thing with a different cat" — the mount assumption is baked into how the path is resolved, how the image is located, and what is available to seek or stat. Treating the image as a streamed file is a different model and wants to be coded as one.

Why

• Removes the NFS/SMB mount as a prerequisite for deploy, and with it a large share of FOG's setup friction and failure modes (mount permissions, firewall rules, protocol version mismatches, stale mounts).
• HTTP is far easier to proxy, cache, load-balance, and secure than NFS.
• One transport that behaves the same everywhere, rather than per-protocol quirks.

Prior art — a 2018 POC that worked

Branch http-stream-deploy, commit c0e1f7e (2018-02-08). The whole change was two lines in writeImage():

-            cat $file >/tmp/pigz1 &
+            #cat $file >/tmp/pigz1 &
+            curl -Lkso /tmp/pigz1 ${web}/api/image/${img}/$(basename ${file}) &

Its commit message records where it stalled:

TODO: Stream the appropriate mbr, and download the partitions files to /tmp so resize could operate.

That is the useful part of the POC: it demonstrated the core stream works, and it identified the actual hard part — the metadata around the image, not the image data itself. The MBR/partition files are read as local files by the resize path, so a transport rewrite has to decide how those are fetched and staged, not just how the bulk data arrives.

The code itself is eight years stale and funcs.sh has been rewritten many times since; it is recorded here as evidence and a starting question list, not as something to rebase. The branch can be retired once this issue exists.

Open questions for the design

• Where do the MBR / partition / metadata files come from, and are they staged to /tmp or streamed too?
• Does resize need random access to anything the stream can't provide?
• What serves the data — an API endpoint on the server, the storage node directly, or both?
• How does this interact with multicast, which is inherently not HTTP?
• Range requests / resume on a dropped connection?
• How is it authenticated? (See https without curl -k in FOG OS? #102 re: curl -k and cert trust in FOS.)

Scope

New transport alongside the existing mount-based one, not a replacement on day one. Mount-based deploy keeps working.

[mastacontrola]

Branch http-stream-deploy has now been deleted (2026-07-16). It held exactly one real commit — c0e1f7e from 2018-02-08 — plus six Merge branch 'master' into ... commits keeping it alive until 2018-02-10.

Nothing is lost by the deletion: the complete diff (two lines) and the TODO are reproduced verbatim in the issue body above. That was the entire content of the branch. The code itself was eight years stale against a funcs.sh that has since been rewritten many times over, so there was nothing to rebase — only an idea, which is what this issue is for.

Note that the c0e1f7e SHA above is now unreachable and may eventually be garbage-collected, so treat the inline diff as the record rather than the link.