IPFS-Gate

From CompleteNoobs
Jump to navigation Jump to search
Please Select a Licence from the LICENCE_HEADERS page
And place at top of your page
If no Licence is Selected/Appended, Default will be CC0

Default Licence IF there is no Licence placed below this notice! When you edit this page, you agree to release your contribution under the CC0 Licence

LICENCE: More information about the cc0 licence can be found here:
https://creativecommons.org/share-your-work/public-domain/cc0

The person who associated a work with this deed has dedicated the work to the public domain by waiving all of his or her rights to the work worldwide under copyright law, including all related and neighboring rights, to the extent allowed by law.

You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission.

Licence:

Statement of Purpose

The laws of most jurisdictions throughout the world automatically confer exclusive Copyright and Related Rights (defined below) upon the creator and subsequent owner(s) (each and all, an "owner") of an original work of authorship and/or a database (each, a "Work").

Certain owners wish to permanently relinquish those rights to a Work for the purpose of contributing to a commons of creative, cultural and scientific works ("Commons") that the public can reliably and without fear of later claims of infringement build upon, modify, incorporate in other works, reuse and redistribute as freely as possible in any form whatsoever and for any purposes, including without limitation commercial purposes. These owners may contribute to the Commons to promote the ideal of a free culture and the further production of creative, cultural and scientific works, or to gain reputation or greater distribution for their Work in part through the use and efforts of others.

For these and/or other purposes and motivations, and without any expectation of additional consideration or compensation, the person associating CC0 with a Work (the "Affirmer"), to the extent that he or she is an owner of Copyright and Related Rights in the Work, voluntarily elects to apply CC0 to the Work and publicly distribute the Work under its terms, with knowledge of his or her Copyright and Related Rights in the Work and the meaning and intended legal effect of CC0 on those rights.

1. Copyright and Related Rights. A Work made available under CC0 may be protected by copyright and related or neighboring rights ("Copyright and Related Rights"). Copyright and Related Rights include, but are not limited to, the following:

   the right to reproduce, adapt, distribute, perform, display, communicate, and translate a Work;
   moral rights retained by the original author(s) and/or performer(s);
   publicity and privacy rights pertaining to a person's image or likeness depicted in a Work;
   rights protecting against unfair competition in regards to a Work, subject to the limitations in paragraph 4(a), below;
   rights protecting the extraction, dissemination, use and reuse of data in a Work;
   database rights (such as those arising under Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, and under any national implementation thereof, including any amended or successor version of such directive); and
   other similar, equivalent or corresponding rights throughout the world based on applicable law or treaty, and any national implementations thereof.

2. Waiver. To the greatest extent permitted by, but not in contravention of, applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and Related Rights and associated claims and causes of action, whether now known or unknown (including existing as well as future claims and causes of action), in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each member of the public at large and to the detriment of Affirmer's heirs and successors, fully intending that such Waiver shall not be subject to revocation, rescission, cancellation, termination, or any other legal or equitable action to disrupt the quiet enjoyment of the Work by the public as contemplated by Affirmer's express Statement of Purpose.

3. Public License Fallback. Should any part of the Waiver for any reason be judged legally invalid or ineffective under applicable law, then the Waiver shall be preserved to the maximum extent permitted taking into account Affirmer's express Statement of Purpose. In addition, to the extent the Waiver is so judged Affirmer hereby grants to each affected person a royalty-free, non transferable, non sublicensable, non exclusive, irrevocable and unconditional license to exercise Affirmer's Copyright and Related Rights in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "License"). The License shall be deemed effective as of the date CC0 was applied by Affirmer to the Work. Should any part of the License for any reason be judged legally invalid or ineffective under applicable law, such partial invalidity or ineffectiveness shall not invalidate the remainder of the License, and in such case Affirmer hereby affirms that he or she will not (i) exercise any of his or her remaining Copyright and Related Rights in the Work or (ii) assert any associated claims and causes of action with respect to the Work, in either case contrary to Affirmer's express Statement of Purpose.

4. Limitations and Disclaimers.

   No trademark or patent rights held by Affirmer are waived, abandoned, surrendered, licensed or otherwise affected by this document.
   Affirmer offers the Work as-is and makes no representations or warranties of any kind concerning the Work, express, implied, statutory or otherwise, including without limitation warranties of title, merchantability, fitness for a particular purpose, non infringement, or the absence of latent or other defects, accuracy, or the present or absence of errors, whether or not discoverable, all to the greatest extent permissible under applicable law.
   Affirmer disclaims responsibility for clearing rights of other persons that may apply to the Work or any use thereof, including without limitation any person's Copyright and Related Rights in the Work. Further, Affirmer disclaims responsibility for obtaining any necessary consents, permissions or other rights required for any use of the Work.
   Affirmer understands and acknowledges that Creative Commons is not a party to this document and has no duty or obligation with respect to this CC0 or use of the Work.

ipfs-gate operator walkthrough

This is a noob-friendly guide to standing up your own ipfs-gate server. It assumes you can SSH into a Linux box and follow recipes — no programming background needed.

> Status: v1 claim-model build. Working software but expect rough edges. Bugs found during real deployment get folded back into the Common Problems section below. Walkthrough last re-verified end-to-end on a fresh Vultr VPS (Ubuntu 24.04.4, 2GB RAM) on 2026-07-02 — every step below was executed as written unless noted.

> Reference OS: Ubuntu Server 24.04 LTS (Noble). The recipes are tested on this OS specifically. Other distros (Ubuntu 22.04, Debian 12, etc.) will likely work but might have subtle differences. For reproducibility, stick to Ubuntu 24.04 LTS on a fresh VPS.

> Reference user: This guide assumes you SSH in as root on a fresh VPS (the default for most providers — Vultr, Hetzner, Linode, DigitalOcean, etc.). If you operate as a non-root user with sudo, swap sudo in where shown.

What you are setting up

A server that:

  • Accepts file uploads from web browsers
  • Charges users in CNOOBS (a Hive-Engine token) for hosting — priced per MB-hour under the v1 claim model (size × time × copies, quoted at reserve time)
  • Stores the files on IPFS (the InterPlanetary File System — a peer-to-peer storage network)
  • Hands users back a "CID" (a fingerprint of their file) that they share with whoever they want to give the file to
  • Auto-deletes files when their claim expires (default 7 days / 168 hours; duration is user-selectable per upload)

You don't need to know how IPFS works internally. ipfs-gate sits in front of it, charges money, and unpins old files automatically.

What you need before starting

  1. A fresh Ubuntu 24.04 LTS VPS with at least:
    • 2 CPU cores, 2GB RAM, 20GB disk (more disk = more pinning capacity)
    • A public IP address and the ability to open ports 80 + 443
  2. A domain name you control (e.g. ipfs.v4call.com). It must point at your VPS's IP via an A record. Set this up at your DNS provider before starting — Let's Encrypt needs the domain to resolve.
  3. A dedicated Hive account for ipfs-gate (separate from any v4call account). Create one if needed. Fund it with about 2 HBD for refunds and Resource Credits.
  4. (Optional) The active private key for that Hive account (NOT the owner key, NOT the posting key — the active key). If you paste it into the server's .env the gate issues refunds automatically; leave it out and the gate still runs, with refunds queued for you to settle manually (see Step 3). Keep it secret.

If you don't have CNOOBS to test with, ask CompleteNoobs in v4call or contact via the Hive blockchain.

This guide installs everything on one VPS (kubo, ipfs-gate, nginx). For v0.1 scale this is fine; splitting onto separate boxes can come later.

Step 1: Install Docker on Ubuntu 24.04

SSH into the VPS as root:

ssh root@your-vps-ip

Update + install Docker (the official one-liner installer):

apt update && apt upgrade -y
curl -fsSL https://get.docker.com | sh

Verify:

docker --version
docker compose version

You should see Docker 27.x or newer and Docker Compose v2.x or newer.

(If you'd rather operate as a non-root user, also run usermod -aG docker your-user then log out and back in. The rest of this guide uses root; adjust paths under /root/... to /home/your-user/... as needed.)

Step 2: Clone ipfs-gate + prepare data directories

We'll deploy from /opt/IPFS-Gate (a standard location for operator-managed services):

cd /opt
git clone https://github.com/CompleteNoobs/IPFS-Gate.git
cd /opt/IPFS-Gate

Critical permission step. The ipfs-gate container runs as UID 1000 (the node user inside the container). Because we're running docker as root, the host data directories get created owned by root, and the container can't write to them. Pre-create them and chown:

mkdir -p data/kubo data/ipfs-gate data/letsencrypt data/letsencrypt-webroot
chown -R 1000:1000 data/

Skipping this leads to SQLITE_CANTOPEN errors in the ipfs-gate logs (see Common Problems #2).

Step 3: Configure .env

cp .env.example .env
nano .env

Fill in at minimum:

  • IPFS_GATE_HIVE_ACCOUNT — your dedicated Hive account name (e.g. v4call-ipfs). The only field the server hard-refuses to boot without.
  • ADMIN_KEY — pick a long random string. Generate one with openssl rand -hex 32 from another terminal. (Ships as change-me-to-something-random — change it.)
  • PUBLIC_GATEWAY_BASEhttps://ipfs.your-domain.com (your real domain). Required in production: without it, clients receive https://ipfs.localhost gateway links that download nothing (Common Problems #14).

Optional but recommended:

  • IPFS_GATE_ACTIVE_KEY — the account's active private key (starts with 5...). Key-optional since the v1 build: leave it blank and the gate still boots — refunds are recorded as pending in the refund ledger and you settle them manually (transfer + /admin/log-refund). The boot log confirms the mode with refunds: MANUAL — IPFS_GATE_ACTIVE_KEY unset. Set the key to enable automatic on-chain refunds. You can add it later: edit .env, then docker compose down && docker compose up -d (Common Problems #4).

Sensible defaults to leave alone:

  • BIND_HOST=0.0.0.0 — required for docker; do not change to 127.0.0.1
  • DISK_LIMIT_GB=5 — increase as your VPS allows
  • DEFAULT_TTL_DAYS=7 — accepts fractions for testing (e.g. 0.001 ≈ 1.4 min)
  • MAX_FILE_SIZE_MB=10 — per-upload cap
  • KUBO_DHT_MODE=none — private hosting; CID only servable from your own gateway. Set to client for public IPFS network availability
  • v1 pricing knobs (PRICE_RATE_PER_MB_HOUR=1, PRICE_MIN_HOURS=1, NODE_COUNT=1, …) — the shipped rate is a TEST-token placeholder, not a tuned price. Read PRICING-V1-DESIGN-NOTES.md before charging a token with real value. For scale: 1 CNOOBS/MB-hour means a 2MB file for the default 168h quotes at 336 CNOOBS.

Save and exit (Ctrl-X, Y, Enter in nano).

Step 4: Edit nginx config for your domain

nano nginx/ipfs-gate.conf

Replace every instance of ipfs.example.com with your actual domain (e.g. ipfs.v4call.com).

Leave the HTTPS server block commented out for now — we'll enable it after the cert is in place in Step 6.

Step 5: Boot HTTP-only and get an SSL cert

Bring up the stack with HTTP only:

docker compose up -d

The first run builds the ipfs-gate image and pulls kubo + nginx — expect several minutes on a small VPS. If your SSH session drops mid-build (broken pipe), don't panic: the build continues server-side. Reconnect and run docker compose ps — you'll likely find everything already up. (Confirmed during the 2026-07-02 re-verification: the SSH session died during the build; the stack was healthy on reconnect.)

Wait ~30 seconds for Kubo's first-boot identity generation. Then verify:

docker compose ps
docker compose logs ipfs-gate | tail -20

You should see lines like:

[quota] applied migration 001_initial.sql
[quota] applied migration 002_uploads_tab.sql
[quota] applied migration 003_claims.sql
[quota] applied migration 004_release_consents.sql
[quota] applied migration 005_receipts.sql
[quota] schema_version = 5
[server] kubo backend OK, version 0.42.0, used <N> bytes
[sweeper] starting, interval = 60000ms
ipfs-gate v1 (claim model) listening on 0.0.0.0:3001
  operator account: @your-hive-account
  pricing: 1 CNOOBS / MB-hour, min 1h, 1 node(s), ≤10MB
  refunds: MANUAL — IPFS_GATE_ACTIVE_KEY unset, refunds recorded pending
  CORS origin: *

(Exact version numbers drift; the refunds: MANUAL line only appears when you left IPFS_GATE_ACTIVE_KEY blank. The migration lines only print on first boot.)

If you see listening on 127.0.0.1:3001 instead, your .env has the old BIND_HOST. Fix it (Common Problems #6) before proceeding — the cert step needs HTTP serving to work.

Don't try to verify with curl http://your-domain/ at this stage. The shipped nginx config already 301-redirects all HTTP traffic (except the Let's Encrypt challenge path) to HTTPS — which doesn't exist until Step 6. A 301 pointing at a dead HTTPS URL is normal here, not a failure. Verify via the container logs above; certbot works regardless because /.well-known/acme-challenge/ is exempt from the redirect.

Now request the cert from Let's Encrypt:

# Replace ipfs.your-domain.com with your actual domain in both places below.
docker run --rm -it \
  -v "$PWD/data/letsencrypt:/etc/letsencrypt" \
  -v "$PWD/data/letsencrypt-webroot:/var/www/certbot" \
  --entrypoint certbot \
  certbot/certbot \
  certonly --webroot --webroot-path=/var/www/certbot \
  -d ipfs.your-domain.com \
  --email you@your-domain.com --agree-tos --no-eff-email

On success: Successfully received certificate. The cert files land in data/letsencrypt/live/ipfs.your-domain.com/.

(Running this from a script or non-interactive session? Drop the -it flags — the command needs no TTY and docker run -it errors without one. Verified 2026-07-02.)

If this fails with a DNS error, double-check that your domain's A record actually points at this VPS's IP (try dig ipfs.your-domain.com from a different machine).

Step 6: Enable HTTPS

nano nginx/ipfs-gate.conf

Uncomment the entire server { listen 443 ssl; ... } block. Make sure every ipfs.example.com in that block is also replaced with your domain (it should already match what you set in Step 4).

Reload nginx:

docker compose restart nginx
docker compose logs nginx | tail -10

Step 7: Verify it's working

From your laptop (NOT the VPS):

curl https://ipfs.your-domain.com/

Expected JSON response (v1 claim model — there is no flat amount any more; cost is quoted per upload at /reserve):

{"service":"ipfs-gate","version":"1.0.0-dev","operator":"your-hive-account",
 "payment":{"model":"claim-mb-hour","currency":"CNOOBS","max_size_mb":10,"default_hours":168,"ttl_days":7},
 "pricing":{"rate_per_mb_hour":1,"min_hours":1,"mb_divisor":1000000,"node_count":1,"copies_max":1,"replication_leeway":2},
 "features":{"public_uploads":true,"uploads_tab":true,"claim_model":true}}

Optional deeper smoke test — ask for a quote (no payment involved; the reservation self-expires after 5 minutes):

curl -s -X POST -H 'Content-Type: application/json' \
  -d '{"uploader":"your-hive-account","size_bytes":2000000}' \
  https://ipfs.your-domain.com/reserve

You should get back a reservation_id, a payment block with the memo to use, and a quote (2MB × 168h × 1 copy = 336 at the placeholder rate).

Test the admin endpoint too (substitute your real ADMIN_KEY):

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' https://ipfs.your-domain.com/admin/stats

You should see a JSON response with disk, pin, payment, moderation, and kubo stats.

Note: an HTTP request (http://...) returns a 301 redirect to HTTPS. That's expected — nginx forces HTTPS for all real traffic. Use curl -L to follow the redirect or hit https:// directly.

The landing page

Open https://ipfs.your-domain.com/ in a browser and you get the standalone landing page (public/index.html) instead of the JSON — same URL, different Accept header. The page is a self-contained client for your gate: log in with a Hive account (Keychain or posting key), then

  • My Uploads — list and delete/unpin your files (same signed-request API v4call's Uploads → IPFS-Gate tab uses)
  • Private Send — encrypt a file in-browser to chosen Hive accounts, pay the gate, get a v4reveal: link
  • Public Upload — plaintext upload → shareable gateway link
  • Reveal — paste a v4reveal: link and decrypt it (links are interchangeable with v4call's Private Send / Reveal tabs)

No configuration needed — the page talks to the gate it's served from by default. Payments always need the Hive Keychain browser extension (they're Active-key token transfers). If you see JSON in the browser instead of the page, your nginx config predates the map $http_accept block — re-copy nginx/ipfs-gate.conf from the repo (Step 4/6) and restart nginx.

Step 8: Schedule cert renewal

Let's Encrypt certs expire after 90 days. Add a cron job for root:

crontab -e

Add this single line (adjust nothing — uses /opt/IPFS-Gate from this walkthrough):

0 3 * * 1 cd /opt/IPFS-Gate && docker run --rm -v "$PWD/data/letsencrypt:/etc/letsencrypt" -v "$PWD/data/letsencrypt-webroot:/var/www/certbot" --entrypoint certbot certbot/certbot renew --quiet && docker compose restart nginx

Runs every Monday at 03:00 UTC. Certbot only actually renews when within 30 days of expiry — earlier runs are no-ops.

Updating to a new version

Recommended: scripts/ipfs-update.sh. Does steps 1–6 below for you, plus safety this manual recipe doesn't have: it validates the restored nginx config against the live container mounts before touching any running container, refuses to run if unexpected local changes exist outside nginx/ipfs-gate.conf, locks against a second overlapping invocation, and no-ops cleanly if already up to date. Install once, then re-run any time:

cp /opt/IPFS-Gate/scripts/ipfs-update.sh /usr/local/bin/ipfs-update.sh
chmod +x /usr/local/bin/ipfs-update.sh
/usr/local/bin/ipfs-update.sh

Written after the incident below (Common Problems #422) where a bare hard reset took the site down.

Manual recipe (what the script automates):

IPFS_GATE_DIR=/opt/IPFS-Gate
cd "$IPFS_GATE_DIR"

# 1. Back up your config + data BEFORE pulling
cp .env .env.bk-$(date +%Y%m%d)
cp -r data data.bk-$(date +%Y%m%d)
cp nginx/ipfs-gate.conf nginx/ipfs-gate.conf.bk-$(date +%Y%m%d)

# 2. Pull latest code (this WILL clobber .env and nginx/ipfs-gate.conf if you've edited them)
git fetch --all && git reset --hard origin/main

# 3. Restore your operator-specific files
cp .env.bk-$(date +%Y%m%d) .env
cp nginx/ipfs-gate.conf.bk-$(date +%Y%m%d) nginx/ipfs-gate.conf

# 4. Re-apply data ownership (in case new directories appeared)
chown -R 1000:1000 data/

# 5. Rebuild and restart (env-change or code-change BOTH need down/up, not restart)
docker compose down
docker compose build --no-cache
docker compose up -d

# 6. Watch logs to confirm clean start
docker compose logs -f ipfs-gate

Always back up .env, nginx/ipfs-gate.conf, and the data/ directory before any update. A git reset --hard will wipe any operator-edited files in the repo.

Admin tasks

All admin endpoints need Authorization: Bearer YOUR_ADMIN_KEY from your .env.

See server stats

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' https://ipfs.your-domain.com/admin/stats

Shows disk usage, pin counts, payment counts, recent moderation actions, Kubo health.

List a user's uploads

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' \
  "https://ipfs.your-domain.com/admin/uploads?account=guest33"

Ban an account

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"hive_account":"badguy","reason":"Spam","refund_policy":"none"}' \
  https://ipfs.your-domain.com/admin/ban

refund_policy can be "none" (default — banned user forfeits unused TTL) or "prorata" (calculate per-pin remaining-time refund; v0.1 records this intent but does NOT auto-issue the on-chain refund. The operator must transfer manually and then /admin/log-refund).

Unban an account

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"hive_account":"badguy"}' \
  https://ipfs.your-domain.com/admin/unban

Note: unban does NOT restore content that was unpinned. The user must re-upload + re-pay.

Take down a specific CID

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"cid":"bafkreig...","reason":"DMCA #12345"}' \
  https://ipfs.your-domain.com/admin/takedown

Adds the CID to the blocklist (can never be re-uploaded) and unpins from Kubo immediately.

Export the takedown list (for backup or peer sharing)

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' https://ipfs.your-domain.com/admin/takedowns > takedowns.json

Import a takedown list

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"takedowns":[{"cid":"bafk1...","reason":"From peer X"}]}' \
  https://ipfs.your-domain.com/admin/takedowns/import

View the audit log

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' \
  "https://ipfs.your-domain.com/admin/moderation/log?limit=50"

Append-only record of every moderation action with admin attribution and timestamps.

Orphan payments (manual refund needed)

If a user paid in wrong currency, wrong amount, or the Hive-Engine sidechain hadn't confirmed the transfer at upload time, the payment ends up in the orphan list:

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' https://ipfs.your-domain.com/admin/orphan-payments

Operator reviews, decides whether to manually refund (out-of-band Hive transfer), and logs the action:

# After manually transferring back to the user on Hive:
curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"payment_id":88,"refund_tx_id":"ef56gh78...","reason":"Wrong amount"}' \
  https://ipfs.your-domain.com/admin/log-refund

Optional: Private / family hosting (whitelist mode)

Everything above builds an open, pay-to-use gate. Whitelist mode flips that: only Hive accounts you explicitly invite can use the server at all, and invited accounts can be made free (no token payment) — the shape you want for a "family server." One knob, three configurations:

  • Whitelist-only, free-for-all — every invited account is fee_exempt. Nobody pays; only invited accounts can upload at all.
  • Admin-free + guest quotas — invited guests pay nothing but each gets a hard per-account storage cap via quota_bytes.
  • Paid-but-still-whitelisted — invited accounts still pay the normal per-MB-hour rate; whitelisting is just a second gate on top (e.g. "only people I've actually met can use my public gate").

Live-tested end-to-end on ipfs.v4call.com on 2026-07-05 — every example below is a copy of a real, working test run against the production server, not a hypothetical.

Turning it on

Edit .env:

WHITELIST_MODE=true
SERVER_ADMIN_HIVE_ACCOUNTS=

SERVER_ADMIN_HIVE_ACCOUNTS is optional — a comma-separated list of Hive accounts (lowercase, no @) that get a second, narrower admin tier authenticated by a Hive-signed request instead of your ADMIN_KEY (useful if you want a trusted family member to manage the guest list without handing them your box's admin password). Leave it blank if you, the box owner, will be the only admin — ADMIN_KEY alone already covers every /admin/* route including whitelist management.

Recreate the container so the env change takes effect (a plain restart does NOT pick up .env changes — Common Problems #4):

docker compose down && docker compose up -d

Confirm it's live:

curl -s https://ipfs.your-domain.com/ | grep whitelist_mode

You should see "whitelist_mode":true inside the features block. With WHITELIST_MODE=false (the default), behavior is byte-for-byte identical to the open gate described earlier in this guide — this mode is entirely opt-in.

Inviting an account

Add each friend/family Hive username with POST /admin/whitelist/add (Bearer ADMIN_KEY, same auth as every other /admin/* route):

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"target_account":"guest33","fee_exempt":true,"note":"family"}' \
  https://ipfs.your-domain.com/admin/whitelist/add

Fields:

  • target_account — required, the Hive username to invite.
  • fee_exempt — optional, default false. true means this account uploads at $0 (the quote still shows the real numbers, just zeroed out — never a silent free ride the account can't explain).
  • quota_bytes — optional, default unlimited. A hard per-account storage cap, on top of (never instead of) your server's overall DISK_LIMIT_GB. Whichever cap is tighter wins.
  • note — optional, freeform (e.g. "cousin's laptop").

List who's currently whitelisted:

curl -H 'Authorization: Bearer YOUR_ADMIN_KEY' https://ipfs.your-domain.com/admin/whitelist

Remove someone (soft-delete — blocks their new reservations only; anything already pinned for them keeps running until it naturally expires, or you ban/take it down separately):

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"target_account":"guest33"}' \
  https://ipfs.your-domain.com/admin/whitelist/remove

What a non-invited stranger sees

Anyone not on the list gets a clean rejection right at the quote stage, before any file bytes move:

$ curl -s -X POST -H 'Content-Type: application/json' \
    -d '{"uploader":"randomstranger999","size_bytes":1000000}' \
    https://ipfs.your-domain.com/reserve
{"error":"forbidden","message":"this server is invite-only — your account is not whitelisted","details":{}}

What an invited, fee-exempt account sees

The quote at /reserve comes back honestly labeled — total: 0 plus fee_exempt: true, never a silent free upload the client can't explain:

"quote": { ... "rate_per_mb_hour": 0, "total": 0, "currency": "TEST", "fee_exempt": true }

The follow-up POST /upload succeeds with no on-chain payment step at all — no Hive Keychain popup, no tx_id. The landing page's upload tabs (and v4call's ipfs-gate tab) show a "Free (fee-exempt)" badge in place of a cost estimate for these accounts.

Add someone with fee_exempt omitted (or explicitly false) and the whitelist becomes a pure access gate — they still pay the normal per-MB-hour rate:

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"target_account":"cnoobz","fee_exempt":false}' \
  https://ipfs.your-domain.com/admin/whitelist/add

/reserve now returns a real non-zero total, and /upload still rejects with 400 {"error":"bad_request","message":"tx_id required"} until a genuine Hive-Engine payment is presented — confirmed live. Useful for "only people I actually know may use this gate at all, but everyone still pays."

Per-account quota caps

Set quota_bytes when inviting someone if you want a hard ceiling per guest (e.g. "cousin gets 500MB, no more"):

curl -X POST -H 'Authorization: Bearer YOUR_ADMIN_KEY' -H 'Content-Type: application/json' \
  -d '{"target_account":"guest33","fee_exempt":true,"quota_bytes":524288000}' \
  https://ipfs.your-domain.com/admin/whitelist/add

Once their pending reservations + active pins hit the cap, further reservations fail with 507 insufficient_storage naming the exact numbers over the cap — tested live, e.g. "per-account quota exceeded (0 + 2000 > 1000 bytes)".

Security note: whitelist membership + on-chain key binding (fee-exempt path)

FIXED 2026-07-16 — this section now documents the hardened behavior; the historical gap is noted at the end. Whitelist mode gates on the Hive account name string supplied to /reserve and /upload. For a normal, paying upload the on-chain payment itself proves who's uploading — spending real CNOOBS/HBD from an account requires that account's real private key. For a fee-exempt whitelisted account there is no on-chain transaction, so /upload now performs the same key→account binding /uploads/by-user and the Hive-signed admin tier already do: the uploader_pubkey that signed upload_proof_sig must be a current posting key of the named Hive account (looked up live via condenser_api.get_accounts). Clients sign the upload proof with the posting key, so a legitimate upload passes unchanged; a throwaway keypair does not.

Behavior now (confirmed live 2026-07-16):

  • A fee-exempt upload signed with the account's real posting key → succeeds and pins (no change for legitimate users).
  • An upload claiming a fee-exempt account but signed with a throwaway keypair → rejected 401 {"error":"unauthorized","message":"uploader_pubkey is not a current posting key of this Hive account"}.
  • If Hive is unreachable at upload time the check fails closed422 unprocessable_entity (an attacker can't bypass by DoSing the RPC path; a legitimate user retries when nodes recover).
  • Whitelisted-but-paying accounts (fee_exempt:false) were never affected — spending that account's real CNOOBS already requires its real keys, verified on-chain. The new check is scoped to the fee-exempt branch only, so the paid path takes no extra Hive round-trip.

Defense-in-depth still worth keeping:

  • Keep DISK_LIMIT_GB and per-account quota_bytes reasonable — good hygiene against a compromised-key or runaway-client scenario, even though the name-spoofing hole is closed.

Historical note (pre-2026-07-16): before this fix, nothing checked uploader_pubkey against the chain on the fee-exempt path, so anyone who knew a whitelisted account name could upload for free under it with a freshly-generated keypair (attributed to the innocent account). See FEE-EXEMPT-UPLOAD-AUTH-HARDENING.md for the full write-up. If you run an old binary, git pull + rebuild.

Federation (deferred to v0.3+)

ipfs-gate is currently standalone. Each operator runs their own server with their own ban list, their own pricing, their own disk allocation. v0.3+ will add optional Nostr-based discovery and opt-in cross-operator banlist sharing — same architectural pattern as v4call's Nostr federation work.

Common problems

(This section grows as bugs are found and fixed during development. Ordered by first-boot likelihood.)

  1. Q: I ran docker compose up and Kubo doesn't respond for the first ~30 seconds. Is it broken?
    A: No. Kubo does a one-time identity generation on first boot. Takes 10–30s on a small VPS. Watch docker compose logs kubo to see when it's ready. The healthcheck in docker-compose.yml gates ipfs-gate startup on Kubo being healthy.
  2. Q: My ipfs-gate container won't start. I see one of:
    • EACCES: permission denied, mkdir '/app/data'
    • SqliteError: unable to open database file with code: 'SQLITE_CANTOPEN'
    • Container in restart loop with errors mentioning /app/data/ipfs-gate.db
    A: Same root cause: the container runs as UID 1000 (node user), but if you ran docker compose up as root, Docker created the host ./data/ directories owned by root. The container can't write there. Fix:
    chown -R 1000:1000 ./data/

docker compose down && docker compose up -d docker compose logs ipfs-gate | tail -20

  1. Confirmed during the v0.1 first-VPS test (2026-05-24). The Dockerfile correctly chowns /app/data at build, but the host volume mount overlays that with the host's ownership, which wins. Step 2 of this walkthrough does the chown proactively to avoid hitting this.
  2. Q: Boot logs say FATAL: IPFS_GATE_HIVE_ACCOUNT not set. Refusing to start.
    A: Your .env is missing or doesn't have IPFS_GATE_HIVE_ACCOUNT. Copy .env.example to .env and fill in the values (Step 3).
  3. Q: I changed something in .env and ran docker compose restart ipfs-gate, but the change didn't take effect (e.g. logs still show the old BIND_HOST or old TTL).
    A: docker compose restart reuses the existing container (and the env vars baked in at create-time). To pick up .env changes you must recreate the container:
    # Either down + up (clean, also recreates the network):

docker compose down && docker compose up -d # Or force-recreate just one service: docker compose up -d --force-recreate ipfs-gate

  1. This is parallel to v4call's "docker compose down required before rebuilding" gotcha — different cause (env change vs code change) but same surprise. Confirmed during v0.1 first-VPS test (2026-05-24).
  2. Q: My response JSON shows "ttl_days":0 even though I set DEFAULT_TTL_DAYS=0.001 for testing fast sweeper expiry. Or pins expire immediately.
    A: Pre-v0.1.1 code used parseInt which truncated 0.001 to 0. Fixed in v0.1.1: now uses parseFloat, so fractional days work. If you're on an old binary, git pull + rebuild. Reference: 0.001 day ≈ 1.44 minutes — handy for testing the sweeper.
  3. Q: Nginx returns 502 Bad Gateway when I curl my domain. The ipfs-gate container is running fine and logs show it's listening.
    A: Your .env has BIND_HOST=127.0.0.1. That binds ipfs-gate to the container's loopback only, which is unreachable from the nginx container. For Docker deployment, set BIND_HOST=0.0.0.0. The container is still isolated by the docker network — only nginx can reach it from inside the stack; the public only sees ports 80/443 on nginx. The shipped .env.example defaults to 0.0.0.0 — but pre-existing .env files from earlier installs still need the manual edit, then a full docker compose down && up -d (Common Problems #4).
  4. Q: Uploads succeed but recipients see "file not found" when fetching from a public IPFS gateway (e.g. ipfs.io).
    A: Check KUBO_DHT_MODE in your .env. If set to none (the v0.1 default), only your own ipfs-gate's gateway serves the file. Other public IPFS gateways won't know about it. This is intentional for privacy; switch to client if you want public-IPFS-network availability. Recipients fetching via https://ipfs.your-domain.com/ipfs/<cid> always work regardless of this setting.
  5. Q: Payment verification fails with memo mismatch.
    A: The browser must send the exact memo string returned by /reserve. Format is ipfs-gate:upload:<16-hex-id>. If the client edited it, this will fail. The reservation_id in the memo also must match the URL parameter.
  6. Q: Upload returns "payload.from undefined doesn't match sender <username>" even though the CNOOBS transfer landed on-chain.
    A: Pre-v0.1.2 had a redundant payload.from check in hive-verify.js that always failed: Hive-Engine's tokens/transfer contractPayload has no from field (the sender is implicit in the wrapping custom_json's required_auths, which extractTokenTransferOp already validates). Fixed in v0.1.2 by removing the check. If you're on an old binary, git pull + docker compose down && docker compose up -d --build. Confirmed during v0.1 first-client integration test (2026-05-25).
  7. Q: Uploads "succeed" with a CID but the gateway returns 404 / "CID not pinned here" a few minutes later (or even within seconds during testing).
    A: Almost always DEFAULT_TTL_DAYS still set to a sweeper-test value like 0.001. That's ~1.44 minutes; the sweeper unpins the file very fast. Check the live config:
    curl -s https://ipfs.your-domain.com/ | grep ttl_days
    If it's not your production value (default 7), edit .env, then docker compose down && docker compose up -d (env changes need recreate per Common Problems #4). Confirmed during v0.1 first-client integration test (2026-05-25) — sweeper-test TTL from earlier session lingered.
  8. Q: Upload returned 2xx with a CID even though my account didn't have enough CNOOBS to pay (so the Hive-Engine sidechain should have rejected the transfer).
    A: Pre-v0.1.3 had a soft balance check: it compared the escrow's current balance to the claimed payment amount. Since the escrow's existing balance always exceeds a single payment, the check passed even when 0 actually landed — and the file got pinned for free. Fixed in v0.1.3: /upload now polls Hive-Engine's getTransactionInfo RPC (at api.hive-engine.com/rpc/blockchain) for an authoritative success/fail signal. If the sidechain rejected the transfer, the upload is now hard-rejected with the actual sidechain error message and the reservation is cancelled. If git pull && docker compose up -d --build doesn't fix, check container logs for sidechain rejected tx ... entries. Confirmed during the same 2026-05-25 session when a user with 0.9 CNOOBS triggered a "successful" upload that was actually unpaid.
  9. Q: I ran git fetch --all && git reset --hard origin/main && docker compose ... --build && docker compose up -d and now the HTTPS site is unreachable (port 443 connection refused; HTTP 80 still works).
    A: git reset --hard clobbered your operator-specific nginx/ipfs-gate.conf (which had the real domain + cert path) back to the placeholder shipped in the repo. Nginx then can't load the cert, fails to bind 443, and the container restart-loops. Always back up nginx config before a hard reset. Working recipe:
    cp nginx/ipfs-gate.conf nginx/bk.ipfs-gate.conf       # one-time backup

git fetch --all && git reset --hard origin/main && \ docker compose down && docker compose build --no-cache && \ cp nginx/bk.ipfs-gate.conf nginx/ipfs-gate.conf && \

docker compose up -dIf you've already lost the file: restore it from nginx/ipfs-gate.conf.example (or the wiki Step 4 instructions) — replace v4call.com/placeholder with your real domain. Same gotcha exists on v4call. Confirmed during 2026-05-25 testing.

  1. Q: I deleted a CID (admin takedown / expiry) but my browser still serves it from cache for hours.
    A: The gateway sets Cache-Control: public, max-age=86400 (1 day) on /ipfs/:cid responses by default. Browsers respect this and skip the network entirely until the cache expires. Use /status/:cid from curl or an incognito window to see the gate's real state. For dev/testing, lower the cache window by setting GATEWAY_CACHE_MAX_AGE=3600 (1h) or even 60 (1min) in .env, then docker compose down && docker compose up -d (env changes need recreate per Common Problems #4). Pin state in the DB is independent of browser cache — the sweeper expires pins on schedule regardless. Documented in v0.1.4.
  2. Q: Uploads work but the gateway URL in the client bubble shows https://ipfs.localhost/ipfs/... instead of my real domain.
    A: You haven't set PUBLIC_GATEWAY_BASE in .env. The server falls back to https://ipfs.localhost when neither PUBLIC_GATEWAY_BASE nor SERVER_DOMAIN is set. Add this to your .env:
    PUBLIC_GATEWAY_BASE=https://ipfs.your-domain.com
    Then docker compose down && docker compose up -d (env changes need recreate per Common Problems #4). v4call's client has a safety net that substitutes the user's chosen gateway URL when the server returns a localhost one, so end-users can still fetch — but new clients won't have that fallback, so fix at the source. Was missing from .env.example pre-v0.1.2; documented in v0.1.2.
  3. Q: Server logs say kubo backend status: unreachable (fetch failed).
    A: Kubo container isn't healthy yet, or the KUBO_API_URL in .env doesn't match the docker-compose service name. Default is http://kubo:5001; only change if you renamed the kubo service.
  4. Q: The cert renewal cron fires but nginx still serves the old cert.
    A: Certbot writes new files but nginx doesn't auto-reload. The cron line in Step 8 includes docker compose restart nginx after renew — make sure you copied that part too.
  5. Q: Let's Encrypt certbot fails with "DNS problem" or "No A/AAAA record for host".
    A: Your domain's A record isn't pointing at this VPS, or DNS hasn't propagated yet. Verify with dig ipfs.your-domain.com from any machine. If it returns the wrong IP, fix at your DNS provider and wait 5-10 minutes for propagation, then retry the certbot command from Step 5.
  6. Q: I set WHITELIST_MODE=true but reservations for my own Hive account (the operator) get rejected with "this server is invite-only".
    A: Turning on whitelist mode does not automatically whitelist the box owner or the IPFS_GATE_HIVE_ACCOUNT escrow account — it's a plain per-account gate with no built-in owner exception. Invite your own personal Hive account the same way you'd invite anyone else (see Whitelist mode above): POST /admin/whitelist/add with your own username.
  7. Q: I flipped WHITELIST_MODE=true in .env but curl https://.../ still shows "whitelist_mode":false.
    A: Same env-reload gotcha as everywhere else in this file (Common Problems #4) — docker compose restart does not re-read .env. Run docker compose down && docker compose up -d to recreate the container with the new value.
  8. Q: I raised MAX_FILE_SIZE_MB in .env (and recreated the containers — curl https://.../ shows the new max_size_mb), small files upload fine, but a big file fails with upload failed: 413after the payment was already taken.
    A: There are TWO size caps and they must move together. .env's MAX_FILE_SIZE_MB is only the app's cap; nginx has its own client_max_body_size (shipped as 12M to match the old 10MB default) in nginx/ipfs-gate.conf. Anything bigger is 413-rejected by nginx before the app ever sees the request — so the app can't refuse it pre-payment, and the payment isn't even recorded in the gate's DB (it won't show in /admin/orphan-payments; the tokens just sit in escrow — refund manually via Keychain if it wasn't your own account). Fix: edit client_max_body_size in nginx/ipfs-gate.conf to the app cap + slop (e.g. MAX_FILE_SIZE_MB=1000client_max_body_size 1024M;), then reload nginx:
    docker exec ipfs-gate-nginx nginx -t && docker exec ipfs-gate-nginx nginx -s reload
    (A conf edit only needs a reload, not a container recreate — the file is bind-mounted.) Since the fix that shipped alongside this wiki entry, /reserve also hard-rejects an over-app-cap size_bytes before handing out payment instructions, so the app-cap side can no longer eat a payment — but the nginx cap is invisible to the app, so keeping the two in sync is on the operator. Also mind RAM: uploads are buffered in memory, so a 1000MB cap means ~1GB RAM per concurrent large upload — a 2GB VPS will OOM on two at once. Confirmed on ipfs.v4call.com 2026-07-05 (125MB file, app cap 1000MB, nginx still 12M).
  9. Q: The upload page shows ⚠ Can't read this gate's upload limit — https://... isn't answering. Check the Gateway URL at the top, then retry. (often right after a server reboot).
    A: The client refuses to upload until it has fetched the gate's real size cap from GET / (JSON), and that fetch just failed. Work through these in order:
    • Did the server just reboot? The stack takes ~30–60s to come back: Kubo does its startup dance first and the healthcheck gates ipfs-gate on it (Common Problems #1). Wait a minute and press the upload button again — ensureGateCap() re-fetches on every attempt, no page reload needed.
    • Is the stack actually up? ssh in and run docker compose ps in the repo dir — all three containers (ipfs-gate, ipfs-gate-kubo, ipfs-gate-nginx) should be Up. If not: docker compose up -d and check docker compose logs ipfs-gate | tail -20.
    • Does the info endpoint answer? From any machine:
      curl -s -H "Accept: application/json" https://ipfs.your-domain.com/
      If you get the JSON (with max_size_mb), the server is fine — the problem is client-side: check the Gateway URL box for a typo, stray whitespace, http:// instead of https://, or a different domain than you think you're testing.
    • Bonus gotcha — a reboot does NOT apply .env edits. If you edited .env (e.g. changed MAX_FILE_SIZE_MB) and then rebooted the VPS expecting the change to land: it didn't. A reboot (like docker compose restart) brings back the same container with the old env baked in (Common Problems #4). Check what's really live with the curl above — if max_size_mb is the old value, run docker compose up -d --force-recreate ipfs-gate. Confirmed on ipfs.v4call.com 2026-07-12: .env had been set back to MAX_FILE_SIZE_MB=10 and the VPS rebooted for OS updates, but the gate still advertised 1000 until the container was recreated (the "isn't answering" message itself was the ~1-minute post-reboot startup window).
  10. Q: I want to raise MAX_FILE_SIZE_MB to something big (500MB, 1GB). Small files work after the change, but files over ~100MB die with ⚠ upload failed (Failed to fetch)after payment. Is big-file hosting supported?
    A: Not reliably yet — this is a known OPEN bug, and it is NOT the nginx 413 problem from the entry above. Read this before raising the cap:
    • Two caps must move together (see the 413 entry above): .env's MAX_FILE_SIZE_MB and client_max_body_size in nginx/ipfs-gate.conf. If nginx is the blocker you get a clean 413. If both caps are raised and the transfer still dies mid-flight with Failed to fetch (connection dropped, no HTTP status), you've hit the open bug.
    • The open bug: confirmed on ipfs.v4call.com 2026-07-06 — 125MB public upload failed after payment; 30MB fine; the same big files upload fine to Pinata from the same browser, so the client side is not the problem. Full investigation state lives in PICKUP-NOTES-2026-07-06.md §2: ranked suspects are (1) node container OOM-kill — most likely, see next bullet, (2) nginx proxy_read_timeout 120s while the app does Hive verify + sidechain confirm + Kubo add after the body lands, (3) KUBO_TIMEOUT_MS (default 60s) aborting the Kubo add. Each leaves a different fingerprint — repro while running docker stats and tailing both containers' logs; check docker inspect ipfs-gate --format 'Template:.State.OOMKilled Template:.RestartCount' and dmesg -T | grep -i oom.
    • Mind your RAM regardless: /upload buffers the whole file in memory (multer memoryStorage), and the Kubo backend copies it again into a Blob — so one upload costs ≥2× the file size in node heap. On a 2GB VPS (which also runs Kubo + nginx), a 1GB cap means a single large upload can OOM the container, and two concurrent ~500MB uploads will. The real fix is server-side streaming uploads (multer diskStorage/busboy → stream into Kubo) — a v0.2 backlog item, design thinking in PICKUP-NOTES-2026-07-06.md §3.
    • Practical operator advice until streaming ships: keep MAX_FILE_SIZE_MB at a value your RAM can eat twice, concurrently — on a 2GB box that means staying in the low-hundreds of MB at most, and the well-tested territory is ≤100MB. Because the failure happens after payment, every crash near the cap is also a manual-refund chore (the tokens sit in escrow; nothing shows in /admin/orphan-payments).
  11. Q: The upload page shows ⚠ Can't read this gate's upload limit — https://… isn't answering. Check the Gateway URL at the top, then retry., but curl -s -H "Accept: application/json" https://your-domain/ returns the JSON just fine, the containers are all Up, and it's not right after a reboot. Uploads (public and private, any logged-in account) all hit the same wall.
    A: Browser HTTP-cache poisoning from content negotiation on /. The gate serves TWO things at the same URL / — the landing-page HTML to a browser navigation (Accept: text/html) and the info JSON to fetch("/") (Accept: application/json), chosen by the map $http_accept block. When the shipped nginx responses carry no Vary: Accept (and no Cache-Control), the browser caches the HTML it got from navigating to /, then hands that same cached HTML back to the page's own fetch("/") that reads the size cap — res.json() throws, gateCapKnown stays false, and every upload button reports the gate "isn't answering." curl never navigates first, so it never caches the HTML — which is exactly why curl says the server is fine while the browser is stuck. Diagnose it in the browser: open DevTools → Console (you'll see [ipfs-gate] gate info fetch failed for …/ — …) and Network (the / request that should be JSON shows Content-Type: text/html, often (from disk cache)). Fingerprint at the CLI — the poisonable response is the one with no Vary and no Cache-Control:
    curl -sI -H 'Accept: application/json' https://your-domain/ | grep -iE 'content-type|vary|cache-control'
    Fixed 2026-07-16 (found on a fresh apex-domain deploy) two ways, both shipped: (1) nginxlocation = / and location = /index.html now send add_header Vary Accept always; + add_header Cache-Control "no-cache" always; so the cache keys on Accept and revalidates; (2) clientrefreshGateInfo() fetches the info with cache: 'no-store' and rejects a non-JSON Content-Type, so the page is robust even against a gate whose nginx predates the fix. If you're on an older checkout, git pull, re-copy the two add_header lines into your live nginx/ipfs-gate.conf (they live in the server { listen 443 … } block), then apply them — but note that if you edited the conf after a git operation you must recreate the nginx container, not just reload it (see the next entry). Already stuck in a browser? One hard refresh (Ctrl/Cmd-Shift-R) clears the poisoned entry; after the fix it self-heals via revalidation.
  12. Q: I edited nginx/ipfs-gate.conf, ran nginx -t (passed) and nginx -s reload, but the change never takes effect — docker exec ipfs-gate-nginx nginx -T doesn't show my new lines, and docker exec ipfs-gate-nginx grep … /etc/nginx/conf.d/default.conf shows the old content even though the host file is clearly edited.
    A: Single-file bind-mount inode trap. docker-compose.yml mounts the conf as a single file (./nginx/ipfs-gate.conf:/etc/nginx/conf.d/default.conf:ro), and Docker binds that mount to the file's inode at container-start time. Editing the file in place (nano, sed -i) keeps the inode, so the container sees it and a reload works. But anything that replaces the file with a new inode — git checkout, git pull, git reset --hard, some editors' atomic-save, mv — leaves the container still bound to the original inode; every later edit lands on the new inode the container can't see, and reload just re-reads the stale one. (The ./public:/var/www/ipfs-gate mount is a directory mount, so file changes there do show up — which is why the landing page updates but the conf doesn't.) Fix: recreate the container so the mount re-binds to the current inode:
    docker compose up -d --force-recreate nginx

docker exec ipfs-gate-nginx nginx -T 2>/dev/null | grep -c add_header # confirm it's loaded nowAvoid it entirely: the standard update recipe (docker compose down && … && up -d) recreates everything, so it never hits this — the trap only bites a hand-edit-then-reload after a git operation. Confirmed 2026-07-16 while hot-patching the Vary: Accept headers above: host file had the lines, container served the pre-git-pull inode, and only --force-recreate picked them up.

Resources