Repository Structure and Metadata

Updated

October 30, 2026

Repository Structure and Metadata

A package manager cannot download thousands of individual packages just to discover what software exists or resolve dependencies. Instead, it queries a centralized index file called repository metadata.

+-----------------------------------------------------------+
|               HTTP / HTTPS Web Server (Nginx)             |
|                                                           |
|  /repo/x86_64/                                            |
|  ├── repodata.json (or Packages.gz / core.db.tar.gz)      |
|  ├── repodata.json.sig (GPG / minisign detached signature)|
|  ├── curl-8.5.0-1-x86_64.pkg                              |
|  ├── openssl-3.2.1-1-x86_64.pkg                           |
|  └── zlib-1.3.1-1-x86_64.pkg                              |
+-----------------------------------------------------------+
                              |
                     HTTPS GET repodata.json
                              |
                              v
+-----------------------------------------------------------+
|                   Client Package Manager                  |
|                                                           |
| 1. Verifies GPG signature against trusted distribution key|
| 2. Parses local cache of repository metadata              |
| 3. Computes dependency graph & presents transaction plan  |
| 4. Downloads only required package binaries               |
+-----------------------------------------------------------+

Repository Metadata Schemas

1. Debian (Packages.gz and Release)

In a Debian repository, the Packages file contains the concatenated control stanzas of every package in the pool, along with SHA256 checksums, package file sizes, and filesystem paths. The Release file contains the cryptographic hashes of the Packages files and is signed with the distribution maintainer’s private GPG key (Release.gpg or in-line InRelease).

2. Arch Linux (core.db.tar.gz)

In Arch repositories, core.db.tar.gz is a tarball containing one directory per package. Inside each directory are two files: desc (package description, version, size, dependencies) and depends (exact dependency version requirements). When pacman -Sy runs, it simply downloads this small archive and uncompresses it into /var/lib/pacman/sync/core/.

3. Modern JSON Metadata (Custom Distro Approach)

For our custom BYOL distribution, we can design a clean, modern metadata format using standard JSON:

{
  "repository": "byol-core",
  "version": 1,
  "updated_at": "2026-10-29T12:00:00Z",
  "packages": {
    "curl": {
      "version": "8.5.0",
      "release": 1,
      "arch": "x86_64",
      "filename": "curl-8.5.0-1-x86_64.byol",
      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
      "size": 492100,
      "depends": ["glibc>=2.38", "openssl>=3.0.0", "zlib>=1.3.0"]
    },
    "openssl": {
      "version": "3.2.1",
      "release": 1,
      "arch": "x86_64",
      "filename": "openssl-3.2.1-1-x86_64.byol",
      "sha256": "4a5e1e...39f1",
      "size": 3410290,
      "depends": ["glibc>=2.38"]
    }
  }
}

Cryptographic Signing of Metadata

Never distribute packages over plain HTTP without cryptographic authentication. If an attacker performs a Man-In-The-Middle (MITM) attack or compromises a mirror server, they could tamper with package payloads.

Signing workflow:

# Generate detached cryptographic signature using minisign or gpg:
$ minisign -Sm repodata.json -s /etc/byol/keys/distro.key

# Client verifies signature before trusting metadata:
$ minisign -Vm repodata.json -p /etc/byol/keys/distro.pub
Signature and comment signature verified