Contents

Writing a Binary Artifact Repository: The Stupid, Simple Way

tl;dr

For my in-house dev projects, I want to be able to upload and download artifacts with a simple command-line call.

So I wrote a dumb one that seems to work : https://gitlab.com/jtrowe/app-artifact-repository .

Why Not...

I really tried to find an open-source alternative. Artifactory Open-Source is nice and I’ve used that for Maven; from what I recall it does not support generic artifacts.

artipie supports generic artifacts, but there does not seem to be any indexing.

That might be it for something open-source that I can self-host.

A More Recent Look at Artifactory

Artifactory Open-Source might support generic artifacts.

I spent 5 minutes looking at the install scripts and it’s complex and it wants the install script to run as root. Perhaps I’ll come back to this.

Use Case

  • needs to handle generic artifacts
  • upload/download via HTTP ( So I can put 1 instance on a server )
  • only need to be able to fetch by:
    • artifact name
    • artifact name, version
    • artifact name, tag
    • artifact name, version, tag
  • deployed via Docker
  • this is for my in-house dev projects only, so i can ignore:
    • authenication
    • authorization
    • throughput
    • monitoring
    • high-availability
    • user interface

Design

I used the Mojolicious Perl framwork because it’s quick and easy and I want to use it again. I used it at a job a few years ago and it seemed like an easy way to get things done.

Getting Started

1. Start The Server

The server will start and you’ll see it’s log output. Enter CTRL-c when you want it to stop. This will be listening on http://localhost:3000/.

$ bin/server daemon
Name "DateTime::TO_JSON" used only once: possible typo at bin/server line 30.
[2022-06-07 16:03:34.67054] [3422] [debug] Creating storage directory "/home/jrowe/projects/App-Artifact-Repository/storage".
[2022-06-07 16:03:34.67095] [3422] [debug] storage_index:
{}
[2022-06-07 16:03:34.67174] [3422] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000

2. Locate An Artifact To Upload

3. Generate A Metadata File

This will create a file name $file.meta.yml in the same directory as $file.

$ bin/build-package-meta-file --name artifact_name --version any_version_string --tag latest --tag my_special_tag $file
2022/06/07 16:09:20 meta:
---
date: 1998-11-11T05:00:00+0000
file: example-1.0.tar.gz
meta_version: 1
name: example
tags:
  - latest
  - my_special_tag
version: 1.0

4. Upload The Files

It will upload the metadata file and the original file.

$ bin/upload $file.meta.yml`
file => example-1.0.tar.gz
RESPONSE:
201
Content-Length: 211
Content-Type: application/json;charset=UTF-8
Date: Tue, 07 Jun 2022 20:14:02 GMT
Server: Mojolicious (Perl)
X-Application-Name: App-Artifact-Repository
X-Application-Version: 0.1.0

{"file":{"filename":"example-1.0.tar.gz","size":217},"meta":{"date":"1998-11-11T05:00:00+0000","file":"example-1.0.tar.gz","meta_version":"1","name":"example","tags":["latest","my_special_tag"],"version":"1.0"}}

5. See The Uploaded Files

The files have now been uploaded to the ‘storage’ directory.

$ find storage -ls
 38232428      4 drwxrwxr-x   3 jrowe    jrowe        4096 Jun  7 16:14 storage/
 38150772      4 -rw-rw-r--   1 jrowe    jrowe         407 Jun  7 16:14 storage/index.json
 38150857      4 -rw-rw-r--   1 jrowe    jrowe         287 Jun  7 16:14 storage/meta.json
 38232430      4 drwxrwxr-x   3 jrowe    jrowe        4096 Jun  7 16:14 storage/example
 38232462      4 drwxrwxr-x   2 jrowe    jrowe        4096 Jun  7 16:14 storage/example/1.0
 38150770      4 -rw-rw-r--   1 jrowe    jrowe         139 Jun  7 16:14 storage/example/1.0/example-1.0.tar.gz.meta.yml
 38150726      4 -rw-rw-r--   1 jrowe    jrowe         217 Jun  7 16:14 storage/example/1.0/example-1.0.tar.gz