Writing a Binary Artifact Repository: The Stupid, Simple Way
tl;dr
For my in-house dev projects, I want to be able to upload and download artifacts with a simple command-line call.
So I wrote a dumb one that seems to work : https://gitlab.com/jtrowe/app-artifact-repository .
I really tried to find an open-source alternative. Artifactory Open-Source is nice and I’ve used that for Maven; from what I recall it does not support generic artifacts.
artipie supports generic artifacts, but there does not seem to be any indexing.
That might be it for something open-source that I can self-host.
Artifactory Open-Source might support generic artifacts.
I spent 5 minutes looking at the install scripts and it’s complex and it wants the install script to run as root. Perhaps I’ll come back to this.
Use Case
- needs to handle generic artifacts
- upload/download via HTTP ( So I can put 1 instance on a server )
- only need to be able to fetch by:
- artifact name
- artifact name, version
- artifact name, tag
- artifact name, version, tag
- deployed via Docker
- this is for my in-house dev projects only, so i can ignore:
- authenication
- authorization
- throughput
- monitoring
- high-availability
- user interface
Design
I used the Mojolicious Perl framwork because it’s quick and easy and I want to use it again. I used it at a job a few years ago and it seemed like an easy way to get things done.
Getting Started
1. Start The Server
The server will start and you’ll see it’s log output. Enter CTRL-c when you want it to stop. This will be listening on http://localhost:3000/.
$ bin/server daemon
Name "DateTime::TO_JSON" used only once: possible typo at bin/server line 30.
[2022-06-07 16:03:34.67054] [3422] [debug] Creating storage directory "/home/jrowe/projects/App-Artifact-Repository/storage".
[2022-06-07 16:03:34.67095] [3422] [debug] storage_index:
{}
[2022-06-07 16:03:34.67174] [3422] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000
2. Locate An Artifact To Upload
3. Generate A Metadata File
This will create a file name $file.meta.yml in the same directory as $file.
$ bin/build-package-meta-file --name artifact_name --version any_version_string --tag latest --tag my_special_tag $file
2022/06/07 16:09:20 meta:
---
date: 1998-11-11T05:00:00+0000
file: example-1.0.tar.gz
meta_version: 1
name: example
tags:
- latest
- my_special_tag
version: 1.0
4. Upload The Files
It will upload the metadata file and the original file.
$ bin/upload $file.meta.yml`
file => example-1.0.tar.gz
RESPONSE:
201
Content-Length: 211
Content-Type: application/json;charset=UTF-8
Date: Tue, 07 Jun 2022 20:14:02 GMT
Server: Mojolicious (Perl)
X-Application-Name: App-Artifact-Repository
X-Application-Version: 0.1.0
{"file":{"filename":"example-1.0.tar.gz","size":217},"meta":{"date":"1998-11-11T05:00:00+0000","file":"example-1.0.tar.gz","meta_version":"1","name":"example","tags":["latest","my_special_tag"],"version":"1.0"}}
5. See The Uploaded Files
The files have now been uploaded to the ‘storage’ directory.
$ find storage -ls
38232428 4 drwxrwxr-x 3 jrowe jrowe 4096 Jun 7 16:14 storage/
38150772 4 -rw-rw-r-- 1 jrowe jrowe 407 Jun 7 16:14 storage/index.json
38150857 4 -rw-rw-r-- 1 jrowe jrowe 287 Jun 7 16:14 storage/meta.json
38232430 4 drwxrwxr-x 3 jrowe jrowe 4096 Jun 7 16:14 storage/example
38232462 4 drwxrwxr-x 2 jrowe jrowe 4096 Jun 7 16:14 storage/example/1.0
38150770 4 -rw-rw-r-- 1 jrowe jrowe 139 Jun 7 16:14 storage/example/1.0/example-1.0.tar.gz.meta.yml
38150726 4 -rw-rw-r-- 1 jrowe jrowe 217 Jun 7 16:14 storage/example/1.0/example-1.0.tar.gz