Package Meta Processing

Monteur has a special metadata processing function for keeping its CI Job sane and maintainable. The inner mechanics are explained here!

The Problem

It was all started with a very messy mathematical problem. Say a build variant linux-amd64 is required to be packaged into 3 different distribution channels:

That's a total of 1 x 3 = 3 SAME CI job recipe files.

Now, try to deploy a full list of build variants for the same list of channels above, let's say:
  1. linux-arm64

  2. linux-arm

  3. darwin-amd64

  4. darwin-arm64

  5. windows-amd64

  6. windows-arm64

That would be 6 x 3 = 18 SAME CI job recipe files! If we scale things in a mathematical way, that's a total of m x n duplicated CI job recipe files for a project supporting m number of build variants for n of distribution channels.

That is A LOT of duplications which is extra ordinary scary and unmaintainable! This means that we need a solution to scale the CI Job not just horizontally across each different distribution channel but also vertically across each build variant.

The Solution

To solve the problem above, Monteur uses this Package Meta Processing approach to facilitate the vertically scaling for any amount of build variants. The idea is to:

  1. Keep the CI Job recipe specific to the distribution channel (means scaling horizontally).

  2. Each recipe lists out all the build variant and package them repeatedly based on the recipe (means scaling vertically).

By doing so, we kept all vertically scaling vector into 1 single CI Job recipe while letting the recipe creator to focus on the distribution channel's requirements.

Current Deployment

Currently, this function is deployed in the following CI Jobs:

  1. Package since Monteur version v0.0.1.

  2. Release since Monteur version v0.0.1.

  3. Prepare since Monteur version v0.0.2.

Built-In Packagers

Monteur offers a list of built-in packagers to provider cross-platform supports as far as possible for applicable CI Job like Package CI Job. To select a packager in the recipe, simply set Metadata.Type as shown below:

Type = 'deb-manual'

Currently, these are the available packagers:

Data Structure

To operate this function, Monteur uses a single unified data structure. Please keep in mind that not all fields are used in a CI Job recipe. You should consult the CI Job's documentation for specific fields' availability and purpose.

The full data structure is shown below:

OS = [ 'linux' ]
Arch = [ 'amd64' ]
Name = '{{- .PkgName -}}-{{- .PkgVersion -}}-{{- .PkgOS -}}-{{- .PkgArch -}}'
Distribution = [
Changelog = '{{- .DataDir -}}/debian/changelog-{{- .PkgArch -}}'
BuildSource = false
Source = '{{- .PackageDir -}}/targz/{{- .PkgName -}}-{{- .PkgVersion -}}-{{- .PkgOS -}}-{{- .PkgArch -}}.tar.gz'
Target = '{{- .ReleaseDir -}}/archives'

'{{- .PackageDir -}}/monteur' = '{{- .BuildDir -}}/{{- .PkgOS -}}-{{- .PkgArch -}}'
'{{- .PackageDir -}}/License.pdf' = '{{- .LicensePath -}}'


That's all for Monteur's Package Meta Processing function. If you have any question, please feel free to raise your question at our Issues Section.