Switch to using nightly images

[ehelms]

The mainline development branch for the installer should use the most up to date images as we iterate. This would change to use the nightly image tags going foward.

[ehelms]

This is included in #280 as foreman_rh_cloud is needed

[pablomh]

Yes, please :)

[ehelms]

@evgeni @stejskalleos @arvind4501 -- #391 presents a solid reason to switch to using nightly images where the code base has an architectural deviation from release-based images. As more developers start contributing here, I think we need to be able to pull in the nightly code bases.

[arvind4501]

I think having nightly pinned is good for development.

Thinking out loud, at some point we need to think how can we ship foremanctl with stable images(3.18, 3.19) or maybe align foremanctl releases with foreman, but not necessarily today.

[evgeni]

Please keep in mind that this change means:

• We are at risk of becoming the CI for nightly, when changes to the nightly container tags are pushed out without proper testing of the containers (which could totally be implemented by running foremanctl pointing at some staged containers, but that's not how things are done today)
• We need to create stable branches for setting the image stream to a stable Foreman release/branch
• The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

That shouldn't mean I am against that change, quite the opposite. But it's more than one line :/

[ehelms]

Please keep in mind that this change means:

• We are at risk of becoming the CI for nightly, when changes to the nightly container tags are pushed out without proper testing of the containers (which could totally be implemented by running foremanctl pointing at some staged containers, but that's not how things are done today)

Arguably, foremanctl and it's test should be the CI. This would just mean right now that PRs to this repository would be that CI rather than PRs to the image repositories themselves. I agree we should close that gap. I suppose this is an order of operations question.

• We need to create stable branches for setting the image stream to a stable Foreman release/branch

Do I translate this correctly to "foremanctl needs to be in the branching procedure for Foreman releases" ?

• The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

This translates to, we gotta make this nifty version thing nightly aware?

That shouldn't mean I am against that change, quite the opposite. But it's more than one line :/

[evgeni]

Please keep in mind that this change means:

• We are at risk of becoming the CI for nightly, when changes to the nightly container tags are pushed out without proper testing of the containers (which could totally be implemented by running foremanctl pointing at some staged containers, but that's not how things are done today)

Arguably, foremanctl and it's test should be the CI. This would just mean right now that PRs to this repository would be that CI rather than PRs to the image repositories themselves. I agree we should close that gap. I suppose this is an order of operations question.

The code should be used in CI, yes.
What I want to avoid is that a PR is opened against foremanctl and that PR is red because a container image changed.

• We need to create stable branches for setting the image stream to a stable Foreman release/branch

Do I translate this correctly to "foremanctl needs to be in the branching procedure for Foreman releases" ?

Correct

• The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

This translates to, we gotta make this nifty version thing nightly aware?

Yes, albeit I am not sure this is actually possible.

[ehelms]

The code should be used in CI, yes.
What I want to avoid is that a PR is opened against foremanctl and that PR is red because a container image changed.

I have an initial attempt at that: theforeman/foreman-oci-images#27
This is failing at the moment because it's a chicken and egg problem.

Nightly has changed behavior, so a test is failing. However, that change can't be made here until the repository relies on the nightly images. However, we can't build the nightly images because the tests fail....

[ehelms]

• The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

This translates to, we gotta make this nifty version thing nightly aware?

Yes, albeit I am not sure this is actually possible.

I am not following how this is affected by using images built based on the "nightly" code bases.

[evgeni]

• The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

This translates to, we gotta make this nifty version thing nightly aware?

Yes, albeit I am not sure this is actually possible.

I am not following how this is affected by using images built based on the "nightly" code bases.

Not the images, but the fact we use stable branches. Probably easiest is to go back to a normal VERSION file.

[evgeni]

The code should be used in CI, yes.
What I want to avoid is that a PR is opened against foremanctl and that PR is red because a container image changed.

I have an initial attempt at that: theforeman/foreman-oci-images#27 This is failing at the moment because it's a chicken and egg problem.

Nightly has changed behavior, so a test is failing. However, that change can't be made here until the repository relies on the nightly images. However, we can't build the nightly images because the tests fail....

In forklift, we worked around this by annotating the tests with data which component versions they apply to. Should we do something similar here? Or just drop the candlepin_events tests knowing it's on the way out anyway

[ehelms]

In forklift, we worked around this by annotating the tests with data which component versions they apply to. Should we do something similar here? Or just drop the candlepin_events tests knowing it's on the way out anyway

I think you are asking a big question here: do we want foremanctl main branch to track Foreman develop or do we want foremanctl main branch to track support across versions?

For tests, I feel like they should be smarter and only execute if the feature is present. This is what I tried to do with iop.

[evgeni]

Yes and no. "Run IOP tests only if IOP feature is enabled" makes sense, but what if the set of tests for said feature vary by version?

[ehelms]

Yes and no. "Run IOP tests only if IOP feature is enabled" makes sense, but what if the set of tests for said feature vary by version?

Good question. I think that goes back to my other question: do we want foremanctl to be multi-version or for it to track the version of Foreman via branching structure?

[ehelms]

This translates to, we gotta make this nifty version thing nightly aware?

Yes, albeit I am not sure this is actually possible.

If I run make, right now I get foremanctl-1.2.0-44-g7aeaf71.tar.gz which seems reasonable as it has the git commit in it and the last tagged version. I don't think we need nightly or such in the name, but this does always use the base of the last release.

I get that means we cannot pre-emptively update the version to indicate what the next target version is. However, if we always work from the viewpoint that if there is a git hash it's saying it's version X + some amount of nightly git bits. I think that works pretty good for a mental model.

[evgeni]

Yes and no. "Run IOP tests only if IOP feature is enabled" makes sense, but what if the set of tests for said feature vary by version?

Good question. I think that goes back to my other question: do we want foremanctl to be multi-version or for it to track the version of Foreman via branching structure?

I think we don't need it to be multi version if we branch accordingly.

This translates to, we gotta make this nifty version thing nightly aware?

Yes, albeit I am not sure this is actually possible.

If I run make, right now I get foremanctl-1.2.0-44-g7aeaf71.tar.gz which seems reasonable as it has the git commit in it and the last tagged version. I don't think we need nightly or such in the name, but this does always use the base of the last release.

It should be 2.0.0-… - you probably are missing a rebase, but that's not important for the argument.

I get that means we cannot pre-emptively update the version to indicate what the next target version is. However, if we always work from the viewpoint that if there is a git hash it's saying it's version X + some amount of nightly git bits. I think that works pretty good for a mental model.

And exactly this won't work (automatically) once we move to stable branches as git describe uses the newest tag it can find in the ancestry, but as the new release tags are on stable branches, it will pick up something old. Try it out in foreman.git or foreman-installer.git to see what I mean.

[ehelms]

Thinking and playing around with this more I see two strategies:

Direct Version Management

This would be reverting to the classical approach of keeping the version number in VERSION file. The version in master would track the "next" version and branches would contain and be updated with the current version for said branch.

Keeping export-subst

Keep the current method of using export-subst and implement the following:

• In master, set VERSION set it to %H$ so that the VERSION file has the commit hash of the point in time the tarball was created
• In branches, set VERSION to %(describe)$ so that VERSION file has the tag commit of tarball

[evgeni]

Given that both strategies involve touching VERSION, plus %H$ not sorting correctly when used as an RPM version (so we'd have to use NEXT_VESION-%H or similar to achieve that), I'd vote for "Direct Version Management"

[ekohl]

The above breaks git describe for master as the tags will be only inside the stable branches (you can see that effect in theforeman/foreman today) and thus breaks the nifty VERSION handling Ewoud built and Packit RPM versioning

For Foreman I already thought about this and we have the same problem. There we have commits like theforeman/foreman@ba8f268 and I'd suggest to re-implement those as git tags when using export-subst.

Big question: what is that develop version? Do we want to start tying the foremanctl version numbers to Foreman or stick to foremanctl's own versions? It would be easier for users to see what Foreman version they will get when they install it.

[ehelms]

For Foreman I already thought about this and we have the same problem. There we have commits like theforeman/foreman@ba8f268 and I'd suggest to re-implement those as git tags when using export-subst.

Seeing if I follow. You are proposing to tag the repo when we "bump" the version to a development version. So we might have:

1. master tagged with 3.18-develop
2. branch to 3.18-stable
3. tag master 3.19-develop
4. 3.18-stable branch gets a 3.18.0 tag on release, etc.

[ekohl]

Seeing if I follow. You are proposing to tag the repo when we "bump" the version to a development version. So we might have:

Kind of yes. Perhaps this helps

---
config:
    gitGraph:
        mainBranchName: develop
---
gitGraph
    commit tag: "3.18-develop"
    commit
    commit
    branch 3.18-stable
    checkout 3.18-stable
    commit
    commit tag: "3.18.0-rc1"
    checkout develop
    commit tag: "3.19-develop"
    commit

Loading

So at the end I'd expect git describe to show 3.19-develop-1-6f07a2d because it's 1 commit after 3.19-develop with git sha 6f07a2d.

[ehelms]

Updated with a release process. I'd like to propose the order of operations:

1. Merge this change here with release documentation
2. Follow up with branch / release updates to https://github.com/theforeman/theforeman-rel-eng

[ekohl]

That plan makes sense to me.

[ekohl]

Perhaps suggest a command?

sed -i '/container_tag_stream/ s/nightly/foreman-3.18/' src/vars/images.yml