Drop Sendmail Support

[arvind4501]

No description provided.

[ekohl]

It may not matter too much, but should this be up with an empty down? Not that we do rollbacks often, but also wondering out of principle how that works.

[arvind4501]

i can do that, also looking at what rails has to say about it. https://guides.rubyonrails.org/active_record_migrations.html#throwing-an-error-to-prevent-reverts mentions that

Sometimes your migration will do something which is just plain irreversible; for example, it might destroy some data.

In such cases, you can raise ActiveRecord::IrreversibleMigration in your down block.

reading this i am leaning to add ActiveRecord::IrreversibleMigration in down?

[ekohl]

In this case I'd argue it is reversible: Foreman will find a new default and work, but you're right it's a bit of an edge case.

[adamruzicka]

It is true that it is irreversible in the true that we remove data and we have no way of getting it back. At the same time, if we have a blank down, it would allow people to "rollback" the migration (which would be a no-op), checkout older version and go on about their day. If we raise, this wouldn't be impossible, but it would become much more involved.

[arvind4501]

Sorry, but i did not got Foreman will find a new default and work this part. I usually see that any data deletion related migrations(be it via active record or raw sql) is something that should not be reverted(there are very less chances it need to be reversible)

[arvind4501]

Ok , so i see that having empty down makes it reversible but its no-op. but then also its a false posiitive, but i would not might having a empty down?

[ekohl]

In Satellite we saw that the majority of users has notifications enabled, but only a much smaller number has modified their settings to use SMTP. I suspect (but we have no telemetry so it's a guess) the same holds true for Foreman. Before we merge this we should think about a smoother migration.

[arvind4501]

Thinking out loud:-
If we are droping sendmail(which means /usr/sbin/sendmail binaries and probabily postfix MTA) completely, then for migration i don't think we can use any data related sendmail settings and seed it in smtp by default? and that then leaves user with no preconfigured smtp settings.
I am wondering is the containerization move making us drop sendmail completely or is there other reasons for that?

[ekohl]

@adamruzicka have you thought about the migration?

[adamruzicka]

There isn't all that much we can do, is there? We can't really take data from an unknown external system and configure it on our end.

is there other reasons for that?

It allows us to simplify the codebase and avoid some issues from ever happening. New deployments will arguably end up in better shape (either explicitly configured on unconfigured), but they won't be in the in-between state they do now (sendmail configured, but postfix not enabled so "test email" succeeds but the emails never really go out)

[ekohl]

There isn't all that much we can do, is there? We can't really take data from an unknown external system and configure it on our end.

If today the sendmail binary is used, is using localhost with port 25 without authentication a reasonable thing to assume? Perhaps the typical scenario to verify: install postfix on EL9 and see if that works.

[adamruzicka]

If today the sendmail binary is used, is using localhost with port 25 without authentication a reasonable thing to assume? Perhaps the typical scenario to verify: install postfix on EL9 and see if that works.

That seems to work (as in, mail gets accepted) with default settings both with postfix on EL9 and with exim4 on debian, but due to the best-effort nature of such a migration, I'd still be worried about two things:

1. Mail being accepted but going nowhere
2. Us setting up wrong configuration rather than leaving things unconfigured when we just don't know

Thinking a bit further ahead, we'll also have to figure out what to do with smtp server being set to localhost:25 when migrating from a "standard" deployment to a container based one.

[arvind4501]

Ok so based on the discussion above, here are my observation:-

Today with sendmail(default mail delivery method), the mails are dependent on local postfix MTA on EL9 to deliver mail, with postfix being stopped/inactive all mails stay in the queue(postqueue -p lists all queued mails) and when we start postfix then only those mail gets delivered

I have tested the SMTP with localhost:25 on a broker satellite instance and here is how it behaved:-

When postfix is not running:

• Immediate error: 'Unable to send email' error (Errno::ECONNREFUSED): Connection refused - connect(2) for localhost port 25
• Email is never getting into any queue so failed emails are lost
• Admin knows right away that email is broken

When postfix is running:

• Email gets delivered successfully.

So i think

Mail being accepted but going nowhere

This is exactly what happens with sendmail today when postfix is not running.

Us setting up wrong configuration rather than leaving things unconfigured when we just don't know

That's a fair concern but at the same time leaving it unconfigured leaves user enviorments broken for mail feature.

[ekohl]

That seems to work (as in, mail gets accepted) with default settings both with postfix on EL9 and with exim4 on debian, but due to the best-effort nature of such a migration, I'd still be worried about two things:

1. Mail being accepted but going nowhere

2. Us setting up wrong configuration rather than leaving things unconfigured when we just don't know

IMHO a release note should cover that.

Thinking a bit further ahead, we'll also have to figure out what to do with smtp server being set to localhost:25 when migrating from a "standard" deployment to a container based one.

In the beginning we'll deploy with host networking where I'd expect that to work. theforeman/foremanctl#403 may change that, so it's good to keep an eye out.

[arvind4501]

@ekohl @adamruzicka , should we skip default to localhost:25 for smtp?

[adamruzicka]

should we skip default to localhost:25 for smtp?

I know I had some objections against that, but seeing we don't really have any alternatives, I wouldn't dismiss it.

Would we make it a new global default or just a default for those who'd be migrated away from smtp?

[arvind4501]

Would we make it a new global default or just a default for those who'd be migrated away from smtp?

You mean migrated away from sendmail? i think it makes sense to only set smtp_address to localhost in case of users migrating from sendmail, because others may have configured diffrent smtp servers if they are already using smtp, so it does not make sense to override their existing smtp configuration

[adamruzicka]

You mean migrated away from sendmail?

Eh, yes, sorry.

it does not make sense to override their existing smtp configuration

I didn't mean to override existing smtp configuration. What I meant was changing the default value for smtp_address in the settings registry here, which would cover both users which had previously using sendmail as well as users who never touched any of those settings, leaving intact configuration of users who were already using smtp.

[arvind4501]

Ah, sorry, i took it in context of updating existing value of smtp_address, yeah changing the default value for smtp_address in the settings registry here, make perfect sense to me.