system:annoyances
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| system:annoyances [2021-04-25T18:49:34+0000] – michael_paoli | system:annoyances [2021-05-06T06:05:20+0000] (current) – reverted the temporarily increase of max queue time from 4 days to 7 days michael_paoli | ||
|---|---|---|---|
| Line 385: | Line 385: | ||
| Analyzed logs further, notably for web and email traffic/ | Analyzed logs further, notably for web and email traffic/ | ||
| Checked the mail queue again - after subtracting out target addresses that have already been successfully delivered to, there remain at the moment only 6 unique email addresses presently showing any delivery issues. | Checked the mail queue again - after subtracting out target addresses that have already been successfully delivered to, there remain at the moment only 6 unique email addresses presently showing any delivery issues. | ||
| + | |||
| + | More anti-spam to do ... SPF ... looks like config files can have that enabled ...\\ | ||
| + | < | ||
| + | conf.d/ | ||
| + | # This is quite costly in terms of DNS lookups (~6 lookups per mail). | ||
| + | # enable if that's an issue. | ||
| + | # install " | ||
| + | # Missing spf-tools-perl will trigger the " | ||
| + | # SPF check" warning. | ||
| + | .ifdef CHECK_RCPT_SPF | ||
| + | deny | ||
| + | message = [SPF] $sender_host_address is not allowed to send mail from \ | ||
| + | ${if def: | ||
| + | Please see \ | ||
| + | http:// | ||
| + | |||
| + | $ dpkg -l spf-tools-perl | grep '^ii ' | ||
| + | ii spf-tools-perl 2.9.0-4 | ||
| + | $ nc -z www.openspf.org. 80 | ||
| + | nc: unable to connect to address www.openspf.org., | ||
| + | $ nc -z www.openspf.org. 443 | ||
| + | nc: unable to connect to address www.openspf.org., | ||
| + | $ | ||
| + | |||
| + | So, is spf-tools-perl still applicable, or is it just the diagnostic that's out-of-date referring to a service that's no longer (at least pesently) | ||
| + | reachable? | ||
| + | |||
| + | $ dpkg -L spf-tools-perl | sort | grep -e bin/ -e '/ | ||
| + | / | ||
| + | / | ||
| + | / | ||
| + | / | ||
| + | $ man spfquery | ||
| + | ... | ||
| + | $ spfquery --scope mfrom --identity balug.org --ip-address $(dig +short balug.org. A) | ||
| + | pass | ||
| + | balug.org: 96.86.170.229 is authorized to use ' | ||
| + | balug.org: 96.86.170.229 is authorized to use ' | ||
| + | Received-SPF: | ||
| + | $ echo $? | ||
| + | 0 | ||
| + | $ spfquery --scope mfrom --identity balug.org --ip-address 8.8.8.8; echo $? | ||
| + | neutral | ||
| + | balug.org: Default neutral result due to no mechanism matches | ||
| + | balug.org: Default neutral result due to no mechanism matches | ||
| + | Received-SPF: | ||
| + | 3 | ||
| + | $ | ||
| + | neutral ? - are we missing something that ought say that should fail??? | ||
| + | Anyway, looks like spfquery probably works fine, but the web site may be no longer available (DDoS from spammers, or ???). | ||
| + | |||
| + | $ spfquery --scope mfrom --identity lists.balug.org --ip-address $(dig +short balug.org. A) | ||
| + | pass | ||
| + | lists.balug.org: | ||
| + | lists.balug.org: | ||
| + | Received-SPF: | ||
| + | $ spfquery --scope mfrom --identity lists.balug.org --ip-address 8.8.8.8 | ||
| + | neutral | ||
| + | lists.balug.org: | ||
| + | lists.balug.org: | ||
| + | Received-SPF: | ||
| + | $ | ||
| + | |||
| + | Again with the neutral. | ||
| + | ... Ah ...: | ||
| + | balug.org. IN TXT " | ||
| + | We're missing the -all at the end. | ||
| + | Should check all our SPF records, and fix as appropriate. | ||
| + | Should probably also add spf version 2, but first things first ... | ||
| + | </ | ||
| + | < | ||
| + | So ... we have ...: | ||
| + | balug.org. | ||
| + | balug.org. | ||
| + | tmp.balug.org. | ||
| + | lists.balug.org. | ||
| + | lists.balug.org. | ||
| + | |||
| + | berkeleylug.com. | ||
| + | berkeleylug.com. | ||
| + | sf-lug.com. | ||
| + | sf-lug.com. | ||
| + | sf-lug.net. | ||
| + | sf-lug.net. | ||
| + | sflug.com. | ||
| + | sflug.com. | ||
| + | sflug.net. | ||
| + | sflug.net. | ||
| + | sflug.org. | ||
| + | sflug.org. | ||
| + | We should: | ||
| + | remove the RRs of type SPF (superseded/ | ||
| + | add trailing " -all" for those that don't have it | ||
| + | Our active sending TTLs look rather short, should probably nudge 'em up to ... 3600 or so? ... at least after they' | ||
| + | </ | ||
| + | |||
| + | < | ||
| + | And after updating, we have: | ||
| + | balug.org. | ||
| + | lists.balug.org. | ||
| + | tmp.balug.org. | ||
| + | berkeleylug.com. | ||
| + | sf-lug.com. | ||
| + | sf-lug.net. | ||
| + | sflug.com. | ||
| + | sflug.net. | ||
| + | sflug.org. | ||
| + | So ... that now looks better. | ||
| + | And let's do a little retest on our earlier: | ||
| + | $ spfquery --scope mfrom --identity balug.org --ip-address $(dig +short balug.org. A); echo " | ||
| + | pass | ||
| + | balug.org: 96.86.170.229 is authorized to use ' | ||
| + | balug.org: 96.86.170.229 is authorized to use ' | ||
| + | Received-SPF: | ||
| + | 0 | ||
| + | $ spfquery --scope mfrom --identity lists.balug.org --ip-address $(dig +short balug.org. A); echo " | ||
| + | pass | ||
| + | lists.balug.org: | ||
| + | lists.balug.org: | ||
| + | Received-SPF: | ||
| + | 0 | ||
| + | $ spfquery --scope mfrom --identity balug.org --ip-address 8.8.8.8; echo " | ||
| + | fail | ||
| + | Please see http:// | ||
| + | balug.org: Sender is not authorized by default to use ' | ||
| + | Received-SPF: | ||
| + | 1 | ||
| + | $ spfquery --scope mfrom --identity lists.balug.org --ip-address 8.8.8.8; echo " | ||
| + | fail | ||
| + | Please see http:// | ||
| + | lists.balug.org: | ||
| + | Received-SPF: | ||
| + | 1 | ||
| + | $ | ||
| + | So, that looks much better now. | ||
| + | </ | ||
| + | < | ||
| + | wordpress also sends mail: | ||
| + | From www-data@balug.org Tue Apr 27 02:12:48 2021 | ||
| + | From: WordPress < | ||
| + | So, @berkeleylug.com needs to be set up to send - and at least minimally receive, email (e.g. postmaster ...) | ||
| + | So, ... SPF first, as that has the longer TTL presently ... | ||
| + | from: | ||
| + | berkeleylug.com. | ||
| + | to: | ||
| + | berkeleylug.com. | ||
| + | |||
| + | And, added bit more for digitalwitness.org. and sf-lug.org. (latter of which thus far still uses @linuxmafia.com for mail), now have: | ||
| + | balug.org. | ||
| + | lists.balug.org. | ||
| + | tmp.balug.org. | ||
| + | berkeleylug.com. | ||
| + | digitalwitness.org. | ||
| + | sf-lug.com. | ||
| + | sf-lug.net. | ||
| + | sf-lug.org. | ||
| + | sflug.com. | ||
| + | sflug.net. | ||
| + | sflug.org. | ||
| + | SPF version 2 could be good/better ... but later, not a top priority. | ||
| + | </ | ||
| + | |||
| + | < | ||
| + | So, let's look into enabling SPF checking upon receipt of incoming ... | ||
| + | I also noticed what looks like something about a daemon - which may be preferable for large volumes/ | ||
| + | let's look at documentation bit more ... | ||
| + | $ man spfd.mail-spf-perl | ||
| + | $ systemctl list-unit-files | fgrep spf | ||
| + | $ | ||
| + | So, nothin' | ||
| + | For now let's presume spfquery (non-daemonized) is quite "good enough" | ||
| + | So ... let's configure that ... | ||
| + | added ...: | ||
| + | # tail -n 1 conf.d/ | ||
| + | CHECK_RCPT_SPF = true | ||
| + | # systemctl restart exim4.service | ||
| + | # That should be enough for that to now be operational - that should stop >> 50% of the incoming spam (attempts). | ||
| + | quite soon (if not already). | ||
| + | </ | ||
| + | < | ||
| + | Not seeing an SPF failure in the logs ... quite yet. | ||
| + | Let's test something that should fail ... | ||
| + | Drats - test made it through, even though the config should' | ||
| + | Oh, let's also add berkeleylug.com to the email domains, so that should work. | ||
| + | # DEBIAN_PRIORITY=medium dpkg-reconfigure exim4-config | ||
| + | # systemctl start exim4.service | ||
| + | Let's try sending to postmaster@berkeleylug.com | ||
| + | and yes, that got delivered fine. | ||
| + | So ... why is SPF check not working? | ||
| + | </ | ||
| + | < | ||
| + | # systemctl stop exim4.service | ||
| + | # ls -d / | ||
| + | / | ||
| + | # update-exim4.conf | ||
| + | # systemctl start exim4.service | ||
| + | SPF check still not working. | ||
| + | </ | ||
| + | < | ||
| + | Wordpress email ... something to circle back on later. | ||
| + | For now, for header it uses: | ||
| + | From: WordPress < | ||
| + | Looks like the only bit of that that's easy to change is the domain. | ||
| + | that's then more complications. | ||
| + | MAIL FROM:< | ||
| + | Again, not simple to change that. More to circle back on for later. | ||
| + | For now, dropped in aliases for www-data and wordpress, so at least attempts to those - and for now at least, won't bounce at those domains if | ||
| + | attempted. | ||
| + | postmaster ... "good enough" | ||
| + | </ | ||
| + | < | ||
| + | Looks like the SPF checks are now working. | ||
| + | I also found an older spdf process running and killed that off - maybe that made the difference? | ||
| + | So, yes, and seeing SPF fail/ | ||
| + | # fgrep -ai spf rejectlog | ||
| + | 2021-04-28 02:29:33 H=(sweja-se.mail.protection.outlook.com) [183.199.220.44] F=< | ||
| + | 2021-04-28 03:50:56 H=(smail1.vub.sk) [222.77.253.120] F=< | ||
| + | # dig +noall +answer +nottl ottawa.ca. TXT ottawa.ca. SPF swebolt.se. TXT swebolt.se. SPF | fgrep \" | ||
| + | ottawa.ca. | ||
| + | swebolt.se. | ||
| + | # spfquery --scope mfrom --id oefydgodea@ottawa.ca --ip 183.199.220.44; | ||
| + | fail | ||
| + | Please see http:// | ||
| + | ottawa.ca: Sender is not authorized by default to use ' | ||
| + | Received-SPF: | ||
| + | 1 | ||
| + | # spfquery --scope mfrom --id jhylunrrhc@swebolt.se --ip 222.77.253.120; | ||
| + | fail | ||
| + | Please see http:// | ||
| + | swebolt.se: Sender is not authorized by default to use ' | ||
| + | Received-SPF: | ||
| + | 1 | ||
| + | # | ||
| + | </ | ||
| + | < | ||
| + | Wrote a handy little program to summarize the exim rejectlog failure from the most recent few such log files: | ||
| + | # Rejectlog_report | ||
| + | 6313 Unrouteable address | ||
| + | 1013 relay not permitted | ||
| + | 8 SPF check failed | ||
| + | 7 SMTP protocol synchronization error (input sent without waiting for greeting) | ||
| + | 7 maximum allowed line length | ||
| + | 3 unqualified address not permitted | ||
| + | 1 SMTP protocol synchronization error (next input sent too soon: pipelining was not advertised) | ||
| + | 1 missing or malformed local part | ||
| + | 1 syntactically invalid | ||
| + | # | ||
| + | Look at least the top couple items would be good candidates for adding configurations for fail2ban. | ||
| + | Some others beyond that may also be worth doing - but not as high a priority. | ||
| + | </ | ||
| + | < | ||
| + | // reverted the temporarily increase of max queue time from 4 days to 7 days: | ||
| + | # awk ' | ||
| + | * * | ||
| + | # systemctl reload exim4.service | ||
| + | # | ||
| + | </ | ||
system/annoyances.1619376574.txt.bz2 · Last modified: 2021-04-25T18:49:34+0000 by michael_paoli