LKML is a very high volume mailing list. I accumulate around 500 emails during the time I spend sleeping. It’s a kitchen sink for every kernel patch and discussion that is going on, which is a good thing and a bad thing. Because I care about only a few of them. Most users subscribe to and read only subsystem specific mailing lists. The trick is to run scripts on top of them to find the emails of interest.
I had a specific need where I wanted to find emails where the sysfs ABI was being modified.
How did I go about this?
It’s easy to run custom scripts of varying complexity if your mails are available locally. I first setup offlineimap to download and keep all my mails offline. It’s never a bad idea to keep a backup. This is the minimal config needed to get started :
[general] # list of accounts, I have only one accounts = aishpant [Account aishpant] # identifier for the local repository localrepository = local-aishpant # identifier for the remote repository remoterepository = remote-aishpant # download emails starting from this date maxage = 2017-12-25 # local repository settings [Repository local-aishpant] type = Maildir # where should the emails go? # this local folder needs to exist localfolders = /home/a/mail/aishpant # remote repository settings [Repository remote-aishpant] type = Gmail # Gmail specific username and password remoteuser = email@example.com remotepass = xxxpass sslcacertfile = /etc/ssl/certs/ca-certificates.crt # synchronise only these remote folders # I have three folders of interest folderfilter = lambda folder: folder in ['INBOX', 'LKML', 'attrDoc']
Then you can just run offlineimap to start fetching all your emails.
Set-up mutt to read and send emails. I use mutt anyway to send patches as the kernel community accepts only text based emails.
## for sending mails set realname = "Aishwarya Pant" set from = "firstname.lastname@example.org" set smtp_url = "smtps://email@example.com@smtp.gmail.com:465/" set smtp_pass = "xxxpass" set sendmail="/usr/bin/esmtp" ## appearance set sort = 'threads' set sort_aux = 'reverse-last-date-received' set charset = "utf-8" ## for reading emails, read from local # needs to be consistent with offlineimap set mbox_type = Maildir set folder = /home/a/mail/aishpant # folder in which to start mutt set spoolfile = +/INBOX/ # cache for even faster set header_cache = /home/a/mail/cache/ ## writing mail set editor = "vim"
If you run offlineimap, it will sync emails only once. After that you will need to set-up a cron job or daemon to keep running the service in background. Do not use the offlineimap autorefresh to sync emails as it takes up a lot of memory. Memory usage on my system was ~800 Mb after it had been running for an hour. I used the systemd user timer to periodically sync emails.
First, create two files -
## offlineimap.service [Unit] Description=Offlineimap Service (oneshot) Documentation=man:offlineimap(1) [Service] # execute the action and stop Type=oneshot ExecStart=/usr/bin/offlineimap -o -u basic # give 120 seconds for offlineimap to gracefully stop before hard killing it TimeoutStopSec=120 [Install] WantedBy=mail.target
## offlineimap.timer [Unit] Description=Offlineimap Query Timer [Timer] # start the service 2 minutes after a boot-up OnBootSec=2m # the timer will run the service every 5 minutes OnUnitInactiveSec=5m [Install] WantedBy=default.target
Put both files in the folder
/etc/systemd/user. Now, to enable the service and
the timer run :
$ systemctl --user enable offlineimap.service offlineimap.timer $ systemctl --user start offlineimap-oneshot.service
The service needs to be started once before the timer can kick in.
You can check the status of the service using the
systemctl command. (The
--user flag has to be passed every time.)
$ systemctl --user list-timers NEXT LEFT LAST PASSED UNIT A Wed 2018-01-10 14:06:50 IST 3min 50s left Wed 2018-01-10 14:01:24 IST 1min 35s ago offlineimap-oneshot.timer o 1 timers listed. Pass --all to see loaded but inactive timers, too.
$ systemctl --user status offlineimap-oneshot ● offlineimap-oneshot.service - Offlineimap Service (oneshot) Loaded: loaded (/etc/systemd/user/offlineimap-oneshot.service; enabled; vendor preset: enabled) Active: inactive (dead) since Wed 2018-01-10 14:01:50 IST; 1min 1s ago Docs: man:offlineimap(1) Process: 6552 ExecStart=/usr/bin/offlineimap -o -u basic (code=exited, status=0/SUCCESS) Main PID: 6552 (code=exited, status=0/SUCCESS) Jan 10 14:01:27 mordor offlineimap: Syncing INBOX: Gmail -> Maildir Jan 10 14:01:27 mordor offlineimap: Syncing attrDoc: Gmail -> Maildir Jan 10 14:01:30 mordor offlineimap: Deleting 2 messages (23726,23729) in Maildir[INBOX] Jan 10 14:01:30 mordor offlineimap: Deleting 1 messages (23703) in Gmail[INBOX] Jan 10 14:01:31 mordor offlineimap: Syncing LKML: Gmail -> Maildir Jan 10 14:01:46 mordor offlineimap: Copy message UID 217124 (1/3) remote-aishpant:LKML -> local-aishpant Jan 10 14:01:46 mordor offlineimap: Copy message UID 217125 (2/3) remote-aishpant:LKML -> local-aishpant Jan 10 14:01:49 mordor offlineimap: Copy message UID 217126 (3/3) remote-aishpant:LKML -> local-aishpant Jan 10 14:01:50 mordor offlineimap: *** Finished account 'aishpant' in 0:25 Jan 10 14:01:50 mordor systemd: Started Offlineimap Service (oneshot).
With all my set-up done, now I can write scripts that parse through the LKML emails and put the emails of my interest in another folder (attrDoc in this case).
The offlineimap mail directory contains three folders:
- ‘tmp’ for messages that are in process of being delivered
- ‘new’ for messages that have not been seen by any mail application such as mutt or the Gmail web client
- ‘cur’ for messages that have been viewed
I have a python script that looks for patch emails in the new messages of the LKML folder, and checks if there are any sysfs attribute declaring macros like DEVICE_ATTR in the diff. If there are any, it will put these emails in the new message folder of attrDoc. Now when I open mutt and navigate to the attrDoc folder, I see saner emails and all this effort has paid off!
In conclusion, this seems like a lot of work for filtering emails. It was. There might be simpler and effective ways for achieving the same result. But this is what works for me currently. This is not recommended for every one. For normal usage, I more than happy with the Gmail web client. I set-up all my mailing list filters over there (i.e. procmail is overkill).
- Modify the script to copy the entire message thread instead of just one email to the attrDoc folder
- Read passwords from GNU Pass
- Configure auto-completion for contacts, colors and vim like navigation in mutt ( or Neomutt ?)
Here are some sources I used to figure this out :