Mailur aims to create useful unique features except general features like good search, conversation view, tags. Let's try to review them here.

#

Manually linking threads

I fell in love with Gmail conversation view when first tried. Gmail has really great threads, but they have some limitation like
- 100 messages in thread is maximum
- not always messages with the same author and subject are combined in one thread

So I really wanted to bring Gmail like conversations into open source webmail, but I also wanted to bring a feature I've always missed in Gmail, a possibility to manually merge threads into one. This feature was initially implemented in Alpha. After intensive usage over two years I realized that it changed the way I organize emails. When new email arrives in Inbox, my first reaction: "Should I put this message in existing thread?" (if it isn't already a part of some thread). I can link all invoices from one service together, link all news from another service into one thread, link almost all messages from a person I talk rarely as one conversation, etc. When next time I need to find an email I'll find not only particular email, but the full thread with all related emails. There is also ability to search through particular thread, it's really useful when I have a huge thread and I want to find a particular message in it.

Sometimes I want to find particular email and I can easily switch from threads view to messages view I just need to remove :threads from search bar.

#

Independent split pane

I use Vim and I always split a window vertically into two panes. I open related file for context in one pane when editing the code in another pane. I introduced two panes mode in first prototype, then removed it from Alpha and now I repaired two panes back again and I really love them. I can write my drafts in the split pane and in the meantime, I can search a thread for context in the main pane.

#

Writing using Markdown

I love a lightweight markup language Markdown. There are many projects that adding Markdown support for Gmail and Thunderbird with some limitations. Markdown is the first citizen in Mailur for composing emails. Also the main pane can be used as live preview when editing a draft in the split pane.

#

Sieve scripts

Gmail has pretty good filters and they can be applied for old messages as well. In open source mail usually Sieve language is used for email filtering, but usually it's hard to apply these Sieve scripts for existing emails. As I use Dovecot as storage, it has IMAP FILTER=SIEVE extension, so Sieve scripts can be applied not only for new emails, but also they can be run for all existing emails from interface.

When I posted the "Alpha" (postgres based) version, it was a good conversation on Reddit. I was thinking about IMAP support after that. I didn't want to write my own IMAP server, so I started to look closer to Dovecot. After first investigation I had an idea that probably I can use Dovecot as main storage, because I didn't want to duplicate emails over Dovecot and Postgres anyway. At summer 2017 I had a chance to evaluate the idea, after some period of deep work I got a first Dovecot based prototype and confidence that I can build a great webmail based on it.

The hardest things to deal with:

• unusual storage (limited IMAP instead of well known and powerful SQL)
• IMAP folders instead of tags
• I needed proper threads with ability to link them together

But at the same time it gives great features:

• already a part of mail stack (often used in home-grown setup)
• simple immutable storage, designed exactly for emails
• a bunch of email related plugins and integrations
• master-master replication

After trying few different approaches for hardest things I got pretty flexible storage.

The current storage

IMAP keywords are used for system tags (like #inbox, #spam and #trash) and user-defined tags.

Just four IMAP folders for:
1. all original messages (no separate folder for Inbox, Spam or Trash)
2. all parsed messages for effective web representation
3. useful metadata (settings, linked threads, cache, etc.)
4. cleaned messages from #spam and #trash (autoexpunge in 30 days)

The most sensitive data is a folder with original messages and settings, parsed messages and metadata can be regenerated. Dovecot ACL is used to deny mailbox operations, expunging messages, etc. I use Dovecot's own high-performance mailbox format single-dbox, so emails are never changed even during applying IMAP flags on them.

As for now, I don't rely on Gmail as storage anymore, so the Dovecot mailboxes are only source of truth, so I should be really careful with them. I use two servers from two different datacenters with master-master replication enabled, so far it has been working pretty well.

From time to time, I want to check Inbox on mobile, so it has limited "general" IMAP support using Dovecot virtual mailboxes and filtering by #inbox, #spam and #trash tags. I use it with FairEmail for Android for fast notifications. Mailur also has a basic mobile view.

As I have only one IMAP folder with all messages, it works pretty well with Sieve scripts using IMAP FILTER=SIEVE, because I don't need to move messages around and just mark them with a proper flag. Also it's easy to run them through all messages (available from UI).

I use Dovecot Lucene for full text search indexing and it works pretty well for me.

In conclusion

I like the current storage and I think it should be pretty stable from that point. It has good flexibility in storing different metadata. Metadata can change over time with new features, but as I said before metadata can be easily regenerated, so it shouldn't be a problem.

I think I'm pretty close to my initial goal to move away from Gmail. Even I still use Gmail as a transport (importing over IMAP and sending over SMTP), but I already can remove emails from Google and I'll do this soon.

public demo :: github page :: short video

Mailur aims to become a powerful Gmail-inspired webmail.

It is already usable as an alternative Gmail client with a set of unique features:
- linking few threads together
- composing emails with Markdown
- internal lightweight tabs

Mailur has got many changes since the first intro, but main points are still the same. It has been developed with simplicity in mind. It should be simple to use, to develop, to deploy and to support. It has as few dependencies as possible, at all levels. It aims to become a Gmail replacement for me in several steps.

The first step is already done, so I (and my wife) can use Mailur as an awesome webmail client. Bidirectional synchronization works very well: new messages appear in several seconds, manually linked threads are saved using Gmail labels. Gmail is still the main storage. I can remove all my emails from the Mailur database, run clean synchronization and get the same state again.

The next big step for me is preparing fault tolerance storage. This is why I chose PostgreSQL as single storage of truth (because it has ACID, different ways of replication, powerful data types, full text search), so I just need a reliable PostgreSQL instance for this purpose.

With such a storage I would be able to prepare Private Beta for people who would want to try it, but don't want to deal with servers and installation process. Mailur uses one database per user, so moving to another instance is as simple as to make a backup and then restore it.

Then, I can remove my email messages from Gmail :), and use something like Mailgun to receive and send messages from my own email address (I still don't want to set up and support a proper mail server with spam filters). This is a cool feature when one can change their email address but still have full access to all email messages from previous addresses and doesn't need to think about transferring these messages from one account to another.

Of course, I will wait a little to be confident in system's stability. Also, Mailur needs some further features, improvements and optimizations. For example, I'm going to make such things: polishing interface, filters for incoming messages, Mailgun support, some kind of mobile application with push notifications, more tests, etc.

I had a long pause in the development of Mailur when I was working on a commercial project accumulating some money. I came back to a nearly full-time development of this project about half a year ago. Now, I have to find a new full-time job again, because I've spent almost all the money I had accumulated before, so it means I will work on Mailur significantly less (or with pause again) in near future.

Discussion: Reddit

Mailr is early in development. There is a lot of work, that has to be done.

Technologies:

• Python 3 with the help of werkzeug, jinja2, sqlalchemy, lxml;
• PostgreSQL with his awesome power;
• lessjs, jquery (yet only) on the frontend.

I want to use minimal scope of dependencies for building Mailr. And I always want to keep in mind simplicity when developing it.

Source code available on github.

A big piece of work I did, but it’s mostly invisible part and about IMAP communication, async synchronization, email parsing. The visible part is some working interface. There is public demo, which works pretty fast (and yep, it is not available to compose email yet). You can send email to demo[at]pusto.org and it probably appears in Inbox.

Screenshot

Vision of first version

Mailr will have web interface, which will be fast and useful on laptop with small screen, on big monitor and on iPad Mini (these devices I have and I want a one customizable web interface for them).

Mailr will have Gmail compatible mode (through IMAP), so return to Gmail will be easy. I will also not plan to implement a phone version and this mode can be used in parallel with any phone client which connects to Gmail as usual. That is also easier to develop using Gmail as the first backend.

Mailr will have many features from Gmail, such as: useful conversations, labels, fast search, filters for sorting incoming messages, detecting and folding quotes, keyboard shortcuts, SSL support…

Also Mailr will have some additional features.

Merging conversations. I really like Gmail conversations, they have made the best implementation (I have tried many others, when I wanted to move away from Gmail). Google are matching well which message belongs to which tread, but sometimes they can’t match:

I think possibility to merge it manually is a good solution in such cases.

Write in Markdown. I like Markdown and reStructuredText and I want to use them for writing my emails. They are also useful for reading after converting to HTML.

Two panels. I really like two panel mode in my VIM(text editor), I started using them and now I feel uncomfortable in one panel editor. Second panel gives me more context when I write in another one. This feature is in progress, as you can see on the screenshot. But I need to implement a possibility to turn that feature off, because I understand, that not all people would like it.

Customizable interface. As I said Mailr will be useful in different resolutions and custom themes and settings for interface will solve this task.

All in one tab. I use Chrome and I like that “Settings”, “Downloads”, “Extensions” just open in new tabs instead of new windows (I used Firefox before and it has been using new separated windows for those things). I really like consistent user interface. All my surfing lives in one window and I want all my emails would probably live in one tab (that also means different accounts, but multiple accounts probably will appear after the first version).

Simple backup. Mailr will have ability to get all your data (including accounts, filters, etc.) and restore it on your own server or on secure server of your trusted friend.