Wild Duck is a scalable no-SPOF IMAP/POP3 mail server. Wild Duck uses a distributed database (sharded + replicated MongoDB) as a backend for storing all data,
Or if you want to override default configuration options with your own, run the following (custom config file is merged with the default, so specify only these
3. Provide Gmail-like features like pushing sent messages automatically to Sent Mail folder or notifying about messages moved to Junk folder so these could be
marked as spam
4. Provide parsed mailbox and message data over HTTP. This should make creating webmail interfaces super easy, no need to parse RFC822 messages to get text
Yes, it does. You can run the server and get working IMAP and POP3 servers for mail store, LMTP server for pushing messages to the mail store and HTTP API
server to create new users. All handled by Node.js, MongoDB and Redis, no additional dependencies needed. Provided services can be disabled and enabled one by
one so, for example you could process just IMAP in one host and LMTP in another.
1.**Stateless.** Start as many instances as you want. You can start multiple Wild Duck instances in different machines and as long as they share the same
MongoDB and Redis settings, users can connect to any instances. This is very different from the traditional IMAP servers where a single user always needs to
connect (or be proxied) to the same IMAP server. Wild Duck keeps all required state information in MongoDB, so it does not matter which IMAP instance you
4.**Centralized authentication** which allows modern features like 2FA, application specific passwords, authentication scopes, revoking authentication tokens,
audit logging and even profile files to auto-configure Apple email clients without master password
8. Access messages both using **IMAP and HTTP API**. The latter serves parsed data, so no need to fetch RFC822 messages and parse out html, plaintext content or
attachments. It is super easy to create a webmail interface on top of this.
12.**Super easy to tweak.** The entire codebase is pure JavaScript, so there's nothing to compile or anything platform specific. If you need to tweak something
then change the code, restart the app and you're ready to go. If it works on one machine then most probably it works in every other machine as well.
15.**Exposed logs.** Users have access to logs concerning their account. This includes security logs (authentication attempts, changes on account) and also
Yes, historically it has [been considered a bad practice](http://www.memoryhole.net/~kyle/databaseemail.html) to store emails in a database. And for a good
reason. The data model of relational databases like MySQL does not work well with tree like structures (email mime tree) or large blobs (email source).
Notice the word "relational"? In fact document stores like MongoDB work very well with emails. Document store is great for storing tree-like structures and
while GridFS is not as good as "real" object storage, it is good enough for storing the raw parts of the message. Additionally there's nothing too GridFS
specific, so (at least in theory) it could be replaced with any object store.
Whenever a message is received Wild Duck parses it into a tree-like structure based on the MIME tree and stores this tree to MongoDB. Attachments are removed
from the tree and stored separately in GridStore. If a message needs to be loaded then Wild Duck fetches the tree structure first and, if needed, loads
attachments from GridStore and then compiles it back into the original RFC822 message. The result should be identical to the original messages unless the
original message used unix newlines, these might be partially replaced with windows newlines.
Wild Duck tries to keep minimal state for sessions (basically just a list of currently known UIDs and latest MODSEQ value) to be able to distribute sessions
between different hosts. Whenever a mailbox is opened the entire message list is loaded as an array of UID values. The first UID in the array element points to
the message nr. 1 in IMAP, second one points to message nr. 2 etc.
Actual update data (information about new and deleted messages, flag updates and such) is stored to a journal log and an update beacon is propagated through
Redis pub/sub whenever something happens. If a session detects that there have been some changes in the current mailbox and it is possible to notify the user
about it (eg. a NOOP call was made), journaled log is loaded from the database and applied to the UID array one action at a time. Once all journaled updates
have applied then the result should match the latest state. If it is not possible to notify the user (eg a FETCH call was made), then journal log is not loaded
* The entire **IMAP4rev1** suite with some minor differences from the spec. See below for [IMAP Protocol Differences](#imap-protocol-differences) for a complete
list
* **IDLE** ([RFC2177](https://tools.ietf.org/html/rfc2177)) – notfies about new and deleted messages and also about flag updates
* **CONDSTORE** ([RFC4551](https://tools.ietf.org/html/rfc4551)) and **ENABLE** ([RFC5161](https://tools.ietf.org/html/rfc5161)) – supports most of the spec,
* **AUTHENTICATE PLAIN** ([RFC4959](https://tools.ietf.org/html/rfc4959)) and **SASL-IR**
* **APPENDLIMIT** ([RFC7889](https://tools.ietf.org/html/rfc7889)) – maximum global allowed message size is advertised in CAPABILITY listing
* **UTF8=ACCEPT** ([RFC6855](https://tools.ietf.org/html/rfc6855)) – this also means that Wild Duck natively supports unicode email usernames. For example
[андрис@уайлддак.орг](mailto:андрис@уайлддак.орг) is a valid email address that is hosted by a test instance of Wild Duck
* **QUOTA** ([RFC2087](https://tools.ietf.org/html/rfc2087)) – Quota size is global for an account, using a single quota root. Be aware that quota size does not
mean actual byte storage in disk, it is calculated as the sum of the [RFC822](https://tools.ietf.org/html/rfc822) sources of stored messages.
* **COMPRESS=DEFLATE** ([RFC4978](https://tools.ietf.org/html/rfc4978)) – Compress traffic between the client and the server
Wild Duck more or less passes the [ImapTest](https://www.imapwiki.org/ImapTest/TestFeatures) Stress Testing run. Common errors that arise in the test are
unknown labels (Wild Duck doesn't send unsolicited `FLAGS` updates even though it does send unsolicited `FETCH FLAGS` updates) and sometimes NO for `STORE`
(messages deleted in one session can not be updated in another).
In comparison Wild Duck is slower in processing single user than Dovecot. Especially when fetching messages, which is expected as Dovecot is reading directly
from filesystem while Wild Duck is recomposing messages from different parts.
Raw read/write speed for a single user is usually not relevant anyway as fetching entire mailbox content is not something that happens often. Wild Duck offers
better parallelization through MongoDB sharding, so more users should not mean slower response times. It is also more important to offer fast synchronization
speeds between clients (eg. notifications about new email and such) where Wild Duck excels due to the write ahead log and the ability to push this log to
All changes to messages like deleting messages or marking messages as seen are stored in storage only in the UPDATE stage (eg. after calling QUIT). Until then
the changes are preserved in memory only. This also means that if a message is downloaded but QUIT is not issued then the message does not get marked as _Seen_.
Wild Duck uses message `_id` value (24 byte hex) as the unique ID. If a message is moved from one mailbox to another then it might _re-appear_ in the listing.
2.`RENAME` does not touch subfolders which is against the spec (RFC3501 6.3.5\. _If the name has inferior hierarchical names, then the inferior hierarchical
names MUST also be renamed._). Wild Duck stores all folders using flat hierarchy, the "/" separator is fake and only used for listing mailboxes
3. Unsolicited `FLAGS` responses (RFC3501 7.2.6.) and `PERMANENTFLAGS` are not sent (except for as part of `SELECT` and `EXAMINE` responses). Wild Duck notifies
about flag updates only with unsolicited FETCH updates.
6. Metadata arguments for `SEARCH MODSEQ` are ignored (RFC7162 3.1.5.). You can define `<entry-name>` and `<entry-type-req>` values but these are not used for
anything
7.`SEARCH TEXT` and `SEARCH BODY` both use MongoDB [$text index](https://docs.mongodb.com/v3.4/reference/operator/query/text/) against decoded plaintext
version of the message. RFC3501 assumes that it should be a string match either against full message (`TEXT`) or body section (`BODY`).
9.**Autoexpunge**, meaning that an EXPUNGE is called on background whenever a messages gets a `\Deleted` flag set. This is not in conflict with IMAP RFCs.
This should "deliver" a new message to the INBOX of _username@example.com_ by using the built-in LMTP maildrop interface. If your email client is connected then
This gives you an outbound SMTP server that uses Wild Duck accounts for authentication. The plugin authenticates user credentials and also rewrites headers if
needed (if the header From: address does not match user address or aliases then it is rewritten). Additionally a copy of the sent message is uploaded to the
Sent Mail folder. Local delivery is done directly to Wild Duck LMTP.
Use [Haraka](http://haraka.github.io/) with [queue/lmtp](http://haraka.github.io/manual/plugins/queue/lmtp.html) plugin to deliver messages to Wild Duck and
[haraka-plugins-wildduck](https://github.com/nodemailer/haraka-plugin-wildduck) to validate recipient addresses and quota usage against the Wild Duck users
* Optimize FETCH queries to load only partial data for BODY subparts
* Parse incoming message into the mime tree as a stream. Currently the entire message is buffered in memory before being parsed.
* Maybe allow some kind of message manipulation through plugins
* Wild Duck does not plan to be the most feature-rich IMAP client in the world. Most IMAP extensions are useless because there aren't too many clients that are
able to benefit from these extensions. There are a few extensions though that would make sense to be added to Wild Duck:
* IMAP4 non-synchronizing literals, LITERAL- ([RFC7888](https://tools.ietf.org/html/rfc7888)). Synchronized literals are needed for APPEND to check mailbox
quota, small values could go with the non-synchronizing version.
Wild Duck supports MongoDB sharding. Consider using sharding only if you know that your data storage is large enough to outgrow single replica. Some actions
require scattered queries to be made that might be a hit on performance on a large cluster but most queries include the shard key by default.
> Attachments collections might be configured to reside in a different database than default. Modify sharding namespaces accordingly (and do not forget to
Wild Duck is able to use Redis Sentinel instead of single Redis master for automatic failover. When using Sentinel and the Redis master fails then it might take
a moment until new master is elected. Pending requests are cached during that window, so most operations should succeed eventually. You might want to test
failover under load though, to see how it behaves.
Redis Sentinel failover does not guarantee consistency. Wild Duck does not store critical information in Redis, so even if some data loss occurs, it should not
You can live-reload updated certificates by sending SIGHUP to the master process. This causes application configuration to be re-read from the disk. Reloading
only affects only some settings, for example all TLS certificates are loaded and updated. In this case existing processes continue as is, while new ones use the
Beware though that if configuration loading fails, then it ends with an exception. Make sure that TLS certificate files are readable for the Wild Duck user.