mirror of
https://github.com/knadh/listmonk.git
synced 2025-09-23 06:49:53 +08:00
High performance, self-hosted, newsletter and mailing list manager with a modern dashboard. Single binary app.
campaigncampaign-managementemail-campaignsemail-marketingemail-subscriptionfcm-messaginglistmonkmailing-listnewsletternewsletter-managementself-hostedsms-gatewaysmtpstarred-knadh-repostarred-repo
cea65c009d
This has been a hair-pulling rabbit hole of an issue. #1931 and others. When the `next-campaign-subscribers` query that fetches $n subscribers per batch for a campaign returns no results, the manager assumes that the campaign is done and marks as finished. Marathon debugging revealed fundamental flaws in qyery's logic that would incorrectly return 0 rows under certain conditions. - Based on the "layout" of subscribers for eg: a series of blocklisted subscribers between confirmed subscribers. A series of unconfirmed subscribers in a batch belonging to a double opt-in list. - Bulk import blocklisting users, but not marking their subscriptions as 'unsubscribed'. - Conditions spread across multiple CTEs resulted in returning an arbitrary number of rows and $N per batch as the selected $N rows would get filtered out elsewhere, possibly even becoming 0. After fixing this and testing it on our prod instance that has 15 million subscribers and ~70 million subscriptions in the `subscriber_lists` table, ended up discovered significant inefficiences in Postgres query planning. When `subscriber_lists` and campaign list IDs are joined dynamically (CTE or ANY() or any kind of JOIN that involves) a query, the Postgres query planner is unable to use the right indexes. After testing dozens of approaches, discovered that statically passing the values to join on (hardcoding or passing via parametrized $1 vars), the query uses the right indexes. The difference is staggering. For the particular scenario on our large prod DB to pull a batch, ~15 seconds vs. ~50ms, a whopping 300x improvement! This patch splits `next-campaign-subscribers` into two separate queries, one which fetches campaign metadata and list_ids, whose values are then passed statically to the next query to fetch subscribers by batch. In addition, it fixes and refactors broken filtering and counting logic in `create-campaign` and `next-campaign` queries. Closes #1931, #1993, #1986. |
||
|---|---|---|
| .github | ||
| cmd | ||
| dev | ||
| docs | ||
| frontend | ||
| i18n | ||
| internal | ||
| models | ||
| scripts | ||
| static | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| .goreleaser.yml | ||
| config-demo.toml | ||
| config.toml.sample | ||
| CONTRIBUTING.md | ||
| docker-compose.yml | ||
| docker-entrypoint.sh | ||
| Dockerfile | ||
| go.mod | ||
| go.sum | ||
| install-demo.sh | ||
| install-prod.sh | ||
| LICENSE | ||
| listmonk-simple.service | ||
| listmonk@.service | ||
| Makefile | ||
| permissions.json | ||
| project.inlang.json | ||
| queries.sql | ||
| README.md | ||
| schema.sql | ||
| VERSION | ||
listmonk is a standalone, self-hosted, newsletter and mailing list manager. It is fast, feature-rich, and packed into a single binary. It uses a PostgreSQL (⩾ 12) database as its data store.
Visit listmonk.app for more info. Check out the live demo.
Installation
Docker
The latest image is available on DockerHub at listmonk/listmonk:latest. Use the sample docker-compose.yml to run manually or use the helper script.
Demo
mkdir listmonk-demo && cd listmonk-demo
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-demo.sh)"
DO NOT use this demo setup in production.
Production
mkdir listmonk && cd listmonk
bash -c "$(curl -fsSL https://raw.githubusercontent.com/knadh/listmonk/master/install-prod.sh)"
Visit http://localhost:9000.
NOTE: Always examine the contents of shell scripts before executing them.
See installation docs.
Binary
- Download the latest release and extract the listmonk binary.
./listmonk --new-configto generate config.toml. Then, edit the file../listmonk --installto setup the Postgres DB (or--upgradeto upgrade an existing DB. Upgrades are idempotent and running them multiple times have no side effects).- Run
./listmonkand visithttp://localhost:9000.
See installation docs.
Developers
listmonk is free and open source software licensed under AGPLv3. If you are interested in contributing, refer to the developer setup. The backend is written in Go and the frontend is Vue with Buefy for UI.
License
listmonk is licensed under the AGPL v3 license.