Commit graph

260 commits

Author SHA1 Message Date
Abhinand KR
4eefd4215e
Remove redundant campaign manager config validations (#2095) 2024-10-27 17:25:48 +05:30
Kailash Nadh
5b3d6e2be0 Add first-time Super Admin setup UI on fresh install.
- Don't setup username+password from config file during fresh install.
- If `LISTMONK_ADMIN_USER` and `LISTMONK_ADMIN_PASSWORD` env vars are
  set during `--install`, use them.
- Otherwise, render new username+password creation UI on `/admin/login`.
- Add Cypress tests.
2024-10-26 19:47:01 +05:30
Kailash Nadh
e7109daaf3 Fix missing email validation in OIDC exchange. 2024-10-21 17:54:58 +05:30
Kailash Nadh
354fb300f7 Replace hardcoded perm literal with const. 2024-10-20 23:50:06 +05:30
Kailash Nadh
30be235e2a Add microseconds to log lines. 2024-10-20 23:50:06 +05:30
Kailash Nadh
cea65c009d Fix and refactor subscriber batch fetching in campaign processing.
This has been a hair-pulling rabbit hole of an issue. #1931 and others.
When the `next-campaign-subscribers` query that fetches $n subscribers
per batch for a campaign returns no results, the manager assumes
that the campaign is done and marks as finished.

Marathon debugging revealed fundamental flaws in qyery's logic that
would incorrectly return 0 rows under certain conditions.
- Based on the "layout" of subscribers for eg: a series of blocklisted
  subscribers between confirmed subscribers.
  A series of unconfirmed subscribers in a batch belonging to a double
  opt-in list.
- Bulk import blocklisting users, but not marking their subscriptions
  as 'unsubscribed'.
- Conditions spread across multiple CTEs resulted in returning an
  arbitrary number of rows and $N per batch as the selected $N rows
  would get filtered out elsewhere, possibly even becoming 0.

After fixing this and testing it on our prod instance that has
15 million subscribers and ~70 million subscriptions in the
`subscriber_lists` table, ended up discovered significant inefficiences
in Postgres query planning. When `subscriber_lists` and campaign list IDs
are joined dynamically (CTE or ANY() or any kind of JOIN that involves)
a query, the Postgres query planner is unable to use the right indexes.

After testing dozens of approaches, discovered that statically passing
the values to join on (hardcoding or passing via parametrized $1 vars),
the query uses the right indexes. The difference is staggering.
For the particular scenario on our large prod DB to pull a batch,
~15 seconds vs. ~50ms, a whopping 300x improvement!

This patch splits `next-campaign-subscribers` into two separate queries,
one which fetches campaign metadata and list_ids, whose values are then
passed statically to the next query to fetch subscribers by batch.

In addition, it fixes and refactors broken filtering and counting logic
in `create-campaign` and `next-campaign` queries.

Closes #1931, #1993, #1986.
2024-10-13 17:03:59 +05:30
Kailash Nadh
a26834196e Refactor subscriber APIs list permission filtering. 2024-10-13 17:03:59 +05:30
Kailash Nadh
d9b4bae0a8 Rename migration to v4.0.0 2024-10-13 17:03:59 +05:30
Kailash Nadh
eb47e80c35 Fix list auth by adding an explicit 'getAll' flag to query. 2024-10-13 17:03:59 +05:30
Kailash Nadh
3671a5254e Update profile UI with new user data structures. 2024-10-13 17:03:58 +05:30
Kailash Nadh
ae2a386193 Add support for "list roles".
This commit splits roles into two, user roles and list roles, both of which
are attached separately to a user.

List roles are collection of lists each with read|write permissions, while
user roles now have all permissions except for per-list ones.

This allows for easier management of roles, eliminating the need to clone and
create new roles just to adjust specific list permissions.
2024-10-13 17:03:58 +05:30
Kailash Nadh
12a6451ed0 Add list permission check to subscriber calls. 2024-10-13 17:03:55 +05:30
Kailash Nadh
d74e067961 Add per-list permission to list management.
- Filter lists by permitted list IDs in DB get calls.
- Split getLists() handlers into two (one, all) for clarity.
- Introduce new `subscribers:get_by_list` permission.
- Tweak UI rendering to work with new per-list permssions.
2024-10-13 16:59:52 +05:30
Kailash Nadh
26c6db0a13 Remove admin user/password from sample config generation. 2024-10-13 16:59:52 +05:30
Kailash Nadh
bf00fd2951 Add support for setting admin user/password via env on --install. 2024-10-13 16:59:52 +05:30
Kailash Nadh
68870ad6b1 Fix update check looping on failed HTTP requests. 2024-10-13 16:59:52 +05:30
Kailash Nadh
2da920d634 Add legacy TOML user+password to API auth on init with warning. 2024-10-13 16:59:52 +05:30
Kailash Nadh
5024ded763 Add API user authentication to auth module with caching of creds on user CRUD. 2024-10-13 16:59:52 +05:30
Kailash Nadh
e6ec1cb9b3 User legacy (TOML) admin credentials as API creds for backwards compatibility. 2024-10-13 16:59:52 +05:30
Kailash Nadh
1e875afa67 Add OIDC auth hooks (init, callback, session) and finish OIDC support. 2024-10-13 16:59:52 +05:30
Kailash Nadh
4eabd967d8 Refactor update check.
- Switch away from GitHub releases API to a statically hosted custom
  JSON message to include richer data.
- Instead of checking 24 hours post-boot, check 15 mins later post boot
  and then every 24 hours.
- Add provision for messages to display on the admin dashboard to
  communicate important / urgent announcements.
  (Fingers crossed, this never has to be used!)
2024-10-13 16:59:52 +05:30
Kailash Nadh
a8c17780f9 Add warning on admin UI for legacy creds in the TOML file. 2024-10-13 16:59:52 +05:30
Kailash Nadh
612c1d6eac Add per-list permission management to roles. 2024-10-13 16:59:52 +05:30
Kailash Nadh
19527f97eb Add new fields to /api/config to remove /settings dependency in camapign UI. 2024-10-13 16:59:52 +05:30
Kailash Nadh
32d5823dfe Refactor 'super' user type to a pre-defined super admin role. 2024-10-13 16:59:52 +05:30
Kailash Nadh
d4e4c5fa99 Add granular permissions and role management to backend and admin UI. 2024-10-13 16:59:51 +05:30
Kailash Nadh
2000e9fa12 Style and add OIDC logo to the login page. 2024-10-13 16:59:51 +05:30
Kailash Nadh
7bb14de42e Upgrade simplesessions to v3. 2024-10-13 16:59:51 +05:30
Kailash Nadh
4997c10b97 Add user profile APIs and update UI. 2024-10-13 16:59:51 +05:30
Kailash Nadh
906e0f57b6 Refactor handler groups and add mising auth features like logout. 2024-10-13 16:59:51 +05:30
Kailash Nadh
57ac9dca4b Add public login page and auth middleware and handlers. 2024-10-13 16:59:51 +05:30
Kailash Nadh
1516bf216f Add api type user. 2024-10-13 16:59:51 +05:30
Kailash Nadh
bf0b500bb0 Add API token authentication. 2024-10-13 16:59:51 +05:30
Kailash Nadh
0968e58766 Add user/password login handler. 2024-10-13 16:59:51 +05:30
Kailash Nadh
435d6d5169 Add create/add/delete user management UI and database schema. 2024-10-13 16:59:51 +05:30
Kailash Nadh
8ca95f6827 Refactor OIDC middleware handler logic. 2024-10-13 16:59:50 +05:30
Kailash Nadh
e406b2516a Add a settings UI for OIDC. 2024-10-13 16:59:50 +05:30
Marc Bärtschi
1b7128abf2 Implement OIDC
This is a simple OIDC implementation. It's very basic and just logs the user in. Access control needs to be done on the IDP side.
2024-10-13 16:59:50 +05:30
Kailash Nadh
58b13afa29 Fix tx template not being cached on update. Closes #2061. 2024-10-05 09:51:56 +05:30
Bowrna
16f4dfd3e9
Fix incorrect bulk blocklisting behaviour (#2041). Fixes #1841 2024-09-19 10:56:56 +05:30
Vinoth Kumar
d7fe13c4b9
Fix typo and formatting (#2028) 2024-08-30 13:24:45 +05:30
Abhinav Raut
679457cb12
Ensure unique upload filenames by adding a suffix (#1963)
Fixes #1957.

Co-authored-by: Abhinav Raut <abhinav.raut@zerodha.com>
2024-08-08 15:42:29 +05:30
Bowrna
1e6e97e750
Add validation for filename with non-ASCII chars in media upload (#1973) 2024-08-05 20:08:33 +05:30
Keshav Gupta
c334d2e6e1
fix #1950:Export of unsubscribed users exports all users (#1965)
Co-authored-by: keshav <keshav.gupta@jarvis.consulting>
2024-08-02 00:05:33 +05:30
Kailash Nadh
c2e7c713ca Increase campaign subject char limit. Closes #1909. 2024-07-17 19:38:11 +05:30
largemouth
c3f4379b17
chore: fix function name in comment (#1836)
Signed-off-by: largemouth <largemouth@aliyun.com>
2024-04-22 18:43:23 +05:30
Jonas Sulzer
1d32d4c368
👌 IMPROVE: make RootURL available in email templates (#1812)
Signed-off-by: Jonas Sulzer <jonas@violoncello.ch>
2024-04-04 22:57:52 +05:30
Kailash Nadh
124af1ee29 Make static e-mail template subjects scriptable. Closes #1727.
This commit introduces optional, custom `<title>` tags that can be
added to `static/email-templates/*.html` so as to make the subjectlines
fully scriptable for system e-mails and notifications.

The tag must look like this.

```
<title data-i18n>Stuff {{ .Subscriber.Name }} here!</title>
```
2024-03-25 15:49:40 +05:30
Muhammed Kadir Tan
2afac2469e
fix: check errors.Is instead of strings.Contains for http: Server closed error (#1779) 2024-03-12 11:06:35 +05:30
tgolang
860009b866
chore: remove repetitive words (#1778)
Signed-off-by: tgolang <seekseat@aliyun.com>
2024-03-11 13:33:50 +05:30