Commit graph

75 commits

Author SHA1 Message Date
restyler
427fad9908 Clean up SQL snippets: remove debug logs and fix defaults
- Remove all console.log debug statements from Subscribers.vue
- Fix SQL snippet default data - use correct subscriber_status values
- Reduce to 2 simple, useful default snippets:
  - Enabled Subscribers (subscribers.status = 'enabled')
  - Recent Signups (last 30 days)
- Removes problematic snippets that used invalid status values

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-28 20:25:54 +04:00
restyler
17e07a5257 Add SQL snippets feature
- Add SQL snippets table and migration
- Create CRUD operations for managing reusable SQL query fragments
- Add frontend UI for SQL snippets management
- Include query validation and syntax highlighting
- Add navigation menu integration
- Requires subscribers:sql_query permission

This is the first part of a larger feature split from dynamic segments.
2025-06-28 20:25:54 +04:00
Kailash Nadh
11f52ee8a3 Switch migration version to v5 2025-04-28 23:32:29 +05:30
Kailash Nadh
db81dc1dad Remove default dummy admin notification email settings. 2025-04-28 18:12:58 +05:30
Kailash Nadh
12bc79866f Make OIDC provider name display on login button configurable. Closes #2211 2025-04-23 00:09:24 +05:30
Kailash Nadh
53d79297ca Prepare first RC. 2025-04-20 20:48:51 +05:30
Kailash Nadh
ffbda019e4 Add sample visual campaign template on install and upgrade. 2025-04-20 20:48:51 +05:30
Kailash Nadh
7786b18473 Add DB migrations for visual editor. 2025-04-20 20:48:51 +05:30
Kailash Nadh
007f4de850 Fix a number of cosmetic inconsistenies across handlers and functions.
- Make the beginning of handlers consistent with uniform variable declaration
  and grouping.
- Add missing comments.
- Fix staticcheck/vet warnings and idiom issues.
2025-04-05 13:41:31 +05:30
Kailash Nadh
a271bf54d5 Introduce per-campaign filter permissions. Closes #2325.
This patch introduces new `campaigns:get_all` and `campaigns:manage_all`
permissions which alter the behaviour of the the old `campaigns:get` and
`campaigns:manage` permissions. This is a subtle breaking behavioural change.

Old:

- `campaigns:get` -> View all campaigns irrespective of a user's list
  permissions.
- `campaigns:manage` -> Manage all campaigns irrespective of a user's list
  permissions.

New:

- `campaigns:get_all` -> View all campaigns irrespective of a user's list
  permissions.
- `campaigns:manage_all` -> Manage all campaigns irrespective of a user's list
  permissions.
- `campaigns:get` -> View only the campaigns that have at least one list to
  which which a user has get or manage access.
- `campaigns:manage` -> Manage only the campaigns that have at list one list
  to which a user has get or manage access.

In addition, this patch refactors and cleans up certain permission related
logic and functions.
2025-03-31 16:39:42 +05:30
Kailash Nadh
b8f50eafa3 Add support for domain allowlists in addition to blocklists. Closes #2230.
This patch introduces a new `Domain allowlist` input in Settings -> Privacy UI
as a new tab alongside domain `Domain blocklist`. If any domains are entered
here, then only subscriptions/imports/additions of e-mails from those particular
domains are accepted. blocklist is mutually exclusive with allowlist when there
are values in the allowlist.
2025-03-29 23:31:34 +05:30
Kailash Nadh
e4a18dabc3 Remove forcing unique filename on all media uploads.
This patch removes the forced suffixing of all media upload filenames with
random strings. The upload handler now checks the `media` table to ensure that
the filename being uploaded doesn't exist before forcing a suffix.

Outright rejecting duplicate filenames cannot be done to maintain backwards
compatibility with the old behaviour.

Closes #2277.
2025-03-29 15:21:32 +05:30
Kailash Nadh
63712285aa Fix schema and add migration for incorrect subscriber count (null) in materialized view. 2024-12-04 23:06:49 +05:30
Kailash Nadh
18edc653a8 Add v4.1.0 migrations. 2024-11-11 16:11:56 +05:30
Kailash Nadh
b8ae4f6f9e Change v4.0.0 migration script to not auto-generate credentials. 2024-10-27 17:18:28 +05:30
Kailash Nadh
cea65c009d Fix and refactor subscriber batch fetching in campaign processing.
This has been a hair-pulling rabbit hole of an issue. #1931 and others.
When the `next-campaign-subscribers` query that fetches $n subscribers
per batch for a campaign returns no results, the manager assumes
that the campaign is done and marks as finished.

Marathon debugging revealed fundamental flaws in qyery's logic that
would incorrectly return 0 rows under certain conditions.
- Based on the "layout" of subscribers for eg: a series of blocklisted
  subscribers between confirmed subscribers.
  A series of unconfirmed subscribers in a batch belonging to a double
  opt-in list.
- Bulk import blocklisting users, but not marking their subscriptions
  as 'unsubscribed'.
- Conditions spread across multiple CTEs resulted in returning an
  arbitrary number of rows and $N per batch as the selected $N rows
  would get filtered out elsewhere, possibly even becoming 0.

After fixing this and testing it on our prod instance that has
15 million subscribers and ~70 million subscriptions in the
`subscriber_lists` table, ended up discovered significant inefficiences
in Postgres query planning. When `subscriber_lists` and campaign list IDs
are joined dynamically (CTE or ANY() or any kind of JOIN that involves)
a query, the Postgres query planner is unable to use the right indexes.

After testing dozens of approaches, discovered that statically passing
the values to join on (hardcoding or passing via parametrized $1 vars),
the query uses the right indexes. The difference is staggering.
For the particular scenario on our large prod DB to pull a batch,
~15 seconds vs. ~50ms, a whopping 300x improvement!

This patch splits `next-campaign-subscribers` into two separate queries,
one which fetches campaign metadata and list_ids, whose values are then
passed statically to the next query to fetch subscribers by batch.

In addition, it fixes and refactors broken filtering and counting logic
in `create-campaign` and `next-campaign` queries.

Closes #1931, #1993, #1986.
2024-10-13 17:03:59 +05:30
Kailash Nadh
d9b4bae0a8 Rename migration to v4.0.0 2024-10-13 17:03:59 +05:30
Kailash Nadh
ae2a386193 Add support for "list roles".
This commit splits roles into two, user roles and list roles, both of which
are attached separately to a user.

List roles are collection of lists each with read|write permissions, while
user roles now have all permissions except for per-list ones.

This allows for easier management of roles, eliminating the need to clone and
create new roles just to adjust specific list permissions.
2024-10-13 17:03:58 +05:30
Kailash Nadh
612c1d6eac Add per-list permission management to roles. 2024-10-13 16:59:52 +05:30
Kailash Nadh
32d5823dfe Refactor 'super' user type to a pre-defined super admin role. 2024-10-13 16:59:52 +05:30
Kailash Nadh
d4e4c5fa99 Add granular permissions and role management to backend and admin UI. 2024-10-13 16:59:51 +05:30
Kailash Nadh
a6e06d92f8 Refactor migration for the latest version. 2024-10-13 16:59:51 +05:30
Kailash Nadh
57ac9dca4b Add public login page and auth middleware and handlers. 2024-10-13 16:59:51 +05:30
Kailash Nadh
1516bf216f Add api type user. 2024-10-13 16:59:51 +05:30
Kailash Nadh
0968e58766 Add user/password login handler. 2024-10-13 16:59:51 +05:30
Kailash Nadh
435d6d5169 Add create/add/delete user management UI and database schema. 2024-10-13 16:59:51 +05:30
Kailash Nadh
83e4f5def4 Add migrations for OIDC db fields. 2024-10-13 16:59:51 +05:30
Kailash Nadh
5a3664aee2 Add support for caching slow queries on large databases.
- Add materialized views for list -> subscriber counts, dashboard chart,
  and dashboard aggregate stats that slow down significantly on large
  databases (with millions or tens of millions of subscribers). These
  slow queries involve full table scan COUNTS().

- Add a toggle to enable caching slow results in Settings -> Performance.

- Add support for setting a cron string that crons and periodically
  refreshes aggregated stats in materialized views.

Closes #1019.
2024-01-27 15:51:12 +05:30
Kailash Nadh
0d319ad9fd Add 'slug' (permalink) support for campaign archives. Closes #1394. 2024-01-09 23:34:08 +05:30
Kailash Nadh
3335171960 Fix (accidentally) broken migration scripts. 2024-01-06 19:55:06 +05:30
Kailash Nadh
fa2c840ddb Rename v2.6.0 migration to v3.0.0, the next major release. 2024-01-06 11:56:08 +05:30
Kailash Nadh
8f2a08b8db Fix invalid suffix 'd' in timestring string in s3 expiry config. 2023-09-19 14:45:51 +05:30
Kailash Nadh
2b95c88188
Add Postmark bounce webhook support (refactor #1385) (#1485)
Co-authored-by: Thomas Siebers <tom@tsiebers.de>
2023-08-31 21:27:34 +05:30
Kailash Nadh
ad80c716f9 Add new privacy option 'Record opt-in IP' to record IP address of optin confirmation.
- Add new 'Subscriptions' table on the subscriber list form that shows subs,
  IP, and other data.
- Add new `meta` JSONB field to `subscriber_lsts` table.

Closes #1329.
2023-07-26 23:00:32 +05:30
Kailash Nadh
3b9a0f782e
Add support for file attachments on campaigns (#1341)
- Adds support for arbitrary file uploads with an admin setting to select allowed file extensions.
- Adds support for attaching media (files) to campaigns.
2023-05-18 16:55:59 +05:30
Kailash Nadh
5fc28a733c Add support for variable bounce processing actions.
- Add support for `complaint` to the SES bounce processor.
- Add support for `hard/soft` to Sendgrid bounce processor.
- Add new bounce actions `None` and `Unsubscribe`.
- Add per type (`soft/hard/complaint`) bounce rule configuration to
  admin settings UI.
- Refactor Cypress bounce tests.
2023-04-11 11:33:40 +05:30
Kailash Nadh
13ad9adb8b Upgrade koanf to v2. 2023-04-10 12:45:25 +05:30
Justin Beaty
476d5bebf2
Add support for publishing full content in public archive RSS feed body (#1262)
- Introduces a new option on the settings UI to optionally publish the full campaign body in
  public archive RSS feeds.

Closes #1033 

Co-authored-by: Kailash Nadh <kailash@nadh.in>
2023-04-08 09:39:10 +05:30
Kailash Nadh
8985e5c24a
Add hCaptcha.com support to public subscription form. (#1152)
Bots easily bypass the simple `nonce` hack. This commit adds support
for the hcaptcha.com widget.

- New `Security` tab in the admin settings UI.
- Enable/disable CAPTCHA.
- Render CAPTCHA on the public subscription form.

Closes #1116.
2023-01-23 21:50:10 +05:30
Kailash Nadh
818f2c9d8e Add public archive on/off toggle to settings. 2022-11-10 23:30:53 +05:30
Kailash Nadh
438568eeb0 Add global site name setting to render name on public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh
9add728b08 WIP: Add support for publishing campaigns to publish archives. 2022-11-10 23:30:11 +05:30
Kailash Nadh
ef1f84ee7c Add new description field to lists. Closes #925. 2022-11-01 21:04:35 +05:30
Kailash Nadh
3b0083190e Add ability for subscribers to manage preferences on the unsub form.
- Ability to change name.
- Ability to unsubscribe from individual lists.
- Toggle option to enable this in Admin Settings -> Privacy.

Closes #455.
2022-10-29 15:23:28 +05:30
Kailash Nadh
c38100427d Add arbitrary meta field to media. Closes #938.
- Add new `meta` JSONB field to `media` table.
- Start storing image width and height as meta with media uploads.
2022-10-02 23:04:51 +05:30
Kailash Nadh
bbbf28c5ce Create default tx template on upgrade. 2022-07-30 22:37:05 +05:30
Kailash Nadh
4de5d53fe4 Refactor upgrade schema to remove column default. 2022-07-09 10:36:12 +05:30
Kailash Nadh
d3774d606a Make tx DB upgrade schema consistent with install schema. 2022-07-09 10:36:12 +05:30
Kailash Nadh
463e92d1e1 Add transactional (tx) messaging capability.
This commit adds a new API `POST /api/tx` that sends an ad-hoc message
to a subscriber based on a pre-defined transactional template. This is
a large commit that adds the following:

- New campaign / tx template types on the UI. tx templates have an
  additional subject field.
- New fields `type` and `subject` to the templates table.
- Refactor template CRUD operations and models.
- Refactor template func assignment in manager.
- Add pre-compiled template caching to manager runtime.
- Pre-compile all tx templates into memory on program boot to avoid
  expensive template compilation on ad-hoc tx messages.
2022-07-09 10:36:12 +05:30
Kailash Nadh
64d2c5aeb9 Add support for custom public S3 URLs. Closes #505. 2022-01-15 21:20:32 +05:30