Commit graph

245 commits

Author SHA1 Message Date
Kailash Nadh b2a5e37ccb Fix incorrect form messages on public re-subscription to double opt-in lists. Closes #1638. 2023-12-30 21:29:34 +05:30
Kailash Nadh bce675870c Fix preconfirm option not working on bulk 'select all' subscriber list management. Closes #1646. 2023-12-30 20:52:14 +05:30
Kailash Nadh 0d74619cac Make providing name in subscriber creation optional and assign internally. Closes #1630. 2023-12-30 15:51:42 +05:30
Kailash Nadh 51af75cfef Add subscription status filter to subscriber list query and admin UI. 2023-12-23 16:43:28 +05:30
Kailash Nadh 01acd386f6 Add tag/type/optin filter options to lists and campaigns APIs. Closes #1631. 2023-12-23 14:45:55 +05:30
Kailash Nadh 5d635744c0 Skip non-existent dirs in --static-dir mode to use default stuffed dirs. Closes #1599. 2023-12-23 10:55:46 +05:30
Kailash Nadh aba71119c3 Fix default non-stuffed frontend dir path. 2023-12-23 10:30:37 +05:30
Kailash Nadh 53eb71a83b Add 404 HTTP handlers to prevent those requests going to BasicAuth endpoints. 2023-11-25 11:53:56 +05:30
guangwu 4577868567
chore: remove refs to deprecated io/ioutil (#1593)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-11-16 13:57:00 +05:30
Kailash Nadh a61a9d8c04 Fix preference management logic to avoid unnecessary DB calls. 2023-09-25 11:14:29 +05:30
Kailash Nadh 82c3c6878b Clean root URL of trailing slashes when updating settings. 2023-09-19 15:23:32 +05:30
Kailash Nadh 8f2a08b8db Fix invalid suffix 'd' in timestring string in s3 expiry config. 2023-09-19 14:45:51 +05:30
Kailash Nadh 717a6362d2 Add Sprig and other tpl functions to static templates as well. Closes #1527. 2023-09-19 11:19:34 +05:30
Kailash Nadh 2b95c88188
Add Postmark bounce webhook support (refactor #1385) (#1485)
Co-authored-by: Thomas Siebers <tom@tsiebers.de>
2023-08-31 21:27:34 +05:30
Kailash Nadh a6a2b69820 Make public subscription API follow the 'enable public' setting. 2023-08-15 21:05:29 +05:30
Kailash Nadh 1164afac5e Fix private lists being unsubscribed from public form and re-factor subscription status view in admin. 2023-08-06 21:09:16 +05:30
Kailash Nadh eafae46409 Fix incorrect unsubscription behaviour in the public 'Manage' flow. Closes #1407. 2023-08-06 11:38:13 +05:30
Kailash Nadh 104c4fc993 Remove non-crossplatform syscalls from /api/about.
The only way to get precise OS information cross platform is to have
a separate file for each platform with a build tag that returns the
info, which seems excessive for this usecase.
2023-08-06 10:49:20 +05:30
Kailash Nadh 2215511f2c Increase the arbitrarily low max-input-length limit. Closes #1416. 2023-08-03 23:52:20 +05:30
Kailash Nadh dcb87a39b7 Removed unused syscall in /about API. Closes #1421, closes #1422.
Co-authored-by: Gabriel Vasile <gabriel.vasile@email.com>
2023-08-03 23:46:46 +05:30
Kailash Nadh e89b9ffb30 Remove dead media cleanup flag. Closes #1423. 2023-08-03 23:35:21 +05:30
Kailash Nadh ad80c716f9 Add new privacy option 'Record opt-in IP' to record IP address of optin confirmation.
- Add new 'Subscriptions' table on the subscriber list form that shows subs,
  IP, and other data.
- Add new `meta` JSONB field to `subscriber_lsts` table.

Closes #1329.
2023-07-26 23:00:32 +05:30
Kailash Nadh a62851915c Mask passwords on the UI accurately with the actual passwords length.
This PR masks all the password fields in the UI with a pseudo dot character
retaining the rune length of the original password so that the password
fields on the UI appear to be containing the entered value as-is.

The earlier implementation would revert to a fixed length dummy password
confusing certain users and making it look like the password they entered
wasn't being saved.
2023-07-21 23:46:46 +05:30
Kailash Nadh c581fe2f3a Add GET /api/about that returns useful system info. Closes #1354. 2023-06-24 13:07:13 +05:30
cui fliter f94c1f34b6
fix function name in comment (#1374)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-06-20 23:40:13 +05:30
Kailash Nadh 0b2da4c664 Add support for streaming async events via HTTP serverside events.
- `GET /api/events?type=error` opens a long-lived HTTP server side
  event connection that streams error messages.
- async (typically SMTP) errors are now streamed to the frontend and
  disaplyed as an error toast on the admin UI.
2023-05-27 15:44:59 +05:30
Kailash Nadh d359ad27aa Refactor media management.
- Change tiled UI to table UI.
- Add support for search and pagination.
- Important: This breaks the `GET /api/media` API to introduce pagination
  fields. Media items are now moved into `{ data: results[] }`.
2023-05-21 15:19:12 +05:30
Kailash Nadh 3b9a0f782e
Add support for file attachments on campaigns (#1341)
- Adds support for arbitrary file uploads with an admin setting to select allowed file extensions.
- Adds support for attaching media (files) to campaigns.
2023-05-18 16:55:59 +05:30
Kailash Nadh 917696a543 Refactor Messenger interface.
Remove `messenger.go` and move the interface definition to `manager`
and the `Message` struct to the `models` package, removing superflous
and redundant message structs used in multiple places.
2023-05-08 22:43:25 +05:30
Kailash Nadh 5fc28a733c Add support for variable bounce processing actions.
- Add support for `complaint` to the SES bounce processor.
- Add support for `hard/soft` to Sendgrid bounce processor.
- Add new bounce actions `None` and `Unsubscribe`.
- Add per type (`soft/hard/complaint`) bounce rule configuration to
  admin settings UI.
- Refactor Cypress bounce tests.
2023-04-11 11:33:40 +05:30
Kailash Nadh 13ad9adb8b Upgrade koanf to v2. 2023-04-10 12:45:25 +05:30
Kailash Nadh 98729f63df Refactor email domain blocklist config field in importer package. 2023-04-09 14:04:02 +05:30
Justin Beaty 476d5bebf2
Add support for publishing full content in public archive RSS feed body (#1262)
- Introduces a new option on the settings UI to optionally publish the full campaign body in
  public archive RSS feeds.

Closes #1033 

Co-authored-by: Kailash Nadh <kailash@nadh.in>
2023-04-08 09:39:10 +05:30
Justin Beaty a95510260d
Add functions to notification templates (#1263) 2023-03-26 12:00:35 +05:30
Kailash Nadh 5aedc3a4f6 Make media upload file extension validation case insensitive. Closes #1256. 2023-03-25 11:46:23 +05:30
Kailash Nadh 55f7eca2e8
Add support for file attachments in the transactional (tx) API. (#1243)
The original PR accepts files to the `/tx` endpoints as Base64 encoded
strings in the JSON payload. This isn't ideal as the payload size
increase caused by Base64 for larger files can be significant,
in addition to the added clientside API complexity.

This PR adds supports for multipart form posts to `/tx` where the
JSON data (name: `data`) and multiple files can be posted simultaenously
(one or more `file` fields).

--- PR: #1166
* Attachment model for TxMessage
* Don't reassign values, just pass the manager.Messgage
* Read attachment info from API; create attachment Header
* Refactor tx attachments to use multipart form files. Closes #1166.
---

Co-authored-by: MatiSSL <matiss.lidaka@nic.lv>
2023-03-19 15:50:44 +05:30
Kailash Nadh 6cf82347bf Add support for SVG files to media. Closes #1217.
- Add support for SVG in media uploader.
- Add provision to exclude vector formats from thumbnail creation.
- Increase default thumb size to 120px from 90px.

Co-authored-by: Ronan <ronan.le_meillat@sctg.eu.org>
2023-03-19 13:58:41 +05:30
Kailash Nadh dda7d44601 Hide private lists from prefs manage page. Closes #1200. 2023-02-26 12:36:46 +05:30
Ronan LE MEILLAT 80592f60c6
Translate the subject of the email sent with personnal datas (#1193) 2023-02-20 20:33:44 +05:30
Kailash Nadh 8985e5c24a
Add hCaptcha.com support to public subscription form. (#1152)
Bots easily bypass the simple `nonce` hack. This commit adds support
for the hcaptcha.com widget.

- New `Security` tab in the admin settings UI.
- Enable/disable CAPTCHA.
- Render CAPTCHA on the public subscription form.

Closes #1116.
2023-01-23 21:50:10 +05:30
Kailash Nadh 62d3782d04 Use send_at date for scheduled campaigns in RSS feed. Closes #1149. 2023-01-21 12:43:30 +05:30
Kailash Nadh 3cfbc646e3 Add support for multiple subscribers in a single transactional message call.
This patch adds new array fields on `POST /tx`: `subscriber_emails[]`, `subscriber_ids[]`.
Either of these array fields can be sent with multiple subscribers.

The individual non-array fields `subscriber_id` and `subscriber_email` are deprecated.

Closes #994.
2022-12-25 17:32:01 +05:30
Kailash Nadh 5d4f1ea0ad Add optional params in DB config to accept arbitrary Postgres params. Closes #1016. 2022-12-25 16:28:19 +05:30
Kailash Nadh 1f693b80f7 Add a default public archive template. 2022-12-25 16:07:12 +05:30
Kailash Nadh 4dee2e9a1b Fix per_page=all in API calls in the paginator lib.
Closes #1098.
2022-12-25 00:43:58 +05:30
Kailash Nadh 8d4a5751d8 Fix broken single list fetch API. 2022-11-22 23:11:20 +05:30
Kailash Nadh 1e90feecaf Show send_at on archive page for scheduled campaigns. Closes #1036. 2022-11-18 23:12:53 +05:30
Kailash Nadh 818f2c9d8e Add public archive on/off toggle to settings. 2022-11-10 23:30:53 +05:30
Kailash Nadh f958f3d24b Add RSS feed to the public mailing list archive. 2022-11-10 23:30:53 +05:30
Kailash Nadh 438568eeb0 Add global site name setting to render name on public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh eac1240437 Exclude opt-in campaign from public archive. 2022-11-10 23:30:53 +05:30
Kailash Nadh 23fb178ec4 Add subscription/archive links to public pages. 2022-11-10 23:30:53 +05:30
Kailash Nadh 56a9836e86 Integrate paginator library in place of custom pagination function. 2022-11-10 23:30:50 +05:30
Kailash Nadh 9add728b08 WIP: Add support for publishing campaigns to publish archives. 2022-11-10 23:30:11 +05:30
Kailash Nadh ef1f84ee7c Add new description field to lists. Closes #925. 2022-11-01 21:04:35 +05:30
Kailash Nadh 3b0083190e Add ability for subscribers to manage preferences on the unsub form.
- Ability to change name.
- Ability to unsubscribe from individual lists.
- Toggle option to enable this in Admin Settings -> Privacy.

Closes #455.
2022-10-29 15:23:28 +05:30
Kailash Nadh 281c47198c Fix go-for-loop reference bug in template caching. 2022-10-21 17:59:56 +05:30
Kailash Nadh c38100427d Add arbitrary meta field to media. Closes #938.
- Add new `meta` JSONB field to `media` table.
- Start storing image width and height as meta with media uploads.
2022-10-02 23:04:51 +05:30
Kailash Nadh c3d04a5490 Refactor models.SubscriberAttribs JSON wrapper to generic name JSON. 2022-10-02 22:53:38 +05:30
Kailash Nadh edb4c9168d Improve HTML check in notif template loading. Closes #903. 2022-09-10 12:21:53 +05:30
Kailash Nadh f266f93cc8 Add Safe() template function to notif templates. 2022-09-10 12:20:27 +05:30
Kailash Nadh 5a9e9209c8 Fix missing list names on optin page. Closes #940. 2022-09-10 12:17:01 +05:30
Kailash Nadh 6d820f4f6e Add maintenance options.
- Add new maintenance UI with options to garbage collect (delete)
  orphan subscriber and analytics records.
2022-09-03 14:58:25 +05:30
Kailash Nadh 8ace25849e Add /api/public/* endpoints.
- Add `/api/public/lists` that returns the list of public lists, same
  information revealed on the `/subscription/form` page.

- Add `/api/public/subscription` that accepts a JSON POST for a
  subscription signup same as `/subscription/form`.

Closes #910.
2022-08-28 15:12:20 +05:30
Kailash Nadh b497f52ae7 Merge branch 'fix-analytics' 2022-07-30 20:12:11 +05:30
Kailash Nadh bfc27def57 Fix regression of public subscriber page behaviour. 2022-07-30 20:11:59 +05:30
Kailash Nadh 3550d5453d Fix incorrect analytics count. Closes #712. 2022-07-30 19:01:20 +05:30
Kailash Nadh 77bc8a7745 Send full media object in upload API response. Closes #770. 2022-07-13 19:48:09 +05:30
Kailash Nadh 278d5bf74e Merge branch 'test-smtp' 2022-07-11 19:46:03 +05:30
Kailash Nadh ee448170ef Add support for testing SMTP connections in the settings UI. 2022-07-11 19:44:44 +05:30
Kailash Nadh e99c8ed86b Disable template type updation after creation to prevent breaking of campaign relations. 2022-07-09 10:36:12 +05:30
Kailash Nadh 5a5caca256 Refactor campaign/template preview functions and component. 2022-07-09 10:36:12 +05:30
Kailash Nadh 0574a1b820 Fix template compilation check on CRUD. 2022-07-09 10:36:12 +05:30
Kailash Nadh 3f5a50fc0d Fix header processing in tx send. 2022-07-09 10:36:12 +05:30
Kailash Nadh 68da86aadc Fix redundant echo/http error wrapping. 2022-07-09 10:36:12 +05:30
Kailash Nadh 4a6e041ca8 Don't break boot on tx template compilation errors. 2022-07-09 10:36:12 +05:30
Kailash Nadh 463e92d1e1 Add transactional (tx) messaging capability.
This commit adds a new API `POST /api/tx` that sends an ad-hoc message
to a subscriber based on a pre-defined transactional template. This is
a large commit that adds the following:

- New campaign / tx template types on the UI. tx templates have an
  additional subject field.
- New fields `type` and `subject` to the templates table.
- Refactor template CRUD operations and models.
- Refactor template func assignment in manager.
- Add pre-compiled template caching to manager runtime.
- Pre-compile all tx templates into memory on program boot to avoid
  expensive template compilation on ad-hoc tx messages.
2022-07-09 10:36:12 +05:30
Kailash Nadh 59c9441b3b Fix subscriber create query to not ignore duplicate e-mail error.
Trying to insert a pre-existing e-mail on POST /api/subscribers
now return a 409 Conflict error.

Closes #718
2022-05-11 21:40:31 +05:30
Kailash Nadh b94da621d7 Fix broken public link redirect. 2022-05-05 18:05:13 +05:30
Kailash Nadh 5fd4d7b44b Refactor paginated bounce query function to return DB total. 2022-05-03 10:50:33 +05:30
Kailash Nadh d2ef23d3fa Refactor paginated campaign query function to return DB total. 2022-05-03 10:50:33 +05:30
Kailash Nadh e303850584 Refactor paginated list query function to return DB total. 2022-05-03 10:50:33 +05:30
Kailash Nadh aa19771307 Refactor bounces package to remove db/queries dependency.
Instead of passing a DB/SQL statement references, instead pass a
callback that inserts a bounce into the DB via the `core` package.
2022-05-03 10:50:33 +05:30
Kailash Nadh b5cd9498b1 Refactore all CRUD functions to a new core package.
This is a long pending refactor. All the DB, query, CRUD, and related
logic scattered across HTTP handlers are now moved into a central
`core` package with clean, abstracted methods, decoupling HTTP
handlers from executing direct DB queries and other business logic.

eg: `core.CreateList()`, `core.GetLists()` etc.

- Remove obsolete subscriber methods.
- Move optin hook queries to core.
- Move campaign methods to `core`.
- Move all campaign methods to `core`.
- Move public page functions to `core`.
- Move all template functions to `core`.
- Move media and settings function to `core`.
- Move handler middleware functions to `core`.
- Move all bounce functions to `core`.
- Move all dashboard functions to `core`.
- Fix GetLists() not honouring type
- Fix unwrapped JSON responses.
- Clean up obsolete pre-core util function.
- Replace SQL array null check with cardinality check.
- Fix missing validations in `core` queries.
- Remove superfluous deps on internal `subimporter`.
- Add dashboard functions to `core`.
- Fix broken domain ban check.
- Fix broken subscriber check middleware.
- Remove redundant error handling.
- Remove obsolete functions.
- Remove obsolete structs.
- Remove obsolete queries and DB functions.
- Document the `core` package.
2022-05-03 10:50:29 +05:30
Kailash Nadh 12b845ef97 Fix incorrect HTTP resp code on public page. Fixes #772. 2022-05-03 10:49:25 +05:30
Yatish Mehta 26483334c5 Fix typos 2022-03-20 11:17:29 -07:00
Kailash Nadh 4e6410ec17 Fix list_ids not being considered in bulk list change on the UI. Fixes #737. 2022-03-20 11:02:43 +05:30
Kailash Nadh ef643a14a3 Add ability to export select subscriber ids.
- Add `id=[]` query param to `/api/subscribers/export` API.
- Add UI export prompt.
- Add Cypress tests.

Closes #739
2022-03-19 13:44:23 +05:30
Gokul Menon 04c4552a9c Fixed typos 2022-02-28 14:19:50 +01:00
Yatish Mehta 6c903239dd Fix typos 2022-02-13 08:54:39 -08:00
Kailash Nadh 0f6a0376da Add accurate realtime message rate counter.
The `rate` field `/api/campaigns/running/stats` returned was computed
based on the total time spent from the start of the campaign to the
current time. This meant that for large campaigns, if there were
pauses or slowdowns in between, the rate would be skewed heavily
making it useless to figure out the current send rate.

This commit introduces a realtime running rate counter in the campaign
manager that returns accurate (running) send rates for the last minute.

The `rate` field in the API now shows the live running rate and a
new `net_rate` field shows the rate from the beginning of the campaign.
2022-02-06 11:38:02 +05:30
Kailash Nadh 48ef3dcb14 Support status in bulk subscriber list update API. Closes #604. 2022-02-05 00:03:28 +05:30
Kailash Nadh da30d4688e Add subscriber status counts to the lists UI.
- Change `query-lists` query to aggregate the subscriber count by
  status (confirmed, unsubscribed etc.) and expose them under a new
  `subscriber_statuses: {}` field in the `GET /lists` API.
- Display the statuses and counts in the lists table on the UI.

Closes #616
2022-02-03 00:03:31 +05:30
Kailash Nadh 2614b072f2 Refactor campaign analytics to show unique / non-unique data.
The analytics page showed non-unique counts for views and clicks which
was misleading and source of confusion: #522, #561, #571, #676, #680
This commit changes this behaviour to pull unique views and clicks when
individual subscriber tracking is turned on in settings, and non-unique
counts when it is turned off (as `subscriber_id` in `campaign_views`
and `link_clicks` will be NULL, rendering unique queries dysfunctional).

This commit changes the stats SQL queries to use string interpolation
to either to SELECT `*` or `DISTINCT subscriber_id` on app boot based
on the setting in the DB. This involves significant changes to how
queries are read and prepared on init.

- Refactor `initQueries()` to `readQueries()` and `prepareQueries()`.
- Read queries first before preparing.
- Load settings from the DB using the read settings query.
- Prepare queries next. Use the privacy setting from the DB to apply
  string interpolation to the analytics queries to pull
  unique/non-unique before preparing the queries.

On the UI:
- Show a note on the analytics page about unique/non-unique counts.
- Hide the % donut charts on the analytics page in non-unique mode.

Closes #676, closes #680
2022-02-01 23:40:03 +05:30
Kailash Nadh 64d2c5aeb9 Add support for custom public S3 URLs. Closes #505. 2022-01-15 21:20:32 +05:30
Kailash Nadh 3f026090ca Add unsubscribe link to opt-in confirmation e-mail. Closes #573. 2022-01-15 16:41:48 +05:30
Kailash Nadh 28efe27cbe Merge branch 'master' of github.com:knadh/listmonk 2022-01-15 15:55:52 +05:30
Kailash Nadh 04ea18c87d Refactor opt-in confirmation behaviour in subscriber update API.
- Updating a subscriber no longer triggers an opt-in confirmation mail
  as `POST /api/subscribers/:id/optin` allows that.
- A "Send opt-in confirmation" option is added to the subscriber
  update UI.

Closes #656.
2022-01-15 15:50:13 +05:30
Kaustubh Maske Patil be1d048e7b Replace whitespace with dash in names of uploaded files 2022-01-15 13:23:58 +05:30
Kailash Nadh 583dab4bc6 Add support for per-campaign custom headers.
- Add new `headers[]` column to the campain table.
- Add new headers box to the campaign UI that takes a JSON array of
  custom headers like the headers on the SMTP settings UI.
- Headers are added to e-mails and messenger postback webhooks.
- Add cypress tests.

Closes #514.
2022-01-04 22:27:40 +05:30