Commit graph

491 commits

Author SHA1 Message Date
Felipe Martin
888d053b2d
Allow JWT authentication into legacy APIs (#651)
* typo: letter in login page

* httpconfig set defaults for secret key with warn

* allow new authentication in old api

* Updated warn log
2023-07-21 07:57:42 +02:00
Felipe Martin
454f21789d
fix: docker buildx tags (#650) 2023-07-19 18:50:57 +02:00
Felipe Martin
a60dbf3dc5
Run legacy API and new API at the same time. (#648)
* config: backwards comptabile dir

* remove duplicated frontend

* frontend: move assets to assets folder

* legacy routes handler in gin

* templates and asset in different embed

* new routes

* frontend routes serve old views

* added DTO for account object

* api auth calls legacy handler

* frontend: handle new error messages

* frontend: update urls

* frontend: login using new api

* updated frontend tests

* chore: remove debug route

* create shiori/gopher user if no owner is present

* server as default command

* serve -> server

* refactored database logic, allow database url

* removed unused configuration

* storage docs

* refactor cli to use cfg and deps

* check errors only in server

* log fatal instead of os exit

* dont default data directory to current dir

* fixed sqlite path

* trigger build on prs

* avoid releasing if lint/test fails

* pull request condition

* event -> event_name

* Get correct pull request number

* added workflow to delete dangling tags

* fix: nil error checking

* set gin mode first

* set gin mode before initialization

* fix logger

* allow version bump from custom ref

* Updated matrix link to workspace
2023-07-19 18:25:41 +02:00
Felipe Martin
6f19c12c95
Start working on new REST API. Refactor logic in domains. (#497)
* added 404 template

* added auth domain

* added embed file for frontend

* added base config and dependencies

* added basic new http server

* added separated server command

* updated go modules

* removed modd file

* Added shortcut to send internal server error response

* Added JWT support to Auth Domain

* Added JWT support to API

* docs: added comments to response struct

* naming

* inline returns

* updated dependencies

* production logger

* bookmarks endpoint

* reverted old views api path

* frontend for api v1

* proper 404 error (not working atm)

* use response

* removed 404 html

* server error handler

* login and basic auth

* adjusted session duration

* properly retrieve tags

* properly delete bookmark

* cleanup

* archiver domain

* debug routes

* bookmark routes

* expiration by parameter

* move to logrus

* logout

* frontend cache

* updated dependencies

* http: migrated to gin

* linted

* Added version command

* unit tests, docs

* response test utils and tests

* remove logout handler

* auth

* createtag

* improved http test utilities

* assert message equals

* Remove 1.19 from test matrix

* moved api to v1 folder

* docs: contribute docs

* updated makefile

* updated usage docs

* warn in server command

* updaed docs with shiori version command

* Updated documentation

* deps: update
2023-07-17 14:30:18 +01:00
Monirzadeh
78a7578f5c
fix download data image fix #645 (#646) 2023-07-15 19:04:14 +01:00
Ryan Gonzalez
e02d3e7739
feat: add SHIORI_PG_SSLMODE (#536)
* feat: add SHIORI_PG_SSLMODE

Some managed hosting providers don't support sslmode=disabled.

* handle empty value

---------

Co-authored-by: Felipe M <me@fmartingr.com>
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-09 08:43:26 +02:00
Felipe Martin
6ab661520e
fix: prevent bookmark overwrite on download (#636) 2023-07-09 08:33:56 +02:00
Monirzadeh
249f4b89c8
Initial Ebook Support (#623)
* generate ebook

* serve ebook file

* Update ebook.go not download same image twice anymore

* download ebook request api/ui part

* fix typo

* add stylesheet

* update hasEbook status

* download link update after ebook generate

update bookmark command in ui

* download ebook with bookmark title

* Apply suggestions from code review for better error handling

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* Update internal/view/js/page/home.js fix typo

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* import error lib and retuen missing error

* move ebook download action to update cache

* replace io/ioutil with io

* add missing error handling

* update Archive now always update ebook

* replace panic error with 404

* remove ebook with delete action

* add download ebook link to content page

* remove tags that not work correctly right now

* if file is pdf not generate ebook

* update style.css

* Revert "update style.css"

This reverts commit 519e10d6ce.

* remove download limit for api

* fix missing fmt.Errorf and change to errors.Wrap

* fix double panic

* return 404 if bookmark not exist

* change function name to GenerateEbook

* not isFatalErr anymore

* add unit test

* remove uneeded field for unit test

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-09 08:29:32 +02:00
Felipe M
ec86febdaa
docs: add matrix link to readme 2023-07-06 21:47:08 +02:00
Felipe M
e839b69875
fix: goreleaser use go version from go.mod file and fix typo 2023-07-02 13:02:46 +02:00
Felipe Martin
8e9324ae8b
deps: update to go 1.20 + dependencies. Fixes CI (#628)
* deps: update go1.20 and dependencies

* fix: goreleaser archives

* ci: removed unused notify irc action

* ci: updated workflow dependency versions

* fix: typo
2023-07-02 12:50:29 +02:00
Monirzadeh
98c2d77257
thumb image will be not accesable anymore if it is private fix #578 (#627)
* thumb image will be not accesable anymore if it is private fix #578

* Update internal/webserver/handler-ui.go better error handling

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-02 10:58:57 +02:00
Tomi lla
dde1b44e77
fix(log): record user real ip from headers (#603)
* fix(real_ip): get user real ip from headers of request

* fix(real_ip): compatible with those header with multiple IP values separated by commas

* test(real_ip): add benchmark for IPv4 and IPv6 private address check

* fix(real_ip): check empty, then remove leading and tailing comma char, finally locate first IP field

* test(real_ip): move checker logic into utils and add more unit test cases

* test(real_ip): write unit tests covering all code branches of the `util-ip` module

* refactor(real_ip): use one-line `testify.assert.Panics` to capture intended panic in test case

* chore(real_ip): add module private variable `UserRealIpHeaderCandidates`

put those headers together, make it easy to manage in one place

* doc(real_ip): write docstring for each function in the `utils-ip` module

* chore(real_ip): choose more concrete and unambiguous name for test helper function

It is to avoid polluting the module name-space with too general names.

* chore(naming): change function names according to code style

* refactor(real_ip): simplify the code indicated by 'gosimple' and `golangci`

* chore(naming): rename the `utils-ip` file to `utils_ip` to match with the rest of the file structure

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-06-11 21:25:23 +02:00
Monirzadeh
a4b92504b0
fix direction in RTL language (#615)
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-06-11 21:18:12 +02:00
Felipe Martin
85534b62f8
Update dependencies (#616)
* Added contribute docs for tests
* deps: update
* mariadb using healthcheck script
* Restored health retries to 5
2023-06-11 21:10:15 +02:00
chain710
888e59dabf
Add log args for root command (#565)
- log-level: specify log level, default info
- log-caller: whether log `file:lineno` or not, default false
- log FullTimestamp
2023-03-17 13:37:23 +01:00
sdvcrx
3c1229d6ce
fix(frontend): prevent archive header covering up the archive content (#562) 2023-02-22 13:05:55 +01:00
Felipe Martin
f6f3faf130
deps: update direct deps (#567)
* deps: update direct deps

* test on 1.20
2023-02-05 11:19:53 +01:00
Ivan Li
47f288fb1d
fix: browser ext cannot archive and update bookmark. (#560) 2023-02-05 10:58:00 +01:00
mox46
52d9a6322f
use UTC for cookie expiry date in backend (#566) 2023-02-05 10:56:34 +01:00
Katarn
257506ae19
feat: improve bookmark image detection (#554) 2023-01-21 09:53:19 +01:00
Felipe Martin
9344bbeb27
ci: fix goreleaser snapshots (#553) 2023-01-10 13:05:00 +01:00
Felipe Martin
6b99e1910c
deps: up to date (#552) 2023-01-10 12:53:29 +01:00
dependabot[bot]
3e1d3cd3aa
chore(deps): bump modernc.org/sqlite from 1.19.5 to 1.20.2 (#551)
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 12:47:06 +01:00
dependabot[bot]
04a79dcf84
chore(deps): bump goreleaser/goreleaser-action from 3 to 4 (#544)
Bumps [goreleaser/goreleaser-action](https://github.com/goreleaser/goreleaser-action) from 3 to 4.
- [Release notes](https://github.com/goreleaser/goreleaser-action/releases)
- [Commits](https://github.com/goreleaser/goreleaser-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: goreleaser/goreleaser-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 12:46:48 +01:00
dependabot[bot]
e5846ca73b
chore(deps): bump github.com/go-sql-driver/mysql from 1.6.0 to 1.7.0 (#541)
Bumps [github.com/go-sql-driver/mysql](https://github.com/go-sql-driver/mysql) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/go-sql-driver/mysql/releases)
- [Changelog](https://github.com/go-sql-driver/mysql/blob/master/CHANGELOG.md)
- [Commits](https://github.com/go-sql-driver/mysql/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/go-sql-driver/mysql
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-10 12:46:38 +01:00
Ryan Welch
1b03cc48d6
Reset page when searching for new tag (#550) 2023-01-10 12:46:13 +01:00
dependabot[bot]
27c2fc74be
chore(deps): bump modernc.org/sqlite from 1.19.4 to 1.19.5 (#539)
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-30 12:12:55 +01:00
Felipe Martin
3fcb295d72
Make migrations transparent to the user (#530)
* refactor: remove migrate command

* reafactor: avoid migration errors on no changes
2022-11-27 15:39:27 +01:00
dependabot[bot]
54fc9399b3
chore(deps): bump github.com/gofrs/uuid (#531)
Bumps [github.com/gofrs/uuid](https://github.com/gofrs/uuid) from 4.3.0+incompatible to 4.3.1+incompatible.
- [Release notes](https://github.com/gofrs/uuid/releases)
- [Commits](https://github.com/gofrs/uuid/compare/v4.3.0...v4.3.1)

---
updated-dependencies:
- dependency-name: github.com/gofrs/uuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-10 08:52:45 +01:00
dependabot[bot]
48fb6f6395
chore(deps): bump modernc.org/sqlite from 1.19.3 to 1.19.4 (#532)
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-10 08:52:33 +01:00
dependabot[bot]
9dac4bf1dc
chore(deps): bump modernc.org/sqlite from 1.19.2 to 1.19.3 (#529)
---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-04 21:26:01 +01:00
dependabot[bot]
278b4ce48d
chore(deps): bump github.com/spf13/cobra from 1.6.0 to 1.6.1 (#528)
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-04 21:25:49 +01:00
dependabot[bot]
23e87f8430
chore(deps): bump actions/download-artifact from 2 to 3 (#523)
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-28 09:10:49 +02:00
dependabot[bot]
8efbc94007
chore(deps): bump actions/setup-go from 2 to 3 (#522)
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 2 to 3.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-28 09:10:31 +02:00
dependabot[bot]
00ef531be8
chore(deps): bump modernc.org/sqlite from 1.19.1 to 1.19.2 (#521)
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.19.1 to 1.19.2.
- [Release notes](https://gitlab.com/cznic/sqlite/tags)
- [Commits](https://gitlab.com/cznic/sqlite/compare/v1.19.1...v1.19.2)

---
updated-dependencies:
- dependency-name: modernc.org/sqlite
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-28 09:09:45 +02:00
Abe
b691abe704
Remove typo in post-run command (#527)
This should be `serve` not `server` 

server was causing it to freak out about the typo.
2022-10-28 09:03:48 +02:00
Felipe Martin
77269b7a64
feat(frontend): username autofocus on login (#519) 2022-10-16 12:17:02 +02:00
Felipe Martin
c86cf12638
fix: remove createnewid usages (#520)
* remove CreateNewID usages

* remove CreateNewID implementations and definition

* added missing sqlite envvar to compose file
2022-10-15 23:01:52 +02:00
Felipe Martin
d1f0ce8dbb
fix(api-ext): use same save logic as the api (#518)
* switched session priority to header then cookie

* fix(db): GetBookmarks handle no rows error

* fix(api-ext): using same save logic as the api
2022-10-15 14:16:12 +02:00
Felipe Martin
5f1adc6c34
fix(db): avoid using dash in fts5 queries (#514)
* fix(sqlite): avoid using dash in fts5 queries

* test(db): get bookmarks and count with dash in keyword

* test(db): common and ad-hoc sqlite test case
2022-10-15 11:33:29 +02:00
Felipe Martin Garcia
4de21eaf40
fix: avoid panic usage when downloading bookmark (#513)
Removed the usage of `panic()` when downloading a bookmark and simply
return an error that has to be checked by implementations.

Right now the API will continue if the bookmark download fails (either
sync or async) but will leave a log with the error cause, so the users
have the bookmark stored even if the archival didn't actually happen
(but can be done manually later on).

Fixes #459
2022-10-14 13:37:24 +02:00
Acelya
ed5a3bcbb9
fix: failed to save bookmark: constraint failed: UNIQUE constraint failed: bookmark.url (#515)
After a first bookmark save, all subsequent ones fail because the update
query does not include an identifier and the update is done on all rows.

Introduced by 05fee53bd0
2022-10-13 19:18:40 +02:00
Felipe Martin Garcia
4ea81a446a
chore(deps): updated go and github actions dependencies (#511)
* deps(go): updated dependencies

* deps(github_actions): updated dependencies
2022-10-12 00:10:55 +02:00
Felipe Martin Garcia
20b4efa49b
Added dependabot configuration 2022-10-11 23:55:18 +02:00
Felipe Martin Garcia
05fee53bd0
fix: saving bookmarks inconsistencies (#500)
* chore: updated go-migrate dependencies

* fix: specify if we're saving bookmarks expecting a creation

up until now the SaveBookmarks method was doing some "magic" to do
"upserts" on the databases, but consistency between engines was scarce
and not knowing if we were expecting saving a new bookmark or updating
an existing one was leading to errors and inconsistencies in logic all
around the place. Now we need to specify a creation boolean when
saving and a differnt query will be make (INSERT vs UPDATE).

* fix(api): using incorrect bookmark for content downlaod

* test(db): added test pipeline for databases

Added functions that will share logic among the engines and will be
called on fresh databases on each test run

* dev: added basic docker-compose for development

* chore: uncommented tests

* ci(test): added mysql service

* typo

* test(mysql): select database after reset

* fix(mysql): ignore empty row errors when parsing tags

* fix(mysql): handle insert errors

* chore: added mysql variables to compose

* ci: explicit mysql service port exposed
2022-10-11 23:47:38 +02:00
Acelya
040dc5c5d1
fix: failed to save bookmark: context canceled (#499)
When the HTTP request completes, the asynchronous bookmarks save may be incomplete.
In this case, the shared context is canceled and saving bookmarks fails.

Introduced by 09f2465065
2022-10-11 18:08:21 +02:00
Felipe Martin Garcia
222fc14fe2
fix: remove CreateNewID usage from api handlers (#486) 2022-10-09 18:56:12 +02:00
Felipe Martin Garcia
158c52a325
fix: data dir permissions having execute for all (#493) 2022-10-09 18:50:42 +02:00
Felipe Martin Garcia
d0210447d2
fix: import should respect add_date or last_modified fields (#490)
* fix: handle last_modified and add_date in imports

* refactor: moved date format to a constant

* fix: use provided modified date when set
2022-10-09 17:05:52 +02:00