Commit graph

230 commits

Author SHA1 Message Date
Felipe Martin
9c7483fd09
fix: override configuration from flags only if set (#865)
* fix: override configuration from flags only if set

* use helper func and test it
2024-03-30 08:33:05 +01:00
Zlendy
5a166906db
fix: use correct endpoint in rename tags operation (#868) 2024-03-25 23:54:55 +01:00
Felipe Martin
1ee762a161
fix: fix problems that may be raised from a 1.5.5 upgrade (#837)
* add missing flag to deprecated serve command

* docs references to initial account

* modify shiori serve references

* update dependencies

* remove serve from shiori -h docs output

* remove migrated routes

* fix faq heading levels

* downgrade go-epub to 1.2.0
2024-02-08 09:26:33 +01:00
Monirzadeh
82aa1e5080
fix: fixes path issues on windows (#829)
* return corrent path in windows with usiing path lib

* ci: run tests in windows and macos too

* avoid testing mysql/psql in windows and macos

* windows & macos matrix

* disable gotestfmt

* replace which in shell script

* handle error in test db removal

* fix expected path baseed on platform

* add leading seprator

* proper temporary storage dir and db cleanup

* fix failed to create destination dir file does not exist in windows

* move temp to /tmp

* update temp folder

* fix config tests in windows

* apply patch for db

* revert temp dir creation.

* unify account db tests pach

* remove TmpDir for sqlite tests

* try to force CGO disable with enviroment variable

* Remove unneeded log

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* remove unneeded comment

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* fix file path for download images

* change way to create temp directory

* use diffrent file name for each test

* fix typo

* fix absolute path in successful download image

* correct filename with png

* change test to download image from internet instead of local machine

* remvoe unneeded import

* remove os.RemoveAll(.env)

* unify variable names in unit test

* return CGO_ENABLED=0

* test other way to set enviroment variable

* try to set enviroment variable sepratly in macos and windows

* set enviroment variable before run commands in windows

* fix windows test name

* combine two workflow for windows and macos again

* fix typo

* remove env

* change env path

* cleanup unneeded env

* general CGO_ENABLED environ

* use absolute path to run fileserver instead of relative

* serve file test from internet shiori repository

* check file existance after download and unify varibale name from temp to tmpDir

* remove unneeded log

---------

Co-authored-by: Felipe M <me@fmartingr.com>
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2024-02-05 10:21:23 +01:00
Felipe Martin
7c13626a5b
fix: strict usage of shiori prefix for environment variables in configuration (#807)
* fix: disable direct os lookuper

* config.setdefaults call config.http.setdefaults

* tests

* log level default in local run server

* store log level in configuration
2023-12-30 00:46:29 +01:00
Felipe Martin
cf7301c78f
Fixed lint errors after refactor (#806)
* removed git diff from swagger check script

* updated swagger docs

* removed ununsed calls
2023-12-28 18:37:19 +01:00
Felipe Martin
cc7c75116d
refactor: migrate bookmark static pages to new http server (#775)
* migrate bookmark content route to new http server

* new archive page

* remove unused go generate comment

* database mock

* utils cleanup

* unused var

* domains refactor and tests

* fixed secret key type

* redirect to login on ui errors

* fixed archive folder with storage domain

* webroot documentation

* some bookmark route tests

* fixed error in bookmark domain for non existant bookmarks

* centralice errors

* add coverage data to unittests

* added tests, refactor storage to use afero

* removed mock to avoid increasing complexity

* using deps to copy files around

* remove config usage (to deps)

* remove handler-ui file
2023-12-28 18:18:32 +01:00
istiak101
f01f2dbc0d
fix: use noreferer to prevent exposing shiori instance url to archived websites (#802) 2023-12-27 11:59:16 +01:00
Monirzadeh
d371ee6037
fix: typo in URL under documentation (#778) 2023-11-17 10:49:47 +01:00
cbe
7765e5bf42
chore: frontend formatting (#764)
* Install `prettier`

Ignore everything in the first step as I want to see the size of changes
that are going to happen first

* Configure prettier

- use tabs as the whole project prefers tabs
- ignore a bunch of files that shouldn't be formatted

* Check & format with prettier when compiling less

* Update docs to use bun

Since bun handles the dependencies used for the CSS compilation this
should be the preferred method from now on.

* Check actual result of prettier

* Use LF line endings

All files where formatted in LF and since it's most common in git repos
we should probably keep it this way

* Limit prettier to `internal/view/`

See pull request review
https://github.com/go-shiori/shiori/pull/764#pullrequestreview-1702935417

* Format code with prettier
2023-11-05 11:02:31 +01:00
Felipe M.
1d58455b53
chore: comment unused functions 2023-10-29 15:42:57 +01:00
cbe
f16dd42e10
Make suggestion clickable in bookmark creation (#765)
This was especially noticeable when using the web app on mobile devices
as the suggestion looks like it's clickable. It should be also a little
bit more convenient UX-wise when it's allowed to be clicked in non-
mobile context as well.

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-29 15:26:39 +01:00
Monirzadeh
5e9847a288
refactor: migrate ebook routes to new http server (#742)
* add new api

* UI use new API

* remove legacy route

* fix request to new api destination and read that response

* feat: allow authentication using cookies

* fix: return proper error on validatesession

* typo: secret key envionment variable in makefile

* serve ebook file with new api

* cache update use new api path

* add skipexist and batch download use same route as cache

* fix bug - update hasebook status

* remove unneeded part of code

* add swagger documentation

* fix swagger documentation

* fix swagger response

* better statuscode

* fix swagger documentation

* covert to snake_case

* recover coverage.txt that remove

---------

Co-authored-by: Felipe M <me@fmartingr.com>
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-29 11:57:07 +01:00
Yuta Hayashibe
c64e858a23
Fix typos (#756)
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-22 17:43:49 +02:00
Monirzadeh
03df5e8399
fix: allow webp thumbnails (#758)
* use webp as thumbnail

* simpler way to use webp

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-22 15:06:08 +02:00
cbe
1ae82f48a3
fix: Actions in overlays on mobile hard to press (#759)
* fix: Actions in overlays on mobile hard to press

Buttons on mobile devices could've been cut off, depending on the browsers
chrome size which rendered them either completely off-screen or at least hard
to reach. Using `dvh` (dynamic viewport height) should fix this issue for
most newer browsers.

* fix: provide fallback values for newer dynamic viewport units

Since `dvh` and `dvw` are rather new and required an up-to-date browser
providing percentage values is a bit safer

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-22 13:37:25 +02:00
Monirzadeh
f82b97dec5
add create ebook by default in settings (#761)
* add create ebook by default in settings

* ebook automatic generate for new bookmark

* fix empty ebook for new bookmark
2023-10-22 13:25:16 +02:00
Monirzadeh
106bf163ca
refactor: remove unneeded variable and unify the way send token in header (#763) 2023-10-17 18:36:55 +02:00
Monirzadeh
c05d617fbd
feat: allow per-user settings and store them in database (#639)
* create needed field in sqlite database

* update account model

* update Account struct for save Account options

* update sqlite database return account settings

* save configure in sqlite as text and return that

* read configure from user account and defualt configure for shiori

* add api/ui for update settings in database user can save settings in database (in sqlite database)

* check configures be in json format before save in database

* support MariaDB

* fix wrong comment

* support PostgreSQL

* revert unneeded change in new logic

* change configures to config

* change SaveAccount to SaveAccountSettings

* add migrate database scripts

* change default in migration scrtipts

* update model

* read config field as json from database

* fix parse value config value & update config update

* update default value for new user

* update settings variable name to reflect database value in UI

* fix typo

* not panic if user not exist and update worng comment

* visitor user can update there settings now

* remove unneeded loading dialog

* fix typo

* update function for pg and mysql

* remove IsJson

* move scan method to model

* simplify jsonify

* simplify assignees value to account.Config

* missing part of function

* fix some typo and unneeded field in struct

* add down migrate script for all database

* change createEbook to CreateEbook

* use json instead of text in mysql and postgres

* implement

* remove unneeded part

* remove unneeded jsonify in code

* return SelectContext and GetContext

* remove defualt config in reques for new user it will be set in backend

* New API

* remove legacy API

* remove validateSessionWithoutOwnerStatus

* remove Jsonify function don't need that anymore

* add unit test for database

* update migrate script name

* change put to patch

* return PUT

* fix Patch problem and now use PATCH instead of PUT

* remove unneeded retuen

* more cleaner code for request new settings

* fix bug to handle string in Scan method thanks to fmartingr

* fix Authorization & use GetAccount & remove username from request

* shiori-settings remove and it read from shiori-account

* add swagger documentation

* API unit test

* fix typo

* remove unneeded coment

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* better Documentation

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* shiori-toke remove on logout

* fix typo

* add unit test check update config in database

* update swag documentation

* fix swag formaing error

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-15 15:27:54 +02:00
Monirzadeh
46c1a6ae8f
feat: allow resize the dialogbox for bigger/hidpi screens (#732)
* resize dialogbox with drag corner of box

* update style.css

* resizable just for more than full HD display

* fix size for initial size for small and big size

* full screen dialog box on mobile device - wider dialogbox on device

* remvoe custom-dialog.css that combine with style.css

* disable resize for moile device

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* use new generated style.css

* not limit minimum size to 400px

* set default value in .custom-dialog

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-10-09 16:07:37 +00:00
Felipe Martin
eb666cd2f5
feat: use new JWT auth in all frontend API calls (#743)
* properly store jwt token

* use a secret in the local dev server

* send jwt token in all api calls
2023-09-30 12:45:35 +02:00
Monirzadeh
9e91029e43
refactor: Migrate ePub generation to go-epub (#679) 2023-09-16 20:45:09 +02:00
Felipe Martin
4df7e1dd8e
feat: build css from less files locally (#735)
* build less using lessc and bun

* build

* make styles-check

* Added workflow to check updates

* avoid embed less files

* docs

* updated tests to reflect changes

* typo

* fixed archive page css

* call styles check on pull requests

* comments in style.less

* fix archive styles affecting other pages

* minify using clean-css

* Split archive.css and style.css

* Updated docs
2023-09-16 19:58:55 +02:00
Monirzadeh
3fb3f6075b
just use one dependency for uuid (#736) 2023-09-16 18:34:51 +02:00
András Rutkai
e722cb6961
fix: properly parse mysql connection string, docs update (#730)
Mysql connection string update, docs update
2023-09-13 18:31:41 +02:00
Felipe Martin
ef1d18d81f
fix(db): handle usage of special characters in searches (#721)
* handle full text search for failing cases

* added test

* test getbookmarkcount too

* replaceall, fix getbookmarkcount
2023-09-11 18:20:09 +02:00
Monirzadeh
f4817cb9c3
fix: Ensure bookmark files are correctly downloaded before deleting current ones (#683)
* generate ebook get dstPath

* Archive and ebook can recover if download faild

* recover thumb if download faild

* thumb image create just if image processing is sucssesful

* create epub in tmp if it sucssesful copy to destination

* archive file create in tmp if it successful move to destination

* move to destination as function

* update ebook download api and remove .epub from file name

* report faild item to user

* not show dialog if error not happen

* update thumbnail based on last status of bookmark fix #524

* better warning massage

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* tmpFile without .epub

* MoveToDestination change to MoveFileToDestination

* return .epub

* log if downloadBookImage return error

* fix bug remove imgPath just if download last image be unsuccessful

* update old unit test

* add processing.go unit test

* small massage for report failded item to the user

* add some more unit test and samplefile

* use sample image in unit test

* use local sample file and unit test not need internet connection anymore

* update error to user and log that too

* add more comment

* update comment

* change variable name parentDir to dstDir

* more simpler error handling

* remove unneeded defer

* remvoe unneeded epubWriter.Close()

* more readable unit test in processing

* more readable unit test for ebooks

* delete all defer os.RemoveAll from unit tests

* Better comment

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* Better Error output

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* fix err.String() method

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-08-20 18:20:13 +02:00
Felipe Martin
8b015a3850
Swagger improvements (#666)
* refactor: swagger docs into a folder

* added scripts for the swaggger tasks

* check version and fmt

* CI

* formatted swag comments

* using custom delims

* revert custom delims

* swag 1.16.1

* update swagger docs

* avoid make swagger output

* swagger check

* test

* swag-fmt

* swagger run

* gofmt

* avoid swag-fmt check for now due to inconsistencies with gofmt

* re-enabled by using go fmt afterwards

* use newer swag in CI

* add gopath to path

* using go binary instead of unset env

* alternative

* correct swag version

* formatted

* formatted

* correct go fmt command

* make swagger

* swagger-check -> swag-check

* run swag-check on lint
2023-08-12 08:38:52 +02:00
arakimo
2e1016e5f1
preserve fragment in URLs (#315) (#687)
* preserve fragment in URLs (#315)

* fix: correctly handle URLs without a fragment

* simplify solution to (#315)
2023-08-05 09:04:25 +02:00
Felipe Martin
307f293430
chore: remove verbose logger (#685) 2023-07-30 07:59:36 +02:00
Felipe Martin
e0d4537b37
default to sqlite, warn dbms users (#667) 2023-07-30 07:41:32 +02:00
Felipe Martin
a60f65f24e
fix: title overwritten if user has defiend it (#684) 2023-07-29 09:24:12 +02:00
Felipe Martin
2cbc592ece
Show Shiori version on server command (#669)
Fixes #668
2023-07-22 18:44:03 +02:00
Felipe Martin
622d556fde
fix: title is never retrieved when adding bookmark (#664)
* fix: title is never retrieved when adding bookmark

Fixes #654
2023-07-21 16:14:34 +02:00
Felipe Martin
c5c7a5f600
Show version in login page (#652) 2023-07-21 09:25:19 +02:00
Felipe Martin
888d053b2d
Allow JWT authentication into legacy APIs (#651)
* typo: letter in login page

* httpconfig set defaults for secret key with warn

* allow new authentication in old api

* Updated warn log
2023-07-21 07:57:42 +02:00
Felipe Martin
a60dbf3dc5
Run legacy API and new API at the same time. (#648)
* config: backwards comptabile dir

* remove duplicated frontend

* frontend: move assets to assets folder

* legacy routes handler in gin

* templates and asset in different embed

* new routes

* frontend routes serve old views

* added DTO for account object

* api auth calls legacy handler

* frontend: handle new error messages

* frontend: update urls

* frontend: login using new api

* updated frontend tests

* chore: remove debug route

* create shiori/gopher user if no owner is present

* server as default command

* serve -> server

* refactored database logic, allow database url

* removed unused configuration

* storage docs

* refactor cli to use cfg and deps

* check errors only in server

* log fatal instead of os exit

* dont default data directory to current dir

* fixed sqlite path

* trigger build on prs

* avoid releasing if lint/test fails

* pull request condition

* event -> event_name

* Get correct pull request number

* added workflow to delete dangling tags

* fix: nil error checking

* set gin mode first

* set gin mode before initialization

* fix logger

* allow version bump from custom ref

* Updated matrix link to workspace
2023-07-19 18:25:41 +02:00
Felipe Martin
6f19c12c95
Start working on new REST API. Refactor logic in domains. (#497)
* added 404 template

* added auth domain

* added embed file for frontend

* added base config and dependencies

* added basic new http server

* added separated server command

* updated go modules

* removed modd file

* Added shortcut to send internal server error response

* Added JWT support to Auth Domain

* Added JWT support to API

* docs: added comments to response struct

* naming

* inline returns

* updated dependencies

* production logger

* bookmarks endpoint

* reverted old views api path

* frontend for api v1

* proper 404 error (not working atm)

* use response

* removed 404 html

* server error handler

* login and basic auth

* adjusted session duration

* properly retrieve tags

* properly delete bookmark

* cleanup

* archiver domain

* debug routes

* bookmark routes

* expiration by parameter

* move to logrus

* logout

* frontend cache

* updated dependencies

* http: migrated to gin

* linted

* Added version command

* unit tests, docs

* response test utils and tests

* remove logout handler

* auth

* createtag

* improved http test utilities

* assert message equals

* Remove 1.19 from test matrix

* moved api to v1 folder

* docs: contribute docs

* updated makefile

* updated usage docs

* warn in server command

* updaed docs with shiori version command

* Updated documentation

* deps: update
2023-07-17 14:30:18 +01:00
Monirzadeh
78a7578f5c
fix download data image fix #645 (#646) 2023-07-15 19:04:14 +01:00
Ryan Gonzalez
e02d3e7739
feat: add SHIORI_PG_SSLMODE (#536)
* feat: add SHIORI_PG_SSLMODE

Some managed hosting providers don't support sslmode=disabled.

* handle empty value

---------

Co-authored-by: Felipe M <me@fmartingr.com>
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-09 08:43:26 +02:00
Felipe Martin
6ab661520e
fix: prevent bookmark overwrite on download (#636) 2023-07-09 08:33:56 +02:00
Monirzadeh
249f4b89c8
Initial Ebook Support (#623)
* generate ebook

* serve ebook file

* Update ebook.go not download same image twice anymore

* download ebook request api/ui part

* fix typo

* add stylesheet

* update hasEbook status

* download link update after ebook generate

update bookmark command in ui

* download ebook with bookmark title

* Apply suggestions from code review for better error handling

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* Update internal/view/js/page/home.js fix typo

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

* import error lib and retuen missing error

* move ebook download action to update cache

* replace io/ioutil with io

* add missing error handling

* update Archive now always update ebook

* replace panic error with 404

* remove ebook with delete action

* add download ebook link to content page

* remove tags that not work correctly right now

* if file is pdf not generate ebook

* update style.css

* Revert "update style.css"

This reverts commit 519e10d6ce.

* remove download limit for api

* fix missing fmt.Errorf and change to errors.Wrap

* fix double panic

* return 404 if bookmark not exist

* change function name to GenerateEbook

* not isFatalErr anymore

* add unit test

* remove uneeded field for unit test

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-09 08:29:32 +02:00
Monirzadeh
98c2d77257
thumb image will be not accesable anymore if it is private fix #578 (#627)
* thumb image will be not accesable anymore if it is private fix #578

* Update internal/webserver/handler-ui.go better error handling

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-07-02 10:58:57 +02:00
Tomi lla
dde1b44e77
fix(log): record user real ip from headers (#603)
* fix(real_ip): get user real ip from headers of request

* fix(real_ip): compatible with those header with multiple IP values separated by commas

* test(real_ip): add benchmark for IPv4 and IPv6 private address check

* fix(real_ip): check empty, then remove leading and tailing comma char, finally locate first IP field

* test(real_ip): move checker logic into utils and add more unit test cases

* test(real_ip): write unit tests covering all code branches of the `util-ip` module

* refactor(real_ip): use one-line `testify.assert.Panics` to capture intended panic in test case

* chore(real_ip): add module private variable `UserRealIpHeaderCandidates`

put those headers together, make it easy to manage in one place

* doc(real_ip): write docstring for each function in the `utils-ip` module

* chore(real_ip): choose more concrete and unambiguous name for test helper function

It is to avoid polluting the module name-space with too general names.

* chore(naming): change function names according to code style

* refactor(real_ip): simplify the code indicated by 'gosimple' and `golangci`

* chore(naming): rename the `utils-ip` file to `utils_ip` to match with the rest of the file structure

---------

Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-06-11 21:25:23 +02:00
Monirzadeh
a4b92504b0
fix direction in RTL language (#615)
Co-authored-by: Felipe Martin <812088+fmartingr@users.noreply.github.com>
2023-06-11 21:18:12 +02:00
chain710
888e59dabf
Add log args for root command (#565)
- log-level: specify log level, default info
- log-caller: whether log `file:lineno` or not, default false
- log FullTimestamp
2023-03-17 13:37:23 +01:00
sdvcrx
3c1229d6ce
fix(frontend): prevent archive header covering up the archive content (#562) 2023-02-22 13:05:55 +01:00
Ivan Li
47f288fb1d
fix: browser ext cannot archive and update bookmark. (#560) 2023-02-05 10:58:00 +01:00
mox46
52d9a6322f
use UTC for cookie expiry date in backend (#566) 2023-02-05 10:56:34 +01:00
Katarn
257506ae19
feat: improve bookmark image detection (#554) 2023-01-21 09:53:19 +01:00