Commit graph

111 commits

Author SHA1 Message Date
Abhishek K
d7bad9865a
NET-2014: Audit Logging (#3455)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* notify peers after settings update

* define schema for activity, add api handler to list network activity

* setup event channel and logger

* setup event logger, add event for user login

* change activity model to event

* add api error constants

* add logout event

* log user crud events

* add login events for oauth

* add user related events

* log events for invites and user approvals

* order user activity event by timestamp

* fix logout api

* add user and network events api, add addtional events triggers

* add filters to all events api

* fix events filter

* add diff to event logs

* update user logout api

* log settigns updates

* log events for network and host updates

* check for diff on events

* log host del event

* add user loc info to desktop app connection events

* fix authorize middleware check

* add gateway events

* resolve merge conflicts

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-05-21 13:13:20 +05:30
Abhishek K
309e4795a1
NET-1950: Persist Server Settings in the DB (#3419)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* publish peer update on settings update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* set verbosity

* sync auto update settings with hosts

* sync auto update settings with hosts

* mask secret and convert jwt duration to minutes

* convert jwt duration to minutes

* notify peers after settings update

* compare with curr settings before updating

* send host update to devices on auto update

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-04-30 02:34:10 +04:00
abhishek9686
b5842b7b06 fix create gateway req 2025-04-03 21:47:41 +04:00
abhishek9686
3bb843e96e fix openapi tag 2025-04-02 19:18:41 +04:00
Abhishek K
8a1f36ee8e
Net 1990 (#3386)
* add peerkey to network egress routes model

* add peerkey to network egress routes model

* filter out conflicting routes from node

* add support for egress HA on relay

* add support for egress HA on relay

* add support for egress HA on relay

* skip if curr node is relay node of the peer

* skip if curr node is relay node of the peer

* fix failover egress HA

* add network to egress route model

* clone before modifying

* check for user policies on uni direction check

* add all network fwd rule

* delete all network policies and tags on network deletion
2025-03-25 00:42:14 +04:00
Abhishek K
3d765f9cf1
NET-1910: Acl controls for Egress Traffic (#3377)
* add support for egress ranges on acl policy

* add egress ranges to acl rules

* add egress ranges to acl policies

* Add egress ranges to acl rules

* add egress ranges to fw update

* fetch acl rules for egress networks

* apply egress policies for devices

* configure user policies for egresss routes

* fix gw tag name migration

* fix egress acl rules for static nodes

* add egress ranges for static nodes on ingress gw

* fileter acl IPs to be unique

* cleanup IOT logic from peer update

* make acl Rule Dst List

* cleanup egress ranges from acl policies

* create user group default acl policy for gateways

* remove remote access name ids

* rm egress ranges removal from acl policies

* simplify user permissions on nodes

* add additional nameservers to extclient dns

* remove debug logs

* fix static checks
2025-03-18 13:25:55 +04:00
Abhishek K
1ad8b8b7b4
Merge pull request #3358 from gravitl/master
Master
2025-03-06 22:17:24 +04:00
abhishek9686
33fafa83e6 add nameservers to dns settins 2025-02-28 21:49:53 +04:00
abhishek9686
21826d41b6 add nameservers to network settings 2025-02-28 21:27:18 +04:00
abhishek9686
225bf37e02 resolve merge conflicts 2025-02-28 12:18:38 +04:00
abhishek9686
c0a88e7c43 add cache internal to the func 2025-02-28 11:09:39 +04:00
abhishek9686
ce6ec25253 fix network ip allocation in HA 2025-02-27 16:26:15 +04:00
Aceix
2fcdd865c8
fix: unlnk deleted networks and tags from enrollment keys (#3328) 2025-02-24 08:22:18 +03:00
Abhishek K
64bc351564
Merge pull request #3315 from gravitl/master
Master
2025-02-04 19:32:14 +04:00
Abhishek K
4431dc99a7
NET-1933: option to force destroy network (#3311)
* option to force destroy network

* fix network tests

* fix network defaults func

* fix network destroy action

* delete network if node count is zero

* push peer update network deletion

* send node update
2025-02-03 15:19:44 +04:00
abhishek9686
68b257b2ec update default acl field 2025-01-30 15:55:41 +04:00
abhishek9686
2ea707fb5c resolve merge conflicts 2024-10-17 10:42:59 +04:00
abhishek9686
272dfda088 create default tags on network 2024-10-17 10:41:33 +04:00
abhishek9686
1d1c033988 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1615 2024-09-26 12:21:22 +04:00
Abhishek K
d58439ac9d
api for network with stats (#3111) 2024-09-26 11:52:21 +04:00
abhishek9686
dfed77906a create default acl policies on network creation 2024-09-25 20:24:04 +04:00
Abhishek K
fe57a6a42d
Net 1227 v1 (#3062)
* generalise smtp config

* copy over smtp vars

* env new line

* fix master key api access

* comment user tests

* fix network and user invite for master key access
2024-08-22 11:55:08 +05:30
Abhishek K
2e8d95e80e
NET-1227: User Mgmt V2 (#3055)
* user mgmt models

* define user roles

* define models for new user mgmt and groups

* oauth debug log

* initialize user role after db conn

* print oauth token in debug log

* user roles CRUD apis

* user groups CRUD Apis

* additional api checks

* add additional scopes

* add additional scopes url

* add additional scopes url

* rm additional scopes url

* setup middlleware permission checks

* integrate permission check into middleware

* integrate permission check into middleware

* check for headers for subjects

* refactor user role models

* refactor user groups models

* add new user to pending user via RAC login

* untracked

* allow multiple groups for an user

* change json tag

* add debug headers

* refer network controls form roles, add debug headers

* refer network controls form roles, add debug headers

* replace auth checks, add network id to role model

* nodes handler

* migration funcs

* invoke sync users migration func

* add debug logs

* comment middleware

* fix get all nodes api

* add debug logs

* fix middleware error nil check

* add new func to get username from jwt

* fix jwt parsing

* abort on error

* allow multiple network roles

* allow multiple network roles

* add migration func

* return err if jwt parsing fails

* set global check to true when accessing user apis

* set netid for acls api calls

* set netid for acls api calls

* update role and groups routes

* add validation checks

* add invite flow apis and magic links

* add invited user via oauth signup automatically

* create invited user on oauth signup, with groups in the invite

* add group validation for user invite

* update create user handler with new role mgmt

* add validation checks

* create user invites tables

* add error logging for email invite

* fix invite singup url

* debug log

* get query params from url

* get query params from url

* add query escape

* debug log

* debug log

* fix user signup via invite api

* set admin field for backward compatbility

* use new role id for user apis

* deprecate use of old admin fields

* deprecate usage of old user fields

* add user role as service user if empty

* setup email sender

* delete invite after user singup

* add plaform user role

* redirect on invite verification link

* fix invite redirect

* temporary redirect

* fix invite redirect

* point invite link to frontend

* fix query params lookup

* add resend support, configure email interface types

* fix groups and user creation

* validate user groups, add check for metrics api in middleware

* add invite url to invite model

* migrate rac apis to new user mgmt

* handle network nodes

* add platform user to default role

* fix user role migration

* add default on rag creation and cleanup after deletion

* fix rac apis

* change to invite code param

* filter nodes and hosts based on user network access

* extend create user group req to accomodate users

* filter network based on user access

* format oauth error

* move user roles and groups

* fix get user v1 api

* move user mgmt func to pro

* add user auth type to user model

* fix roles init

* remove platform role from group object

* list only platform roles

* add network roles to invite req

* create default groups and roles

* fix middleware for global access

* create default role

* fix nodes filter with global network roles

* block selfupdate of groups and network roles

* delete netID if net roles are empty

* validate user roles nd groups on update

* set extclient permission scope when rag vpn access is set

* allow deletion of roles and groups

* replace _ with - in role naming convention

* fix failover middleware mgmt

* format oauth templates

* fetch route temaplate

* return err if user wrong login type

* check user groups on rac apis

* fix rac apis

* fix resp msg

* add validation checks for admin invite

* return oauth type

* format group err msg

* fix html tag

* clean up default groups

* create default rag role

* add UI name to roles

* remove default net group from user when deleted

* reorder migration funcs

* fix duplicacy of hosts

* check old field for migration

* from pro to ce make all secondary users admins

* from pro to ce make all secondary users admins

* revert: from pro to ce make all secondary users admins

* make sure downgrades work

* fix pending users approval

* fix duplicate hosts

* fix duplicate hosts entries

* fix cache reference issue

* feat: configure FRONTEND_URL during installation

* disable user vpn access when network roles are modified

* rm vpn acces when roles or groups are deleted

* add http to frontend url

* revert crypto version

* downgrade crytpo version

* add platform id check on user invites

---------

Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2024-08-20 17:08:56 +05:30
Abhishek K
5a561b3835
Net 1440 batchpeerupdate (#3042)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* batch peerUpdate

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* add peerUpdate batch size to config

* update batch peerUpdate

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed

* add batch peerUpdate switch

* set batch peerUpdate to true by default

---------

Co-authored-by: Max Ma <mayabin@gmail.com>
2024-08-16 15:35:43 +05:30
Max Ma
46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Sayan Mallick
c551c487ca
New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
Max Ma
6fa084006a
NET-1355:peerUpdate for default host after new network created (#2974)
* peerUpdate for default host after new network created

* remove mq check for peerUpdate
2024-06-12 17:33:26 +05:30
Aceix
faf50c8c3d
chore(NET-1245): remove check for private network address (#2934) 2024-05-14 19:54:00 +05:30
Aceix
100b778449
fix(NET-897): uniform client and node acls (#2803) 2024-02-13 20:25:27 +07:00
abhishek9686
43e82a2936 make defaul host failover and remote access gw on joining new network 2024-02-05 11:18:28 +05:30
Abhishek K
5bf30b2c10
NET-877: Replace peers on Refreshkeys peer update (#2761)
* replace peers on key refresh

* add peer conf to metrics map only when allowed
2024-01-11 15:59:19 +05:30
Abhishek K
797f3a4533
Send deleted extclient mq update when disabled (#2760) 2024-01-10 14:57:29 +05:30
Aceix
c8ae70bd69
fix(NET-760): check for private network range (#2722) 2023-12-13 14:02:57 +04:00
Matthew R Kasun
876778a1cc
open api spec file (#2595)
* remove usergroup

* superadmin

* superadmin creation

* generate openapi spec file

* statticcheck

* review comments
2023-10-04 10:26:38 +04:00
Abhishek K
719e0c254d
NET-551: User Mgmt Re-Design (#2547)
* add superadmin role, apis to create superadmin user

* apis to attach and remove user from remote access gateways

* add api to list user's remote client has gateway clients

* remove code related user groups

* remove networks and groups from user model

* refactor user CRUD operations

* fix network permission test

* add superadmin to authorize func

* remove user network and groups from cli

* api to transfer superadmin role

* add api to list users on a ingress gw

* restrict user access to resources on server

* deny request from remote access client if extclient is already created

* fix user tests

* fix static checks

* fix static checks

* add limits to extclient create handler

* set username to superadmin on if masterkey is used

* allow creation of extclients using masterkey

* add migration func to assign superadmin role for existing admin user

* check for superadmin on migration if users are present

* allowe masterkey to extcleint apis

* check ownerid

* format error, on jwt token verification failure return unauthorized rather than forbidden

* user update fix

* move user remote functionality to ee

* fix update user api

* security patch

* initalise ee user handlers

* allow user to use master key to update any user

* use slog

* fix auth user test

* table headers

* remove user role, it's covered in middleware

* setuser defaults fix
2023-09-01 14:27:08 +05:30
Gabriel de Souza Seibel
8ce7da2ce9
[NET-494 / ACC-322] New free tier limits (#2495)
* Rename var

* Rename consts and use iota

* Use switch instead of repeated else if

* Rename limits related vars

* Introduce new free tier limits

* Measure new limits and report on license validation

* Separate usage and limits, have new ones

* Don't check for hosts and clients limits, but for machines instead

* Error on egress creation @ free tier w/ internet gateways

* Remove clients and hosts limit from code

* Rename var

* Rename consts and use iota

* Use switch instead of repeated else if

* Rename limits related vars

* Introduce new free tier limits

* Measure new limits and report on license validation

* Separate usage and limits, have new ones

* Don't check for hosts and clients limits, but for machines instead

* Error on egress creation @ free tier w/ internet gateways

* Remove clients and hosts limit from code
2023-08-08 23:17:49 +05:30
Farukh Khan
449f3f947b
Net 509 515 (#2496)
* NET-509

* External client config files with IPv6 endpoints now have the [] separating the address and port.

* NET-515

* Increased network name max length to 32

* NET-509-515

* Updated unit test for network name max length check.

* Updated extclient endpoint ip string manipulation to use sprintf

* Added proper error message for network name length more than max allowed.

* NET-515 small typo fix for error strings should not be capitalized
2023-08-08 23:17:22 +05:30
Tobias Cudnik
e286adda77
NET-291 PUT /api/networks/{networkname} (#2380)
* PUT /api/networks/{networkname}

* review
2023-06-08 17:22:35 -04:00
Tobias Cudnik
723375b334
NET-152 enrollment keys for non admins (#2346)
* return 401 instead of 403

* fixed http.StatusForbidden

* Tagged build version (temp)

* Unauthorized_Err when applicable

* untagged version

* fixed PUT /api/users/networks/user1

* - expired token redirs to login
- added `/api/enrollment_keys` for non-admins
- unit test for enrollment keys for non-admins

* handle user perms in `/hosts`

* removed debug

* misc

* - support masteradmin
- return hosts with partial access

* added `ismaster` to middleware
2023-05-31 13:11:54 +05:30
Matthew R Kasun
3e0d7720c8 return {} if no records found for acls/metrics 2023-05-04 16:50:03 -04:00
Matthew R Kasun
de111181bf restore sortNetworks 2023-04-19 14:06:39 -04:00
Matthew R Kasun
57723b7aae merge conflicts 2023-04-19 13:18:03 -04:00
Matthew R Kasun
c510846444
Merge branch 'develop' into GRA-1529-key-update 2023-04-19 13:16:09 -04:00
dcarns
700147e3ca
Merge pull request #2171 from walkerwmanuel/GRA-1336-sort-before-sending
GRA-1336: sort before sending
2023-04-19 13:08:08 -04:00
Matthew R Kasun
920590b651 fix build issue 2023-04-19 09:58:16 -04:00
Matthew R Kasun
754ab44a48 Merge remote-tracking branch 'origin/develop' into GRA-1529-key-update 2023-04-19 09:48:04 -04:00
Matthew R Kasun
34eb699752 deprecated ability to edit networks 2023-04-14 10:39:36 -04:00
Matthew R Kasun
63552891ee refactor key updates on host basis vice node 2023-04-13 15:02:14 -04:00
walkerwmanuel
9e56135424 Added user sorting and fixed error 2023-04-03 14:36:38 -04:00
walkerwmanuel
09872e9ec5 added sort logic 2023-03-30 16:04:12 -04:00