Commit graph

1217 commits

Author SHA1 Message Date
Abhishek K
d7bad9865a
NET-2014: Audit Logging (#3455)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* notify peers after settings update

* define schema for activity, add api handler to list network activity

* setup event channel and logger

* setup event logger, add event for user login

* change activity model to event

* add api error constants

* add logout event

* log user crud events

* add login events for oauth

* add user related events

* log events for invites and user approvals

* order user activity event by timestamp

* fix logout api

* add user and network events api, add addtional events triggers

* add filters to all events api

* fix events filter

* add diff to event logs

* update user logout api

* log settigns updates

* log events for network and host updates

* check for diff on events

* log host del event

* add user loc info to desktop app connection events

* fix authorize middleware check

* add gateway events

* resolve merge conflicts

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-05-21 13:13:20 +05:30
Abhishek K
307a3d1e4b
NET-1932: Merge egress and internet gateways (#3436)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* update gorm tag for primary key on egress model

* create user policies for egress resources

* resolve merge conflicts

* get egress info on failover apis, add egress src validation for inet gws

* add additional validation checks on egress req

* add additional validation checks on egress req

* skip all resources for inet policy

* delete associated egress acl policies

* fix failover of inetclient

* avoid setting inet client asd inet gw

* fix all resource egress policy

* fix inet gw egress rule

* check for node egress on relay req

* fix egress acl rules comms

* add new field for egress info on node

* check acl policy in failover ctx

* avoid default host to be set as inet client

* fix relayed egress node

* add valid error messaging for egress validate func

* return if inet default host

* jump port detection to 51821

* check host ports on pull

* check user access gws via acls

* add validation check for default host and failover for inet clients

* add error messaging for acl policy check

* fix inet gw status

* ignore failover req for peer using inet gw

* check for allowed egress ranges for a peer

* add egress routes to static nodes by access

* avoid setting failvoer as inet client

* fix egress error messaging

* fix extclients egress comms

* fix inet gw acting as inet client

* return formatted error on update acl validation

* add default route for static nodes on inetclient

* check relay node acting as inetclient

* move inet node info to separate field, fix all resouces policy

* remove debug logs

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-05-21 12:50:21 +05:30
Aceix
506f73ebb9
Merge pull request #3441 from gravitl/nmctl-access-token
feat: add support for user access tokens in nmctl
2025-05-20 09:16:24 +05:30
Abhishek K
f9bc3a5386
Merge pull request #3440 from gravitl/master
Master
2025-04-30 03:02:05 +04:00
Abhishek K
85084f9dfe
check host ports on join (#3437) 2025-04-30 03:00:36 +04:00
Aceix
b0a8a07ebd
fix: add node status in response to create/delete gateway (#3425) 2025-04-30 02:57:53 +04:00
alingse
eb0dbf7ee4
fix call logic.FormatError with a nil value error (#3406)
* fix call logic.FormatError with a nil value error

Signed-off-by: alingse <alingse@foxmail.com>

* fix call slog.Error with wrong error

---------

Signed-off-by: alingse <alingse@foxmail.com>
2025-04-30 02:39:35 +04:00
Aceix
c9f3750499
fix: improve error response message (#3426) 2025-04-30 02:34:45 +04:00
Abhishek K
309e4795a1
NET-1950: Persist Server Settings in the DB (#3419)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* publish peer update on settings update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* set verbosity

* sync auto update settings with hosts

* sync auto update settings with hosts

* mask secret and convert jwt duration to minutes

* convert jwt duration to minutes

* notify peers after settings update

* compare with curr settings before updating

* send host update to devices on auto update

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-04-30 02:34:10 +04:00
Abhishek K
262803c234
add ctx to DB funcs (#3435) 2025-04-29 00:22:02 +04:00
Abhishek K
119ef4e17e
move api tokens models to schema pkg (#3434) 2025-04-29 00:11:56 +04:00
Abhishek K
ca95954fb5
NET-2000: Api access tokens (#3418)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* fix token removal on user update

* fix token removal on user update
2025-04-23 20:21:42 +04:00
abhishek9686
b5842b7b06 fix create gateway req 2025-04-03 21:47:41 +04:00
abhishek9686
3bb843e96e fix openapi tag 2025-04-02 19:18:41 +04:00
Abhishek K
4c26ae5197
Merge pull request #3391 from gravitl/master
Master
2025-03-25 11:51:06 +04:00
Abhishek K
8a1f36ee8e
Net 1990 (#3386)
* add peerkey to network egress routes model

* add peerkey to network egress routes model

* filter out conflicting routes from node

* add support for egress HA on relay

* add support for egress HA on relay

* add support for egress HA on relay

* skip if curr node is relay node of the peer

* skip if curr node is relay node of the peer

* fix failover egress HA

* add network to egress route model

* clone before modifying

* check for user policies on uni direction check

* add all network fwd rule

* delete all network policies and tags on network deletion
2025-03-25 00:42:14 +04:00
Abhishek K
5e1ef5e366
Net 1990 (#3385)
* add peerkey to network egress routes model

* add peerkey to network egress routes model

* filter out conflicting routes from node

* add support for egress HA on relay

* add support for egress HA on relay

* add support for egress HA on relay

* skip if curr node is relay node of the peer

* skip if curr node is relay node of the peer

* fix failover egress HA

* add network to egress route model

* clone before modifying

* check for user policies on uni direction check

* add all network fwd rule
2025-03-25 00:29:35 +04:00
Abhishek K
3d765f9cf1
NET-1910: Acl controls for Egress Traffic (#3377)
* add support for egress ranges on acl policy

* add egress ranges to acl rules

* add egress ranges to acl policies

* Add egress ranges to acl rules

* add egress ranges to fw update

* fetch acl rules for egress networks

* apply egress policies for devices

* configure user policies for egresss routes

* fix gw tag name migration

* fix egress acl rules for static nodes

* add egress ranges for static nodes on ingress gw

* fileter acl IPs to be unique

* cleanup IOT logic from peer update

* make acl Rule Dst List

* cleanup egress ranges from acl policies

* create user group default acl policy for gateways

* remove remote access name ids

* rm egress ranges removal from acl policies

* simplify user permissions on nodes

* add additional nameservers to extclient dns

* remove debug logs

* fix static checks
2025-03-18 13:25:55 +04:00
Aceix
372c797e60
fix: add status to updated node response during update (#3363) 2025-03-18 09:34:42 +04:00
Vishal Dalwadi
4c14cfd099
NET-1986: Only report online hosts and external clients (#3375)
* feat(go): only report online hosts.

* feat(go): only report online external clients.

* feat(go): only report online hosts and external clients.
2025-03-17 18:35:17 +04:00
Aceix
7c262abbb9
Merge pull request #3356 from gravitl/fix/allow-multiple-client-gw-dns
Fix/allow-multiple-client-dns
2025-03-11 00:53:38 +04:00
abhishek9686
bb25739bd9 resolve merge conflicts 2025-03-06 22:41:56 +04:00
Abhishek K
1ad8b8b7b4
Merge pull request #3358 from gravitl/master
Master
2025-03-06 22:17:24 +04:00
abhishek9686
b45a926649 optimise static node rules, fix traffic flows for static nodes 2025-03-05 14:26:35 +04:00
abhishek9686
d21411392b fix integration tests 2025-03-03 12:05:23 +04:00
abhishek9686
48b805bbc2 add loadbalance endpoint for extclients 2025-03-02 20:59:53 +04:00
abhishek9686
33fafa83e6 add nameservers to dns settins 2025-02-28 21:49:53 +04:00
abhishek9686
21826d41b6 add nameservers to network settings 2025-02-28 21:27:18 +04:00
abhishek9686
225bf37e02 resolve merge conflicts 2025-02-28 12:18:38 +04:00
abhishek9686
c0a88e7c43 add cache internal to the func 2025-02-28 11:09:39 +04:00
abhishek9686
ce6ec25253 fix network ip allocation in HA 2025-02-27 16:26:15 +04:00
abhishek9686
e50da805d4 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1911 2025-02-24 08:53:49 +03:00
Abhishek K
48535f7ef1
NET-1956: Async Node Status API (#3341)
* add node status api

* upsate node status api to return map data

* resolve merge conflicts
2025-02-24 08:48:24 +03:00
Aceix
2fcdd865c8
fix: unlnk deleted networks and tags from enrollment keys (#3328) 2025-02-24 08:22:18 +03:00
Aceix
f46030e40f
fix: add user group struct to get user response (#3336)
* fix: add user group struct to user response

* fix: set auto-created network groups as default
2025-02-24 08:17:55 +03:00
abhishek9686
9590f9e5a4 resolve merge conflicts 2025-02-19 09:33:23 +04:00
Abhishek K
ee6bea117f
Merge branch 'develop' into master 2025-02-19 01:14:06 +04:00
abhishek9686
7dd4c048c3 add mutex around peer tags 2025-02-18 22:10:03 +04:00
abhishek9686
41fa0b1bce resolve merge conflicts 2025-02-14 18:24:56 +04:00
Abhishek K
689b9b6b52
NET-1941: failover signaling optimized, new endpoint for metrics peer info (#3322)
* optimise acl cache with network map

* fix mutex clash

* fix acl mutex

* add new endpoint for metrics peer info

* fix static check

* add new endpoint for metrics peer info

* fix host peer info api params

* cache failover data

* add json tag

* avoid duplicate calls to failover

* add failover check ctx

* add failover check ctx

* fix failover debug log

* optimise failover operations

* remove debug logs

* rm unused model

* rm unused model
2025-02-14 16:06:26 +04:00
abhishek9686
61bf479e50 upsert on color update 2025-02-07 17:23:53 +04:00
abhishek9686
9cf2ad4fc5 add color coding to tags 2025-02-06 15:14:15 +04:00
abhishek9686
a9c14c4643 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1911 2025-02-04 19:32:25 +04:00
Abhishek K
64bc351564
Merge pull request #3315 from gravitl/master
Master
2025-02-04 19:32:14 +04:00
Abhishek K
e13bf2c0eb
NET-1923: Add Metric Port to server config (#3306)
* set default metrics port 8889

* set default metrics port 51821

* add metrics port to server config

* bind caddy only on tcp

* add var for pulling files

* add new line

* update peer update model

* check if port is not zero

* set replace peer to false on pull

* do not replace peers on failover sync

* remove debug log

* add old peer update fields for backwards compatibility

* add old json tag

* add debug log in caller trace func
2025-02-04 08:44:24 +04:00
Abhishek K
4431dc99a7
NET-1933: option to force destroy network (#3311)
* option to force destroy network

* fix network tests

* fix network defaults func

* fix network destroy action

* delete network if node count is zero

* push peer update network deletion

* send node update
2025-02-03 15:19:44 +04:00
abhishek9686
d2d5658a01 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1911 2025-01-31 20:26:24 +04:00
abhishek9686
68b257b2ec update default acl field 2025-01-30 15:55:41 +04:00
Vishal Dalwadi
cec48be354
NET-1906: Add endpoints for Sync All Hosts and Upgrade All Hosts. (#3302)
* feat(go): add endpoint to sync all hosts.

* feat(go): add endpoint to upgrade all hosts.

* feat(go): allow force upgrade of hosts.

* fix(go): config yaml tag.
2025-01-28 11:33:38 +05:30
Abhishek K
8297642b90
NET-1914: add gw apis, move relays into CE (#3309)
* add gw apis, move relays into CE

* set gw field on relay and ingress creation

* add gw handlers to relay and ingress apis

* if node is inetgw and gw add dns

* remove pro check on relays

* fetch node before updating
2025-01-28 11:28:31 +05:30