Commit graph

263 commits

Author SHA1 Message Date
abhishek9686
b8320ae1a3 make idp and event hooks aysnc 2025-05-22 19:32:14 +05:30
abhishek9686
7b483f8c73 add clear all invites event 2025-05-22 15:50:23 +05:30
Abhishek K
7840eedc7b
convert invited emails to lowercase (#3457) 2025-05-21 17:05:53 +05:30
Vishal Dalwadi
614cf77b5a
NET-1991: Add IDP sync functionality. (#3428)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* feat(go): add types for idp package;

* feat(go): import azure sdk;

* feat(go): add stub for google workspace client;

* feat(go): implement azure ad client;

* feat(go): sync users and groups using idp client;

* publish peer update on settings update

* feat(go): read creds from env vars;

* feat(go): add api endpoint to trigger idp sync;

* fix(go): sync member changes;

* fix(go): handle error;

* fix(go): set correct response type;

* feat(go): support disabling user accounts;

1. Add api endpoints to enable and disable user accounts.
2. Add checks in authenticators to prevent disabled users from logging in.
3. Add checks in middleware to prevent api usage by disabled users.

* feat(go): use string slice for group members;

* feat(go): sync user account status from idp;

* feat(go): import google admin sdk;

* feat(go): add support for google workspace idp;

* feat(go): initialize idp client on sync;

* feat(go): sync from idp periodically;

* feat(go): improvements for google idp;

1. Use the impersonate package to authenticate.
2. Use Pages method to get all data.

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* feat(go): fetch idp creds from server settings;

* feat(go): add filters for users and groups;

* feat(go): skip sync from idp if disabled;

* feat(go): add endpoint to remove idp integration;

* feat(go): import all users if no filters;

* feat(go): assign service-user role on sync;

* feat(go): remove microsoft-go-sdk;

* feat(go): add display name field for user;

* fix(go): set account disabled correctly;

* fix(go): update user if display name changes;

* fix(go): remove auth provider when removing idp integration;

* fix(go): ignore display name if empty;

* feat(go): add idp sync interval setting;

* fix(go): error on invalid auth provider;

* fix(go): no error if no user on group delete;

* fix(go): check superadmin using platform role id;

* feat(go): add display name and account disabled to return user as well;

* feat(go): tidy go mod after merge;

* feat(go): reinitialize auth provider and idp sync hook;

* fix(go): merge error;

* fix(go): merge error;

* feat(go): use id as the external provider id;

* fix(go): comments;

* feat(go): add function to return pending users;

* feat(go): prevent external id erasure;

* fix(go): user and group sync errors;

* chore(go): cleanup;

* fix(go): delete only oauth users;

* feat(go): use uuid group id;

* export ipd id to in rest api

* feat(go): don't use uuid for default groups;

* feat(go): migrate group only if id not uuid;

* chore(go): go mod tidy;

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <abhishek@netmaker.io>
Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2025-05-21 13:48:15 +05:30
Abhishek K
d7bad9865a
NET-2014: Audit Logging (#3455)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* notify peers after settings update

* define schema for activity, add api handler to list network activity

* setup event channel and logger

* setup event logger, add event for user login

* change activity model to event

* add api error constants

* add logout event

* log user crud events

* add login events for oauth

* add user related events

* log events for invites and user approvals

* order user activity event by timestamp

* fix logout api

* add user and network events api, add addtional events triggers

* add filters to all events api

* fix events filter

* add diff to event logs

* update user logout api

* log settigns updates

* log events for network and host updates

* check for diff on events

* log host del event

* add user loc info to desktop app connection events

* fix authorize middleware check

* add gateway events

* resolve merge conflicts

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-05-21 13:13:20 +05:30
Abhishek K
307a3d1e4b
NET-1932: Merge egress and internet gateways (#3436)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* egress model

* fix revoked tokens to be unauthorized

* update egress model

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* setup api handlers for egress revamp

* use single DB, fix update nat boolean field

* extend validaiton checks for egress ranges

* add migration to convert to new egress model

* fix panic interface conversion

* publish peer update on settings update

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* remove nat check on egress gateway request

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* add inet gw status to egress model

* fetch node ids in the tag, add inet gw info clients

* add inet gw info to node from egress list

* add migration logic internet gws

* create default acl policies

* add egress info

* add egress TODO

* add egress TODO

* fix user auth api:

* add reference id to acl policy

* add egress response from DB

* publish peer update on egress changes

* re initalise oauth and email config

* set verbosity

* normalise cidr on egress req

* add egress id to acl group

* change acls to use egress id

* resolve merge conflicts

* fix egress reference errors

* move egress model to schema

* add api context to DB

* sync auto update settings with hosts

* sync auto update settings with hosts

* check acl for egress node

* check for egress policy in the acl dst groups

* fix acl rules for egress policies with new models

* add status to egress model

* fix inet node func

* mask secret and convert jwt duration to minutes

* enable egress policies on creation

* convert jwt duration to minutes

* add relevant ranges to inet egress

* skip non active egress routes

* resolve merge conflicts

* fix static check

* update gorm tag for primary key on egress model

* create user policies for egress resources

* resolve merge conflicts

* get egress info on failover apis, add egress src validation for inet gws

* add additional validation checks on egress req

* add additional validation checks on egress req

* skip all resources for inet policy

* delete associated egress acl policies

* fix failover of inetclient

* avoid setting inet client asd inet gw

* fix all resource egress policy

* fix inet gw egress rule

* check for node egress on relay req

* fix egress acl rules comms

* add new field for egress info on node

* check acl policy in failover ctx

* avoid default host to be set as inet client

* fix relayed egress node

* add valid error messaging for egress validate func

* return if inet default host

* jump port detection to 51821

* check host ports on pull

* check user access gws via acls

* add validation check for default host and failover for inet clients

* add error messaging for acl policy check

* fix inet gw status

* ignore failover req for peer using inet gw

* check for allowed egress ranges for a peer

* add egress routes to static nodes by access

* avoid setting failvoer as inet client

* fix egress error messaging

* fix extclients egress comms

* fix inet gw acting as inet client

* return formatted error on update acl validation

* add default route for static nodes on inetclient

* check relay node acting as inetclient

* move inet node info to separate field, fix all resouces policy

* remove debug logs

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-05-21 12:50:21 +05:30
Abhishek K
309e4795a1
NET-1950: Persist Server Settings in the DB (#3419)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* add server settings apis, add db table for settigs

* handle server settings updates

* switch to using settings from DB

* fix sever settings migration

* revet force migration for settings

* fix server settings database write

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* fix merge conflicts

* fix server settings types

* bypass basic auth setting for super admin

* add TODO comment

* publish peer update on settings update

* chore(go): import style changes from migration branch;

1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.

* remove nat check on egress gateway request

* Revert "remove nat check on egress gateway request"

This reverts commit 0aff12a189.

* feat(go): add db middleware;

* feat(go): restore method;

* feat(go): add user access token schema;

* fix user auth api:

* re initalise oauth and email config

* set verbosity

* sync auto update settings with hosts

* sync auto update settings with hosts

* mask secret and convert jwt duration to minutes

* convert jwt duration to minutes

* notify peers after settings update

* compare with curr settings before updating

* send host update to devices on auto update

---------

Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
2025-04-30 02:34:10 +04:00
Abhishek K
ca95954fb5
NET-2000: Api access tokens (#3418)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* fix token removal on user update

* fix token removal on user update
2025-04-23 20:21:42 +04:00
Abhishek K
4c26ae5197
Merge pull request #3391 from gravitl/master
Master
2025-03-25 11:51:06 +04:00
Abhishek K
4cc56fd3be
NET-1990: add peerkey to network egress routes model (#3379)
* add peerkey to network egress routes model

* add peerkey to network egress routes model

* filter out conflicting routes from node

* add support for egress HA on relay

* add support for egress HA on relay

* add support for egress HA on relay

* skip if curr node is relay node of the peer

* skip if curr node is relay node of the peer

* fix failover egress HA

* add network to egress route model

* clone before modifying
2025-03-24 15:33:39 +04:00
Aceix
39d812f137
feat: send gateway dns and private address (#3378) 2025-03-18 13:26:29 +04:00
Abhishek K
3d765f9cf1
NET-1910: Acl controls for Egress Traffic (#3377)
* add support for egress ranges on acl policy

* add egress ranges to acl rules

* add egress ranges to acl policies

* Add egress ranges to acl rules

* add egress ranges to fw update

* fetch acl rules for egress networks

* apply egress policies for devices

* configure user policies for egresss routes

* fix gw tag name migration

* fix egress acl rules for static nodes

* add egress ranges for static nodes on ingress gw

* fileter acl IPs to be unique

* cleanup IOT logic from peer update

* make acl Rule Dst List

* cleanup egress ranges from acl policies

* create user group default acl policy for gateways

* remove remote access name ids

* rm egress ranges removal from acl policies

* simplify user permissions on nodes

* add additional nameservers to extclient dns

* remove debug logs

* fix static checks
2025-03-18 13:25:55 +04:00
Aceix
57c910ba69
patch pro LICENSE broken link (#3364) 2025-03-17 18:50:47 +04:00
Vishal Dalwadi
4b3f1fd58a
feat(go): improve error message. (#3368) 2025-03-17 18:50:08 +04:00
Vishal Dalwadi
346f09ce39
NET-1986: Only report online hosts. (#3370)
* feat(go): only report online hosts.

* feat(go): only report online external clients.
2025-03-14 17:16:31 +04:00
abhishek9686
f3fd10326f migrate remote access gw tags 2025-03-11 00:50:44 +04:00
abhishek9686
bb25739bd9 resolve merge conflicts 2025-03-06 22:41:56 +04:00
Abhishek K
1ad8b8b7b4
Merge pull request #3358 from gravitl/master
Master
2025-03-06 22:17:24 +04:00
abhishek9686
b45a926649 optimise static node rules, fix traffic flows for static nodes 2025-03-05 14:26:35 +04:00
abhishek9686
225bf37e02 resolve merge conflicts 2025-02-28 12:18:38 +04:00
abhishek9686
3dedb20f94 fix user email validation 2025-02-27 18:42:09 +04:00
abhishek9686
50c3e3aaed import dns field from gw node 2025-02-27 12:29:01 +04:00
abhishek9686
aa84d43f1c import dns field from gw node 2025-02-27 12:12:13 +04:00
abhishek9686
bc1eed3745 fix azure email claims 2025-02-26 16:52:47 +04:00
abhishek9686
e50da805d4 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1911 2025-02-24 08:53:49 +03:00
Abhishek K
48535f7ef1
NET-1956: Async Node Status API (#3341)
* add node status api

* upsate node status api to return map data

* resolve merge conflicts
2025-02-24 08:48:24 +03:00
Aceix
a805901a73
feat: add node status to rac response (#3327) 2025-02-24 08:23:45 +03:00
Aceix
f46030e40f
fix: add user group struct to get user response (#3336)
* fix: add user group struct to user response

* fix: set auto-created network groups as default
2025-02-24 08:17:55 +03:00
abhishek9686
41fa0b1bce resolve merge conflicts 2025-02-14 18:24:56 +04:00
Abhishek K
689b9b6b52
NET-1941: failover signaling optimized, new endpoint for metrics peer info (#3322)
* optimise acl cache with network map

* fix mutex clash

* fix acl mutex

* add new endpoint for metrics peer info

* fix static check

* add new endpoint for metrics peer info

* fix host peer info api params

* cache failover data

* add json tag

* avoid duplicate calls to failover

* add failover check ctx

* add failover check ctx

* fix failover debug log

* optimise failover operations

* remove debug logs

* rm unused model

* rm unused model
2025-02-14 16:06:26 +04:00
abhishek9686
d2d5658a01 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1911 2025-01-31 20:26:24 +04:00
Abhishek K
8297642b90
NET-1914: add gw apis, move relays into CE (#3309)
* add gw apis, move relays into CE

* set gw field on relay and ingress creation

* add gw handlers to relay and ingress apis

* if node is inetgw and gw add dns

* remove pro check on relays

* fetch node before updating
2025-01-28 11:28:31 +05:30
Abhishek K
c02ec193db
NET-1920: Add disconnected node status (#3300)
* create peer ack table

* add restricted status

* add disconnected status
2025-01-28 11:26:12 +05:30
abhishek9686
8de1f4f6ca single node acl group 2025-01-20 17:43:13 +05:30
abhishek9686
04f2dc2d6a fix status for new added nodes 2025-01-09 22:16:02 +05:30
abhishek9686
4b41e86901 improvise status check 2025-01-09 09:34:27 +05:30
abhishek9686
739f6ade90 check last check in against metrics interval 2025-01-07 21:23:49 +05:30
abhishek9686
088b4ef82b check peer status 2025-01-06 17:54:04 +04:00
Abhishek K
629f6d3d70
Merge pull request #3273 from gravitl/NET-1784-release
add all proto to default user policies
2024-12-20 20:11:57 +04:00
abhishek9686
27088a3595 add all proto to default user policies 2024-12-20 20:10:57 +04:00
Abhishek K
93219dd984
Merge pull request #3265 from gravitl/NET-1784-release
Net 1784 release
2024-12-18 23:09:02 +04:00
abhishek9686
73eed6e583 add service type to default acl polices 2024-12-18 16:18:24 +04:00
Aceix
b1d813a906
refactor: update metadata descriptions for Network User permissions (#3254) 2024-12-17 15:32:47 +04:00
Yabin Ma
1d9e0f71df
add format int64 for swagger doc (#3247)
* add format int64 for swagger doc

* add format int64 for swagger doc

* add global override

* swagger doc update

* add new line for .swaggo file
2024-12-12 09:52:52 +04:00
Abhishek K
f124b10c35
NET-1784: Adv Acl Rules (#3239)
* define direction on acl req

* define protocol types and rule model

* get rules for node

* fetch acl rule for a node

* redine acl firewall model

* add json tags

* update port,protocol, and direction

* add json tags to acl options

* convert protocol to string

* simplify acl map

* add json tags to acl rules

* add networks to fw update

* add acls rules

* NET-1784: add allow all field

* add allow all field on fw udpate

* remove debug logs

* fix port and protocol types

* migrate default acl policies

* define constants for service types

* add adv options for user rules on ingress gw

* debug log

* allow whole network

* add static nodes to acl rules

* replace peers on acl updates

* initiliase rule map

* add user acl rules on target node

* revert acl check on extclient

* handle static node rules on ingress gw

* update multiple policies for users

* check allowed direction

* remove debug logs
2024-12-10 11:21:14 +04:00
Abhishek K
31c2311bef
NET-1782: Fetch Node Connection Status from metrics (#3237)
* add live status of node

* handle static node status

* add public IP field to server configuration

* get public Ip from config

* improve node status logic

* improvise status check

* use only checkin status on old nodes

---------

Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2024-12-10 10:46:05 +04:00
Vishal Dalwadi
496d541822
NET-1833: add retries to license key validation. (#3222)
* feat(go): add retries to license key validation.

* feat(go): increase the number of retries.
2024-12-10 10:15:54 +04:00
Yabin Ma
c56f1cab15
fix swagger generation issue (#3241) 2024-12-10 08:41:41 +04:00
Abhishek K
dbb44c2867
Merge pull request #3217 from gravitl/failover_mutex
Failover mutex
2024-11-23 18:18:36 +04:00
abhishek9686
44769aff31 ignore request to failover peer 2024-11-22 19:31:50 +04:00