Commit graph

169 commits

Author SHA1 Message Date
Max Ma
19d4fbaa24
NET-1497:set node lastcheckin after reboot (#3102)
* set node lastcheckin after reboot

* set node connected after reboot
2024-09-16 18:41:11 +04:00
Abhishek K
2e8d95e80e
NET-1227: User Mgmt V2 (#3055)
* user mgmt models

* define user roles

* define models for new user mgmt and groups

* oauth debug log

* initialize user role after db conn

* print oauth token in debug log

* user roles CRUD apis

* user groups CRUD Apis

* additional api checks

* add additional scopes

* add additional scopes url

* add additional scopes url

* rm additional scopes url

* setup middlleware permission checks

* integrate permission check into middleware

* integrate permission check into middleware

* check for headers for subjects

* refactor user role models

* refactor user groups models

* add new user to pending user via RAC login

* untracked

* allow multiple groups for an user

* change json tag

* add debug headers

* refer network controls form roles, add debug headers

* refer network controls form roles, add debug headers

* replace auth checks, add network id to role model

* nodes handler

* migration funcs

* invoke sync users migration func

* add debug logs

* comment middleware

* fix get all nodes api

* add debug logs

* fix middleware error nil check

* add new func to get username from jwt

* fix jwt parsing

* abort on error

* allow multiple network roles

* allow multiple network roles

* add migration func

* return err if jwt parsing fails

* set global check to true when accessing user apis

* set netid for acls api calls

* set netid for acls api calls

* update role and groups routes

* add validation checks

* add invite flow apis and magic links

* add invited user via oauth signup automatically

* create invited user on oauth signup, with groups in the invite

* add group validation for user invite

* update create user handler with new role mgmt

* add validation checks

* create user invites tables

* add error logging for email invite

* fix invite singup url

* debug log

* get query params from url

* get query params from url

* add query escape

* debug log

* debug log

* fix user signup via invite api

* set admin field for backward compatbility

* use new role id for user apis

* deprecate use of old admin fields

* deprecate usage of old user fields

* add user role as service user if empty

* setup email sender

* delete invite after user singup

* add plaform user role

* redirect on invite verification link

* fix invite redirect

* temporary redirect

* fix invite redirect

* point invite link to frontend

* fix query params lookup

* add resend support, configure email interface types

* fix groups and user creation

* validate user groups, add check for metrics api in middleware

* add invite url to invite model

* migrate rac apis to new user mgmt

* handle network nodes

* add platform user to default role

* fix user role migration

* add default on rag creation and cleanup after deletion

* fix rac apis

* change to invite code param

* filter nodes and hosts based on user network access

* extend create user group req to accomodate users

* filter network based on user access

* format oauth error

* move user roles and groups

* fix get user v1 api

* move user mgmt func to pro

* add user auth type to user model

* fix roles init

* remove platform role from group object

* list only platform roles

* add network roles to invite req

* create default groups and roles

* fix middleware for global access

* create default role

* fix nodes filter with global network roles

* block selfupdate of groups and network roles

* delete netID if net roles are empty

* validate user roles nd groups on update

* set extclient permission scope when rag vpn access is set

* allow deletion of roles and groups

* replace _ with - in role naming convention

* fix failover middleware mgmt

* format oauth templates

* fetch route temaplate

* return err if user wrong login type

* check user groups on rac apis

* fix rac apis

* fix resp msg

* add validation checks for admin invite

* return oauth type

* format group err msg

* fix html tag

* clean up default groups

* create default rag role

* add UI name to roles

* remove default net group from user when deleted

* reorder migration funcs

* fix duplicacy of hosts

* check old field for migration

* from pro to ce make all secondary users admins

* from pro to ce make all secondary users admins

* revert: from pro to ce make all secondary users admins

* make sure downgrades work

* fix pending users approval

* fix duplicate hosts

* fix duplicate hosts entries

* fix cache reference issue

* feat: configure FRONTEND_URL during installation

* disable user vpn access when network roles are modified

* rm vpn acces when roles or groups are deleted

* add http to frontend url

* revert crypto version

* downgrade crytpo version

* add platform id check on user invites

---------

Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2024-08-20 17:08:56 +05:30
Max Ma
46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Max Ma
f63dfaf4b5
NET-1288:add egress range check with netmaker network address (#2978)
* add egress range check with netmaker network address

* add egerssrange check for delete egressGW and extClientGW

* remove egress range check for delete
2024-07-05 15:53:46 +05:30
abhishek9686
b2361be29b track failover usage 2024-04-17 17:22:47 +05:30
abhishek9686
8e0580764b remove debug logs 2024-03-04 13:14:51 +07:00
abhishek9686
c25a732015 fix acl mutex lock 2024-03-04 13:11:51 +07:00
abhishek9686
0dae81438f add debug logs to removing host from network 2024-03-04 12:38:21 +07:00
abhishek9686
e3b6b060c3 add debug logs to removing host from network 2024-03-04 12:06:36 +07:00
Abhishek K
0638dcac49
NET-940: Inet Gws (#2828)
* internet gws apis

* add validate check for inet request

* add default gw changes to peer update

* update json tag

* add OS checks for inet gws

* add set defaul gw pro func

* allow disable and enable inet gw

* add inet handlers to pro

* add fields to api node

* add inet allowed ips

* add default gw to pull

* unset node inet details on deletion

* unset internet gw on network nodes

* unset inet gw fix

* unset inet gw fix

* send default gw ip

* fix inet node endpoint

* add default gw endpoint ip to pull resp

* validate after unset gws

* add inet client peer allowedips to inet node

* validate after unset gws

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix inet gw and relayed conflict

* fix inet gw and relayed conflict

* fix update req

* fix update inet gw api

* when inet gw is peer ignore other allowedIps

* test relay

* revert test relay

* revert inet peer update changes

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* add check for relayed node

* add inet info to peer update

* add inet info to peer update

* fix update node to persist inet info

* fix go tests

* egress ranges with inet gw fix

* egress ranges with inet gw fix

* disallow node acting using inet gw to act as inet gw

* add check to validate inet gw

* fix typos

* add firewall check

* set inetgw on ingress req on community

* set inetgw to false on community on ingress del
2024-02-28 08:46:51 +07:00
Abhishek K
718f739b7d
NET-891: Send deleted node peer update when expired (#2763)
* send deleted node peer update when expired

* set default expiration to 100 years

* pruge expired nodes

* send mq node delete update before deleting node

* initalise to new var

* revert expiration time interval
2024-01-12 16:01:40 +05:30
Abhishek K
2c4a27c53b
NET-735: HA Support (#2701)
* cache enabled option, cache hosts data if only enabled

* cache nodes only when enabled

* cache extclients only when enabled

* cache acls only when enabled
2023-12-05 12:01:35 +04:00
Abhishek K
b78cc0a8a1
NET-725: Failovers (#2685)
* api to  to get host relayed from client

* add auto relay to api host

* add peer nat type

* set pro field on signal

* rm net check on relay me handler

* return success response

* re-establish failover logic

* set failOver ctx

* failOver with peer pub key

* failovered peer updates

* failover handlers, reset failovered peer on deletion

* rm unused funcs

* initialize failover handler on EE

* ignore failover node on signal

* failover changes

* set host id on signal

* extend signal model to include node ids

* add backwards compatibility

* add failover as node api

* set json response on failover handers

* add failover field to api node

* fix signal data check

* initialize failover peer map

* reset failovered status when relayed or deleted

* add failover info to api node

* reset network failover

* only proceed furtuer if failover exists in the network

* set failOver node defaults

* cannot set failover node as relayed

* debug log

* debug log

* debug changes

* debug changes

* debug changes

* revert debug changes

* don't add peers to idmap when removed

* reset failed Over

* fix static checks

* rm debug log

* add check for linux host
2023-11-29 20:10:07 +04:00
Abhishek K
7169db78ea
NET-717: DNS Fixes (#2654)
* send dns with complete entry name

* publish dns records on adding host to network from UI

* publish node dns updates when host is removed
2023-11-08 17:43:41 +04:00
Abhishek K
7a9dc3458f
NET-667: force delete daemon node (#2645)
* force delete zombie node

* return correct resp

* fix zombie hosts processing

* add nil check rather checking error

* pr comments
2023-10-31 14:34:28 +04:00
Gabriel de Souza Seibel
cb4b99ffcb
[NET-562] Persistent Keep Alive from node to host (#2604)
* Move PKA field from models node to host level

* Move PKA field from api models node to host level

* Adapt logic package to node->host PKA

* Adapt migration-related code to node->host PKA

* Adapt cli code to node->host PKA

* Change host PKA default to 20s

* On IfaceDelta, check for PKA on host

* On handleHostRegister, set default PKA

* Use a default PKA

* Use int64 for api host pka

* Reorder imports

* Don't use host pka in iface delta

* Fix ConvertAPIHostToNMHost

* Add swagger doc for host PKA field

* Fix swagger.yml

* Set default PKA only for new hosts

* Remove TODO comment

* Remove redundant check

* Have api-host pka be specified in seconds
2023-10-06 10:09:19 +04:00
Farukh Khan
9870920aac
NET-541 (#2544)
* NET-541

* Updated both zombie cleanup and expired nodes cleanup go routines to use tick timers for execution.

* NET-541 updated to ticker

* Changed from timer to ticker for periodic execution of zombie cleanup and expired nodes cleanup go routines.

---------

Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
2023-09-11 09:47:01 +05:30
Abhishek K
719e0c254d
NET-551: User Mgmt Re-Design (#2547)
* add superadmin role, apis to create superadmin user

* apis to attach and remove user from remote access gateways

* add api to list user's remote client has gateway clients

* remove code related user groups

* remove networks and groups from user model

* refactor user CRUD operations

* fix network permission test

* add superadmin to authorize func

* remove user network and groups from cli

* api to transfer superadmin role

* add api to list users on a ingress gw

* restrict user access to resources on server

* deny request from remote access client if extclient is already created

* fix user tests

* fix static checks

* fix static checks

* add limits to extclient create handler

* set username to superadmin on if masterkey is used

* allow creation of extclients using masterkey

* add migration func to assign superadmin role for existing admin user

* check for superadmin on migration if users are present

* allowe masterkey to extcleint apis

* check ownerid

* format error, on jwt token verification failure return unauthorized rather than forbidden

* user update fix

* move user remote functionality to ee

* fix update user api

* security patch

* initalise ee user handlers

* allow user to use master key to update any user

* use slog

* fix auth user test

* table headers

* remove user role, it's covered in middleware

* setuser defaults fix
2023-09-01 14:27:08 +05:30
Gabriel de Souza Seibel
1a1ba1ccf4
[NET-546] Move ee code to ee package, unify ee status and terminology (#2538)
* Move ee code to ee package and unify ee status to IsPro

* Consolidate naming for paid/professional/enterprise version as "pro". Notes:

- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"

* Revert docker image tag being -pro, back to -ee

* Revert go build tag being pro, back to ee

* Add build tags for some ee content

* [2] Revert go build tag being pro, back to ee

* Fix test workflow

* Add a json tag to be backwards compatible with frontend "IsEE" check

* Add a json tag for the serverconfig struct for IsEE

* Ammend json tag to Is_EE

* fix ee tags

---------

Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
2023-09-01 07:42:05 +05:30
Farukh Khan
a9a8596d71
NET-507 (#2506)
* NET-507

* Fixed server restart changing the node expiration date to the defaults.

* Removed expired nodes removal from zombie cleanup routine.

* Added a new expired nodes deletion routine which removes expired nodes every hour.

* NET-507 suggested changes

* Possible fix for zombie nodes upon node deletion from the UI.

* Suggested changes implemented for expired nodes deletion go routine.

* NET-507 typo fix

* typo fix for DeleteNode purge parameter
2023-08-10 20:27:44 +05:30
Gabriel de Souza Seibel
7e8b66e03d
[NET-353] Solve race condition with "unique" ips (#2461)
* Use a lock for getting unique ips

* Make getting & setting unique ips atomic

* Remove some blank lines

* Set addressLock to &sync.Mutex on declaration
2023-07-27 11:34:15 +05:30
Abhishek Kondur
27ac920069 cache acls 2023-06-27 16:12:06 +05:30
Abhishek Kondur
307203105d unexport cache funcs 2023-06-27 16:10:44 +05:30
Abhishek Kondur
4071bb2bac db cache nodes and hosts 2023-06-27 16:10:04 +05:30
Abhishek K
6c2582651c
Net-137: Node relays (#2401)
* revert relays

* initial relay commit

* get relayed allowed ips

* add more relay validation checks, peer logic cleanup

* rm relayed nodes from relay node when relayed node is deleted

* fix egress updates for relayed nodes

* rm unused func

* remove  debug logs

* avoid adding egress ranges on the relayed gw node

---------

Co-authored-by: Matthew R Kasun <mkasun@nusak.ca>
2023-06-19 12:26:14 -04:00
Abhishek K
78e26c4673
add nodes to pull response (#2371) 2023-06-06 22:16:21 -04:00
walkerwmanuel
9e56135424 Added user sorting and fixed error 2023-04-03 14:36:38 -04:00
walkerwmanuel
09872e9ec5 added sort logic 2023-03-30 16:04:12 -04:00
0xdcarns
f25421f6c4 fixed update issue 2023-03-22 10:48:54 -04:00
0xdcarns
28119c22ee added context to peer updates, moved nodes to memory 2023-03-06 12:21:51 -05:00
0xdcarns
91373953e7 added ability to force delete if node is requested twice 2023-03-03 11:11:02 -05:00
0xdcarns
5014c389ca adjusted main to use one single context 2023-02-24 15:46:06 -05:00
0xdcarns
697450884d remove unused func 2023-02-21 12:02:09 -05:00
Matthew R Kasun
82cac8fa30 use channels for sync 2023-02-14 07:01:38 -05:00
Matthew R Kasun
c3e253fb79 updates to zombie processing 2023-02-13 14:52:39 -05:00
Matthew R Kasun
77cfae41ff remove postup/down 2023-02-02 11:16:54 -05:00
0xdcarns
f034c74838 fixed default expiration date setting and removed unused func 2023-02-01 11:21:30 -05:00
dcarns
cdd8527b51
Merge pull request #1971 from gravitl/bugfix_node_update
Fix address range check
2023-01-26 08:33:31 -05:00
Anish Mukherjee
1b43d6bf1c increase log verbosity for legacy nodes 2023-01-26 14:41:50 +05:30
Anish Mukherjee
131f70b2a4 fix address range check 2023-01-25 16:01:46 +05:30
Abhishek Kondur
73386e8f2e set node defaults 2023-01-24 12:37:12 +04:00
dcarns
0d99b507d6
Merge pull request #1938 from gravitl/mkasun/gra-989-delete_node
when deleting node from UI/API add node to zombie list
2023-01-16 16:41:36 -05:00
Matthew R Kasun
d3b5672029 when deleting node from UI/API add node to zombie list 2023-01-16 13:33:39 -05:00
0xdcarns
e4d44b5fab ee update 2023-01-14 12:00:44 -05:00
Anish Mukherjee
797fd5b211 ignore legacy nodes 2023-01-12 14:10:07 +05:30
Abhishek Kondur
ca3d9dc40b fix node update 2023-01-06 16:58:41 +05:30
Abhishek Kondur
9f0ef1628c node update fix, nil pointer errors fix 2023-01-06 00:29:55 +05:30
Abhishek Kondur
8a9241d5e2 fix keepalive conversion type 2022-12-30 11:43:48 +05:30
Abhishek Kondur
ba2466aa25 delete node fix, mq modify client if host is already created 2022-12-29 19:25:35 +05:30
Matthew R. Kasun
319193c18a update IsIPUnique
also:
 - remove unnecessary call to UpertHost in create node
 - remove debugging logs
2022-12-23 18:06:59 -05:00