Commit graph

59 commits

Author SHA1 Message Date
abhishek9686 6a1eb76633 add return response for user invites 2024-09-14 11:53:42 +04:00
abhishek9686 30309a4f9a add email validation 2024-09-11 15:43:58 +04:00
abhishek9686 b3a9ffd260 Merge branch 'develop' of https://github.com/gravitl/netmaker into ACC-638 2024-09-06 12:04:22 +04:00
Max Ma bbca20e463
NET-1565:fix extClient ip conflict issue (#3082)
* fix extClient ip conflict issue

* Update users.go

---------

Co-authored-by: Abhishek K <abhishek@netmaker.io>
2024-08-28 18:58:07 +05:30
Abhishek K a39da31fa6
add list roles to pro and ce (#3072) 2024-08-27 11:51:13 +05:30
Abhishek K 936e1b4d45
Net 1227 v1 (#3066)
* generalise smtp config

* copy over smtp vars

* env new line

* fix master key api access

* comment user tests

* fix network and user invite for master key access

* remove email sender type

* user mgmt commands

* check user role on CE

* user role nmtcl cmds

* user groups commands

* fix role and groups command

* fix user create cmd

* add usage info

* rm user role check

* fix user update cmd

* fix static check

* add backwards comptability support for extclient api for mobile

* rm debug logs

* set frontend url from base domain if empty
2024-08-26 18:27:45 +05:30
abhishek9686 fc2fff9f9d Merge branch 'develop' of https://github.com/gravitl/netmaker into ACC-638 2024-08-22 12:43:38 +05:30
Abhishek K fe57a6a42d
Net 1227 v1 (#3062)
* generalise smtp config

* copy over smtp vars

* env new line

* fix master key api access

* comment user tests

* fix network and user invite for master key access
2024-08-22 11:55:08 +05:30
abhishek9686 27ceaed739 resolve merge conflicts 2024-08-20 18:32:49 +05:30
abhishek9686 bc42f1ebf0 fix saas frontend url 2024-08-20 17:53:56 +05:30
Abhishek K 2e8d95e80e
NET-1227: User Mgmt V2 (#3055)
* user mgmt models

* define user roles

* define models for new user mgmt and groups

* oauth debug log

* initialize user role after db conn

* print oauth token in debug log

* user roles CRUD apis

* user groups CRUD Apis

* additional api checks

* add additional scopes

* add additional scopes url

* add additional scopes url

* rm additional scopes url

* setup middlleware permission checks

* integrate permission check into middleware

* integrate permission check into middleware

* check for headers for subjects

* refactor user role models

* refactor user groups models

* add new user to pending user via RAC login

* untracked

* allow multiple groups for an user

* change json tag

* add debug headers

* refer network controls form roles, add debug headers

* refer network controls form roles, add debug headers

* replace auth checks, add network id to role model

* nodes handler

* migration funcs

* invoke sync users migration func

* add debug logs

* comment middleware

* fix get all nodes api

* add debug logs

* fix middleware error nil check

* add new func to get username from jwt

* fix jwt parsing

* abort on error

* allow multiple network roles

* allow multiple network roles

* add migration func

* return err if jwt parsing fails

* set global check to true when accessing user apis

* set netid for acls api calls

* set netid for acls api calls

* update role and groups routes

* add validation checks

* add invite flow apis and magic links

* add invited user via oauth signup automatically

* create invited user on oauth signup, with groups in the invite

* add group validation for user invite

* update create user handler with new role mgmt

* add validation checks

* create user invites tables

* add error logging for email invite

* fix invite singup url

* debug log

* get query params from url

* get query params from url

* add query escape

* debug log

* debug log

* fix user signup via invite api

* set admin field for backward compatbility

* use new role id for user apis

* deprecate use of old admin fields

* deprecate usage of old user fields

* add user role as service user if empty

* setup email sender

* delete invite after user singup

* add plaform user role

* redirect on invite verification link

* fix invite redirect

* temporary redirect

* fix invite redirect

* point invite link to frontend

* fix query params lookup

* add resend support, configure email interface types

* fix groups and user creation

* validate user groups, add check for metrics api in middleware

* add invite url to invite model

* migrate rac apis to new user mgmt

* handle network nodes

* add platform user to default role

* fix user role migration

* add default on rag creation and cleanup after deletion

* fix rac apis

* change to invite code param

* filter nodes and hosts based on user network access

* extend create user group req to accomodate users

* filter network based on user access

* format oauth error

* move user roles and groups

* fix get user v1 api

* move user mgmt func to pro

* add user auth type to user model

* fix roles init

* remove platform role from group object

* list only platform roles

* add network roles to invite req

* create default groups and roles

* fix middleware for global access

* create default role

* fix nodes filter with global network roles

* block selfupdate of groups and network roles

* delete netID if net roles are empty

* validate user roles nd groups on update

* set extclient permission scope when rag vpn access is set

* allow deletion of roles and groups

* replace _ with - in role naming convention

* fix failover middleware mgmt

* format oauth templates

* fetch route temaplate

* return err if user wrong login type

* check user groups on rac apis

* fix rac apis

* fix resp msg

* add validation checks for admin invite

* return oauth type

* format group err msg

* fix html tag

* clean up default groups

* create default rag role

* add UI name to roles

* remove default net group from user when deleted

* reorder migration funcs

* fix duplicacy of hosts

* check old field for migration

* from pro to ce make all secondary users admins

* from pro to ce make all secondary users admins

* revert: from pro to ce make all secondary users admins

* make sure downgrades work

* fix pending users approval

* fix duplicate hosts

* fix duplicate hosts entries

* fix cache reference issue

* feat: configure FRONTEND_URL during installation

* disable user vpn access when network roles are modified

* rm vpn acces when roles or groups are deleted

* add http to frontend url

* revert crypto version

* downgrade crytpo version

* add platform id check on user invites

---------

Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2024-08-20 17:08:56 +05:30
Abhishek K 5a561b3835
Net 1440 batchpeerupdate (#3042)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* batch peerUpdate

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* add peerUpdate batch size to config

* update batch peerUpdate

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed

* add batch peerUpdate switch

* set batch peerUpdate to true by default

---------

Co-authored-by: Max Ma <mayabin@gmail.com>
2024-08-16 15:35:43 +05:30
abhishek9686 d4af26b289 handle invite link for saas tenants 2024-08-16 15:32:23 +05:30
abhishek9686 f4a293ab27 fix pending users approval 2024-08-16 13:31:46 +05:30
abhishek9686 2799dc95d0 resolve merge conflicts 2024-08-16 11:43:17 +05:30
Max Ma 46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Sayan Mallick c551c487ca
New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
abhishek9686 b160445a18 add validation checks for admin invite 2024-08-09 17:06:26 +05:30
abhishek9686 30e1272ae9 fix rac apis 2024-08-07 14:07:01 +05:30
abhishek9686 43a0ca20d7 set extclient permission scope when rag vpn access is set 2024-08-02 14:25:58 +05:30
abhishek9686 9adb36b82c delete netID if net roles are empty 2024-08-02 13:53:41 +05:30
abhishek9686 5f53887c0e fix middleware for global access 2024-08-01 17:42:37 +05:30
abhishek9686 3820e7dcfe create default groups and roles 2024-08-01 08:21:46 +05:30
abhishek9686 ce226122ef add network roles to invite req 2024-08-01 00:01:01 +05:30
abhishek9686 ad4b6f9cfc list only platform roles 2024-07-31 12:56:15 +05:30
abhishek9686 eed4a7fa18 remove platform role from group object 2024-07-31 12:33:46 +05:30
abhishek9686 a8c796b735 fix roles init 2024-07-31 11:33:09 +05:30
abhishek9686 a7a431b36b move user roles and groups 2024-07-29 16:41:57 +05:30
abhishek9686 f6ae4788a1 fix rac apis 2024-07-24 16:08:59 +05:30
abhishek9686 9abc892c5a migrate rac apis to new user mgmt 2024-07-22 00:00:26 +05:30
abhishek9686 d8d82b933a fix merge conflicts 2024-07-16 15:54:29 +05:30
Max Ma 65faf73fe9
NET-1226: Scalability Improvements (#2987)
* add api to check if failover node existed

* remove 5 minute peerUpdate

* update peerUpdate to trigger pull

* update Action name to SignalPull

* revert the peerUpdate from SignalPull

* fix getfailover error issue

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* NET-1165: Remove creation of acls on emqx (#2996)

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* comment out mq reconnect logic

* configure mq conn params

* add metric_interval in ENV for publishing metrics

* add metric_interval in ENV for publishing metrics

* update PUBLISH_METRIC_INTERVAL env name

* revert the mq setttings back

* fix error nil issue

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
2024-07-09 18:56:55 +05:30
abhishek9686 db3deba181 deprecate use of old admin fields 2024-07-07 13:53:22 +05:30
Abhishek K 7ff30599ed
NET-1252: Restrict inetGws, Relays from getting failedOver (#2937)
* add additional checks to avoid failovers

* add failover defence check on signal handler

* only add check for victim node

* avoid failover reset on pull

* add relayed for failoverme

* misc changes for failover

* remove resetfailoverpeers for InetNode

* add egress route back to allowedip list if relayed is egressGW

* add extclient back to allowedip list if peer is ingressGW

* reset failover on pull

---------

Co-authored-by: Max Ma <mayabin@gmail.com>
2024-06-03 10:49:02 +04:00
Aceix d9237da2f6
feat(NET-1207): show network info on RAC (#2949) 2024-06-03 10:25:07 +04:00
Abhishek K 4a355ee9f3
add gw listen port (#2931) 2024-05-09 13:00:21 +05:30
Aceix abf3f4f55d
feat(NET-1106): support additional RAG endpoint IPs (#2907) 2024-05-08 14:45:05 +05:30
Max Ma da11dc8a87
NET-1134:move oauth from CE build block to pro (#2919)
* move oauth from CE build block to pro

* move oauth code and api handler under pro

* move common func back to auth from pro/auth

* change log level to Info for information logs

* fix import issue
2024-05-08 12:44:39 +05:30
Max Ma 1ad837ad72
NET-1164:PublishDeletedClientPeerUpdate when deleting user from RAGW (#2912)
* PublishDeletedClientPeerUpdate when deleting user from RAGW

* update ACLs after extClient is removed
2024-05-03 12:25:38 +05:30
Max Ma 961f8eab6e
NET-1119 (#2886)
* exclude IngressGW in failover

* resetfailoverpeer when adding IngressGw if failover enabled

* exclude InetGW in failover

* get egress ranges of failedover peer

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
2024-04-12 18:22:03 +05:30
Abhishek K 66069fbc34
NET-1082: Scale Testing Fixes (#2894)
* add additional mutex lock on node acls func

* increase verbosity

* disable acls on cloud emqx

* add emqx creds creation to go routine

* add debug log of mq client id

* comment port check

* uncomment port check

* check for connection mq connection open

* use username for client id

* add write mutex on acl is allowed

* add mq connection lost handler on server

* spin off zombie init as go routine

* get whole api path from config

* Revert "get whole api path from config"

This reverts commit 392f5f4c5f.

* update extclient acls async

* add additional mutex lock on node acls func

(cherry picked from commit 5325f0e7d7)

* increase verbosity

(cherry picked from commit 705b3cf0bf)

* add emqx creds creation to go routine

(cherry picked from commit c8e65f4820)

* add debug log of mq client id

(cherry picked from commit 29c5d6ceca)

* comment port check

(cherry picked from commit db8d6d95ea)

* check for connection mq connection open

(cherry picked from commit 13b11033b0)

* use username for client id

(cherry picked from commit e90c7386de)

* add write mutex on acl is allowed

(cherry picked from commit 4cae1b0bb4)

* add mq connection lost handler on server

(cherry picked from commit c82918ad35)

* spin off zombie init as go routine

(cherry picked from commit 6d65c44c43)

* update extclient acls async

(cherry picked from commit 6557ef1ebe)

* additionl logs for oauth user flow

(cherry picked from commit 61703038ae)

* add more debug logs

(cherry picked from commit 5980beacd1)

* add more debug logs

(cherry picked from commit 4d001f0d27)

* add set auth secret

(cherry picked from commit f41cef5da5)

* fix fetch pass

(cherry picked from commit 825caf4b60)

* make sure auth secret is set only once

(cherry picked from commit ba33ed02aa)

* make sure auth secret is set only once

(cherry picked from commit 920ac4c507)

* comment usage of emqx acls

* replace  read lock with write lock on acls

* replace  read lock with write lock on acls

(cherry picked from commit 808d2135c8)

* use deadlock pkg for visibility

* add additional mutex locks

* remove race flag

* on mq re-connecting donot exit if failed

* on mq re-connecting donot exit if failed

* revert mutex package change

* set mq clean session

* remove debug log

* go mod tidy

* revert on prem emqx acls del
2024-04-11 21:18:57 +05:30
Aceix 4871a0dd1c
fix(NET-1009): fix client connectivity metrics data (#2858) 2024-03-20 15:11:11 +07:00
Abhishek K c7e673fb9f
ACC-532: set mq clean session to true (#2865)
* set clean session

* delete emqx hosts creds api

* add emqx hosts del api to limited middleware controller

* add emqx hosts del api to limited middleware controller

* remove server creds from emqx
2024-03-20 15:03:41 +07:00
Abhishek K 0638dcac49
NET-940: Inet Gws (#2828)
* internet gws apis

* add validate check for inet request

* add default gw changes to peer update

* update json tag

* add OS checks for inet gws

* add set defaul gw pro func

* allow disable and enable inet gw

* add inet handlers to pro

* add fields to api node

* add inet allowed ips

* add default gw to pull

* unset node inet details on deletion

* unset internet gw on network nodes

* unset inet gw fix

* unset inet gw fix

* send default gw ip

* fix inet node endpoint

* add default gw endpoint ip to pull resp

* validate after unset gws

* add inet client peer allowedips to inet node

* validate after unset gws

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix inet gw and relayed conflict

* fix inet gw and relayed conflict

* fix update req

* fix update inet gw api

* when inet gw is peer ignore other allowedIps

* test relay

* revert test relay

* revert inet peer update changes

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* add check for relayed node

* add inet info to peer update

* add inet info to peer update

* fix update node to persist inet info

* fix go tests

* egress ranges with inet gw fix

* egress ranges with inet gw fix

* disallow node acting using inet gw to act as inet gw

* add check to validate inet gw

* fix typos

* add firewall check

* set inetgw on ingress req on community

* set inetgw to false on community on ingress del
2024-02-28 08:46:51 +07:00
Abhishek K ecd769ea07
Merge pull request #2780 from gravitl/kwesi/net-709-add-metadata-field-to-gateway-node
feat(NET-709): add node metadata for remote gateways
2024-02-06 14:59:59 +05:30
abhishek9686 43e82a2936 make defaul host failover and remote access gw on joining new network 2024-02-05 11:18:28 +05:30
the_aceix 5882b86d40 feat(NET-709): add node metadata for remote gws 2024-01-26 10:06:36 +00:00
Abhishek K 5bf30b2c10
NET-877: Replace peers on Refreshkeys peer update (#2761)
* replace peers on key refresh

* add peer conf to metrics map only when allowed
2024-01-11 15:59:19 +05:30
Abhishek K 8bfd791135
Net 822 (#2759)
* add extclient allowed ips to rac resp

* add gw peer public key

* fix static check

* add param for remote access client id

* don't decode body

* debug

* debug

* change user gws req to POST

* debug

* change api to GET

* send array for user gws if req came from mobile
2024-01-09 13:25:18 +05:30
Abhishek K 9fcefd7911
Net 822 (#2745)
* add extclient allowed ips to rac resp

* add gw peer public key

* fix static check

* add param for remote access client id

* don't decode body

* debug

* debug

* change user gws req to POST

* debug

* change api to GET
2023-12-20 13:17:34 +04:00