Commit graph

208 commits

Author SHA1 Message Date
Abhishek K
96e2c29783
v1.0.0 release notes (#3530)
* add v1.0.0 release notes

* add v1.0.0 release notes

* update version tags
2025-06-26 11:35:35 +05:30
Vishal Dalwadi
d9f910c29e feat(go): close the db; 2025-06-10 11:06:39 +05:30
Abhishek K
81d8e61eca
Merge branch 'release-v0.99.0' into patch/db-conn-pool 2025-06-10 09:51:13 +05:30
Abhishek K
238c68eb2f
v0.99.0 Release Notes (#3489)
* add v0.99 release notes

* version bump

* add formating on release notes

* add formating on release notes

* Update release.md
2025-06-06 17:29:11 +05:30
Vishal Dalwadi
48957c02fa feat(go): use single db handle; use connection pool; 2025-06-06 15:26:23 +05:30
Abhishek K
a1304b43d8
NET-2054: Auto Removal of Offline Nodes, fix enrollment key relay function (#3458)
* check host ports on join

* if 443 not available fallback to 51821

* if 443 not available fallback to 51821

* add config for auto delete of offline nodes

* autocleanup offline nodes

* delete offline nodes on startup

* fix relay via join token
2025-05-24 08:21:47 +05:30
Abhishek K
ca95954fb5
NET-2000: Api access tokens (#3418)
* feat: api access tokens

* revoke all user tokens

* redefine access token api routes, add auto egress option to enrollment keys

* fix revoked tokens to be unauthorized

* remove unused functions

* convert access token to sql schema

* switch access token to sql schema

* revoke token generated by an user

* add user token creation restriction by user role

* add forbidden check for access token creation

* revoke user token when group or role is changed

* add default group to admin users on update

* fix token removal on user update

* fix token removal on user update
2025-04-23 20:21:42 +04:00
abhishek9686
137303a687 resolve merge conflicts 2025-03-28 16:14:07 +04:00
abhishek9686
25d07964d7 disable server start notify 2025-03-28 12:41:14 +04:00
Abhishek K
06f6f5ca98
update release versions (#3387)
* update release versions

* update release versions

* update release versions
2025-03-25 11:08:54 +04:00
Abhishek K
70ffdef387
update release version (#3248) 2024-12-10 15:16:54 +04:00
Yabin Ma
5f21c8bb1d
NET-1778: scale test code changes (#3203)
* comment ACL call and add debug message

* add cache for network nodes

* fix load node to network cache issue

* add peerUpdate call 1 min limit

* add debug log for scale test

* release maps

* avoid default policy for node

* 1 min limit for peerUpdate trigger

* mq options

* Revert "mq options"

This reverts commit 10b93d0118.

* set peerUpdate run in sequence

* update for emqx 5.8.2

* remove batch peer update

* change the sleep to 10 millisec to avoid timeout

* add compress and change encrypt for peerUpdate message

* add mem profiling and automaxprocs

* add failover ctx mutex

* ignore request to failover peer

* remove code without called

* remove debug logs

* update emqx to v5.8.2

* change broker keepalive

* add OLD_ACL_SUPPORT setting

* add host version check for message encrypt

* remove debug message

* remove peerUpdate call control

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
2024-12-10 10:15:31 +04:00
Max Ma
e8d3e80b08 initialize cache in startup 2024-11-29 11:35:34 +01:00
abhishek9686
486da75786 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-1615 2024-10-30 20:56:41 +04:00
abhishek9686
019f179b85 update api docs version 2024-10-30 20:55:51 +04:00
abhishek9686
046e88366f update release version v0.26.0 2024-10-30 12:42:29 +04:00
abhishek9686
4eacadbb27 convert network roles to groups, deprecate RAG roles 2024-10-28 16:18:50 +04:00
Abhishek K
2e8d95e80e
NET-1227: User Mgmt V2 (#3055)
* user mgmt models

* define user roles

* define models for new user mgmt and groups

* oauth debug log

* initialize user role after db conn

* print oauth token in debug log

* user roles CRUD apis

* user groups CRUD Apis

* additional api checks

* add additional scopes

* add additional scopes url

* add additional scopes url

* rm additional scopes url

* setup middlleware permission checks

* integrate permission check into middleware

* integrate permission check into middleware

* check for headers for subjects

* refactor user role models

* refactor user groups models

* add new user to pending user via RAC login

* untracked

* allow multiple groups for an user

* change json tag

* add debug headers

* refer network controls form roles, add debug headers

* refer network controls form roles, add debug headers

* replace auth checks, add network id to role model

* nodes handler

* migration funcs

* invoke sync users migration func

* add debug logs

* comment middleware

* fix get all nodes api

* add debug logs

* fix middleware error nil check

* add new func to get username from jwt

* fix jwt parsing

* abort on error

* allow multiple network roles

* allow multiple network roles

* add migration func

* return err if jwt parsing fails

* set global check to true when accessing user apis

* set netid for acls api calls

* set netid for acls api calls

* update role and groups routes

* add validation checks

* add invite flow apis and magic links

* add invited user via oauth signup automatically

* create invited user on oauth signup, with groups in the invite

* add group validation for user invite

* update create user handler with new role mgmt

* add validation checks

* create user invites tables

* add error logging for email invite

* fix invite singup url

* debug log

* get query params from url

* get query params from url

* add query escape

* debug log

* debug log

* fix user signup via invite api

* set admin field for backward compatbility

* use new role id for user apis

* deprecate use of old admin fields

* deprecate usage of old user fields

* add user role as service user if empty

* setup email sender

* delete invite after user singup

* add plaform user role

* redirect on invite verification link

* fix invite redirect

* temporary redirect

* fix invite redirect

* point invite link to frontend

* fix query params lookup

* add resend support, configure email interface types

* fix groups and user creation

* validate user groups, add check for metrics api in middleware

* add invite url to invite model

* migrate rac apis to new user mgmt

* handle network nodes

* add platform user to default role

* fix user role migration

* add default on rag creation and cleanup after deletion

* fix rac apis

* change to invite code param

* filter nodes and hosts based on user network access

* extend create user group req to accomodate users

* filter network based on user access

* format oauth error

* move user roles and groups

* fix get user v1 api

* move user mgmt func to pro

* add user auth type to user model

* fix roles init

* remove platform role from group object

* list only platform roles

* add network roles to invite req

* create default groups and roles

* fix middleware for global access

* create default role

* fix nodes filter with global network roles

* block selfupdate of groups and network roles

* delete netID if net roles are empty

* validate user roles nd groups on update

* set extclient permission scope when rag vpn access is set

* allow deletion of roles and groups

* replace _ with - in role naming convention

* fix failover middleware mgmt

* format oauth templates

* fetch route temaplate

* return err if user wrong login type

* check user groups on rac apis

* fix rac apis

* fix resp msg

* add validation checks for admin invite

* return oauth type

* format group err msg

* fix html tag

* clean up default groups

* create default rag role

* add UI name to roles

* remove default net group from user when deleted

* reorder migration funcs

* fix duplicacy of hosts

* check old field for migration

* from pro to ce make all secondary users admins

* from pro to ce make all secondary users admins

* revert: from pro to ce make all secondary users admins

* make sure downgrades work

* fix pending users approval

* fix duplicate hosts

* fix duplicate hosts entries

* fix cache reference issue

* feat: configure FRONTEND_URL during installation

* disable user vpn access when network roles are modified

* rm vpn acces when roles or groups are deleted

* add http to frontend url

* revert crypto version

* downgrade crytpo version

* add platform id check on user invites

---------

Co-authored-by: the_aceix <aceixsmartx@gmail.com>
2024-08-20 17:08:56 +05:30
Max Ma
46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Sayan Mallick
c551c487ca
New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
Christopher Blaha
66dbaf1bc2
update develop to v0.25.0 (#3009)
* update develop to v0.24.4

* update develop to v0.25.0
2024-07-25 10:19:31 +05:30
Christopher Blaha
7d001b81a7
update develop to v0.24.3 (#2977) 2024-06-18 07:23:20 +05:30
Christopher Blaha
ee6cdc604b
update develop to v0.24.2 (#2938) 2024-05-17 12:47:54 +05:30
Max Ma
da11dc8a87
NET-1134:move oauth from CE build block to pro (#2919)
* move oauth from CE build block to pro

* move oauth code and api handler under pro

* move common func back to auth from pro/auth

* change log level to Info for information logs

* fix import issue
2024-05-08 12:44:39 +05:30
Christopher Blaha
8b1294b5fb
update develop to v0.24.1 (#2904) 2024-04-26 09:06:40 +05:30
Abhishek K
66069fbc34
NET-1082: Scale Testing Fixes (#2894)
* add additional mutex lock on node acls func

* increase verbosity

* disable acls on cloud emqx

* add emqx creds creation to go routine

* add debug log of mq client id

* comment port check

* uncomment port check

* check for connection mq connection open

* use username for client id

* add write mutex on acl is allowed

* add mq connection lost handler on server

* spin off zombie init as go routine

* get whole api path from config

* Revert "get whole api path from config"

This reverts commit 392f5f4c5f.

* update extclient acls async

* add additional mutex lock on node acls func

(cherry picked from commit 5325f0e7d7)

* increase verbosity

(cherry picked from commit 705b3cf0bf)

* add emqx creds creation to go routine

(cherry picked from commit c8e65f4820)

* add debug log of mq client id

(cherry picked from commit 29c5d6ceca)

* comment port check

(cherry picked from commit db8d6d95ea)

* check for connection mq connection open

(cherry picked from commit 13b11033b0)

* use username for client id

(cherry picked from commit e90c7386de)

* add write mutex on acl is allowed

(cherry picked from commit 4cae1b0bb4)

* add mq connection lost handler on server

(cherry picked from commit c82918ad35)

* spin off zombie init as go routine

(cherry picked from commit 6d65c44c43)

* update extclient acls async

(cherry picked from commit 6557ef1ebe)

* additionl logs for oauth user flow

(cherry picked from commit 61703038ae)

* add more debug logs

(cherry picked from commit 5980beacd1)

* add more debug logs

(cherry picked from commit 4d001f0d27)

* add set auth secret

(cherry picked from commit f41cef5da5)

* fix fetch pass

(cherry picked from commit 825caf4b60)

* make sure auth secret is set only once

(cherry picked from commit ba33ed02aa)

* make sure auth secret is set only once

(cherry picked from commit 920ac4c507)

* comment usage of emqx acls

* replace  read lock with write lock on acls

* replace  read lock with write lock on acls

(cherry picked from commit 808d2135c8)

* use deadlock pkg for visibility

* add additional mutex locks

* remove race flag

* on mq re-connecting donot exit if failed

* on mq re-connecting donot exit if failed

* revert mutex package change

* set mq clean session

* remove debug log

* go mod tidy

* revert on prem emqx acls del
2024-04-11 21:18:57 +05:30
Christopher Blaha
0b2422b848
update to 0.24.0 (#2893) 2024-04-11 21:05:06 +05:30
Christopher Blaha
83cb39db5a
update develop to 0.23.1 (#2835) 2024-03-05 18:08:54 +07:00
Christopher Blaha
eaa62d8f22
update develop to v0.23.0 (#2816) 2024-02-08 23:30:10 +05:30
Christopher Blaha
8449f5487c
update develop to v0.22.1 (#2779) 2024-01-25 13:21:29 +05:30
Abhishek K
718f739b7d
NET-891: Send deleted node peer update when expired (#2763)
* send deleted node peer update when expired

* set default expiration to 100 years

* pruge expired nodes

* send mq node delete update before deleting node

* initalise to new var

* revert expiration time interval
2024-01-12 16:01:40 +05:30
Christopher Blaha
6a6c55c21a
update develop (#2725) 2023-12-11 20:01:31 +04:00
Christopher Blaha
b26a18d425
update develop (#2679) 2023-11-22 11:50:49 +04:00
Gabriel de Souza Seibel
4acba03b62
Have hooks be initialized after apis (#2626) 2023-10-20 10:41:17 +04:00
Abhishek K
675c4ec34c
NET-641: Enable/Disable Auto Update From UI (#2622)
* add support to disable/enable autoupdate on a host

* setting to older version for QA to test
2023-10-19 08:59:36 +04:00
Christopher Blaha
64d2a4820c
update develop (#2633) 2023-10-18 22:51:36 +04:00
Christopher Blaha
5074e56e6a
update develop to v0.21.1 (#2567) 2023-09-12 15:43:08 +05:30
Abhishek K
719e0c254d
NET-551: User Mgmt Re-Design (#2547)
* add superadmin role, apis to create superadmin user

* apis to attach and remove user from remote access gateways

* add api to list user's remote client has gateway clients

* remove code related user groups

* remove networks and groups from user model

* refactor user CRUD operations

* fix network permission test

* add superadmin to authorize func

* remove user network and groups from cli

* api to transfer superadmin role

* add api to list users on a ingress gw

* restrict user access to resources on server

* deny request from remote access client if extclient is already created

* fix user tests

* fix static checks

* fix static checks

* add limits to extclient create handler

* set username to superadmin on if masterkey is used

* allow creation of extclients using masterkey

* add migration func to assign superadmin role for existing admin user

* check for superadmin on migration if users are present

* allowe masterkey to extcleint apis

* check ownerid

* format error, on jwt token verification failure return unauthorized rather than forbidden

* user update fix

* move user remote functionality to ee

* fix update user api

* security patch

* initalise ee user handlers

* allow user to use master key to update any user

* use slog

* fix auth user test

* table headers

* remove user role, it's covered in middleware

* setuser defaults fix
2023-09-01 14:27:08 +05:30
Gabriel de Souza Seibel
1a1ba1ccf4
[NET-546] Move ee code to ee package, unify ee status and terminology (#2538)
* Move ee code to ee package and unify ee status to IsPro

* Consolidate naming for paid/professional/enterprise version as "pro". Notes:

- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"

* Revert docker image tag being -pro, back to -ee

* Revert go build tag being pro, back to ee

* Add build tags for some ee content

* [2] Revert go build tag being pro, back to ee

* Fix test workflow

* Add a json tag to be backwards compatible with frontend "IsEE" check

* Add a json tag for the serverconfig struct for IsEE

* Ammend json tag to Is_EE

* fix ee tags

---------

Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
2023-09-01 07:42:05 +05:30
Christopher Blaha
fe470e92dc
update develop to 0.20.7 (#2522)
* update develop to 0.20.7

* change version to 0.21.0
2023-08-18 11:55:12 +05:30
Farukh Khan
a9a8596d71
NET-507 (#2506)
* NET-507

* Fixed server restart changing the node expiration date to the defaults.

* Removed expired nodes removal from zombie cleanup routine.

* Added a new expired nodes deletion routine which removes expired nodes every hour.

* NET-507 suggested changes

* Possible fix for zombie nodes upon node deletion from the UI.

* Suggested changes implemented for expired nodes deletion go routine.

* NET-507 typo fix

* typo fix for DeleteNode purge parameter
2023-08-10 20:27:44 +05:30
Christopher Blaha
44bc2461c4
update develop to v0.20.6 (#2485) 2023-08-01 11:45:17 +05:30
Christopher Blaha
30a557cc69
update develop to v0.20.5 (#2458) 2023-07-19 21:52:10 +05:30
Christopher Blaha
184011f1f2
upgrade develop to v0.20.4 (#2444) 2023-07-10 18:21:08 +05:30
Alex Feiszli
e4895ae4a1
Merge branch 'develop' into remove-stun 2023-06-28 13:53:46 -04:00
Abhishek K
230e062c84
GRA-1298: License check changes, free tier limits for saas (#2418)
* set free tier limits through config

* add host limit to config

* check for host limit on free tier

* fix license validation, replace node limit with hosts

* add hosts to telemetry data

* debug init

* validate license every 1hr

* hook manager, api to fetch server usage

* hook manager, server usage api

* encode json server usage api

* update ngork url

* update license validation endpoint

* avoid setting limits on eer

* adding hotfix

* correct users limits env var

* add comments to exported funcs

---------

Co-authored-by: afeiszli <alex.feiszli@gmail.com>
2023-06-28 20:33:06 +05:30
Christopher Blaha
2cc0da344f
update develop to v0.20.3 (#2412) 2023-06-21 19:45:44 +05:30
afeiszli
b9a18cf73d remove stun 2023-06-13 15:53:24 -04:00
Matthew R Kasun
f94cda11f6
Net 262 - structured logging (#2366)
* structured logging infra

* structured logging mq handlers
2023-06-06 13:47:16 -04:00
Christopher Blaha
c6b1ec494c
update version to 0.20.2 for develop (#2363) 2023-06-06 13:46:46 -04:00