Commit graph

35 commits

Author SHA1 Message Date
Max Ma
46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Sayan Mallick
c551c487ca
New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
Max Ma
65faf73fe9
NET-1226: Scalability Improvements (#2987)
* add api to check if failover node existed

* remove 5 minute peerUpdate

* update peerUpdate to trigger pull

* update Action name to SignalPull

* revert the peerUpdate from SignalPull

* fix getfailover error issue

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* NET-1165: Remove creation of acls on emqx (#2996)

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* comment out mq reconnect logic

* configure mq conn params

* add metric_interval in ENV for publishing metrics

* add metric_interval in ENV for publishing metrics

* update PUBLISH_METRIC_INTERVAL env name

* revert the mq setttings back

* fix error nil issue

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
2024-07-09 18:56:55 +05:30
momantech
7e2b5b196a
chore: fix some comments (#2918) 2024-05-08 12:17:23 +05:30
Abhishek K
66069fbc34
NET-1082: Scale Testing Fixes (#2894)
* add additional mutex lock on node acls func

* increase verbosity

* disable acls on cloud emqx

* add emqx creds creation to go routine

* add debug log of mq client id

* comment port check

* uncomment port check

* check for connection mq connection open

* use username for client id

* add write mutex on acl is allowed

* add mq connection lost handler on server

* spin off zombie init as go routine

* get whole api path from config

* Revert "get whole api path from config"

This reverts commit 392f5f4c5f.

* update extclient acls async

* add additional mutex lock on node acls func

(cherry picked from commit 5325f0e7d7)

* increase verbosity

(cherry picked from commit 705b3cf0bf)

* add emqx creds creation to go routine

(cherry picked from commit c8e65f4820)

* add debug log of mq client id

(cherry picked from commit 29c5d6ceca)

* comment port check

(cherry picked from commit db8d6d95ea)

* check for connection mq connection open

(cherry picked from commit 13b11033b0)

* use username for client id

(cherry picked from commit e90c7386de)

* add write mutex on acl is allowed

(cherry picked from commit 4cae1b0bb4)

* add mq connection lost handler on server

(cherry picked from commit c82918ad35)

* spin off zombie init as go routine

(cherry picked from commit 6d65c44c43)

* update extclient acls async

(cherry picked from commit 6557ef1ebe)

* additionl logs for oauth user flow

(cherry picked from commit 61703038ae)

* add more debug logs

(cherry picked from commit 5980beacd1)

* add more debug logs

(cherry picked from commit 4d001f0d27)

* add set auth secret

(cherry picked from commit f41cef5da5)

* fix fetch pass

(cherry picked from commit 825caf4b60)

* make sure auth secret is set only once

(cherry picked from commit ba33ed02aa)

* make sure auth secret is set only once

(cherry picked from commit 920ac4c507)

* comment usage of emqx acls

* replace  read lock with write lock on acls

* replace  read lock with write lock on acls

(cherry picked from commit 808d2135c8)

* use deadlock pkg for visibility

* add additional mutex locks

* remove race flag

* on mq re-connecting donot exit if failed

* on mq re-connecting donot exit if failed

* revert mutex package change

* set mq clean session

* remove debug log

* go mod tidy

* revert on prem emqx acls del
2024-04-11 21:18:57 +05:30
Jonathan Roth
f370a2923c
Add missing enrollment key swagger parameters (#2767)
* Make swagger param private

* Add missing swagger params for create/update enrollment key

* Use same string in swagger doc path as the parameter

* Re-generate swagger

swagger generate spec -t ee -o swagger.yml
2024-03-12 08:30:44 +07:00
abhishek9686
82de96d9a3 if emqx, avoid sending mq creds from server to client 2024-01-26 11:23:50 +05:30
abhishek9686
fb0fead2f0 create emqx for server, get app creds from env 2024-01-26 10:24:29 +05:30
abhishek9686
155f2887b2 implement emqx interface methods for cloud and on-prem 2024-01-25 15:11:16 +05:30
Aceix
033e203d91
fix(NET-786): enhance enrollment key validation (#2726) 2024-01-03 12:23:04 +05:30
Abhishek K
98c01c4325
NET-814: Deprecating TURN (#2723)
* deprecate turn

* process signals through mq
2023-12-13 22:46:57 +04:00
Aceix
61ef6142ff
feat(NET-688): auto relaying via enrollment keys (#2647)
* feat(NET-688): auto relaying via enrollment keys

* feat(NET-688): address pr comments
2023-11-04 16:28:57 +04:00
Gabriel de Souza Seibel
cb4b99ffcb
[NET-562] Persistent Keep Alive from node to host (#2604)
* Move PKA field from models node to host level

* Move PKA field from api models node to host level

* Adapt logic package to node->host PKA

* Adapt migration-related code to node->host PKA

* Adapt cli code to node->host PKA

* Change host PKA default to 20s

* On IfaceDelta, check for PKA on host

* On handleHostRegister, set default PKA

* Use a default PKA

* Use int64 for api host pka

* Reorder imports

* Don't use host pka in iface delta

* Fix ConvertAPIHostToNMHost

* Add swagger doc for host PKA field

* Fix swagger.yml

* Set default PKA only for new hosts

* Remove TODO comment

* Remove redundant check

* Have api-host pka be specified in seconds
2023-10-06 10:09:19 +04:00
Matthew R Kasun
876778a1cc
open api spec file (#2595)
* remove usergroup

* superadmin

* superadmin creation

* generate openapi spec file

* statticcheck

* review comments
2023-10-04 10:26:38 +04:00
Abhishek K
3f1211795c
NET-383: Set Additional Host Fields From Client Side (#2566)
* update static,mtu from client side host update

* update host fields if host exists already on registration
2023-10-02 19:36:31 +04:00
Abhishek K
719e0c254d
NET-551: User Mgmt Re-Design (#2547)
* add superadmin role, apis to create superadmin user

* apis to attach and remove user from remote access gateways

* add api to list user's remote client has gateway clients

* remove code related user groups

* remove networks and groups from user model

* refactor user CRUD operations

* fix network permission test

* add superadmin to authorize func

* remove user network and groups from cli

* api to transfer superadmin role

* add api to list users on a ingress gw

* restrict user access to resources on server

* deny request from remote access client if extclient is already created

* fix user tests

* fix static checks

* fix static checks

* add limits to extclient create handler

* set username to superadmin on if masterkey is used

* allow creation of extclients using masterkey

* add migration func to assign superadmin role for existing admin user

* check for superadmin on migration if users are present

* allowe masterkey to extcleint apis

* check ownerid

* format error, on jwt token verification failure return unauthorized rather than forbidden

* user update fix

* move user remote functionality to ee

* fix update user api

* security patch

* initalise ee user handlers

* allow user to use master key to update any user

* use slog

* fix auth user test

* table headers

* remove user role, it's covered in middleware

* setuser defaults fix
2023-09-01 14:27:08 +05:30
Tobias Cudnik
723375b334
NET-152 enrollment keys for non admins (#2346)
* return 401 instead of 403

* fixed http.StatusForbidden

* Tagged build version (temp)

* Unauthorized_Err when applicable

* untagged version

* fixed PUT /api/users/networks/user1

* - expired token redirs to login
- added `/api/enrollment_keys` for non-admins
- unit test for enrollment keys for non-admins

* handle user perms in `/hosts`

* removed debug

* misc

* - support masteradmin
- return hosts with partial access

* added `ismaster` to middleware
2023-05-31 13:11:54 +05:30
Abhishek Kondur
423abf0fe4 add use_turn option to config,check if server is using turn 2023-04-28 14:06:28 +05:30
Abhishek Kondur
9f5239ec79 re-register host with turn 2023-04-26 13:12:16 +04:00
0xdcarns
18c9bcc58f adapted sso to host registration 2023-04-17 22:23:17 -04:00
Anish Mukherjee
8a9f569c4f add emqx acls 2023-03-21 17:47:57 +05:30
afeiszli
0bbf1dbe54 hotfixing logging and model issues found during IoT testing 2023-03-16 13:41:56 -04:00
0xdcarns
ee9df20b05 changed register response, simplified host port check dereference 2023-03-10 11:43:26 -05:00
0xdcarns
9a7407f635 updated logic to add new nodes 2023-02-27 19:18:41 -05:00
0xdcarns
977c9c8c19 send peer update after request + fix pass update issue 2023-02-27 12:32:07 -05:00
0xdcarns
0335e258ad added TODO comment and allowed using enrollment key more than once 2023-02-24 12:08:32 -05:00
0xdcarns
541e232ad7 update comments 2023-02-17 12:09:18 -05:00
0xdcarns
9103efa88f added request ack on register 2023-02-17 11:54:25 -05:00
0xdcarns
6b30cef968 handled node additions in more elegant manner 2023-02-17 11:32:02 -05:00
0xdcarns
08248e1b35 added log 2023-02-16 19:34:25 -05:00
0xdcarns
6e1db0bb3f removed admin security check 2023-02-16 16:42:08 -05:00
0xdcarns
d8c7ab980e fixed nil pointer from dereference in loop 2023-02-16 15:41:23 -05:00
0xdcarns
9078608bd1 fix initial map allocation 2023-02-16 15:13:40 -05:00
0xdcarns
607198d563 added host registration endpoint 2023-02-16 14:27:57 -05:00
0xdcarns
71ce2caabd added tokenization + detokenization 2023-02-15 16:32:16 -05:00