Commit graph

1073 commits

Author SHA1 Message Date
Max Ma
46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Sayan Mallick
c551c487ca
New Docs (#3034)
* New Docs

CSS update and Dockerfile to include docs folder

flash of unrendered text fix

markdown docs

ignore docs/docs.go

improving the docs generation

github actions for docs generation

go runner version fix

updated docs.yml

update repo action updated

updated actions and dns docs

dns complete

More docs update

Complete docs and updated workflow

Update documentation Tue Aug  6 11:17:42 UTC 2024

Update documentation Thu Aug  8 12:26:57 UTC 2024

clean up

clean up

Dockerfile clean up

Updated workflow

Updated workflow

Update docs.yml

Update docs.yml

* requested changes

* changed ingress gateway to remote access gateway
2024-08-15 11:55:01 +05:30
Christopher Blaha
66dbaf1bc2
update develop to v0.25.0 (#3009)
* update develop to v0.24.4

* update develop to v0.25.0
2024-07-25 10:19:31 +05:30
Max Ma
3dce62a7b6
add peerUpdate when calling hostupdate api (#3005) 2024-07-17 13:07:07 +05:30
Max Ma
023956d361
NET-1288:Fix egress node id issue (#3004)
* add egress range check with netmaker network address

* add egerssrange check for delete egressGW and extClientGW

* remove egress range check for delete

* fix egress nodeid issue
2024-07-10 15:03:25 +05:30
Max Ma
65faf73fe9
NET-1226: Scalability Improvements (#2987)
* add api to check if failover node existed

* remove 5 minute peerUpdate

* update peerUpdate to trigger pull

* update Action name to SignalPull

* revert the peerUpdate from SignalPull

* fix getfailover error issue

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* NET-1165: Remove creation of acls on emqx (#2996)

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* comment out mq reconnect logic

* configure mq conn params

* add metric_interval in ENV for publishing metrics

* add metric_interval in ENV for publishing metrics

* update PUBLISH_METRIC_INTERVAL env name

* revert the mq setttings back

* fix error nil issue

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
2024-07-09 18:56:55 +05:30
Aceix
2de3fde872
fix(NET-1280): remove RAG metadata length restriction (#2990) 2024-07-05 15:54:05 +05:30
Max Ma
f63dfaf4b5
NET-1288:add egress range check with netmaker network address (#2978)
* add egress range check with netmaker network address

* add egerssrange check for delete egressGW and extClientGW

* remove egress range check for delete
2024-07-05 15:53:46 +05:30
Christopher Blaha
7d001b81a7
update develop to v0.24.3 (#2977) 2024-06-18 07:23:20 +05:30
Max Ma
6fa084006a
NET-1355:peerUpdate for default host after new network created (#2974)
* peerUpdate for default host after new network created

* remove mq check for peerUpdate
2024-06-12 17:33:26 +05:30
Abhishek K
50d8da2dc2
Merge pull request #2958 from gravitl/NET-1268
NET-1268:seperate static port and static endpoint
2024-06-05 12:47:45 +05:30
Max Ma
c7469c79d0 rename isstaticendpoint to isstatic 2024-06-03 10:37:19 +02:00
Max Ma
35a8528030
NET-1154:fix stale node issue on multinet when deleting host (#2940)
* fix stale node issue on multinet when deleting host

* fix stale node when deleting host or uninstall
2024-06-03 10:28:21 +04:00
Max Ma
81a6c9cf8a seperate static port and static endpoint 2024-05-28 10:16:11 +02:00
Christopher Blaha
ee6cdc604b
update develop to v0.24.2 (#2938) 2024-05-17 12:47:54 +05:30
Aceix
faf50c8c3d
chore(NET-1245): remove check for private network address (#2934) 2024-05-14 19:54:00 +05:30
Aceix
abf3f4f55d
feat(NET-1106): support additional RAG endpoint IPs (#2907) 2024-05-08 14:45:05 +05:30
Max Ma
da11dc8a87
NET-1134:move oauth from CE build block to pro (#2919)
* move oauth from CE build block to pro

* move oauth code and api handler under pro

* move common func back to auth from pro/auth

* change log level to Info for information logs

* fix import issue
2024-05-08 12:44:39 +05:30
momantech
7e2b5b196a
chore: fix some comments (#2918) 2024-05-08 12:17:23 +05:30
Max Ma
1ad837ad72
NET-1164:PublishDeletedClientPeerUpdate when deleting user from RAGW (#2912)
* PublishDeletedClientPeerUpdate when deleting user from RAGW

* update ACLs after extClient is removed
2024-05-03 12:25:38 +05:30
Christopher Blaha
8b1294b5fb
update develop to v0.24.1 (#2904) 2024-04-26 09:06:40 +05:30
abhishek9686
b2361be29b track failover usage 2024-04-17 17:22:47 +05:30
abhishek9686
84fa3a1406 change err description 2024-04-17 15:41:22 +05:30
abhishek9686
00838cab52 add user id check on extclient 2024-04-17 15:31:34 +05:30
Abhishek K
66069fbc34
NET-1082: Scale Testing Fixes (#2894)
* add additional mutex lock on node acls func

* increase verbosity

* disable acls on cloud emqx

* add emqx creds creation to go routine

* add debug log of mq client id

* comment port check

* uncomment port check

* check for connection mq connection open

* use username for client id

* add write mutex on acl is allowed

* add mq connection lost handler on server

* spin off zombie init as go routine

* get whole api path from config

* Revert "get whole api path from config"

This reverts commit 392f5f4c5f.

* update extclient acls async

* add additional mutex lock on node acls func

(cherry picked from commit 5325f0e7d7)

* increase verbosity

(cherry picked from commit 705b3cf0bf)

* add emqx creds creation to go routine

(cherry picked from commit c8e65f4820)

* add debug log of mq client id

(cherry picked from commit 29c5d6ceca)

* comment port check

(cherry picked from commit db8d6d95ea)

* check for connection mq connection open

(cherry picked from commit 13b11033b0)

* use username for client id

(cherry picked from commit e90c7386de)

* add write mutex on acl is allowed

(cherry picked from commit 4cae1b0bb4)

* add mq connection lost handler on server

(cherry picked from commit c82918ad35)

* spin off zombie init as go routine

(cherry picked from commit 6d65c44c43)

* update extclient acls async

(cherry picked from commit 6557ef1ebe)

* additionl logs for oauth user flow

(cherry picked from commit 61703038ae)

* add more debug logs

(cherry picked from commit 5980beacd1)

* add more debug logs

(cherry picked from commit 4d001f0d27)

* add set auth secret

(cherry picked from commit f41cef5da5)

* fix fetch pass

(cherry picked from commit 825caf4b60)

* make sure auth secret is set only once

(cherry picked from commit ba33ed02aa)

* make sure auth secret is set only once

(cherry picked from commit 920ac4c507)

* comment usage of emqx acls

* replace  read lock with write lock on acls

* replace  read lock with write lock on acls

(cherry picked from commit 808d2135c8)

* use deadlock pkg for visibility

* add additional mutex locks

* remove race flag

* on mq re-connecting donot exit if failed

* on mq re-connecting donot exit if failed

* revert mutex package change

* set mq clean session

* remove debug log

* go mod tidy

* revert on prem emqx acls del
2024-04-11 21:18:57 +05:30
Christopher Blaha
0b2422b848
update to 0.24.0 (#2893) 2024-04-11 21:05:06 +05:30
Abhishek K
dccb6b5da7
NET-1075: Endpoint detection server config (#2876)
* add config for endpoint detection

* add config to netmaker env file

* fix config value check for endpoint detection
2024-04-03 23:51:09 +05:30
Jonathan Roth
80e775d5b4
Fix ACL, host and node swagger API definitions (#2864)
* Fix get all hosts response swagger type

* Remove body parameter from GET ACL request

* Use ApiNode response in requests that return it

* Redact net.Address field from API Iface

This field doesn't get declared correctly in the swagger file, and seems to break clients

* Re-generate swagger

swagger generate spec -t ee -o swagger.yml
2024-04-03 15:07:20 +05:30
Abhishek K
0d4552db5e
NET-1064: Oauth User SignUp Approval Flow (#2874)
* add pending users api

* insert user to pending users on first time oauth login

* add pending user check on headless login

* fix conflicting apis

* no records error

* add allowed emails domains for oauth singup to config

* check if user is allowed to signup
2024-04-03 11:20:19 +05:30
Abhishek K
c7e673fb9f
ACC-532: set mq clean session to true (#2865)
* set clean session

* delete emqx hosts creds api

* add emqx hosts del api to limited middleware controller

* add emqx hosts del api to limited middleware controller

* remove server creds from emqx
2024-03-20 15:03:41 +07:00
Jonathan Roth
f370a2923c
Add missing enrollment key swagger parameters (#2767)
* Make swagger param private

* Add missing swagger params for create/update enrollment key

* Use same string in swagger doc path as the parameter

* Re-generate swagger

swagger generate spec -t ee -o swagger.yml
2024-03-12 08:30:44 +07:00
Christopher Blaha
83cb39db5a
update develop to 0.23.1 (#2835) 2024-03-05 18:08:54 +07:00
Abhishek K
e846881bf2
Merge pull request #2843 from gravitl/ACC-468
ACC-468: EMQX Migration Helpers
2024-03-04 21:15:08 +07:00
abhishek9686
8ac864fec8 add update check for validate relay 2024-03-04 13:39:33 +07:00
abhishek9686
d69d0ed2e2 validate relay req for inet gws 2024-03-02 12:42:04 +07:00
abhishek9686
83cdc1abf6 migrate emqx 2024-03-01 14:26:07 +07:00
abhishek9686
8e240ff701 migrate emqx 2024-03-01 13:35:59 +07:00
Aceix
b79a457ce2
fix: fix extclient creation and ACL migration (#2831) 2024-02-28 09:01:07 +07:00
Abhishek K
0638dcac49
NET-940: Inet Gws (#2828)
* internet gws apis

* add validate check for inet request

* add default gw changes to peer update

* update json tag

* add OS checks for inet gws

* add set defaul gw pro func

* allow disable and enable inet gw

* add inet handlers to pro

* add fields to api node

* add inet allowed ips

* add default gw to pull

* unset node inet details on deletion

* unset internet gw on network nodes

* unset inet gw fix

* unset inet gw fix

* send default gw ip

* fix inet node endpoint

* add default gw endpoint ip to pull resp

* validate after unset gws

* add inet client peer allowedips to inet node

* validate after unset gws

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix allowed ips for inet peer and gw node

* fix inet gw and relayed conflict

* fix inet gw and relayed conflict

* fix update req

* fix update inet gw api

* when inet gw is peer ignore other allowedIps

* test relay

* revert test relay

* revert inet peer update changes

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* channel internet traffic of relayed node to relay's inetgw

* add check for relayed node

* add inet info to peer update

* add inet info to peer update

* fix update node to persist inet info

* fix go tests

* egress ranges with inet gw fix

* egress ranges with inet gw fix

* disallow node acting using inet gw to act as inet gw

* add check to validate inet gw

* fix typos

* add firewall check

* set inetgw on ingress req on community

* set inetgw to false on community on ingress del
2024-02-28 08:46:51 +07:00
Abhishek K
2d8f67db11
ignore post cmd for qr code and if empty (#2825) 2024-02-19 19:53:45 +07:00
Aceix
100b778449
fix(NET-897): uniform client and node acls (#2803) 2024-02-13 20:25:27 +07:00
Christopher Blaha
eaa62d8f22
update develop to v0.23.0 (#2816) 2024-02-08 23:30:10 +05:30
Aceix
39fbb45cfe
feat(NET-817): add postup/down scripts for clients (#2810) 2024-02-08 23:29:43 +05:30
Abhishek K
ecd769ea07
Merge pull request #2780 from gravitl/kwesi/net-709-add-metadata-field-to-gateway-node
feat(NET-709): add node metadata for remote gateways
2024-02-06 14:59:59 +05:30
Abhishek K
e685e3ca45
Merge pull request #2805 from gravitl/NET-908
NET-908: EMQX cloud Apis support
2024-02-06 14:59:18 +05:30
abhishek9686
98d9979e8b create failover and remoteaccess gw 2024-02-05 11:34:50 +05:30
abhishek9686
43e82a2936 make defaul host failover and remote access gw on joining new network 2024-02-05 11:18:28 +05:30
the_aceix
5882b86d40 feat(NET-709): add node metadata for remote gws 2024-01-26 10:06:36 +00:00
abhishek9686
82de96d9a3 if emqx, avoid sending mq creds from server to client 2024-01-26 11:23:50 +05:30
abhishek9686
8a777205d5 if emqx, avoid sending mq creds from server to client 2024-01-26 11:15:18 +05:30