Commit graph

122 commits

Author SHA1 Message Date
Abhishek K 5a561b3835
Net 1440 batchpeerupdate (#3042)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* batch peerUpdate

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* add peerUpdate batch size to config

* update batch peerUpdate

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed

* add batch peerUpdate switch

* set batch peerUpdate to true by default

---------

Co-authored-by: Max Ma <mayabin@gmail.com>
2024-08-16 15:35:43 +05:30
Max Ma 46b8fd21c8
NET-1440: scale test changes (#3014)
* NET-1440 scale test changes

* fix UT error and add error info

* load metric data into cacha in startup

* remove debug info for metric

* add server telemetry and hasSuperAdmin to cache

* fix user UT case

* update sqlite connection string for performance

* update check-in TS in cache only if cache enabled

* update metric data in cache only if cache enabled and write to DB once in stop

* update server status in mq topic

* add failover existed to server status update

* only send mq messsage when there is server status change

* batch peerUpdate

* code changes for scale for review

* update UT case

* update mq client check

* mq connection code change

* revert server status update changes

* revert batch peerUpdate

* remove server status update info

* code changes based on review and setupmqtt in keepalive

* set the mq message order to false for PIN

* remove setupmqtt in keepalive

* recycle ip in node deletion

* update ip allocation logic

* remove ip addr cap

* remove ippool file

* update get extClient func

* remove ip from cache map when extClient is removed
2024-08-15 11:59:00 +05:30
Max Ma 65faf73fe9
NET-1226: Scalability Improvements (#2987)
* add api to check if failover node existed

* remove 5 minute peerUpdate

* update peerUpdate to trigger pull

* update Action name to SignalPull

* revert the peerUpdate from SignalPull

* fix getfailover error issue

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* NET-1165: Remove creation of acls on emqx (#2996)

* rm acls creation for on-prem emqx

* remove use of acls

* add additional broker status field on status api

* comment out mq reconnect logic

* configure mq conn params

* add metric_interval in ENV for publishing metrics

* add metric_interval in ENV for publishing metrics

* update PUBLISH_METRIC_INTERVAL env name

* revert the mq setttings back

* fix error nil issue

---------

Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
2024-07-09 18:56:55 +05:30
Max Ma 86fac41868
fix metric exporter mq connection issue (#2925) 2024-05-08 12:13:11 +05:30
Abhishek K 465f2bd5be
NET-896: Scale test bug fixes (#2764)
* send peer update in async

* update metrics on fallback

* return http json response
2024-01-15 23:17:36 +05:30
Abhishek K 5bf30b2c10
NET-877: Replace peers on Refreshkeys peer update (#2761)
* replace peers on key refresh

* add peer conf to metrics map only when allowed
2024-01-11 15:59:19 +05:30
Abhishek K 7e3bf3baea
NET-812: Deprecate Setting /etc/hosts entries (#2733)
* deprecate setting etc host dns entries, set coredns entries at neccessary places

* remove dns mq topics

* only set dns when set on

* add extcleints dns entries to coredns

* fix static checks

* run coredns with host network mode

* add bind address to coredns container
2023-12-20 10:24:53 +04:00
Abhishek K 27378b9c52
NET-765: publish deleted node update on disconnect (#2668)
* publish deleted node update on disconnect

* delete any daemon node

* set peer update for disconnected Node

* set peer update for disconnected Node
2023-11-15 19:53:52 +04:00
Abhishek K 7169db78ea
NET-717: DNS Fixes (#2654)
* send dns with complete entry name

* publish dns records on adding host to network from UI

* publish node dns updates when host is removed
2023-11-08 17:43:41 +04:00
pwillis7 fe842e0f05
feat(NET-591): allow generic DNS entries (#2568) 2023-09-20 14:20:08 +02:00
Gabriel de Souza Seibel 1a1ba1ccf4
[NET-546] Move ee code to ee package, unify ee status and terminology (#2538)
* Move ee code to ee package and unify ee status to IsPro

* Consolidate naming for paid/professional/enterprise version as "pro". Notes:

- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"

* Revert docker image tag being -pro, back to -ee

* Revert go build tag being pro, back to ee

* Add build tags for some ee content

* [2] Revert go build tag being pro, back to ee

* Fix test workflow

* Add a json tag to be backwards compatible with frontend "IsEE" check

* Add a json tag for the serverconfig struct for IsEE

* Ammend json tag to Is_EE

* fix ee tags

---------

Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
2023-09-01 07:42:05 +05:30
Farukh Khan 8be0409b6a
NET-555 (#2524)
* IoT peer updates remove flag is now only triggering on relay deletion. Not triggering on extclients deletion anymore.

* Small typo fix on publish dns update for new extclient function name.
2023-08-25 14:23:01 +05:30
Abhishek K 97f21d8b45
don't reference host on err (#2493) 2023-08-02 22:01:41 +05:30
Abhishek K 645bc4c126
NET-447: Removed proxy related fields and code (#2459)
* remove related fields and code

* remover metrics collection from server code

* fw update struct

* add ext client flag to metrics data

* simply nat types

* rm proxy update from cli
2023-07-26 11:22:49 +05:30
Abhishek Kondur 086ef25ac6 remove peer update context 2023-06-29 00:17:42 +05:30
Abhishek Kondur 950a303f66 Merge branch 'develop' of https://github.com/gravitl/netmaker into NET-390-scale-latest 2023-06-28 23:10:00 +05:30
Abhishek K 38454ece79
IOT-66: Single Peer update for IOT client (#2424)
* send peer update to IOT client only when it is relayed

* move node check

* send relay del update for iot client

* fix relay delete logic for iot

* set relay node to true for iot peer update

* add node addrs to peer update

* revert tag
2023-06-28 22:08:49 +05:30
Abhishek Kondur 4efbe6256f stop context cancel on peer updates 2023-06-28 21:05:21 +05:30
0xdcarns 9d230d3540 initial commit 2023-04-06 12:27:33 -04:00
Abhishek Kondur 45ea3be347 set proxy action based on proxy status 2023-03-29 10:39:18 +04:00
Abhishek Kondur 20cf3b00bc remove additional port field 2023-03-29 05:24:07 +04:00
Abhishek Kondur 60ddad3189 proxy only required connections 2023-03-28 14:04:27 +04:00
0xdcarns 0f3faceb93 added logic for ext client ACLs 2023-03-14 13:39:33 -04:00
0xdcarns 58adef9e12 address non-functional comment 2023-03-07 11:00:21 -05:00
0xdcarns 28119c22ee added context to peer updates, moved nodes to memory 2023-03-06 12:21:51 -05:00
0xdcarns 2be3ff747a Merge branch 'develop' into GRA-1198-enrollment 2023-02-27 15:55:25 -05:00
dcarns 26df6f4016
Merge pull request #2076 from walkerwmanuel/GRA-1270-startup-log-rework
Added more accurate startup logs
2023-02-27 15:22:54 -05:00
walkerwmanuel b6e0a860ae Added more accurate startup logs 2023-02-27 14:52:50 -05:00
0xdcarns e759637750 Merge branch 'develop' into GRA-1198 2023-02-24 11:23:41 -05:00
0xdcarns adf3967e0d handled forced deleted peer 2023-02-24 10:41:04 -05:00
0xdcarns 80fda83c0c adjusted peer updates to remove peers properly, slowed zombie check 2023-02-23 10:55:42 -05:00
0xdcarns 16f2093004 added empty string to params 2023-02-21 11:17:48 -05:00
0xdcarns 6b30cef968 handled node additions in more elegant manner 2023-02-17 11:32:02 -05:00
dcarns 92dbfa52a5
Merge pull request #2001 from gravitl/gra-1088_dns_updates
Gra 1088 dns updates
2023-02-15 12:55:58 -05:00
Matthew R Kasun c7c864b229 break into smaller funcs 2023-02-14 13:29:20 -05:00
0xdcarns 93473a4e70 updated node topic for publishing 2023-02-14 13:00:08 -05:00
Matthew R Kasun 6e061c49d0 fix typo 2023-02-10 11:03:08 -05:00
Matthew R Kasun 7c1784e74f log message on succesful alldns publish 2023-02-10 10:58:06 -05:00
Matthew R Kasun c7338888e4 error handling 2023-02-10 06:58:00 -05:00
Matthew R Kasun 98e66b7292 include ipv6 for extclients 2023-02-09 11:20:50 -05:00
Matthew R Kasun 0b0db1c0e8 add network name to dns 2023-02-09 10:49:56 -05:00
Matthew R Kasun e0478b9b54 ensure network add to new extclient for dns update 2023-02-08 15:58:32 -05:00
Matthew R Kasun 574583b3b1 include network on extcient dns update 2023-02-08 13:50:34 -05:00
Matthew R Kasun d4cd24e778 extclient update 2023-02-08 11:32:28 -05:00
Matthew R Kasun 88358f517c include old/new ip in dnsupdate for node ip change 2023-02-08 10:33:27 -05:00
Matthew R Kasun 8d6728bc41 add network when publishing custom dns creation 2023-02-08 10:11:36 -05:00
Matthew R Kasun 322bdb0af8 update dns mq topics to have four fields 2023-02-08 08:35:36 -05:00
Matthew R Kasun e1f644a453 add server name to dns publish topics 2023-02-07 15:17:11 -05:00
Matthew R Kasun 488ada670c cleanup: function comments, etc 2023-02-07 15:17:11 -05:00
Matthew R Kasun 5cac655f04 publish dns update on host name change 2023-02-07 15:17:11 -05:00