* feat: api access tokens
* revoke all user tokens
* redefine access token api routes, add auto egress option to enrollment keys
* add server settings apis, add db table for settigs
* handle server settings updates
* switch to using settings from DB
* fix sever settings migration
* revet force migration for settings
* fix server settings database write
* fix revoked tokens to be unauthorized
* remove unused functions
* convert access token to sql schema
* switch access token to sql schema
* fix merge conflicts
* fix server settings types
* bypass basic auth setting for super admin
* add TODO comment
* publish peer update on settings update
* chore(go): import style changes from migration branch;
1. Singular file names for table schema.
2. No table name method.
3. Use .Model instead of .Table.
4. No unnecessary tagging.
* remove nat check on egress gateway request
* Revert "remove nat check on egress gateway request"
This reverts commit 0aff12a189.
* feat(go): add db middleware;
* feat(go): restore method;
* feat(go): add user access token schema;
* fix user auth api:
* re initalise oauth and email config
* set verbosity
* sync auto update settings with hosts
* sync auto update settings with hosts
* mask secret and convert jwt duration to minutes
* convert jwt duration to minutes
* notify peers after settings update
* compare with curr settings before updating
* send host update to devices on auto update
---------
Co-authored-by: Vishal Dalwadi <dalwadivishal26@gmail.com>
* set default metrics port 8889
* set default metrics port 51821
* add metrics port to server config
* bind caddy only on tcp
* add var for pulling files
* add new line
* update peer update model
* check if port is not zero
* set replace peer to false on pull
* do not replace peers on failover sync
* remove debug log
* add old peer update fields for backwards compatibility
* add old json tag
* add debug log in caller trace func
* add live status of node
* handle static node status
* add public IP field to server configuration
* get public Ip from config
* improve node status logic
* improvise status check
* use only checkin status on old nodes
---------
Co-authored-by: the_aceix <aceixsmartx@gmail.com>
* add switch for manage dns
* manage DNS sync publish
* add dns sync api
* add manageDNS field in peerUpdate
* add default dns for extClent if manage dns enabled
* add DEFAULT_DOMAIN for internal DNS lookup
* move DNSSync to peerUpdate
* fix empty host in network issue
* sync up dns when custom dns add/delete
* fix custom DNS ip4/ipv6 validator issue
* NET-1440 scale test changes
* fix UT error and add error info
* load metric data into cacha in startup
* remove debug info for metric
* add server telemetry and hasSuperAdmin to cache
* fix user UT case
* update sqlite connection string for performance
* update check-in TS in cache only if cache enabled
* update metric data in cache only if cache enabled and write to DB once in stop
* update server status in mq topic
* add failover existed to server status update
* only send mq messsage when there is server status change
* batch peerUpdate
* code changes for scale for review
* update UT case
* update mq client check
* mq connection code change
* revert server status update changes
* revert batch peerUpdate
* remove server status update info
* batch peerUpdate
* code changes based on review and setupmqtt in keepalive
* set the mq message order to false for PIN
* remove setupmqtt in keepalive
* add peerUpdate batch size to config
* update batch peerUpdate
* recycle ip in node deletion
* update ip allocation logic
* remove ip addr cap
* remove ippool file
* update get extClient func
* remove ip from cache map when extClient is removed
* add batch peerUpdate switch
* set batch peerUpdate to true by default
---------
Co-authored-by: Max Ma <mayabin@gmail.com>
* NET-1440 scale test changes
* fix UT error and add error info
* load metric data into cacha in startup
* remove debug info for metric
* add server telemetry and hasSuperAdmin to cache
* fix user UT case
* update sqlite connection string for performance
* update check-in TS in cache only if cache enabled
* update metric data in cache only if cache enabled and write to DB once in stop
* update server status in mq topic
* add failover existed to server status update
* only send mq messsage when there is server status change
* batch peerUpdate
* code changes for scale for review
* update UT case
* update mq client check
* mq connection code change
* revert server status update changes
* revert batch peerUpdate
* remove server status update info
* code changes based on review and setupmqtt in keepalive
* set the mq message order to false for PIN
* remove setupmqtt in keepalive
* recycle ip in node deletion
* update ip allocation logic
* remove ip addr cap
* remove ippool file
* update get extClient func
* remove ip from cache map when extClient is removed
* add api to check if failover node existed
* remove 5 minute peerUpdate
* update peerUpdate to trigger pull
* update Action name to SignalPull
* revert the peerUpdate from SignalPull
* fix getfailover error issue
* rm acls creation for on-prem emqx
* remove use of acls
* add additional broker status field on status api
* NET-1165: Remove creation of acls on emqx (#2996)
* rm acls creation for on-prem emqx
* remove use of acls
* add additional broker status field on status api
* comment out mq reconnect logic
* configure mq conn params
* add metric_interval in ENV for publishing metrics
* add metric_interval in ENV for publishing metrics
* update PUBLISH_METRIC_INTERVAL env name
* revert the mq setttings back
* fix error nil issue
---------
Co-authored-by: abhishek9686 <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
* add additional checks to avoid failovers
* add failover defence check on signal handler
* only add check for victim node
* avoid failover reset on pull
* add relayed for failoverme
* misc changes for failover
* remove resetfailoverpeers for InetNode
* add egress route back to allowedip list if relayed is egressGW
* add extclient back to allowedip list if peer is ingressGW
* reset failover on pull
---------
Co-authored-by: Max Ma <mayabin@gmail.com>
* exclude IngressGW in failover
* resetfailoverpeer when adding IngressGw if failover enabled
* exclude InetGW in failover
* get egress ranges of failedover peer
---------
Co-authored-by: abhishek9686 <abhi281342@gmail.com>
* add additional mutex lock on node acls func
* increase verbosity
* disable acls on cloud emqx
* add emqx creds creation to go routine
* add debug log of mq client id
* comment port check
* uncomment port check
* check for connection mq connection open
* use username for client id
* add write mutex on acl is allowed
* add mq connection lost handler on server
* spin off zombie init as go routine
* get whole api path from config
* Revert "get whole api path from config"
This reverts commit 392f5f4c5f.
* update extclient acls async
* add additional mutex lock on node acls func
(cherry picked from commit 5325f0e7d7)
* increase verbosity
(cherry picked from commit 705b3cf0bf)
* add emqx creds creation to go routine
(cherry picked from commit c8e65f4820)
* add debug log of mq client id
(cherry picked from commit 29c5d6ceca)
* comment port check
(cherry picked from commit db8d6d95ea)
* check for connection mq connection open
(cherry picked from commit 13b11033b0)
* use username for client id
(cherry picked from commit e90c7386de)
* add write mutex on acl is allowed
(cherry picked from commit 4cae1b0bb4)
* add mq connection lost handler on server
(cherry picked from commit c82918ad35)
* spin off zombie init as go routine
(cherry picked from commit 6d65c44c43)
* update extclient acls async
(cherry picked from commit 6557ef1ebe)
* additionl logs for oauth user flow
(cherry picked from commit 61703038ae)
* add more debug logs
(cherry picked from commit 5980beacd1)
* add more debug logs
(cherry picked from commit 4d001f0d27)
* add set auth secret
(cherry picked from commit f41cef5da5)
* fix fetch pass
(cherry picked from commit 825caf4b60)
* make sure auth secret is set only once
(cherry picked from commit ba33ed02aa)
* make sure auth secret is set only once
(cherry picked from commit 920ac4c507)
* comment usage of emqx acls
* replace read lock with write lock on acls
* replace read lock with write lock on acls
(cherry picked from commit 808d2135c8)
* use deadlock pkg for visibility
* add additional mutex locks
* remove race flag
* on mq re-connecting donot exit if failed
* on mq re-connecting donot exit if failed
* revert mutex package change
* set mq clean session
* remove debug log
* go mod tidy
* revert on prem emqx acls del
* set clean session
* delete emqx hosts creds api
* add emqx hosts del api to limited middleware controller
* add emqx hosts del api to limited middleware controller
* remove server creds from emqx
* NET-655
* Updated HostPull structure to include EgressRoutes and FirewallUpdate models.
* added ServerVersion structure to hostpull model
* added ServerVersion structure to hostpull model
* removed ServerVersion structure
* removed ServerVersion structure
* added egressroute and fwupdate to hostpull handler
* add host update fallback handler
* set broker type on server cfg
* use actual host password to create emqx user
---------
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
* deprecate setting etc host dns entries, set coredns entries at neccessary places
* remove dns mq topics
* only set dns when set on
* add extcleints dns entries to coredns
* fix static checks
* run coredns with host network mode
* add bind address to coredns container
* api to to get host relayed from client
* add auto relay to api host
* add peer nat type
* set pro field on signal
* rm net check on relay me handler
* return success response
* re-establish failover logic
* set failOver ctx
* failOver with peer pub key
* failovered peer updates
* failover handlers, reset failovered peer on deletion
* rm unused funcs
* initialize failover handler on EE
* ignore failover node on signal
* failover changes
* set host id on signal
* extend signal model to include node ids
* add backwards compatibility
* add failover as node api
* set json response on failover handers
* add failover field to api node
* fix signal data check
* initialize failover peer map
* reset failovered status when relayed or deleted
* add failover info to api node
* reset network failover
* only proceed furtuer if failover exists in the network
* set failOver node defaults
* cannot set failover node as relayed
* debug log
* debug log
* debug changes
* debug changes
* debug changes
* revert debug changes
* don't add peers to idmap when removed
* reset failed Over
* fix static checks
* rm debug log
* add check for linux host
* Move ee code to ee package and unify ee status to IsPro
* Consolidate naming for paid/professional/enterprise version as "pro". Notes:
- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"
* Revert docker image tag being -pro, back to -ee
* Revert go build tag being pro, back to ee
* Add build tags for some ee content
* [2] Revert go build tag being pro, back to ee
* Fix test workflow
* Add a json tag to be backwards compatible with frontend "IsEE" check
* Add a json tag for the serverconfig struct for IsEE
* Ammend json tag to Is_EE
* fix ee tags
---------
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
* IoT peer updates remove flag is now only triggering on relay deletion. Not triggering on extclients deletion anymore.
* Small typo fix on publish dns update for new extclient function name.
* remove related fields and code
* remover metrics collection from server code
* fw update struct
* add ext client flag to metrics data
* simply nat types
* rm proxy update from cli
* send peer update to IOT client only when it is relayed
* move node check
* send relay del update for iot client
* fix relay delete logic for iot
* set relay node to true for iot peer update
* add node addrs to peer update
* revert tag
