* NET-655
* Updated HostPull structure to include EgressRoutes and FirewallUpdate models.
* added ServerVersion structure to hostpull model
* added ServerVersion structure to hostpull model
* removed ServerVersion structure
* removed ServerVersion structure
* added egressroute and fwupdate to hostpull handler
* add host update fallback handler
* set broker type on server cfg
* use actual host password to create emqx user
---------
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
* deprecate setting etc host dns entries, set coredns entries at neccessary places
* remove dns mq topics
* only set dns when set on
* add extcleints dns entries to coredns
* fix static checks
* run coredns with host network mode
* add bind address to coredns container
* block normal user login from accessing dashboard
* header change
* allow from ui header
* allow from ui header
* check for user role after decoding
* block oauth login for normal user
* handle other oauth provider callback funcs for user login
* add internet gateway to client gateway
* migration func to remove internet egress range from egress gateway
* add internet gateways ranges to firewall update
* add internet gw ranges to extcleint conf
* add ipv6 internet address
* remove failover field from ingress req
* only let normal to be created on PRO (#2716)
* feat(NET-805): send internet gw props to rac
* set inet gw field on node update api
* move internet gws to EE
---------
Co-authored-by: the_aceix <aceixsmartx@gmail.com>
* cache enabled option, cache hosts data if only enabled
* cache nodes only when enabled
* cache extclients only when enabled
* cache acls only when enabled
* api to to get host relayed from client
* add auto relay to api host
* add peer nat type
* set pro field on signal
* rm net check on relay me handler
* return success response
* re-establish failover logic
* set failOver ctx
* failOver with peer pub key
* failovered peer updates
* failover handlers, reset failovered peer on deletion
* rm unused funcs
* initialize failover handler on EE
* ignore failover node on signal
* failover changes
* set host id on signal
* extend signal model to include node ids
* add backwards compatibility
* add failover as node api
* set json response on failover handers
* add failover field to api node
* fix signal data check
* initialize failover peer map
* reset failovered status when relayed or deleted
* add failover info to api node
* reset network failover
* only proceed furtuer if failover exists in the network
* set failOver node defaults
* cannot set failover node as relayed
* debug log
* debug log
* debug changes
* debug changes
* debug changes
* revert debug changes
* don't add peers to idmap when removed
* reset failed Over
* fix static checks
* rm debug log
* add check for linux host
* create emqx creds for host on pull if not found
* create emqx creds for host on pull if not found.
* emqx creds creation changed to host authenticate function
* emqx creds creation changed to host authenticate function.
* added node acls update
* added node acls update.
* remove client gw from user when deleted
* clear extclient of a user if deleted
* check if ingress gw on user gws
* debug log
* log change
* pr comments
* Send relays and internet gws count to amb
* Unify usage funcs in logic pkg
* Fix ee build
* Revert "Unify usage funcs in logic pkg"
This reverts commit 28afc91f7e.
* Add more resource metrics on getUsage handler
* Use pro module's function to get server usage on lic val
* Move GetRelays to pro package
* Move PKA field from models node to host level
* Move PKA field from api models node to host level
* Adapt logic package to node->host PKA
* Adapt migration-related code to node->host PKA
* Adapt cli code to node->host PKA
* Change host PKA default to 20s
* On IfaceDelta, check for PKA on host
* On handleHostRegister, set default PKA
* Use a default PKA
* Use int64 for api host pka
* Reorder imports
* Don't use host pka in iface delta
* Fix ConvertAPIHostToNMHost
* Add swagger doc for host PKA field
* Fix swagger.yml
* Set default PKA only for new hosts
* Remove TODO comment
* Remove redundant check
* Have api-host pka be specified in seconds
* feat(NET-584): wip: session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): session mgmt for RAC
* feat(NET-584): only enable if client is disabled
* feat(NET-584): check only for normal users
* feat(NET-584): fix condition
* NET-600
* Ext clients dns now properly set from ingress dns value provided that the individual ext client dns are not present.
* NET-468 & NET-600
* When individual extclient dns is changed or removed, it reflects on the UI properly and also gets updated on backend.
* add superadmin role, apis to create superadmin user
* apis to attach and remove user from remote access gateways
* add api to list user's remote client has gateway clients
* remove code related user groups
* remove networks and groups from user model
* refactor user CRUD operations
* fix network permission test
* add superadmin to authorize func
* remove user network and groups from cli
* api to transfer superadmin role
* add api to list users on a ingress gw
* restrict user access to resources on server
* deny request from remote access client if extclient is already created
* fix user tests
* fix static checks
* fix static checks
* add limits to extclient create handler
* set username to superadmin on if masterkey is used
* allow creation of extclients using masterkey
* add migration func to assign superadmin role for existing admin user
* check for superadmin on migration if users are present
* allowe masterkey to extcleint apis
* check ownerid
* format error, on jwt token verification failure return unauthorized rather than forbidden
* user update fix
* move user remote functionality to ee
* fix update user api
* security patch
* initalise ee user handlers
* allow user to use master key to update any user
* use slog
* fix auth user test
* table headers
* remove user role, it's covered in middleware
* setuser defaults fix
* if ingress node is deleted, cleanup gateway clients
* delete ext clients in a go routine
* remove response writer from go routine
* add superadmin role, apis to create superadmin user
* apis to attach and remove user from remote access gateways
* add api to list user's remote client has gateway clients
* remove code related user groups
* remove networks and groups from user model
* refactor user CRUD operations
* fix network permission test
* add superadmin to authorize func
* remove user network and groups from cli
* api to transfer superadmin role
* add api to list users on a ingress gw
* restrict user access to resources on server
* deny request from remote access client if extclient is already created
* fix user tests
* fix static checks
* fix static checks
* add limits to extclient create handler
* set username to superadmin on if masterkey is used
* allow creation of extclients using masterkey
* add migration func to assign superadmin role for existing admin user
* check for superadmin on migration if users are present
* allowe masterkey to extcleint apis
* check ownerid
* format error, on jwt token verification failure return unauthorized rather than forbidden
* user update fix
* move user remote functionality to ee
* fix update user api
* security patch
* initalise ee user handlers
* allow user to use master key to update any user
* use slog
* fix auth user test
* table headers
* remove user role, it's covered in middleware
* setuser defaults fix
* Move ee code to ee package and unify ee status to IsPro
* Consolidate naming for paid/professional/enterprise version as "pro". Notes:
- Changes image tags
- Changes build tags
- Changes package names
- Doesn't change links to docs that mention "ee"
- Doesn't change parameters sent to PostHog that mention "ee"
* Revert docker image tag being -pro, back to -ee
* Revert go build tag being pro, back to ee
* Add build tags for some ee content
* [2] Revert go build tag being pro, back to ee
* Fix test workflow
* Add a json tag to be backwards compatible with frontend "IsEE" check
* Add a json tag for the serverconfig struct for IsEE
* Ammend json tag to Is_EE
* fix ee tags
---------
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
* IoT peer updates remove flag is now only triggering on relay deletion. Not triggering on extclients deletion anymore.
* Small typo fix on publish dns update for new extclient function name.
* use NewHosts to init hostfile
* Revert "Bump github.com/txn2/txeh from 1.4.0 to 1.5.3 (#2464)" (#2475)
This reverts commit aafaa91af0.
* NET-433: Only Set Routes for NAT enabled egress GW (#2469)
* remove related fields and code
* remover metrics collection from server code
* fw update struct
* add ext client flag to metrics data
* simply nat types
* rm proxy update from cli
* remove ingress routes from firewall update
* check if egress ranges are present
* rm unused func
* ignore network parameter passed to delete ingress endpoint (#2471)
* fix(NET-388): fix to update node dns toggle (#2472)
* use local port for metrics pinger (#2479)
* v0.20.5 release notes (#2482)
* v0.20.5 release notes
* add a known issue
* update develop to v0.20.6 (#2485)
* enforce unique names for ext client names (#2476)
* enforce unique names for ext client names
* only check for unique id on creation
* check for unique id if changed
* prune(NET-483): remove defunct host.internetgateway field (#2487)
* don't reference host on err (#2493)
* deprecrate netclient install scripts (#2490)
* Net 500: validate network parameter passed to node endpoints (#2480)
* enforce unique names for ext client names
* only check for unique id on creation
* check for unique id if changed
* validate network parameter passed to node endpoints
---------
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
* NET-513 (#2492)
nm-certs.sh now requests certificate for EE and CE edition domains accordingly.
* [NET-404] Run in limited mode when ee checks fail (#2474)
* Add limited http handlers functionality to rest handler
* Export ee.errValidation (ee.ErrValidation)
* Export a fatal error handled by the hook manager
* Export a new status variable for unlicensed server
* Mark server as unlicensed when ee checks fail
* Handle license validation failures with a (re)boot in a limited state
* Revert "Export a fatal error handled by the hook manager"
This reverts commit 069c21974a8d36e889c73ad78023448d787d62a5.
* Revert "Export ee.errValidation (ee.ErrValidation)"
This reverts commit 59dbab8c79773ca5d879f28cbaf53f3dd4297b9b.
* Revert "Add limited http handlers functionality to rest handler"
This reverts commit e2f1f28facaca54713db76a588839cd2733cf673.
* Revert "Handle license validation failures with a (re)boot in a limited state"
This reverts commit 58cfbbaf522a1345aac1fa67964ebff0a6d60cd8.
* Revert "Mark server as unlicensed when ee checks fail"
This reverts commit 77c6dbdd3c9cfa6e7d6becedef6251e8617ae367.
* Handle license validation failures with a middleware
* Forbid responses if unlicensed ee and not in status api
* Remove unused func
* feat(NET-449): add sync feature to request a host pull from server (#2491)
* fix(NET-486): change client name length validation (#2498)
set limit to 5<=x<=32
* [NET-477] Pick AMB URL dynamically (#2489)
* Introduce config for environment
* Introduce func to get environment
* Choose accounts api host from environment
* Test the ee package on workflows
* Use build tag ee for license_test.go
* [Feature]: nm-quick script tackling arm TODO support (#2488)
* domain flag for auto installs
* use static servers with custom domain (#2421)
* send delete peer update always
* fix add/remove host api calls
* keep mq updates in a single go func
* move branch test logic to devops (#2443)
* handle IOT OS
* save server name to env (#2460)
* ensure branch test servers available after test runs (#2467)
* save server name to env
* free server always; add PR to discord messages
* use correct method to delete droplets (#2468)
* quick fix for the launcher
* removed exit when triggering not supported exit and removed the TODO comments related to this issue
---------
Co-authored-by: Matthew R Kasun <mkasun@nusak.ca>
Co-authored-by: Alex Feiszli <31018251+afeiszli@users.noreply.github.com>
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
* Bump github.com/go-playground/validator/v10 from 10.14.1 to 10.15.0 (#2503)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.14.1 to 10.15.0.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.14.1...v10.15.0)
---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump golang.org/x/oauth2 from 0.10.0 to 0.11.0 (#2502)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.10.0 to 0.11.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.10.0...v0.11.0)
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump alpine from 3.18.2 to 3.18.3 (#2500)
Bumps alpine from 3.18.2 to 3.18.3.
---
updated-dependencies:
- dependency-name: alpine
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Net 509 515 (#2496)
* NET-509
* External client config files with IPv6 endpoints now have the [] separating the address and port.
* NET-515
* Increased network name max length to 32
* NET-509-515
* Updated unit test for network name max length check.
* Updated extclient endpoint ip string manipulation to use sprintf
* Added proper error message for network name length more than max allowed.
* NET-515 small typo fix for error strings should not be capitalized
* [NET-494 / ACC-322] New free tier limits (#2495)
* Rename var
* Rename consts and use iota
* Use switch instead of repeated else if
* Rename limits related vars
* Introduce new free tier limits
* Measure new limits and report on license validation
* Separate usage and limits, have new ones
* Don't check for hosts and clients limits, but for machines instead
* Error on egress creation @ free tier w/ internet gateways
* Remove clients and hosts limit from code
* Rename var
* Rename consts and use iota
* Use switch instead of repeated else if
* Rename limits related vars
* Introduce new free tier limits
* Measure new limits and report on license validation
* Separate usage and limits, have new ones
* Don't check for hosts and clients limits, but for machines instead
* Error on egress creation @ free tier w/ internet gateways
* Remove clients and hosts limit from code
* NET-507 (#2506)
* NET-507
* Fixed server restart changing the node expiration date to the defaults.
* Removed expired nodes removal from zombie cleanup routine.
* Added a new expired nodes deletion routine which removes expired nodes every hour.
* NET-507 suggested changes
* Possible fix for zombie nodes upon node deletion from the UI.
* Suggested changes implemented for expired nodes deletion go routine.
* NET-507 typo fix
* typo fix for DeleteNode purge parameter
* NET-517 (#2510)
* Fixed emqx docker compose file syntax and added volumes for persistence.
* Added helper comments for docker-compose.yml and netmaker.env files to enable emqx broker support.
* migration (#2509)
* create gateways during migration
* set version for testing
* restruct migration
* debug logging
* enforce unique names for ext client names (#2476)
* enforce unique names for ext client names
* only check for unique id on creation
* check for unique id if changed
* prune(NET-483): remove defunct host.internetgateway field (#2487)
* don't reference host on err (#2493)
* deprecrate netclient install scripts (#2490)
* Net 500: validate network parameter passed to node endpoints (#2480)
* enforce unique names for ext client names
* only check for unique id on creation
* check for unique id if changed
* validate network parameter passed to node endpoints
---------
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
* NET-513 (#2492)
nm-certs.sh now requests certificate for EE and CE edition domains accordingly.
* [NET-404] Run in limited mode when ee checks fail (#2474)
* Add limited http handlers functionality to rest handler
* Export ee.errValidation (ee.ErrValidation)
* Export a fatal error handled by the hook manager
* Export a new status variable for unlicensed server
* Mark server as unlicensed when ee checks fail
* Handle license validation failures with a (re)boot in a limited state
* Revert "Export a fatal error handled by the hook manager"
This reverts commit 069c21974a8d36e889c73ad78023448d787d62a5.
* Revert "Export ee.errValidation (ee.ErrValidation)"
This reverts commit 59dbab8c79773ca5d879f28cbaf53f3dd4297b9b.
* Revert "Add limited http handlers functionality to rest handler"
This reverts commit e2f1f28facaca54713db76a588839cd2733cf673.
* Revert "Handle license validation failures with a (re)boot in a limited state"
This reverts commit 58cfbbaf522a1345aac1fa67964ebff0a6d60cd8.
* Revert "Mark server as unlicensed when ee checks fail"
This reverts commit 77c6dbdd3c9cfa6e7d6becedef6251e8617ae367.
* Handle license validation failures with a middleware
* Forbid responses if unlicensed ee and not in status api
* Remove unused func
* feat(NET-449): add sync feature to request a host pull from server (#2491)
* fix(NET-486): change client name length validation (#2498)
set limit to 5<=x<=32
* [NET-477] Pick AMB URL dynamically (#2489)
* Introduce config for environment
* Introduce func to get environment
* Choose accounts api host from environment
* Test the ee package on workflows
* Use build tag ee for license_test.go
* [Feature]: nm-quick script tackling arm TODO support (#2488)
* domain flag for auto installs
* use static servers with custom domain (#2421)
* send delete peer update always
* fix add/remove host api calls
* keep mq updates in a single go func
* move branch test logic to devops (#2443)
* handle IOT OS
* save server name to env (#2460)
* ensure branch test servers available after test runs (#2467)
* save server name to env
* free server always; add PR to discord messages
* use correct method to delete droplets (#2468)
* quick fix for the launcher
* removed exit when triggering not supported exit and removed the TODO comments related to this issue
---------
Co-authored-by: Matthew R Kasun <mkasun@nusak.ca>
Co-authored-by: Alex Feiszli <31018251+afeiszli@users.noreply.github.com>
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
* rebase conflict
* include pass and os in mirgration data
* node network ranges
* remove debugging logs
* add gateways
* use sent node
* upgrade shell script
* associate node to host during migration
* add node to host.Nodes and publish peer update
* save host outside loop
* fix script name
* simplify upgrade script
* don't migrate relays
* simplify upgrade script even more
* guard against blank address or address6
* typos
---------
Co-authored-by: Aceix <aceixsmartX@gmail.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
Co-authored-by: Farukh Khan <farukhkhan21@gmail.com>
Co-authored-by: Gabriel de Souza Seibel <gabrielseibel1@gmail.com>
Co-authored-by: bornav <51048565+bornav@users.noreply.github.com>
Co-authored-by: Alex Feiszli <31018251+afeiszli@users.noreply.github.com>
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
* dns tests
* go mod tidy
* set dns dir when testing
* troubleshoot test failure
* test fail troubleshooting
* use txeh.RenderHostFile to compare
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Abhishek K <32607604+abhishek9686@users.noreply.github.com>
Co-authored-by: Aceix <aceixsmartX@gmail.com>
Co-authored-by: Christopher Blaha <crispspiceguitar@gmail.com>
Co-authored-by: Farukh Khan <farukhkhan21@gmail.com>
Co-authored-by: Gabriel de Souza Seibel <gabrielseibel1@gmail.com>
Co-authored-by: bornav <51048565+bornav@users.noreply.github.com>
Co-authored-by: Alex Feiszli <31018251+afeiszli@users.noreply.github.com>
Co-authored-by: Abhishek Kondur <abhi281342@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
