This commit is contained in:
mdecimus 2024-04-09 12:54:49 +02:00
parent 62aa3be182
commit 0caaf43d31
9 changed files with 207 additions and 777 deletions

View file

@ -2,6 +2,28 @@
All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
## [0.7.0] - 2024-04-09
This version uses a different database layout and introduces multiple breaking changes in the configuration files. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.
## Added
- Web-based administration interface.
- REST API for management and configuration.
- Automatic RSA and ED25519 DKIM key generation.
- Support for compressing binaries in the blob store (#227).
- Improved performance accessing IMAP mailboxes with a large number of messages.
- Support for custom DNS resolvers.
- Support for multiple loggers with different levels and outputs.
### Changed
### Fixed
- Store quotas as `u64` rather than `u32`.
- Second IDLE connections disconnects the first one (#280).
- Use relaxed DNS parsing, allowing underscores in DNS labels (#172).
- Escape regexes within `matches()` expressions (#155).
- ManageSieve LOGOUT should reply with `OK` instead of `BYE`.
## [0.6.0] - 2024-02-14
This version introduces breaking changes in the configuration file. Please read the [UPGRADING.md](UPGRADING.md) file for more information on how to upgrade from previous versions.

View file

@ -1,26 +1,35 @@
# Contributing
Thank you for your interest in contributing to the Stalwart Mail Server project! We appreciate your support and enthusiasm. This document provides guidelines for contributing to the project.
When contributing to this repository, please first discuss the change you wish to make via issue,
email, or any other method with the owners of this repository before making a change.
## Not accepting Pull Requests at this time
As of now, the Stalwart Mail Server project is in a phase of rapid development and evolution. Until we reach version 1.0, **we are not open to direct code contributions** in the form of pull requests. This is due to the following reasons:
- The project is undergoing significant changes which might make integrating external contributions challenging.
- Our current resources do not allow for the efficient evaluation and testing of submitted pull requests.
Please note we have a code of conduct, please follow it in all your interactions with the project.
We understand that this might be disappointing, but rest assured, this measure is temporary and is aimed at maintaining the integrity and consistency of the project during its critical development phase.
## Any contributions you make will be under AGPL
## Other Ways to Contribute
While we're not accepting pull requests at this moment, there are several other valuable ways you can contribute to the Stalwart Mail Server project:
This software is licensed under the Affero General Public License (AGPL). Any contributions made to
this project will be under this license. Before any contributions can be made, contributors are
required to sign a Contributor License Agreement (CLA). The purpose of the agreement is to clarify
and document the rights granted by contributors to us.
- **Enhancement Requests**: If you have an idea for a new feature or an improvement, we encourage you to file an enhancement request on GitHub. This way, your idea can be evaluated and potentially included in future versions of the project. To do this, create a new issue on our GitHub repository and label it as an 'enhancement'.
- **Extensive Testing**: One of the most helpful contributions is to rigorously test the software in various environments and use-cases. Your testing can uncover important issues and help improve the stability of the project.
- **Writing Unit Tests**: Contributing new unit tests or improving existing ones is a great way to ensure the reliability and robustness of our codebase.
- **Reviewing Documentation**: As the project evolves, so does the need for accurate and up-to-date documentation. Reviewing, updating, or writing new documentation is a critical contribution that helps new and existing users understand and effectively use Stalwart Mail Server.
## Pull Request Process
## Stay Updated
We anticipate opening up for code contributions once version 1.0 is launched. We recommend keeping an eye on our project announcements for any updates regarding contribution opportunities.
1. Ensure any install or build dependencies are removed before the end of the layer when doing a
build.
2. Update the README.md with details of changes to the interface, this includes new environment
variables, exposed ports, useful file locations and container parameters.
3. Increase the version numbers in any examples files and the README.md to the new version that this
Pull Request would represent. The versioning scheme we use is [SemVer](http://semver.org/).
4. You may merge the Pull Request in once you have the sign-off of two other developers, or if you
do not have permission to do that, you may request the second reviewer to merge it for you.
## Questions or Suggestions?
If you have any questions or suggestions regarding the contribution process, feel free to open a discussion on our GitHub repository.
## Code of Conduct
Thank you for your support and understanding. Together, we are building a robust and reliable Stalwart Mail Server!
We as members, contributors, and leaders pledge to make participation in our community a harassment-free
experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex
characteristics, gender identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive,
and healthy community.
You can read the full Code of Conduct [here](https://github.com/stalwartlabs/.github/blob/main/CODE_OF_CONDUCT.md).

View file

@ -75,6 +75,13 @@ Key features:
- Sieve scripting language with support for all [registered extensions](https://www.iana.org/assignments/sieve-extensions/sieve-extensions.xhtml).
- Email aliases, mailing lists, subaddressing and catch-all addresses support.
- Integration with **OpenTelemetry** to enable monitoring, tracing, and performance analysis.
- **Web-based administration**:
- Account, domain, group and mailing list management.
- SMTP queue management for messages and outbound DMARC and TLS reports.
- Report visualization interface for received DMARC, TLS-RPT and Failure (ARF) reports.
- Configuration of every aspect of the mail server.
- Log viewer with search and filtering capabilities.
- Self-service portal for password reset and encryption-at-rest key management.
- **Secure and robust**:
- Encryption at rest with **S/MIME** or **OpenPGP**.
- Automatic TLS certificate provisioning with [ACME](https://datatracker.ietf.org/doc/html/rfc8555).
@ -106,7 +113,7 @@ Additionally you may become a sponsor to obtain priority support from Stalwart L
- [x] Performance enhancements
- [x] Distributed SMTP queues
- [ ] Web-based admin panel
- [x] Web-based admin panel
- [ ] JMAP Calendar, Contacts and Tasks support
- [ ] CalDAV and CardDAV support

View file

@ -6,9 +6,9 @@ We provide security updates for the following versions of Stalwart Mail Server:
| Version | Supported |
| ------- | ------------------ |
| 0.5.x | :white_check_mark: |
| 0.4.x | :white_check_mark: |
| < 0.3 | :x: |
| 0.7.x | :white_check_mark: |
| 0.6.x | :white_check_mark: |
| < 0.5 | :x: |
## Reporting a Vulnerability

View file

@ -1,3 +1,38 @@
Upgrading from `v0.6.0` to `v0.7.0`
-----------------------------------
Version `0.7.0` of Stalwart Mail Server introduces significant improvements and features that enhance performance and functionality. However, it also comes with multiple breaking changes in the configuration files and a revamped database layout optimized for accessing large mailboxes. Additionally, Stalwart now supports compression for binaries stored in the blob store, further increasing efficiency.
Due to these extensive changes, the recommended approach for upgrading is to perform a clean reinstallation of Stalwart and manually migrate your accounts to the new version.
## Pre-Upgrade Steps
- Download the `v0.7.0` mail-server and CLI binaries for your platform from the [releases page](https://github.com/stalwartlabs/mail-server/releases/latest/).
- Initialize the setup on a distinct directory using the command `sudo ./stalwart-mail -- init /path/to/new-install`. This command will print the administrator password required to access the web-admin.
- Create the `bin` directory using `mkdir /path/to/new-install/bin`.
- Move the downloaded binaries to the `bin` directory using the command `mv stalwart-mail stalwart-cli /path/to/new-install/bin`.
- Open `/path/to/new-install/etc/config.toml` in a text editor and comment out all listeners except the HTTP listener for port `8080`.
- Start the new installation from the terminal using the command `sudo /path/to/new-install/bin/stalwart-mail --config /path/to/new-install/etc/config.toml`.
- Point your browser to the web-admin at `http://yourserver.org:8080` and login using the auto-generated administrator password.
- Configure the new installation with your domain, hostname, certificates, and other settings following the instructions at [stalw.art/docs/get-started](https://stalw.art/docs/get-started). Ignore the part about using the installation script, we are performing a manual installation.
- Add your user accounts.
- Configure Stalwart to run as the `stalwart-mail` user and `stalwart-mail` group from `Settings` > `Server` > `System`. This is not necessary if you are using Docker.
- Stop the new installation by pressing `Ctrl+C` in the terminal.
## Upgrade Steps
- On your `v0.6.0` installation, open in a text editor the `smtp/listener.toml`, `imap/listener.toml` files and comment out all listeners except the JMAP/HTTP listener (we are going to need it to export the user accounts) and then restart the service.
- If you are using an external store, backup the database using the appropriate method for your database system.
- Create the `~/exports` directory, here we will store the exported accounts.
- Using the existing CLI tool (not the one you just downloaded as it is not compatible), export each user account using the command `./stalwart-cli -u https://your-old-server.org -c <ADMIN_PASSWORD> export account <ACCOUNT_NAME> ~/exports`.
- Stop the `v0.6.0` installation using the command `sudo systemctl stop stalwart-mail`.
- Move the old `v0.6.0` installation to a backup directory, for example `mv /opt/stalwart-mail /opt/stalwart-mail-backup`.
- Move the new `v0.7.0` installation to the old installation directory, for example `mv /path/to/new-install /opt/stalwart-mail`.
- Set the right permissions for the new installation using the command `sudo chown -R stalwart:stalwart /opt/stalwart-mail`.
- Start the new installation using the command `sudo systemctl start stalwart-mail`.
- Import the accounts using the new CLI tool with the command `./stalwart-cli -u http://yourserver.org:8080-c <ADMIN_PASSWORD> import <ACCOUNT> ~/exports/<ACCOUNT>`.
- Using the admin tool, reactivate all the necessary listener (SMTP, IMAP, etc.)
- Restart the service using the command `sudo systemctl restart stalwart-mail`.
We apologize for the complexity of the upgrade process associated with this version of Stalwart. We understand the challenges and inconveniences that the requirement for a clean reinstallation and manual account migration poses. Moving forward, an automated migration tool will be included in any future releases that necessitate changes to the database layout, aiming to streamline the upgrade process for you. Furthermore, as we approach the milestone of version 1.0.0, we anticipate that such foundational changes will become increasingly infrequent, leading to more straightforward updates. We appreciate your patience and commitment to Stalwart during this upgrade.
Upgrading from `v0.5.3` to `v0.6.0`
-----------------------------------

View file

@ -1,7 +1,7 @@
#!/usr/bin/env sh
# shellcheck shell=dash
# Stalwart Mail install script -- based on the rustup installation script.
# Stalwart Mail Server install script -- based on the rustup installation script.
set -e
set -u
@ -34,26 +34,43 @@ main() {
_account="_stalwart-mail"
fi
# Start configuration mode
if [ "$#" -eq 1 ] && [ "$1" = "--init" ] ; then
init
configure
return 0
fi
# Read arguments
local _dir="/opt/stalwart-mail"
# Default component setting
local _component="stalwart-mail"
# Loop through the arguments
for arg in "$@"; do
case "$arg" in
--fdb)
_component="stalwart-mail-foundationdb"
;;
*)
if [ -n "$arg" ]; then
_dir=$arg
fi
;;
esac
done
# Detect platform architecture
get_architecture || return 1
local _arch="$RETVAL"
assert_nz "$_arch" "arch"
# Create directories
ensure mkdir -p "$_dir" "$_dir/bin" "$_dir/etc" "$_dir/logs"
# Download latest binary
say "⏳ Downloading installer for ${_arch}..."
local _dir
_dir="$(ensure mktemp -d)"
local _file="${_dir}/stalwart-install.tar.gz"
local _url="${BASE_URL}/stalwart-install-${_arch}.tar.gz"
say "⏳ Downloading ${_component} for ${_arch}..."
local _file="${_dir}/bin/stalwart-mail.tar.gz"
local _url="${BASE_URL}/${_component}-${_arch}.tar.gz"
ensure mkdir -p "$_dir"
ensure downloader "$_url" "$_file" "$_arch"
ensure tar zxvf "$_file" -C "$_dir/bin"
ignore chmod +x "$_dir/bin/stalwart-mail"
ignore rm "$_file"
# Create system account
if ! id -u ${_account} > /dev/null 2>&1; then
@ -86,16 +103,92 @@ main() {
fi
fi
# Copy binary
say "⬇️ Running installer..."
ensure tar zxvf "$_file" -C "$_dir"
ignore $_dir/stalwart-install
ignore rm "$_file"
ignore rm "$_dir/stalwart-install"
# Run init
ignore $_dir/bin/stalwart-mail --init "$_dir"
# Set permissions
say "🔐 Setting permissions..."
ensure chown -R ${_account}:${_account} "$_dir"
ensure chmod -R 755 "$_dir"
ensure chmod 700 "$_dir/etc/config.toml"
# Create service file
say "🚀 Starting service..."
if [ "${_os}" = "linux" ]; then
printf "\n[server.run-as]\nuser = \"stalwart-mail\"\ngroup = \"stalwart-mail\"\n" >> "$_dir/etc/config.toml"
create_service_linux "$_dir"
elif [ "${_os}" = "macos" ]; then
create_service_macos "$_dir"
fi
# Installation complete
local _host=$(hostname)
say "🎉 Installation complete! Continue the setup at http://$_host:8080/login"
return 0
}
# Functions to create service files
create_service_linux() {
local _dir="$1"
cat <<EOF | sed "s|__PATH__|$_dir|g" > /etc/systemd/system/stalwart-mail.service
[Unit]
Description=Stalwart Mail Server Server
Conflicts=postfix.service sendmail.service exim4.service
ConditionPathExists=__PATH__/etc/config.toml
After=network-online.target
[Service]
Type=simple
LimitNOFILE=65536
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
ExecStart=__PATH__/bin/stalwart-mail --config=__PATH__/etc/config.toml
PermissionsStartOnly=true
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=stalwart-mail
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable stalwart-mail.service
systemctl restart stalwart-mail.service
}
create_service_macos() {
local _dir="$1"
cat <<EOF | sed "s|__PATH__|$_dir|g" > /Library/LaunchAgents/stalwart.mail.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>stalwart.mail</string>
<key>ServiceDescription</key>
<string>Stalwart Mail Server</string>
<key>ProgramArguments</key>
<array>
<string>__PATH__/bin/stalwart-mail</string>
<string>--config=__PATH__/etc/config.toml</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
</dict>
</plist>
EOF
launchctl load /Library/LaunchAgents/stalwart.mail.plist
launchctl enable system/stalwart.mail
launchctl start system/stalwart.mail
}
get_architecture() {
local _ostype _cputype _bitness _arch _clibtype
_ostype="$(uname -s)"
@ -369,7 +462,7 @@ get_endianness() {
}
say() {
printf 'stalwart-mail: %s\n' "$1"
printf '%s\n' "$1"
}
err() {

View file

@ -1,734 +0,0 @@
#!/usr/bin/env sh
# shellcheck shell=dash
# Stalwart Mail Server install script -- based on the rustup installation script.
set -e
set -u
readonly BASE_URL="https://github.com/stalwartlabs/mail-server/releases/latest/download"
main() {
downloader --check
need_cmd uname
need_cmd mktemp
need_cmd chmod
need_cmd mkdir
need_cmd rm
need_cmd rmdir
need_cmd tar
# Make sure we are running as root
if [ "$(id -u)" -ne 0 ] ; then
err "❌ Install failed: This program needs to run as root."
fi
# Detect OS
local _os="unknown"
local _uname="$(uname)"
_account="stalwart-mail"
if [ "${_uname}" = "Linux" ]; then
_os="linux"
elif [ "${_uname}" = "Darwin" ]; then
_os="macos"
_account="_stalwart-mail"
fi
# Read arguments
local _dir="/opt/stalwart-mail"
# Default component setting
local _component="stalwart-mail"
# Loop through the arguments
for arg in "$@"; do
case "$arg" in
--fdb)
_component="stalwart-mail-foundationdb"
;;
*)
if [ -n "$arg" ]; then
_dir=$arg
fi
;;
esac
done
# Detect platform architecture
get_architecture || return 1
local _arch="$RETVAL"
assert_nz "$_arch" "arch"
# Create directories
ensure mkdir -p "$_dir" "$_dir/bin" "$_dir/etc" "$_dir/logs"
# Download latest binary
say "⏳ Downloading ${_component} for ${_arch}..."
local _file="${_dir}/bin/stalwart-mail.tar.gz"
local _url="${BASE_URL}/${_component}-${_arch}.tar.gz"
ensure mkdir -p "$_dir"
ensure downloader "$_url" "$_file" "$_arch"
ensure tar zxvf "$_file" -C "$_dir/bin"
ignore chmod +x "$_dir/bin/stalwart-mail"
ignore rm "$_file"
# Create system account
if ! id -u ${_account} > /dev/null 2>&1; then
say "🖥️ Creating '${_account}' account..."
if [ "${_os}" = "macos" ]; then
local _last_uid="$(dscacheutil -q user | grep uid | awk '{print $2}' | sort -n | tail -n 1)"
local _last_gid="$(dscacheutil -q group | grep gid | awk '{print $2}' | sort -n | tail -n 1)"
local _uid="$((_last_uid+1))"
local _gid="$((_last_gid+1))"
ensure dscl /Local/Default -create Groups/_stalwart-mail
ensure dscl /Local/Default -create Groups/_stalwart-mail Password \*
ensure dscl /Local/Default -create Groups/_stalwart-mail PrimaryGroupID $_gid
ensure dscl /Local/Default -create Groups/_stalwart-mail RealName "Stalwart Mail service"
ensure dscl /Local/Default -create Groups/_stalwart-mail RecordName _stalwart-mail stalwart-mail
ensure dscl /Local/Default -create Users/_stalwart-mail
ensure dscl /Local/Default -create Users/_stalwart-mail NFSHomeDirectory /Users/_stalwart-mail
ensure dscl /Local/Default -create Users/_stalwart-mail Password \*
ensure dscl /Local/Default -create Users/_stalwart-mail PrimaryGroupID $_gid
ensure dscl /Local/Default -create Users/_stalwart-mail RealName "Stalwart Mail service"
ensure dscl /Local/Default -create Users/_stalwart-mail RecordName _stalwart-mail stalwart-mail
ensure dscl /Local/Default -create Users/_stalwart-mail UniqueID $_uid
ensure dscl /Local/Default -create Users/_stalwart-mail UserShell /bin/bash
ensure dscl /Local/Default -delete /Users/_stalwart-mail AuthenticationAuthority
ensure dscl /Local/Default -delete /Users/_stalwart-mail PasswordPolicyOptions
else
ensure useradd ${_account} -s /sbin/nologin -M
fi
fi
# Run init
ignore $_dir/bin/stalwart-mail --init "$_dir"
# Set permissions
say "🔐 Setting permissions..."
ensure chown -R ${_account}:${_account} "$_dir"
ensure chmod -R 755 "$_dir"
ensure chmod 700 "$_dir/etc/config.toml"
# Create service file
say "🚀 Starting service..."
if [ "${_os}" = "linux" ]; then
printf "\n[server.run-as]\nuser = \"stalwart-mail\"\ngroup = \"stalwart-mail\"\n" >> "$_dir/etc/config.toml"
create_service_linux "$_dir"
elif [ "${_os}" = "macos" ]; then
create_service_macos "$_dir"
fi
# Installation complete
local _host=$(hostname)
say "🎉 Installation complete! Continue the setup at http://$_host:8080/login"
return 0
}
# Functions to create service files
create_service_linux() {
local _dir="$1"
cat <<EOF | sed "s|__PATH__|$_dir|g" > /etc/systemd/system/stalwart-mail.service
[Unit]
Description=Stalwart Mail Server Server
Conflicts=postfix.service sendmail.service exim4.service
ConditionPathExists=__PATH__/etc/config.toml
After=network-online.target
[Service]
Type=simple
LimitNOFILE=65536
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
ExecStart=__PATH__/bin/stalwart-mail --config=__PATH__/etc/config.toml
PermissionsStartOnly=true
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=stalwart-mail
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable stalwart-mail.service
systemctl restart stalwart-mail.service
}
create_service_macos() {
local _dir="$1"
cat <<EOF | sed "s|__PATH__|$_dir|g" > /Library/LaunchAgents/stalwart.mail.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>stalwart.mail</string>
<key>ServiceDescription</key>
<string>Stalwart Mail Server</string>
<key>ProgramArguments</key>
<array>
<string>__PATH__/bin/stalwart-mail</string>
<string>--config=__PATH__/etc/config.toml</string>
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
</dict>
</plist>
EOF
launchctl load /Library/LaunchAgents/stalwart.mail.plist
launchctl enable system/stalwart.mail
launchctl start system/stalwart.mail
}
get_architecture() {
local _ostype _cputype _bitness _arch _clibtype
_ostype="$(uname -s)"
_cputype="$(uname -m)"
_clibtype="gnu"
if [ "$_ostype" = Linux ]; then
if [ "$(uname -o)" = Android ]; then
_ostype=Android
fi
if ldd --version 2>&1 | grep -q 'musl'; then
_clibtype="musl"
fi
fi
if [ "$_ostype" = Darwin ] && [ "$_cputype" = i386 ]; then
# Darwin `uname -m` lies
if sysctl hw.optional.x86_64 | grep -q ': 1'; then
_cputype=x86_64
fi
fi
if [ "$_ostype" = SunOS ]; then
# Both Solaris and illumos presently announce as "SunOS" in "uname -s"
# so use "uname -o" to disambiguate. We use the full path to the
# system uname in case the user has coreutils uname first in PATH,
# which has historically sometimes printed the wrong value here.
if [ "$(/usr/bin/uname -o)" = illumos ]; then
_ostype=illumos
fi
# illumos systems have multi-arch userlands, and "uname -m" reports the
# machine hardware name; e.g., "i86pc" on both 32- and 64-bit x86
# systems. Check for the native (widest) instruction set on the
# running kernel:
if [ "$_cputype" = i86pc ]; then
_cputype="$(isainfo -n)"
fi
fi
case "$_ostype" in
Android)
_ostype=linux-android
;;
Linux)
check_proc
_ostype=unknown-linux-$_clibtype
_bitness=$(get_bitness)
;;
FreeBSD)
_ostype=unknown-freebsd
;;
NetBSD)
_ostype=unknown-netbsd
;;
DragonFly)
_ostype=unknown-dragonfly
;;
Darwin)
_ostype=apple-darwin
;;
illumos)
_ostype=unknown-illumos
;;
MINGW* | MSYS* | CYGWIN* | Windows_NT)
_ostype=pc-windows-gnu
;;
*)
err "unrecognized OS type: $_ostype"
;;
esac
case "$_cputype" in
i386 | i486 | i686 | i786 | x86)
_cputype=i686
;;
xscale | arm)
_cputype=arm
if [ "$_ostype" = "linux-android" ]; then
_ostype=linux-androideabi
fi
;;
armv6l)
_cputype=arm
if [ "$_ostype" = "linux-android" ]; then
_ostype=linux-androideabi
else
_ostype="${_ostype}eabihf"
fi
;;
armv7l | armv8l)
_cputype=armv7
if [ "$_ostype" = "linux-android" ]; then
_ostype=linux-androideabi
else
_ostype="${_ostype}eabihf"
fi
;;
aarch64 | arm64)
_cputype=aarch64
;;
x86_64 | x86-64 | x64 | amd64)
_cputype=x86_64
;;
mips)
_cputype=$(get_endianness mips '' el)
;;
mips64)
if [ "$_bitness" -eq 64 ]; then
# only n64 ABI is supported for now
_ostype="${_ostype}abi64"
_cputype=$(get_endianness mips64 '' el)
fi
;;
ppc)
_cputype=powerpc
;;
ppc64)
_cputype=powerpc64
;;
ppc64le)
_cputype=powerpc64le
;;
s390x)
_cputype=s390x
;;
riscv64)
_cputype=riscv64gc
;;
*)
err "unknown CPU type: $_cputype"
esac
# Detect 64-bit linux with 32-bit userland
if [ "${_ostype}" = unknown-linux-gnu ] && [ "${_bitness}" -eq 32 ]; then
case $_cputype in
x86_64)
if [ -n "${RUSTUP_CPUTYPE:-}" ]; then
_cputype="$RUSTUP_CPUTYPE"
else {
# 32-bit executable for amd64 = x32
if is_host_amd64_elf; then {
echo "This host is running an x32 userland; as it stands, x32 support is poor," 1>&2
echo "and there isn't a native toolchain -- you will have to install" 1>&2
echo "multiarch compatibility with i686 and/or amd64, then select one" 1>&2
echo "by re-running this script with the RUSTUP_CPUTYPE environment variable" 1>&2
echo "set to i686 or x86_64, respectively." 1>&2
echo 1>&2
echo "You will be able to add an x32 target after installation by running" 1>&2
echo " rustup target add x86_64-unknown-linux-gnux32" 1>&2
exit 1
}; else
_cputype=i686
fi
}; fi
;;
mips64)
_cputype=$(get_endianness mips '' el)
;;
powerpc64)
_cputype=powerpc
;;
aarch64)
_cputype=armv7
if [ "$_ostype" = "linux-android" ]; then
_ostype=linux-androideabi
else
_ostype="${_ostype}eabihf"
fi
;;
riscv64gc)
err "riscv64 with 32-bit userland unsupported"
;;
esac
fi
# Detect armv7 but without the CPU features Rust needs in that build,
# and fall back to arm.
# See https://github.com/rust-lang/rustup.rs/issues/587.
if [ "$_ostype" = "unknown-linux-gnueabihf" ] && [ "$_cputype" = armv7 ]; then
if ensure grep '^Features' /proc/cpuinfo | grep -q -v neon; then
# At least one processor does not have NEON.
_cputype=arm
fi
fi
_arch="${_cputype}-${_ostype}"
RETVAL="$_arch"
}
check_proc() {
# Check for /proc by looking for the /proc/self/exe link
# This is only run on Linux
if ! test -L /proc/self/exe ; then
err "fatal: Unable to find /proc/self/exe. Is /proc mounted? Installation cannot proceed without /proc."
fi
}
get_bitness() {
need_cmd head
# Architecture detection without dependencies beyond coreutils.
# ELF files start out "\x7fELF", and the following byte is
# 0x01 for 32-bit and
# 0x02 for 64-bit.
# The printf builtin on some shells like dash only supports octal
# escape sequences, so we use those.
local _current_exe_head
_current_exe_head=$(head -c 5 /proc/self/exe )
if [ "$_current_exe_head" = "$(printf '\177ELF\001')" ]; then
echo 32
elif [ "$_current_exe_head" = "$(printf '\177ELF\002')" ]; then
echo 64
else
err "unknown platform bitness"
fi
}
is_host_amd64_elf() {
need_cmd head
need_cmd tail
# ELF e_machine detection without dependencies beyond coreutils.
# Two-byte field at offset 0x12 indicates the CPU,
# but we're interested in it being 0x3E to indicate amd64, or not that.
local _current_exe_machine
_current_exe_machine=$(head -c 19 /proc/self/exe | tail -c 1)
[ "$_current_exe_machine" = "$(printf '\076')" ]
}
get_endianness() {
local cputype=$1
local suffix_eb=$2
local suffix_el=$3
# detect endianness without od/hexdump, like get_bitness() does.
need_cmd head
need_cmd tail
local _current_exe_endianness
_current_exe_endianness="$(head -c 6 /proc/self/exe | tail -c 1)"
if [ "$_current_exe_endianness" = "$(printf '\001')" ]; then
echo "${cputype}${suffix_el}"
elif [ "$_current_exe_endianness" = "$(printf '\002')" ]; then
echo "${cputype}${suffix_eb}"
else
err "unknown platform endianness"
fi
}
say() {
printf '%s\n' "$1"
}
err() {
say "$1" >&2
exit 1
}
need_cmd() {
if ! check_cmd "$1"; then
err "need '$1' (command not found)"
fi
}
check_cmd() {
command -v "$1" > /dev/null 2>&1
}
assert_nz() {
if [ -z "$1" ]; then err "assert_nz $2"; fi
}
# Run a command that should never fail. If the command fails execution
# will immediately terminate with an error showing the failing
# command.
ensure() {
if ! "$@"; then err "command failed: $*"; fi
}
# This wraps curl or wget. Try curl first, if not installed,
# use wget instead.
downloader() {
local _dld
local _ciphersuites
local _err
local _status
local _retry
if check_cmd curl; then
_dld=curl
elif check_cmd wget; then
_dld=wget
else
_dld='curl or wget' # to be used in error message of need_cmd
fi
if [ "$1" = --check ]; then
need_cmd "$_dld"
elif [ "$_dld" = curl ]; then
check_curl_for_retry_support
_retry="$RETVAL"
get_ciphersuites_for_curl
_ciphersuites="$RETVAL"
if [ -n "$_ciphersuites" ]; then
_err=$(curl $_retry --proto '=https' --tlsv1.2 --ciphers "$_ciphersuites" --silent --show-error --fail --location "$1" --output "$2" 2>&1)
_status=$?
else
echo "Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure"
if ! check_help_for "$3" curl --proto --tlsv1.2; then
echo "Warning: Not enforcing TLS v1.2, this is potentially less secure"
_err=$(curl $_retry --silent --show-error --fail --location "$1" --output "$2" 2>&1)
_status=$?
else
_err=$(curl $_retry --proto '=https' --tlsv1.2 --silent --show-error --fail --location "$1" --output "$2" 2>&1)
_status=$?
fi
fi
if [ -n "$_err" ]; then
if echo "$_err" | grep -q 404; then
err "❌ Binary for platform '$3' not found, this platform may be unsupported."
else
echo "$_err" >&2
fi
fi
return $_status
elif [ "$_dld" = wget ]; then
if [ "$(wget -V 2>&1|head -2|tail -1|cut -f1 -d" ")" = "BusyBox" ]; then
echo "Warning: using the BusyBox version of wget. Not enforcing strong cipher suites for TLS or TLS v1.2, this is potentially less secure"
_err=$(wget "$1" -O "$2" 2>&1)
_status=$?
else
get_ciphersuites_for_wget
_ciphersuites="$RETVAL"
if [ -n "$_ciphersuites" ]; then
_err=$(wget --https-only --secure-protocol=TLSv1_2 --ciphers "$_ciphersuites" "$1" -O "$2" 2>&1)
_status=$?
else
echo "Warning: Not enforcing strong cipher suites for TLS, this is potentially less secure"
if ! check_help_for "$3" wget --https-only --secure-protocol; then
echo "Warning: Not enforcing TLS v1.2, this is potentially less secure"
_err=$(wget "$1" -O "$2" 2>&1)
_status=$?
else
_err=$(wget --https-only --secure-protocol=TLSv1_2 "$1" -O "$2" 2>&1)
_status=$?
fi
fi
fi
if [ -n "$_err" ]; then
if echo "$_err" | grep -q ' 404 Not Found'; then
err "❌ Binary for platform '$3' not found, this platform may be unsupported."
else
echo "$_err" >&2
fi
fi
return $_status
else
err "Unknown downloader" # should not reach here
fi
}
# Check if curl supports the --retry flag, then pass it to the curl invocation.
check_curl_for_retry_support() {
local _retry_supported=""
# "unspecified" is for arch, allows for possibility old OS using macports, homebrew, etc.
if check_help_for "notspecified" "curl" "--retry"; then
_retry_supported="--retry 3"
fi
RETVAL="$_retry_supported"
}
check_help_for() {
local _arch
local _cmd
local _arg
_arch="$1"
shift
_cmd="$1"
shift
local _category
if "$_cmd" --help | grep -q 'For all options use the manual or "--help all".'; then
_category="all"
else
_category=""
fi
case "$_arch" in
*darwin*)
if check_cmd sw_vers; then
case $(sw_vers -productVersion) in
10.*)
# If we're running on macOS, older than 10.13, then we always
# fail to find these options to force fallback
if [ "$(sw_vers -productVersion | cut -d. -f2)" -lt 13 ]; then
# Older than 10.13
echo "Warning: Detected macOS platform older than 10.13"
return 1
fi
;;
11.*)
# We assume Big Sur will be OK for now
;;
*)
# Unknown product version, warn and continue
echo "Warning: Detected unknown macOS major version: $(sw_vers -productVersion)"
echo "Warning TLS capabilities detection may fail"
;;
esac
fi
;;
esac
for _arg in "$@"; do
if ! "$_cmd" --help $_category | grep -q -- "$_arg"; then
return 1
fi
done
true # not strictly needed
}
# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites
# if support by local tools is detected. Detection currently supports these curl backends:
# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.
get_ciphersuites_for_curl() {
if [ -n "${RUSTUP_TLS_CIPHERSUITES-}" ]; then
# user specified custom cipher suites, assume they know what they're doing
RETVAL="$RUSTUP_TLS_CIPHERSUITES"
return
fi
local _openssl_syntax="no"
local _gnutls_syntax="no"
local _backend_supported="yes"
if curl -V | grep -q ' OpenSSL/'; then
_openssl_syntax="yes"
elif curl -V | grep -iq ' LibreSSL/'; then
_openssl_syntax="yes"
elif curl -V | grep -iq ' BoringSSL/'; then
_openssl_syntax="yes"
elif curl -V | grep -iq ' GnuTLS/'; then
_gnutls_syntax="yes"
else
_backend_supported="no"
fi
local _args_supported="no"
if [ "$_backend_supported" = "yes" ]; then
# "unspecified" is for arch, allows for possibility old OS using macports, homebrew, etc.
if check_help_for "notspecified" "curl" "--tlsv1.2" "--ciphers" "--proto"; then
_args_supported="yes"
fi
fi
local _cs=""
if [ "$_args_supported" = "yes" ]; then
if [ "$_openssl_syntax" = "yes" ]; then
_cs=$(get_strong_ciphersuites_for "openssl")
elif [ "$_gnutls_syntax" = "yes" ]; then
_cs=$(get_strong_ciphersuites_for "gnutls")
fi
fi
RETVAL="$_cs"
}
# Return cipher suite string specified by user, otherwise return strong TLS 1.2-1.3 cipher suites
# if support by local tools is detected. Detection currently supports these wget backends:
# GnuTLS and OpenSSL (possibly also LibreSSL and BoringSSL). Return value can be empty.
get_ciphersuites_for_wget() {
if [ -n "${RUSTUP_TLS_CIPHERSUITES-}" ]; then
# user specified custom cipher suites, assume they know what they're doing
RETVAL="$RUSTUP_TLS_CIPHERSUITES"
return
fi
local _cs=""
if wget -V | grep -q '\-DHAVE_LIBSSL'; then
# "unspecified" is for arch, allows for possibility old OS using macports, homebrew, etc.
if check_help_for "notspecified" "wget" "TLSv1_2" "--ciphers" "--https-only" "--secure-protocol"; then
_cs=$(get_strong_ciphersuites_for "openssl")
fi
elif wget -V | grep -q '\-DHAVE_LIBGNUTLS'; then
# "unspecified" is for arch, allows for possibility old OS using macports, homebrew, etc.
if check_help_for "notspecified" "wget" "TLSv1_2" "--ciphers" "--https-only" "--secure-protocol"; then
_cs=$(get_strong_ciphersuites_for "gnutls")
fi
fi
RETVAL="$_cs"
}
# Return strong TLS 1.2-1.3 cipher suites in OpenSSL or GnuTLS syntax. TLS 1.2
# excludes non-ECDHE and non-AEAD cipher suites. DHE is excluded due to bad
# DH params often found on servers (see RFC 7919). Sequence matches or is
# similar to Firefox 68 ESR with weak cipher suites disabled via about:config.
# $1 must be openssl or gnutls.
get_strong_ciphersuites_for() {
if [ "$1" = "openssl" ]; then
# OpenSSL is forgiving of unknown values, no problems with TLS 1.3 values on versions that don't support it yet.
echo "TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384"
elif [ "$1" = "gnutls" ]; then
# GnuTLS isn't forgiving of unknown values, so this may require a GnuTLS version that supports TLS 1.3 even if wget doesn't.
# Begin with SECURE128 (and higher) then remove/add to build cipher suites. Produces same 9 cipher suites as OpenSSL but in slightly different order.
echo "SECURE128:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-DTLS-ALL:-CIPHER-ALL:-MAC-ALL:-KX-ALL:+AEAD:+ECDHE-ECDSA:+ECDHE-RSA:+AES-128-GCM:+CHACHA20-POLY1305:+AES-256-GCM"
fi
}
# This is just for indicating that commands' results are being
# intentionally ignored. Usually, because it's being executed
# as part of error handling.
ignore() {
"$@"
}
main "$@" || exit 1

Binary file not shown.

View file

@ -5,9 +5,7 @@ edition = "2021"
resolver = "2"
[features]
default = ["sqlite", "foundationdb", "postgres", "mysql", "rocks", "elastic", "s3", "redis"]
#default = ["sqlite", "postgres", "mysql", "rocks", "elastic", "s3", "redis"]
#default = ["foundationdb", "postgres"]
default = ["sqlite", "postgres", "mysql", "rocks", "elastic", "s3", "redis"]
sqlite = ["store/sqlite"]
foundationdb = ["store/foundation"]
postgres = ["store/postgres"]