mirror of
https://github.com/tgbot-collection/ytdlbot.git
synced 2024-09-20 06:55:55 +08:00
Add K8S Deloyment and Readme
This commit is contained in:
parent
1b245471dc
commit
c8c04b4321
157
README.md
157
README.md
|
@ -197,6 +197,7 @@ You can also limit CPU and RAM usage by adding an `deploy' key:
|
|||
|
||||
Be sure to use `--compatibility` when deploying.
|
||||
|
||||
|
||||
## 4. run
|
||||
|
||||
### 4.1. standalone mode
|
||||
|
@ -231,6 +232,162 @@ docker-compose -f worker.yml up -d
|
|||
|
||||
**⚠️ Bear in mind don't publish redis directly on the internet! You can use WireGuard to wrap it up.**
|
||||
|
||||
## Kubernetes
|
||||
|
||||
Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications
|
||||
|
||||
# Complete deployment guide for k8s deloyment
|
||||
|
||||
* contains every functionality
|
||||
* compatible with amd64, arm64 and armv7l
|
||||
|
||||
## First. Get all file in k8s folder
|
||||
|
||||
Download `k8s` file to a directory on your k8s server and go to this folder
|
||||
|
||||
## 1. Create Redis deloyment
|
||||
|
||||
```shell
|
||||
kubectl apply -f 01.redis.yml
|
||||
```
|
||||
|
||||
This command will create ytdl namespace, redis pod and redis service
|
||||
|
||||
## 2. Creat MariaDB deloyment
|
||||
|
||||
```shell
|
||||
kubectl apply -f 02.mariadb.yml
|
||||
```
|
||||
This deloyment will claim 10GB storage from storageClassName: longhorn. Please replace longhorn with your storageClassName before apply.
|
||||
|
||||
## 3. Set environment variables
|
||||
|
||||
Create configMap for env
|
||||
### 3.1 Edit configmap.yml
|
||||
|
||||
```shell
|
||||
vim 03.configmap.yml
|
||||
```
|
||||
|
||||
you can configure all the following environment variables:
|
||||
|
||||
* PYRO_WORKERS: number of workers for pyrogram, default is 100
|
||||
* WORKERS: workers count for celery
|
||||
* APP_ID: **REQUIRED**, get it from https://core.telegram.org/
|
||||
* APP_HASH: **REQUIRED**
|
||||
* TOKEN: **REQUIRED**
|
||||
* REDIS: **REQUIRED if you need VIP mode and cache** ⚠️ Don't publish your redis server on the internet. ⚠️
|
||||
|
||||
* OWNER: owner username
|
||||
* QUOTA: quota in bytes
|
||||
* EX: quota expire time
|
||||
* MULTIPLY: vip quota comparing to normal quota
|
||||
* USD2CNY: exchange rate
|
||||
* VIP: VIP mode, default: disable
|
||||
* AFD_LINK
|
||||
* COFFEE_LINK
|
||||
* COFFEE_TOKEN
|
||||
* AFD_TOKEN
|
||||
* AFD_USER_ID
|
||||
|
||||
* AUTHORIZED_USER: users that could use this bot, user_id, separated with `,`
|
||||
* REQUIRED_MEMBERSHIP: group or channel username, user must join this group to use the bot. Could be use with
|
||||
above `AUTHORIZED_USER`
|
||||
|
||||
* ENABLE_CELERY: Distribution mode, default: disable. You'll can setup workers in different locations.
|
||||
* ENABLE_FFMPEG: enable ffmpeg so Telegram can stream
|
||||
* MYSQL_HOST: you'll have to setup MySQL if you enable VIP mode
|
||||
* MYSQL_USER
|
||||
* MYSQL_PASS
|
||||
* GOOGLE_API_KEY: YouTube API key, required for YouTube video subscription.
|
||||
* AUDIO_FORMAT: audio format, default is m4a. You can set to any known and supported format for ffmpeg. For
|
||||
example,`mp3`, `flac`, etc. ⚠️ m4a is the fastest. Other formats may affect performance.
|
||||
* ARCHIVE_ID: group or channel id/username. All downloads will send to this group first and then forward to end user.
|
||||
**Inline button will be lost during the forwarding.**
|
||||
|
||||
### 3.2 Apply configMap for environment variables
|
||||
|
||||
```shell
|
||||
kubectl apply -f 03.configmap.yml
|
||||
```
|
||||
|
||||
## 4. Run Master Celery
|
||||
|
||||
```shell
|
||||
kubectl apply -f 04.ytdl-master.yml
|
||||
```
|
||||
This deloyment will create ytdl-pvc PersistentVolumeClaim on storageClassName: longhorn. This clain will contain vnstat, cookies folder and flower database. Please replace longhorn with your storageClassName before apply
|
||||
|
||||
### 4.1 Setup instagram cookies
|
||||
Required if you want to support instagram.
|
||||
|
||||
You can use this extension
|
||||
[Get cookies.txt](https://chrome.google.com/webstore/detail/get-cookiestxt/bgaddhkoddajcdgocldbbfleckgcbcid)
|
||||
to get instagram cookies
|
||||
|
||||
Get pod running ytdl master:
|
||||
```shell
|
||||
kubectl get pods --namespace ytdl
|
||||
```
|
||||
Name should be ytdl-xxxxxxxx
|
||||
|
||||
Access to pod
|
||||
```shell
|
||||
kubectl --namespace=ytdl exec --stdin --tty ytdl-xxx -- sh
|
||||
```
|
||||
(replace ytdl-xxx by your pod name)
|
||||
|
||||
Go to ytdl-pvc mounted folder
|
||||
```shell
|
||||
cd /ytdlbot/ytdlbot/data/
|
||||
vim instagram.com_cookies.txt
|
||||
# paste your cookies
|
||||
```
|
||||
## 5. Run Worker Celery
|
||||
|
||||
```shell
|
||||
kubectl apply -f 05.ytdl-worker.yml
|
||||
```
|
||||
|
||||
## 6. Run Flower image (OPTIONAL)
|
||||
|
||||
|
||||
### 6.1 Setup flower db
|
||||
|
||||
Get pod running ytdl master:
|
||||
```shell
|
||||
kubectl get pods --namespace ytdl
|
||||
```
|
||||
Name should be ytdl-xxxxxxxx
|
||||
|
||||
Access to pod
|
||||
```shell
|
||||
kubectl --namespace=ytdl exec --stdin --tty ytdl-xxx -- sh
|
||||
```
|
||||
(replace ytdl-xxx by your pod name)
|
||||
|
||||
Go to ytdl-pvc mounted folder
|
||||
```shel
|
||||
cd /var/lib/vnstat/
|
||||
```
|
||||
|
||||
Create flower database file
|
||||
```shell
|
||||
{} ~ python3
|
||||
Python 3.9.9 (main, Nov 21 2021, 03:22:47)
|
||||
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
|
||||
Type "help", "copyright", "credits" or "license" for more information.
|
||||
>>> import dbm;dbm.open("flower","n");exit()
|
||||
```
|
||||
### 6.2 Config Flower Ingress
|
||||
This step need config ingress from line 51 with your ingress service for access from internet
|
||||
YML file should be adjusted depending on your load balancing, ingress and network system
|
||||
### 6.2 Apply Flower deloyment
|
||||
|
||||
```shell
|
||||
kubectl apply -f 06.flower.yml
|
||||
```
|
||||
|
||||
# Command
|
||||
|
||||
```
|
||||
|
|
53
k8s/01.redis.yml
Normal file
53
k8s/01.redis.yml
Normal file
|
@ -0,0 +1,53 @@
|
|||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ytdl
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: redis
|
||||
name: redis
|
||||
namespace: ytdl
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
ytdl: redis
|
||||
strategy: {}
|
||||
template:
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: redis
|
||||
spec:
|
||||
containers:
|
||||
- image: redis:7-alpine
|
||||
name: redis
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
resources: {}
|
||||
restartPolicy: Always
|
||||
status: {}
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: redis
|
||||
name: redis
|
||||
namespace: ytdl
|
||||
spec:
|
||||
ports:
|
||||
- name: "6379"
|
||||
port: 6379
|
||||
targetPort: 6379
|
||||
selector:
|
||||
ytdl: redis
|
||||
status:
|
||||
loadBalancer: {}
|
80
k8s/02.mariadb.yml
Normal file
80
k8s/02.mariadb.yml
Normal file
|
@ -0,0 +1,80 @@
|
|||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: mariadb-pvc
|
||||
name: mariadb-pvc
|
||||
namespace: ytdl
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
storageClassName: longhorn
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
status: {}
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
annotations:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: mariadb
|
||||
name: mariadb
|
||||
namespace: ytdl
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
ytdl: mariadb
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: mariadb
|
||||
spec:
|
||||
containers:
|
||||
- env:
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
value: ro0tP4sSworD
|
||||
- name: MYSQL_DATABASE
|
||||
value: ytdl
|
||||
image: mariadb:latest
|
||||
name: mariadb
|
||||
ports:
|
||||
- containerPort: 3306
|
||||
resources: {}
|
||||
volumeMounts:
|
||||
- mountPath: /var/lib/mysql
|
||||
name: "mariadb-persistent-storage"
|
||||
restartPolicy: Always
|
||||
volumes:
|
||||
- name: mariadb-persistent-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: mariadb-pvc
|
||||
status: {}
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: mariadb
|
||||
name: mariadb-svc
|
||||
namespace: ytdl
|
||||
spec:
|
||||
ports:
|
||||
- name: "3306"
|
||||
port: 3306
|
||||
targetPort: 3306
|
||||
selector:
|
||||
ytdl: mariadb
|
||||
status:
|
||||
loadBalancer: {}
|
||||
|
17
k8s/03.configmap.yml
Normal file
17
k8s/03.configmap.yml
Normal file
|
@ -0,0 +1,17 @@
|
|||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: ytdlenv
|
||||
namespace: ytdl
|
||||
annotations:
|
||||
data:
|
||||
APP_HASH:
|
||||
APP_ID:
|
||||
TOKEN:
|
||||
ARCHIVE_ID:
|
||||
ENABLE_CELERY: 'True'
|
||||
ENABLE_FFMPEG: 'True'
|
||||
MYSQL_HOST: mariadb-svc
|
||||
MYSQL_PASS: ro0tP4sSworD
|
||||
MYSQL_USER: root
|
||||
REDIS: redis
|
65
k8s/04.ytdl-master.yml
Normal file
65
k8s/04.ytdl-master.yml
Normal file
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: ytdl-pvc
|
||||
namespace: ytdl
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: ytdl-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteMany
|
||||
storageClassName: longhorn
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
status: {}
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: ytdl
|
||||
namespace: ytdl
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: ytdl
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
ytdl: ytdl
|
||||
template:
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: ytdl
|
||||
spec:
|
||||
volumes:
|
||||
- name: ytdl-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: ytdl-pvc
|
||||
containers:
|
||||
- name: ytdl
|
||||
image: bennythink/ytdlbot
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: ytdlenv
|
||||
resources: {}
|
||||
volumeMounts:
|
||||
- name: ytdl-pvc
|
||||
mountPath: /var/lib/vnstat/
|
||||
subPath: vnstat/
|
||||
- name: ytdl-pvc
|
||||
mountPath: /ytdlbot/ytdlbot/data/
|
||||
subPath: data/
|
||||
terminationMessagePath: /dev/termination-log
|
||||
terminationMessagePolicy: File
|
||||
imagePullPolicy: Always
|
||||
restartPolicy: Always
|
||||
terminationGracePeriodSeconds: 30
|
||||
dnsPolicy: ClusterFirst
|
||||
securityContext: {}
|
||||
schedulerName: default-scheduler
|
||||
status: {}
|
47
k8s/05.ytdl-worker.yml
Normal file
47
k8s/05.ytdl-worker.yml
Normal file
|
@ -0,0 +1,47 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: ytdl-worker
|
||||
name: ytdl-worker
|
||||
namespace: ytdl
|
||||
spec:
|
||||
replicas: 4
|
||||
selector:
|
||||
matchLabels:
|
||||
ytdl: ytdl-worker
|
||||
template:
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: ytdl-worker
|
||||
spec:
|
||||
volumes:
|
||||
- name: ytdl-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: ytdl-pvc
|
||||
containers:
|
||||
- name: ytdl-worker
|
||||
image: bennythink/ytdlbot
|
||||
args:
|
||||
- /usr/local/bin/supervisord
|
||||
- '-c'
|
||||
- /ytdlbot/conf/supervisor_worker.conf
|
||||
envFrom:
|
||||
- configMapRef:
|
||||
name: ytdlenv
|
||||
resources: {}
|
||||
volumeMounts:
|
||||
- name: ytdl-pvc
|
||||
mountPath: /ytdlbot/ytdlbot/data/
|
||||
subPath: data/
|
||||
terminationMessagePath: /dev/termination-log
|
||||
terminationMessagePolicy: File
|
||||
imagePullPolicy: Always
|
||||
restartPolicy: Always
|
||||
terminationGracePeriodSeconds: 30
|
||||
dnsPolicy: ClusterFirst
|
||||
securityContext: {}
|
||||
schedulerName: default-scheduler
|
||||
status: {}
|
101
k8s/06.flower.yml
Normal file
101
k8s/06.flower.yml
Normal file
|
@ -0,0 +1,101 @@
|
|||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: flower
|
||||
name: flower
|
||||
namespace: ytdl
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
ytdl: flower
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: flower
|
||||
spec:
|
||||
containers:
|
||||
- envFrom:
|
||||
- configMapRef:
|
||||
name: ytdlenv
|
||||
args:
|
||||
- /usr/local/bin/celery
|
||||
- -A
|
||||
- flower_tasks
|
||||
- flower
|
||||
- --basic_auth=bennythink:123456
|
||||
- --address=0.0.0.0
|
||||
- --persistent
|
||||
- --purge_offline_workers=3600
|
||||
image: bennythink/ytdlbot
|
||||
name: flower
|
||||
ports:
|
||||
- containerPort: 5555
|
||||
resources: {}
|
||||
volumeMounts:
|
||||
- name: ytdl-pvc
|
||||
mountPath: /ytdlbot/ytdlbot/flower
|
||||
subPath: vnstat/flower
|
||||
restartPolicy: Always
|
||||
volumes:
|
||||
- name: ytdl-pvc
|
||||
persistentVolumeClaim:
|
||||
claimName: ytdl-pvc
|
||||
status: {}
|
||||
|
||||
# THIS IS OPTION IF YOU WANT PUBLIC FLOWER PAGE TO INTERNET.
|
||||
# should be adjusted depending on your load balancing system machine
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
creationTimestamp: null
|
||||
labels:
|
||||
ytdl: flower
|
||||
name: flower-svc
|
||||
namespace: ytdl
|
||||
spec:
|
||||
type: NodePort
|
||||
ports:
|
||||
- name: "5555"
|
||||
protocol: TCP
|
||||
port: 5555
|
||||
targetPort: 5555
|
||||
selector:
|
||||
ytdl: flower
|
||||
status:
|
||||
loadBalancer: {}
|
||||
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: nginx-flower-ingress
|
||||
namespace: ytdl
|
||||
annotations:
|
||||
# cert-manager.io/cluster-issuer: letsencrypt-prod
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /
|
||||
# nginx.ingress.kubernetes.io/whitelist-source-range: 14.161.27.151 limit by ipaddresss
|
||||
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- your-domain
|
||||
secretName: flower-tls
|
||||
rules:
|
||||
- host: your-domain
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: flower-svc
|
||||
port:
|
||||
number: 5555
|
Loading…
Reference in a new issue