Add K8S Deloyment and Readme

2024-09-20 06:55:55 +08:00 · 2022-11-25 14:09:56 +07:00 · 2022-11-25 14:09:56 +07:00 · c8c04b4321
parent 1b245471dc
commit c8c04b4321
7 changed files with 521 additions and 1 deletions
--- a/README.md
+++ b/README.md
@ -197,6 +197,7 @@ You can also limit CPU and RAM usage by adding an `deploy' key:

 Be sure to use `--compatibility` when deploying.

+
 ## 4. run

 ### 4.1. standalone mode
@ -231,6 +232,162 @@ docker-compose -f worker.yml up -d

 **⚠️ Bear in mind don't publish redis directly on the internet! You can use WireGuard to wrap it up.**

+## Kubernetes
+
+Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications 
+
+# Complete deployment guide for k8s deloyment
+
+* contains every functionality
+* compatible with amd64, arm64 and armv7l
+
+## First. Get all file in k8s folder
+
+Download `k8s` file to a directory on your k8s server and go to this folder
+
+## 1. Create Redis deloyment
+
+```shell
+kubectl apply -f 01.redis.yml
+```
+
+This command will create ytdl namespace, redis pod and redis service
+
+## 2. Creat MariaDB deloyment
+
+```shell
+kubectl apply -f 02.mariadb.yml
+```
+This deloyment will claim 10GB storage from storageClassName: longhorn. Please replace longhorn with your storageClassName before apply.
+
+## 3. Set environment variables
+
+Create configMap for env
+### 3.1 Edit configmap.yml
+
+```shell
+vim 03.configmap.yml
+```
+
+you can configure all the following environment variables:
+
+* PYRO_WORKERS: number of workers for pyrogram, default is 100
+* WORKERS: workers count for celery
+* APP_ID: **REQUIRED**, get it from https://core.telegram.org/
+* APP_HASH: **REQUIRED**
+* TOKEN: **REQUIRED**
+* REDIS: **REQUIRED if you need VIP mode and cache** ⚠️ Don't publish your redis server on the internet. ⚠️
+
+* OWNER: owner username
+* QUOTA: quota in bytes
+* EX: quota expire time
+* MULTIPLY: vip quota comparing to normal quota
+* USD2CNY: exchange rate
+* VIP: VIP mode, default: disable
+* AFD_LINK
+* COFFEE_LINK
+* COFFEE_TOKEN
+* AFD_TOKEN
+* AFD_USER_ID
+
+* AUTHORIZED_USER: users that could use this bot, user_id, separated with `,`
+* REQUIRED_MEMBERSHIP: group or channel username, user must join this group to use the bot. Could be use with
+  above `AUTHORIZED_USER`
+
+* ENABLE_CELERY: Distribution mode, default: disable. You'll can setup workers in different locations.
+* ENABLE_FFMPEG: enable ffmpeg so Telegram can stream
+* MYSQL_HOST: you'll have to setup MySQL if you enable VIP mode
+* MYSQL_USER
+* MYSQL_PASS
+* GOOGLE_API_KEY: YouTube API key, required for YouTube video subscription.
+* AUDIO_FORMAT: audio format, default is m4a. You can set to any known and supported format for ffmpeg. For
+  example,`mp3`, `flac`, etc. ⚠️ m4a is the fastest. Other formats may affect performance.
+* ARCHIVE_ID: group or channel id/username. All downloads will send to this group first and then forward to end user.
+  **Inline button will be lost during the forwarding.**
+
+### 3.2 Apply configMap for environment variables
+
+```shell
+kubectl apply -f 03.configmap.yml
+```
+
+## 4. Run Master Celery
+
+```shell
+kubectl apply -f 04.ytdl-master.yml
+```
+This deloyment will create ytdl-pvc PersistentVolumeClaim on storageClassName: longhorn. This clain will contain vnstat, cookies folder and flower database. Please replace longhorn with your storageClassName before apply
+
+### 4.1 Setup instagram cookies
+Required if you want to support instagram.
+
+You can use this extension
+[Get cookies.txt](https://chrome.google.com/webstore/detail/get-cookiestxt/bgaddhkoddajcdgocldbbfleckgcbcid)
+to get instagram cookies
+
+Get pod running ytdl master:
+```shell
+kubectl get pods --namespace ytdl
+```
+Name should be ytdl-xxxxxxxx
+
+Access to pod
+```shell
+kubectl --namespace=ytdl exec --stdin --tty ytdl-xxx -- sh
+```
+(replace ytdl-xxx by your pod name)
+
+Go to ytdl-pvc mounted folder
+```shell
+cd /ytdlbot/ytdlbot/data/
+vim  instagram.com_cookies.txt
+# paste your cookies
+```
+## 5. Run Worker Celery
+
+```shell
+kubectl apply -f 05.ytdl-worker.yml
+```
+
+## 6. Run Flower image (OPTIONAL)
+
+
+### 6.1 Setup flower db
+
+Get pod running ytdl master:
+```shell
+kubectl get pods --namespace ytdl
+```
+Name should be ytdl-xxxxxxxx
+
+Access to pod
+```shell
+kubectl --namespace=ytdl exec --stdin --tty ytdl-xxx -- sh
+```
+(replace ytdl-xxx by your pod name)
+
+Go to ytdl-pvc mounted folder
+```shel
+cd /var/lib/vnstat/
+```
+
+Create flower database file
+```shell
+{} ~ python3
+Python 3.9.9 (main, Nov 21 2021, 03:22:47)
+[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
+Type "help", "copyright", "credits" or "license" for more information.
+>>> import dbm;dbm.open("flower","n");exit()
+```
+### 6.2 Config Flower Ingress
+This step need config ingress from line 51 with your ingress service for access from internet
+YML file should be adjusted depending on your load balancing, ingress and network system
+### 6.2 Apply Flower deloyment
+
+```shell
+kubectl apply -f 06.flower.yml
+```
+
 # Command

 ```
@ -275,4 +432,4 @@ https://twitter.com/BennyThinks/status/1475836588542341124

 # License

-Apache License 2.0
+Apache License 2.0
--- a/k8s/01.redis.yml
+++ b/k8s/01.redis.yml
@ -0,0 +1,53 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: ytdl
+  
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: redis
+  name: redis
+  namespace: ytdl
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      ytdl: redis
+  strategy: {}
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        ytdl: redis
+    spec:
+      containers:
+        - image: redis:7-alpine
+          name: redis
+          ports:
+            - containerPort: 6379
+          resources: {}
+      restartPolicy: Always
+status: {}
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: redis
+  name: redis
+  namespace: ytdl
+spec:
+  ports:
+    - name: "6379"
+      port: 6379
+      targetPort: 6379
+  selector:
+    ytdl: redis
+status:
+  loadBalancer: {}
--- a/k8s/02.mariadb.yml
+++ b/k8s/02.mariadb.yml
@ -0,0 +1,80 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: mariadb-pvc
+  name: mariadb-pvc
+  namespace: ytdl
+spec:
+  accessModes:
+    - ReadWriteOnce
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 10Gi
+status: {}
+
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  annotations:
+  creationTimestamp: null
+  labels:
+    ytdl: mariadb
+  name: mariadb
+  namespace: ytdl
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      ytdl: mariadb
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        ytdl: mariadb
+    spec:
+      containers:
+        - env:
+            - name: MYSQL_ROOT_PASSWORD
+              value: ro0tP4sSworD
+            - name: MYSQL_DATABASE
+              value: ytdl
+          image: mariadb:latest
+          name: mariadb
+          ports:
+            - containerPort: 3306
+          resources: {}
+          volumeMounts:
+            - mountPath: /var/lib/mysql
+              name: "mariadb-persistent-storage"
+      restartPolicy: Always
+      volumes:
+        - name: mariadb-persistent-storage
+          persistentVolumeClaim:
+            claimName: mariadb-pvc
+status: {}
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: mariadb
+  name: mariadb-svc
+  namespace: ytdl
+spec:
+  ports:
+    - name: "3306"
+      port: 3306
+      targetPort: 3306
+  selector:
+    ytdl: mariadb
+status:
+  loadBalancer: {}
+
--- a/k8s/03.configmap.yml
+++ b/k8s/03.configmap.yml
@ -0,0 +1,17 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: ytdlenv
+  namespace: ytdl
+  annotations:
+data:
+  APP_HASH: 
+  APP_ID: 
+  TOKEN:
+  ARCHIVE_ID: 
+  ENABLE_CELERY: 'True'
+  ENABLE_FFMPEG: 'True'
+  MYSQL_HOST: mariadb-svc
+  MYSQL_PASS: ro0tP4sSworD
+  MYSQL_USER: root
+  REDIS: redis
--- a/k8s/04.ytdl-master.yml
+++ b/k8s/04.ytdl-master.yml
@ -0,0 +1,65 @@
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: ytdl-pvc
+  namespace: ytdl
+  creationTimestamp: null
+  labels:
+    ytdl: ytdl-pvc
+spec:
+  accessModes:
+    - ReadWriteMany
+  storageClassName: longhorn
+  resources:
+    requests:
+      storage: 10Gi
+status: {}
+
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ytdl
+  namespace: ytdl
+  creationTimestamp: null
+  labels:
+    ytdl: ytdl
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      ytdl: ytdl
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        ytdl: ytdl
+    spec:
+      volumes:
+        - name: ytdl-pvc
+          persistentVolumeClaim:
+            claimName: ytdl-pvc
+      containers:
+        - name: ytdl
+          image: bennythink/ytdlbot
+          envFrom:
+            - configMapRef:
+                name: ytdlenv
+          resources: {}
+          volumeMounts:
+            - name: ytdl-pvc
+              mountPath: /var/lib/vnstat/
+              subPath: vnstat/
+            - name: ytdl-pvc
+              mountPath: /ytdlbot/ytdlbot/data/
+              subPath: data/
+          terminationMessagePath: /dev/termination-log
+          terminationMessagePolicy: File
+          imagePullPolicy: Always
+      restartPolicy: Always
+      terminationGracePeriodSeconds: 30
+      dnsPolicy: ClusterFirst
+      securityContext: {}
+      schedulerName: default-scheduler
+status: {}
--- a/k8s/05.ytdl-worker.yml
+++ b/k8s/05.ytdl-worker.yml
@ -0,0 +1,47 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: ytdl-worker
+  name: ytdl-worker
+  namespace: ytdl
+spec:
+  replicas: 4
+  selector:
+    matchLabels:
+      ytdl: ytdl-worker
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        ytdl: ytdl-worker
+    spec:
+      volumes:
+        - name: ytdl-pvc
+          persistentVolumeClaim:
+            claimName: ytdl-pvc
+      containers:
+        - name: ytdl-worker
+          image: bennythink/ytdlbot
+          args:
+            - /usr/local/bin/supervisord
+            - '-c'
+            - /ytdlbot/conf/supervisor_worker.conf
+          envFrom:
+            - configMapRef:
+                name: ytdlenv
+          resources: {}
+          volumeMounts:
+            - name: ytdl-pvc
+              mountPath: /ytdlbot/ytdlbot/data/
+              subPath: data/
+          terminationMessagePath: /dev/termination-log
+          terminationMessagePolicy: File
+          imagePullPolicy: Always
+      restartPolicy: Always
+      terminationGracePeriodSeconds: 30
+      dnsPolicy: ClusterFirst
+      securityContext: {}
+      schedulerName: default-scheduler
+status: {}
--- a/k8s/06.flower.yml
+++ b/k8s/06.flower.yml
@ -0,0 +1,101 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: flower
+  name: flower
+  namespace: ytdl
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      ytdl: flower
+  strategy:
+    type: Recreate
+  template:
+    metadata:
+      creationTimestamp: null
+      labels:
+        ytdl: flower
+    spec:
+      containers:
+        - envFrom:
+            - configMapRef:
+                name: ytdlenv
+          args:
+           - /usr/local/bin/celery
+           - -A
+           - flower_tasks
+           - flower
+           - --basic_auth=bennythink:123456
+           - --address=0.0.0.0
+           - --persistent
+           - --purge_offline_workers=3600
+          image: bennythink/ytdlbot
+          name: flower
+          ports:
+            - containerPort: 5555
+          resources: {}
+          volumeMounts:
+            - name: ytdl-pvc
+              mountPath: /ytdlbot/ytdlbot/flower
+              subPath: vnstat/flower
+      restartPolicy: Always
+      volumes:
+        - name: ytdl-pvc
+          persistentVolumeClaim:
+            claimName: ytdl-pvc
+status: {}
+
+# THIS IS OPTION IF YOU WANT PUBLIC FLOWER PAGE TO INTERNET.
+# should be adjusted depending on your load balancing system machine
+---
+apiVersion: v1
+kind: Service
+metadata:
+  creationTimestamp: null
+  labels:
+    ytdl: flower
+  name: flower-svc
+  namespace: ytdl
+spec:
+  type: NodePort
+  ports:
+    - name: "5555"
+      protocol: TCP
+      port: 5555
+      targetPort: 5555
+  selector:
+    ytdl: flower
+status:
+  loadBalancer: {}
+
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  name: nginx-flower-ingress
+  namespace: ytdl
+  annotations:
+    # cert-manager.io/cluster-issuer: letsencrypt-prod
+    nginx.ingress.kubernetes.io/rewrite-target: /
+    # nginx.ingress.kubernetes.io/whitelist-source-range: 14.161.27.151 limit by ipaddresss
+ 
+spec:
+  ingressClassName: nginx
+  tls:
+    - hosts:
+        - your-domain
+      secretName: flower-tls
+  rules:
+    - host: your-domain
+      http:
+        paths:
+          - path: /
+            pathType: Prefix
+            backend:
+              service:
+                name: flower-svc
+                port:
+                  number: 5555