JuiceFS on Kubernetes 面试题库

graph TD
    subgraph APP["Application / Pod"]
        direction LR
        C1["FUSE"]
        C2["CSI"]
        C3["S3 GW"]
        C4["Hadoop"]
        C5["SDK"]
    end
    APP --> JFS
    subgraph JFS["JuiceFS Client (jfs)"]
        MC["Metadata Client"]
        DC["Data Client"]
        CM["Cache Manager"]
    end
    MC --> ME["Metadata Engine<br/>Redis/TiKV/MySQL/PG/FDB"]
    DC --> OS["Object Storage<br/>S3/OSS/COS/MinIO"]
    CM --> LD["Local Disk Cache"]

文件数 < 1000 万，时延敏感 ──► Redis（最高性能）
文件数 > 1 亿 ──► TiKV 或 FoundationDB（水平扩展）
已有 MySQL/PG 运维体系 ──► MySQL 或 PostgreSQL（运维成本低）
需要跨 DC 容灾 ──► TiKV（原生 Raft 多副本）

文件
  └── Chunk 1 (64MB)
        ├── Slice 1 (数据在连续写入时产生)
        │     ├── Block 1 (4MB)  → 对象: chunks/0/0_0_1
        │     ├── Block 2 (4MB)  → 对象: chunks/0/0_1_1
        │     └── Block 3 (4MB)  → 对象: chunks/0/0_2_1
        └── Slice 2 (flush 或 seek 后产生新 Slice)
              ├── Block 1 (4MB)
              └── Block 2 (4MB)
  └── Chunk 2 (64MB)
        └── ...

flowchart TD
    A["Read Request"] --> B1{"1. 内核页缓存命中?"}
    B1 -->|是| R1["直接返回"]
    B1 -->|否| B2{"2. 内存缓存命中?<br/>(jfs process)"}
    B2 -->|是| R2["返回"]
    B2 -->|否| B3{"3. 本地磁盘缓存命中?"}
    B3 -->|是| S1["读入内存"] --> R3["返回"]
    B3 -->|否| B4{"4. 缓存组查询命中?"}
    B4 -->|是| S2["从对端节点拉取"] --> R4["返回"]
    B4 -->|否| B5["5. 从 Object Storage 下载"]

# CSI Driver 中配置本地磁盘缓存
apiVersion: v1
kind: Secret
metadata:
  name: juicefs-cache
stringData:
  cache-dir: /var/jfs-cache
  cache-size: "102400"  # 100GB
  buffer-size: "300"    # 300MB 写缓冲
  prefetch: "1"         # 启用顺序读预取
  writeback: "false"    # 写回模式（生产慎用）

打开文件 (open)
  └── 获取读锁 (FLock / POSIX Lock)
        ├── 读锁共享，多个 Client 可同时持有
        └── 写锁互斥，排他性占用

写入流程:
  1. Client write() → 数据进本地 Buffer
  2. Client flush() / fsync() → 数据切 Block → 写 Object Storage
  3. Object Storage 写入完成 → 更新 Meta Engine 元数据
  4. Meta Engine 递增 inode 版本号

其他 Client 读取:
  1. 检查 inode 版本号（通过 Meta Engine）
  2. 版本号变更 → 丢弃本地缓存 → 重新读取元数据 → 从 Object Storage 加载 Block

graph TD
    subgraph Cluster["Kubernetes Cluster"]
        subgraph Controller["CSI Controller (Deployment)"]
            CP["- Provisioner"]
            CA["- Attacher"]
            CR["- Resizer"]
            CS["- Snapshotter"]
        end
        subgraph Node["CSI Node Plugin (DaemonSet)"]
            ND["- Node DriverReg"]
            MP["- Mount Pod Manager"]
            VC["- Volume Cleaner"]
        end
        Controller --> PVC["PVC to PV Bind<br/>Secret Read<br/>Format/Create"]
        Node --> MountPod
        subgraph MountPod["Mount Pod (per PVC)"]
            MD["juicefs mount...<br/>FUSE Daemon<br/>Mount Point"]
        end
    end

# 1. 安装 CSI Driver
kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml

# 2. 创建 Secret（包含文件系统连接信息）
kubectl create secret generic juicefs-secret \
  --from-literal=name=myjfs \
  --from-literal=metaurl=redis://10.0.0.1:6379/0 \
  --from-literal=storage=s3 \
  --from-literal=bucket=https://s3.amazonaws.com/mybucket \
  --from-literal=access-key=AKIAIOSFODNN7EXAMPLE \
  --from-literal=secret-key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

# 静态 PV
apiVersion: v1
kind: PersistentVolume
metadata:
  name: juicefs-static-pv
spec:
  capacity:
    storage: 10Pi
  accessModes:
    - ReadWriteMany
  csi:
    driver: csi.juicefs.com
    volumeHandle: myjfs
    fsType: juicefs
    volumeAttributes:
      subPath: /data/team-a
    nodePublishSecretRef:
      name: juicefs-secret
      namespace: default
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: juicefs-static-pvc
spec:
  storageClassName: ""
  volumeName: juicefs-static-pv
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Pi

# 动态 StorageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
  subPathPattern: "${pvc.namespace}-${pvc.name}"
reclaimPolicy: Delete
allowVolumeExpansion: true
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: juicefs-dynamic-pvc
spec:
  storageClassName: juicefs-sc
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Pi

graph LR
    subgraph AppPod["Application Pod"]
        BM["/data (bind mount)"]
    end
    BM -->|"Bind Mount"| VolDir
    subgraph MountPod["Mount Pod"]
        JFS["juicefs mount...<br/>FUSE Daemon"]
    end
    subgraph Node["Node"]
        VolDir["/var/lib/kubelet/pods/<br/>xxx/volumes/.../mount"]
    end
    MountPod --> VolDir

graph LR
    subgraph AppPod["Application Pod"]
        subgraph AppContainer["App Container"]
            Data["/data"]
        end
        subgraph Sidecar["JuiceFS Sidecar Container"]
            FUSE["FUSE Daemon"]
        end
        Data --> FUSE
    end

graph TD
    subgraph AppPod["Application Pod"]
        JF["juicefs mount + App<br/>同一进程空间"]
    end

graph TD
    A["Client A 写 file.txt"]
    B["Client B 写 file.txt"]
    C["Meta Engine: Chunk 0"]
    D["Meta Engine: Chunk 1"]
    E["Object Storage:<br/>chunks/0/0_0_1<br/>chunks/0/0_1_1"]
    F["Object Storage:<br/>chunks/1/0_0_1<br/>chunks/1/0_1_1"]
    A --> C --> E
    B --> D --> F

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: shared-data
spec:
  accessModes:
    - ReadWriteMany  # 多 Pod 同时读写
  storageClassName: juicefs-sc
  resources:
    requests:
      storage: 100Gi

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-a
spec:
  template:
    spec:
      containers:
      - name: main
        volumeMounts:
        - name: data
          mountPath: /shared
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: shared-data
# Deployment B 使用同一 PVC，实现 RWX
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-b
spec:
  template:
    spec:
      containers:
      - name: main
        volumeMounts:
        - name: data
          mountPath: /shared
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: shared-data

juicefs mount --prefetch=1 redis://host:6379/0 /mnt/jfs

juicefs mount \
  --max-uploads=20 \          # 并发上传数（写密集场景增大）
  --max-deletes=10 \          # 并发删除数
  --io-retries=10 \           # I/O 重试次数
  --get-timeout=60 \          # 对象下载超时（秒）
  --put-timeout=60            # 对象上传超时（秒）

apiVersion: v1
kind: Secret
metadata:
  name: juicefs-perf-secret
stringData:
  name: myjfs
  metaurl: redis://10.0.0.1:6379/0
  storage: s3
  bucket: https://s3.amazonaws.com/mybucket
  access-key: AKIAIOSFODNN7EXAMPLE
  secret-key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
  cache-dir: /var/jfs-cache
  cache-size: "512000"
  buffer-size: "1000"
  prefetch: "1"
  metacache: "60"
  attr-cache: "3600"
  entry-cache: "3600"
  max-uploads: "50"
  max-deletes: "20"

# 挂载时启用回收站（默认启用）
juicefs mount --trash-days=7 redis://host:6379/0 /mnt/jfs

# 查看待清理对象
juicefs gc redis://host:6379/0

# 执行清理
juicefs gc --delete redis://host:6379/0

1. 扫描 Meta Engine 中所有文件的 Chunk / Slice 引用
2. 列举 Object Storage 中所有已存储的对象
3. 比对两者的差集 → 孤儿对象列表
4. --delete 模式下删除孤儿对象

apiVersion: batch/v1
kind: CronJob
metadata:
  name: juicefs-gc
spec:
  schedule: "0 2 * * 0"  # 每周日 02:00
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: gc
            image: juicedata/juicefs-csi-driver:latest
            command:
            - juicefs
            - gc
            - --delete
            - redis://redis.default.svc.cluster.local:6379/0
          restartPolicy: OnFailure

graph TD
    subgraph Before["创建快照 snap-20250101"]
        FS1["文件系统当前状态<br/>/dir/file1.dat<br/>Chunk 0, 1, 2<br/>共享 Chunk"]
        FS1 -->|"COW"| SNAP1["快照 snap-20250101<br/>/dir/file1.dat<br/>Chunk 0, 1, 2<br/>共享 Chunk"]
    end
    subgraph After["修改 file1.dat 后"]
        FS2["文件系统当前状态<br/>/dir/file1.dat<br/>Chunk 0, 3, 2<br/>新写 Chunk 3"]
        SNAP2["快照 snap-20250101<br/>/dir/file1.dat<br/>Chunk 0, 1, 2<br/>快照保留旧 Chunk"]
    end
    Before -->|"修改 file1.dat"| After

# 语法: juicefs snapshot <METAURL> <dir> <snapshot-name>
juicefs snapshot redis://host:6379/0 /mnt/jfs/data snap-20250101

# 从快照创建克隆
juicefs clone redis://host:6379/0 snap-20250101 new-volume

# 挂载克隆文件系统
juicefs mount redis://host:6379/0 /mnt/jfs-new --subdir new-volume

# VolumeSnapshotClass
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
  name: juicefs-snapclass
driver: csi.juicefs.com
deletionPolicy: Delete
# VolumeSnapshot
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: data-snapshot-20250101
spec:
  volumeSnapshotClassName: juicefs-snapclass
  source:
    persistentVolumeClaimName: data-pvc
# 从快照恢复 PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-restore
spec:
  dataSource:
    name: data-snapshot-20250101
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
    - ReadWriteMany
  storageClassName: juicefs-sc
  resources:
    requests:
      storage: 100Gi

juicefs format \
  --storage s3 \
  --bucket https://s3.amazonaws.com/mybucket \  # HTTPS
  redis://host:6379/0 myjfs

# Redis TLS
juicefs mount rediss://host:6379/0 /mnt/jfs

# MySQL TLS
juicefs mount "mysql://user:pass@(host:3306)/db?tls=true" /mnt/jfs

# 生成 RSA 私钥
openssl genrsa -out private.pem 2048

# 创建加密文件系统
juicefs format \
  --encrypt-rsa-key private.pem \
  --storage s3 \
  --bucket https://s3.amazonaws.com/mybucket \
  redis://host:6379/0 secure-jfs

# 挂载时指定密钥
juicefs mount \
  --encrypt-rsa-key private.pem \
  redis://host:6379/0 /mnt/jfs

apiVersion: v1
kind: Secret
metadata:
  name: juicefs-encrypt-secret
stringData:
  name: secure-jfs
  metaurl: redis://redis:6379/0
  storage: s3
  bucket: https://s3.amazonaws.com/mybucket
  access-key: xxx
  secret-key: xxx
  encrypt-rsa-key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEpAIBAAKCAQEA...
    -----END RSA PRIVATE KEY-----    

# 导出元数据到 JSON（可读备份）
juicefs dump redis://host:6379/0 > metadata-20250101.json

# 导入元数据（到新的 Meta Engine）
juicefs load redis://new-host:6379/0 < metadata-20250101.json

灾难恢复场景:
  1. 验证 Object Storage 数据完整（对象存储自身通常有 99.999999999% 耐久）
  2. 创建新的 Meta Engine 实例
  3. juicefs load 导入备份元数据
  4. juicefs mount 挂载验证
  5. 更新 CSI Driver Secret 的 metaurl
  6. 重建应用 Pod

# 自动备份 CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: juicefs-dump
spec:
  schedule: "0 3 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: dump
            image: juicedata/juicefs-csi-driver:latest
            command:
            - /bin/sh
            - -c
            - |
              juicefs dump redis://redis:6379/0 > /backup/dump-$(date +%Y%m%d-%H%M).json
              # 可选：上传到对象存储作为异地备份
            volumeMounts:
            - name: backup
              mountPath: /backup
          volumes:
          - name: backup
            persistentVolumeClaim:
              claimName: backup-pvc
          restartPolicy: OnFailure

graph TD
    N1["Node 1<br/>JFS Client"]
    N2["Node 2<br/>JFS Client"]
    N3["Node 3<br/>JFS Client"]
    N1 --> ME
    N2 --> ME
    N3 --> ME
    ME["Meta Engine<br/>Redis / TiKV..."]
    ME --> OS["Object Storage<br/>S3 / OSS..."]

# 不同节点使用相同的 Meta Engine 地址
# Node 1
juicefs mount redis://10.0.0.1:6379/0 /mnt/jfs

# Node 2
juicefs mount redis://10.0.0.1:6379/0 /mnt/jfs

# Node 3
juicefs mount redis://10.0.0.1:6379/0 /mnt/jfs

# Kubernetes 示例：3 个 Deployment 共享同一 JuiceFS PVC
# 场景：AI 训练时，多个 Worker Pod 并发从共享文件系统读取训练数据

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: training-worker
spec:
  replicas: 4
  template:
    spec:
      containers:
      - name: worker
        image: my-training-image
        volumeMounts:
        - name: dataset
          mountPath: /data/dataset
        - name: checkpoint
          mountPath: /data/checkpoints
      volumes:
      - name: dataset
        persistentVolumeClaim:
          claimName: dataset-pvc    # 只读，共享数据集
      - name: checkpoint
        persistentVolumeClaim:
          claimName: checkpoint-pvc # RWX，写检查点

graph LR
    API["API Server<br/>PVC Bound"] -->|"1"| CSI["CSI Controller<br/>CreateVolume"]
    CSI -->|"2"| Node["CSI Node<br/>NodePublish"]
    Node -->|"3"| MountPod["Mount Pod<br/>FUSE Mount /jfs/mount"]
    MountPod -->|"4"| Kubelet["Kubelet Pod Volume Dir<br/>bind mount to<br/>Application Pod"]

/var/lib/kubelet/pods/<pod-uid>/volumes/kubernetes.io~csi/<pv-name>/mount

Pod 删除
  → Kubelet 调用 CSI NodeUnpublishVolume
    → unmount Kubelet Volume 目录
    → 检查 Mount Pod 是否还有其他 PVC 引用
    → 无引用时，删除 Mount Pod
PVC 删除（reclaimPolicy: Delete）
  → CSI Controller DeleteVolume
    → 删除 subPath 目录内容
    → 释放文件系统空间
    → 删除 PV 对象

# 手动清理残留的 Mount Pod
kubectl delete pod -l app.kubernetes.io/name=juicefs-mount -n kube-system

# 清理 Kubelet 残留目录
kubectl delete pod -A --all --force --grace-period=0
# 然后登录节点
rm -rf /var/lib/kubelet/pods/*/volumes/kubernetes.io~csi/*/mount

graph TD
    subgraph CG["Cache Group: training-cache"]
        W1["Worker 1<br/>Cache<br/>SSD-1"]
        W2["Worker 2<br/>Cache<br/>SSD-2"]
        W3["Worker 3<br/>Cache<br/>SSD-3"]
    end
    Hash["一致性哈希分布"]
    W1 --> Hash
    W2 --> Hash
    W3 --> Hash
    Hash --> BA["Block A to Worker 1"]
    Hash --> BB["Block B to Worker 3"]
    Hash --> BC["Block C to Worker 2"]

# 所有节点使用相同的 cache-group 名称
juicefs mount \
  --cache-dir=/var/jfs-cache \
  --cache-size=500000 \
  --cache-group=training-cache \
  redis://host:6379/0 /mnt/jfs

apiVersion: v1
kind: Secret
metadata:
  name: juicefs-cachegroup
stringData:
  name: myjfs
  metaurl: redis://redis:6379/0
  storage: s3
  bucket: https://s3.amazonaws.com/mybucket
  access-key: xxx
  secret-key: xxx
  cache-dir: /var/jfs-cache
  cache-size: "512000"
  cache-group: training-cache

1. Client 请求 Block X
2. 检查本地缓存 → 命中则返回
3. 如果未命中，计算哈希 → 确定 Block X 归属的节点
4. 向归属节点发送请求 → 命中则从对端拉取
5. 归属节点也未命中 → 从 Object Storage 下载 → 存入本地缓存

graph TD
    S3["S3 Client<br/>(mc/s3cmd)"]
    Boto3["Boto3 Client<br/>(Python)"]
    AWS["AWS CLI<br/>(aws s3api)"]
    S3 --> GW
    Boto3 --> GW
    AWS --> GW
    subgraph GW["JuiceFS Gateway"]
        MG["MinIO Gateway 兼容层<br/>S3 API to JFS API"]
        subgraph JFS["JuiceFS Client (内部)"]
            MC["Metadata Client"]
            DC["Data Client"]
        end
        MG --> JFS
    end
    JFS --> ME["Meta Engine"]
    JFS --> OS["Object Storage"]

juicefs gateway redis://host:6379/0 0.0.0.0:9000 \
  --access-key=admin \
  --secret-key=admin123

apiVersion: apps/v1
kind: Deployment
metadata:
  name: juicefs-gateway
spec:
  replicas: 2
  selector:
    matchLabels:
      app: juicefs-gateway
  template:
    metadata:
      labels:
        app: juicefs-gateway
    spec:
      containers:
      - name: gateway
        image: juicedata/juicefs-csi-driver:latest
        command:
        - juicefs
        - gateway
        - redis://redis:6379/0
        - 0.0.0.0:9000
        env:
        - name: MINIO_ROOT_USER
          value: admin
        - name: MINIO_ROOT_PASSWORD
          value: admin123
        ports:
        - containerPort: 9000
apiVersion: v1
kind: Service
metadata:
  name: juicefs-s3
spec:
  selector:
    app: juicefs-gateway
  ports:
  - port: 9000
    targetPort: 9000

# AWS CLI
aws configure --profile juicefs set aws_access_key_id admin
aws configure --profile juicefs set aws_secret_access_key admin123
aws --endpoint-url http://juicefs-s3:9000 --profile juicefs s3 ls

# MinIO Client
mc alias set juicefs http://juicefs-s3:9000 admin admin123
mc ls juicefs/

# Python Boto3
import boto3
s3 = boto3.client('s3',
  endpoint_url='http://juicefs-s3:9000',
  aws_access_key_id='admin',
  aws_secret_access_key='admin123'
)
s3.list_buckets()

graph TD
    subgraph Apps["Spark / Flink / Hive / Presto / MapReduce"]
    end
    Apps --> HDFS["Hadoop FileSystem API"]
    HDFS --> JFS
    subgraph JFS["JuiceFS Hadoop SDK (jfs-hadoop)"]
        JI["JuiceFileSystemImpl<br/>- Metadata Client<br/>- Data Client<br/>- Cache Manager"]
    end
    JFS --> ME["Meta Engine"]
    JFS --> OS["Object Storage"]

<!-- core-site.xml -->
<configuration>
  <property>
    <name>fs.jfs.impl</name>
    <value>io.juicefs.JuiceFileSystem</value>
  </property>
  <property>
    <name>fs.defaultFS</name>
    <value>jfs://myjfs</value>
  </property>
  <property>
    <name>juicefs.meta</name>
    <value>redis://10.0.0.1:6379/0</value>
  </property>
</configuration>

// 启动 Spark 时指定 JuiceFS SDK
spark-shell \
  --packages io.juicefs:juicefs-hadoop:1.1.2 \
  --conf spark.hadoop.fs.jfs.impl=io.juicefs.JuiceFileSystem \
  --conf spark.hadoop.fs.defaultFS=jfs://myjfs \
  --conf spark.hadoop.juicefs.meta=redis://10.0.0.1:6379/0

// 读取 Parquet 数据
val df = spark.read.parquet("jfs://myjfs/data/warehouse/table")
df.groupBy("key").count().write.parquet("jfs://myjfs/data/output/result")

# Flink SQL 中使用 JuiceFS 作为 Checkpoint 存储
state.checkpoints.dir: jfs://myjfs/flink/checkpoints
state.savepoints.dir: jfs://myjfs/flink/savepoints

juicefs mount --metrics=0.0.0.0:9567 redis://host:6379/0 /mnt/jfs

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: juicefs-metrics
spec:
  selector:
    matchLabels:
      app: juicefs-mount
  endpoints:
  - port: metrics
    interval: 30s
    path: /metrics

groups:
- name: juicefs
  rules:
  - alert: JuiceFSHighMetaLatency
    expr: histogram_quantile(0.99, rate(juicefs_meta_ops_durations_histogram_seconds_bucket[5m])) > 0.1
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "JuiceFS 元数据操作 P99 延迟 > 100ms"

  - alert: JuiceFSLowCacheHitRate
    expr: rate(juicefs_blockcache_hit_bytes_total[5m]) / (rate(juicefs_blockcache_hit_bytes_total[5m]) + rate(juicefs_blockcache_miss_bytes_total[5m])) < 0.8
    for: 10m
    labels:
      severity: warning
    annotations:
      summary: "JuiceFS 缓存命中率 < 80%"

# 创建文件系统时指定所有者
juicefs format \
  --storage s3 \
  --bucket https://s3.amazonaws.com/mybucket \
  --inode-limit 10000000 \
  redis://host:6379/0 myjfs

# 挂载后使用标准 Linux 权限管理
chown 1000:1000 /mnt/jfs/data
chmod 755 /mnt/jfs/data

# 挂载时启用 UID/GID 映射
juicefs mount \
  --uid-map=1000:2000 \
  --gid-map=1000:2000 \
  redis://host:6379/0 /mnt/jfs

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  securityContext:
    fsGroup: 2000        # Mount Pod 挂载点的组权限
    runAsUser: 1000      # 应用容器运行用户
    runAsGroup: 3000
  containers:
  - name: main
    image: myapp:latest
    volumeMounts:
    - name: data
      mountPath: /data
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: juicefs-pvc

# 限制 Secret 访问（RBAC）
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: juicefs-user
  namespace: team-a
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["juicefs-secret"]
  verbs: ["get"]
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: juicefs-user-binding
  namespace: team-a
subjects:
- kind: ServiceAccount
  name: team-a-sa
  namespace: team-a
roleRef:
  kind: Role
  name: juicefs-user
  apiGroup: rbac.authorization.k8s.io

graph TD
    subgraph JuiceFS["JuiceFS: 计算存储分离"]
        P1["Pod"] --> F1["FUSE"]
        P2["Pod"] --> F2["FUSE"]
        F1 --> S3["S3<br/>数据在外部对象存储"]
        F2 --> S3
    end

    subgraph Longhorn["Longhorn: 计算存储耦合"]
        P3["Pod<br/>Volume"] --> I1["iSCSI"]
        P4["Pod<br/>Volume"] --> I2["iSCSI"]
        I1 --> R1["Replica<br/>(Disk)<br/>数据在节点本地"]
        I2 --> R2["Replica<br/>(Disk)<br/>数据在节点本地"]
    end

# 在源云挂载 JuiceFS
juicefs mount redis://source:6379/0 /mnt/source

# 在目标云挂载 JuiceFS
juicefs mount redis://target:6379/0 /mnt/target

# 同步数据（并行、断点续传、增量同步）
juicefs sync /mnt/source/data/ /mnt/target/data/ \
  --threads=20 \
  --update \
  --delete

# 使用 rclone 在两个对象存储桶之间同步
rclone sync source-bucket:myjfs/chunks/ target-bucket:myjfs/chunks/ \
  --transfers=32 \
  --checksum

# 然后迁移元数据
juicefs dump redis://source:6379/0 > metadata.json
juicefs load redis://target:6379/0 < metadata.json

# 将源文件系统配置为只读，挂载后同步
juicefs mount --read-only redis://source:6379/0 /mnt/source
juicefs mount redis://target:6379/0 /mnt/target
juicefs sync /mnt/source/ /mnt/target/ --update --delete

graph LR
    subgraph SRC["源云 (AWS)"]
        MetaSrc["Redis / TiKV<br/>(Meta)"]
        DataSrc["AWS S3<br/>(Data)"]
    end
    subgraph TGT["目标云 (Aliyun)"]
        MetaTgt["Redis / TiKV<br/>(Meta)"]
        DataTgt["Aliyun OSS<br/>(Data)"]
    end
    MetaSrc -->|"dump/load"| MetaTgt
    DataSrc -->|"juicefs sync / rclone"| DataTgt
    MetaSrc --> DataSrc
    MetaTgt --> DataTgt

1. 目标云新建 JuiceFS 文件系统
2. 增量同步（juicefs sync --update）：源 → 目标
3. 停写源端 → 最终全量同步（juicefs sync --delete）
4. 切换应用 DNS/配置指向目标云 CSI Secret
5. 验证数据完整性
6. 退役源端文件系统

juicefs mount \
  --buffer-size=4000 \           # 增大写缓冲（高吞吐写入）
  --prefetch=1 \                 # 启用预取
  --cache-size=500000 \          # 大容量 SSD 缓存
  --cache-dir=/nvme/jfscache \   # NVMe SSD 缓存
  --max-uploads=100 \            # 提高并发上传
  --io-retries=30 \              # 增加重试次数
  --attr-cache=3600 \            # 长缓存时间（读多写少）
  --entry-cache=3600 \           # 长缓存时间
  --metacache=120 \              # 元数据缓存
  --writeback \                  # 写回模式（慎用，需评估风险）
  redis://host:6379/0 /mnt/jfs

# 系统级网络参数
sysctl -w net.core.rmem_max=134217728
sysctl -w net.core.wmem_max=134217728
sysctl -w net.ipv4.tcp_rmem="4096 87380 134217728"
sysctl -w net.ipv4.tcp_wmem="4096 65536 134217728"

# 挂载时启用详细日志
juicefs mount --verbose redis://host:6379/0 /mnt/jfs
# 日志输出到 syslog 或 STDERR

# 调整日志级别
juicefs mount --log-level=trace redis://host:6379/0 /mnt/jfs
# 支持: trace, debug, info, warn, error, fatal

juicefs stats /mnt/jfs

# 输出示例：
# uptime: 7h53m5.5s
# cpu: 12.3%, memory: 2.1G
# used buffer: 128.0M / 500.0M
# FUSE ops: read=12,345 write=5,678 ...
# meta ops: lookup=50,123 getattr=23,456 ...
# blockcache read: hit=98,765 miss=2,345 (97.7%)
# object storage: get=15,678 put=3,456 ...

# 启用 pprof 性能分析
juicefs mount --pprof=0.0.0.0:6060 redis://host:6379/0 /mnt/jfs

# 分析 CPU 热点
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# 分析内存分配
go tool pprof http://localhost:6060/debug/pprof/heap

# 批量诊断
juicefs doctor redis://host:6379/0
# 检查内容：Meta Engine 连接、对象存储可达性、
# 缓存目录读写权限、系统内核版本等

# CSI Controller 日志
kubectl logs -n kube-system -l app=juicefs-csi-controller --tail=200

# CSI Node Plugin 日志
kubectl logs -n kube-system -l app=juicefs-csi-node --tail=200

# Mount Pod 日志
kubectl logs <mount-pod-name> -c jfs-mount --tail=200

# 进入 Mount Pod 调试
kubectl exec -it <mount-pod-name> -c jfs-mount -- juicefs status /jfs/mount
kubectl exec -it <mount-pod-name> -c jfs-mount -- juicefs stats /jfs/mount

# 创建文件系统时设置容量显示值
juicefs format --capacity=102400 redis://host:6379/0 myjfs  # 100TB

# 查看文件系统实际使用量
juicefs info redis://host:6379/0
# 输出: used: 15.3 TiB, inodes: 2,345,678

# StorageClass 开启扩容
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: juicefs-sc
provisioner: csi.juicefs.com
allowVolumeExpansion: true  # 启用扩容
# 在线扩容 PVC（JuiceFS 仅更新逻辑容量标识）
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: juicefs-pvc
spec:
  resources:
    requests:
      storage: 200Gi  # 从 100Gi 扩展到 200Gi

# 缓存自动清理策略
juicefs mount \
  --cache-size=500000 \        # 缓存最大 500GB
  --free-space-ratio=0.1 \     # 磁盘剩余空间 < 10% 时自动清理
  redis://host:6379/0 /mnt/jfs

# HPA 无法直接扩缩容 JuiceFS，通过 KEDA 等监控驱动响应
# 示例: 监控 Redis 内存使用率，触发告警

- alert: RedisMemoryHigh
  expr: redis_memory_used_bytes / redis_memory_max_bytes > 0.8
  for: 10m
  labels:
    severity: critical
  annotations:
    summary: "Redis 内存使用率 > 80%，需扩容 Meta Engine"

apiVersion: v1
kind: Secret
metadata:
  name: juicefs-production
stringData:
  name: prod-jfs
  metaurl: redis://redis-sentinel:26379/0?route-read=replica&route-write=master
  storage: s3
  bucket: https://s3.ap-northeast-1.amazonaws.com/prod-bucket
  access-key: xxx
  secret-key: xxx
  # 缓存配置
  cache-dir: /var/jfs-cache
  cache-size: "512000"          # 500GB
  free-space-ratio: "0.15"
  # 性能配置
  buffer-size: "2000"           # 2GB 写缓冲
  prefetch: "1"
  max-uploads: "50"
  max-deletes: "20"
  # 元数据缓存（根据场景调整）
  metacache: "60"
  attr-cache: "60"
  entry-cache: "60"
  # 安全
  encrypt-rsa-key: ""           # 按需启用客户端加密

# Mount Pod 默认自动分配资源，可通过 values.yaml 调整
# Helm 安装 CSI Driver 时配置
helm install juicefs-csi juicefs/juicefs-csi-driver \
  --set node.resources.requests.cpu=100m \
  --set node.resources.requests.memory=256Mi \
  --set node.resources.limits.cpu=2000m \
  --set node.resources.limits.memory=4Gi \
  --set controller.resources.requests.cpu=100m \
  --set controller.resources.requests.memory=256Mi

# Redis Sentinel 示例
# metaurl: redis://:password@sentinel-1:26379,sentinel-2:26379,sentinel-3:26379/0?master=myMaster

# TiKV 集群
# metaurl: tikv://pd-1:2379,pd-2:2379,pd-3:2379/jfs-prod

# MySQL 主从
# metaurl: mysql://user:pass@(master:3306,slave:3306)/juicefs?readTimeout=30s

apiVersion: batch/v1
kind: CronJob
metadata:
  name: juicefs-backup
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: juicedata/juicefs-csi-driver:latest
            command:
            - /bin/sh
            - -c
            - |
              juicefs dump $METAURL > /backup/metadata-$(date +%Y%m%d%H%M).json
              aws s3 cp /backup/metadata-*.json s3://backup-bucket/juicefs/              
            env:
            - name: METAURL
              valueFrom:
                secretKeyRef:
                  name: juicefs-production
                  key: metaurl
          restartPolicy: OnFailure

# 使用 ServiceMonitor 自动发现
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: juicefs-mounts
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: juicefs-mount
  endpoints:
  - port: metrics
    interval: 15s

# 关键告警规则
# - Meta Engine 连接失败
# - 缓存命中率 < 80%
# - FUSE 操作 P99 > 500ms
# - Mount Pod 频繁重启

1. 阅读 Release Notes，确认 Breaking Changes
2. 在测试环境验证新版本
3. 滚动更新 CSI Controller（先升级 Controller，再升级 Node Plugin）
   kubectl apply -f deploy/k8s.yaml
4. 观察 Mount Pod 是否自动重建（配置变更时）
5. 验证已有 PVC 读写正常
6. 生产验证通过后再升级下一批集群