Migrate a PVC to EBS CSI from In-tree EBS
This is how you convert a legacy In-tree EBS volume to EBS CSI.
I am going to assume you already applied this method to your EKS cluster. But now you want to schedule backups for an old non-CSI PVC associated with an old pod.
These are the high level steps.
- Find the volume ID from the legacy PVC
- Take a snapshot of the legacy volume with AWS CLI
- Wait for the snapshot to complete
- Create a VolumeSnapshotContent object with the snapshot ID
- Create a VolumeSnapshot object referencing the VolumeSnapshotContent object
- Create a PersistentVolumeClaim (PVC) that references the VolumeSnapshot object
- Pass the new PVC to your Pod/Deployment/Release, etc.
In the following examples I’ll use what I did for my legacy Grafana helm release. If you want another perspective on the process, this is the guide I followed.
Find the legacy volume ID
First get the legacy In-tree EBS PVC ID
➜ kubectl get pvc -n grafana
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
grafana Bound pvc-fda7b565-01a6-43aa-9122-2f47444de173 256Gi RWO gp2 4d19h
➜ Then use the PVC ID to get the volume ID from the PV
➜ kubectl get pv pvc-fda7b565-01a6-43aa-9122-2f47444de173 -n grafana -o jsonpath='{.spec.awsElasticBlockStore.volumeID}'
aws://us-east-1b/vol-0e3874465e9377743
➜ Take a snapshot of the legacy volume
Use the volume ID from the step before and tag the snapshot for EBS CSI.
➜ aws ec2 create-snapshot --volume-id vol-0e3874465e9377743 --tag-specifications 'ResourceType=snapshot,Tags=[{Key="ec2:ResourceTag/ebs.csi.aws.com/cluster",Value="true"}]'
...
{
"Description": "",
"Encrypted": false,
"OwnerId": "redacted",
"Progress": "",
"SnapshotId": "snap-052b62c9a1495046c",
"StartTime": "2023-04-05T17:05:49.763000+00:00",
"State": "pending",
"VolumeId": "vol-0e3874465e9377743",
"VolumeSize": 256,
"Tags": [
{
"Key": "ec2:ResourceTag/ebs.csi.aws.com/cluster",
"Value": "true"
}
]
}Now check the snapshot State until it changes from pending to completed.
➜ aws ec2 describe-snapshots --snapshot-ids snap-052b62c9a1495046c
...
{
"Snapshots": [
{
"Description": "",
"Encrypted": false,
"OwnerId": "redacted",
"Progress": "100%",
"SnapshotId": "snap-052b62c9a1495046c",
"StartTime": "2023-04-05T17:05:49.763000+00:00",
"State": "completed",
"VolumeId": "vol-0e3874465e9377743",
"VolumeSize": 256,
"Tags": [
{
"Key": "ec2:ResourceTag/ebs.csi.aws.com/cluster",
"Value": "true"
}
],
"StorageTier": "standard"
}
]
}Create a VolumeSnapshotContent object with the snapshot ID
Since this is a 1-time operation just apply this and other manifests— no need for Flux. Note that in addition to specifying the snapshot ID from the step before I also chose my own namespace.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
name: imported-grafana-content
namespace: grafana
spec:
volumeSnapshotRef:
kind: VolumeSnapshot
name: imported-grafana-snapshot
namespace: grafana
source:
snapshotHandle: snap-052b62c9a1495046c # <-- snapshot to import
driver: ebs.csi.aws.com
deletionPolicy: Delete
volumeSnapshotClassName: ebs-csi-aws➜ kubectl apply -f volume_snapshot_content.yaml
volumesnapshotcontent.snapshot.storage.k8s.io/imported-grafana-content created
➜ kubectl get volumesnapshotcontent imported-grafana-content -n grafana
NAME READYTOUSE RESTORESIZE DELETIONPOLICY DRIVER VOLUMESNAPSHOTCLASS VOLUMESNAPSHOT VOLUMESNAPSHOTNAMESPACE AGE
imported-grafana-content true 274877906944 Delete ebs.csi.aws.com ebs-csi-aws imported-grafana-snapshot grafana 38s
➜Note that this references a VolumeSnapshot which doesn’t exist yet. Thats okay. We are about to create it.
Create a VolumeSnapshot
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: imported-grafana-snapshot
namespace: grafana
spec:
volumeSnapshotClassName: ebs-csi-aws
source:
volumeSnapshotContentName: imported-grafana-content➜ kubectl apply -f volume_snapshot.yaml
volumesnapshot.snapshot.storage.k8s.io/imported-grafana-snapshot created
➜ kubectl get volumesnapshot imported-grafana-snapshot -n grafana
NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE
imported-grafana-snapshot true imported-grafana-content 256Gi ebs-csi-aws imported-grafana-content 10m 16s
➜Create a new PersistentVolumeClaim from the VolumeSnapshot
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: imported-grafana-pvc
namespace: grafana
spec:
accessModes:
- ReadWriteOnce
storageClassName: gp3
resources:
requests:
storage: 256Gi
dataSource:
name: imported-grafana-snapshot
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io➜ kubectl apply -f pvc.yaml
persistentvolumeclaim/imported-grafana-pvc created
➜ kubectl get pvc -n grafana
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
grafana Bound pvc-fda7b565-01a6-43aa-9122-2f47444de173 256Gi RWO gp2 4d19h
imported-grafana-pvc Pending gp3 5s
➜ Note that the legacy PVC is still bound and the new PVC will stay pending until we bind it instead.
Use the new PVC
Now in my example I am passing it to a Grafana helm release which will add the PVC to a deployment. But you could do the exact same thing with your own Deployment or Pod manifests.
This is the diff for my helm release in Flux. I just specified the name of my new PVC, imported-grafana-vpc, as an existingClaim. This is a field in the values for this particular helm chart which eventually gets interpolated into the Deployment template as the new PVC.
enabled: true
type: pvc
size: 256Gi
+ existingClaim: imported-grafana-pvc
serviceAccount:
name: grafana
annotations:After the release the old PVC disappears and the new PVC shows as bound.
➜ kubectl get pvc -n grafana
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
imported-grafana-pvc Bound pvc-1ac1c6a5-768b-4035-ab94-0e472a80ab1b 256Gi RWO gp3 31m
➜Now make a point of enabling EBS CSI as the default in all of your new clusters. This manual migration process is annoying and you won’t want to do it often.
