구글 Cloud 스토리지 N일 지난 파일 삭제

1 개요[ | ]

GCS N일 지난 파일 삭제
구글 Cloud 스토리지 N일 지난 파일 삭제

7일 지난 파일목록 조회

gsutil ls -l gs://버킷명/경로 | awk -v t=$(date --date=-7day +%Y-%m-%dT%H:%M:%SZ) '$2<t'

gsutil ls -l gs://버킷명/경로 | awk -v t=$(date -d @$(( $(date +%s) - 86400*3 )) +%Y-%m-%dT%H:%M:%SZ) '$2<t'

→ gsutil ls 파일목록 조회

7일 지난 파일 삭제

gsutil ls -l gs://버킷명/경로 | awk -v t=$(date --date=-7day +%Y-%m-%dT%H:%M:%SZ) '$2<t' | awk '{print $3}' | gsutil -m rm -I

→ gsutil rm 파일 삭제

→ -m 멀티스레드로 수행

→ -I 표준입력으로부터 파일목록 받아서 수행

2 실행예시[ | ]

전체 파일 조회

user01@localhost:~$ gsutil ls -l gs://mybackup/
  11630788  2020-03-10T14:03:57Z  gs://mybackup/db_20200310-140013.sql.gz
  11633260  2020-03-11T14:04:13Z  gs://mybackup/db_20200311-140011.sql.gz
  11638372  2020-03-12T14:03:52Z  gs://mybackup/db_20200312-140007.sql.gz
  11641792  2020-03-13T14:03:45Z  gs://mybackup/db_20200313-140011.sql.gz
  11645968  2020-03-14T14:03:39Z  gs://mybackup/db_20200314-140006.sql.gz
  11651099  2020-03-15T14:03:34Z  gs://mybackup/db_20200315-140007.sql.gz
  11655220  2020-03-16T14:05:30Z  gs://mybackup/db_20200316-140006.sql.gz
  11661400  2020-03-17T14:04:21Z  gs://mybackup/db_20200317-140003.sql.gz
  11668137  2020-03-18T14:05:16Z  gs://mybackup/db_20200318-140011.sql.gz
  11673629  2020-03-19T14:03:18Z  gs://mybackup/db_20200319-140007.sql.gz
  11678940  2020-03-20T14:04:19Z  gs://mybackup/db_20200320-140038.sql.gz
TOTAL: 11 objects, 128178605 bytes (122.24 MiB)

5일 지난 파일 조회

user01@localhost:~$ gsutil ls -l gs://mybackup/ | awk -v t=$(date --date=-5day +%Y-%m-%dT%H:%M:%SZ) '$2<t'
  11630788  2020-03-10T14:03:57Z  gs://mybackup/db_20200310-140013.sql.gz
  11633260  2020-03-11T14:04:13Z  gs://mybackup/db_20200311-140011.sql.gz
  11638372  2020-03-12T14:03:52Z  gs://mybackup/db_20200312-140007.sql.gz
  11641792  2020-03-13T14:03:45Z  gs://mybackup/db_20200313-140011.sql.gz
  11645968  2020-03-14T14:03:39Z  gs://mybackup/db_20200314-140006.sql.gz
  11651099  2020-03-15T14:03:34Z  gs://mybackup/db_20200315-140007.sql.gz
TOTAL: 11 objects, 128178605 bytes (122.24 MiB)

5일 지난 파일 삭제

user01@localhost:~$ gsutil ls -l gs://mybackup/ | awk -v t=$(date --date=-5day +%Y-%m-%dT%H:%M:%SZ) '$2<t' | awk '{print $3}' | gsutil -m rm -I
Removing gs://mybackup/db_20200310-140013.sql.gz...
Removing gs://mybackup/db_20200311-140011.sql.gz...
Removing gs://mybackup/db_20200312-140007.sql.gz...
Removing gs://mybackup/db_20200313-140011.sql.gz...
Removing gs://mybackup/db_20200314-140006.sql.gz...
Removing gs://mybackup/db_20200315-140007.sql.gz...
/ [6/6 objects] 100% Done
Operation completed over 6 objects.
CommandException: 1 files/objects could not be removed.

전체 파일 조회

user01@localhost:~$ gsutil ls -l gs://mybackup/
  11655220  2020-03-16T14:05:30Z  gs://mybackup/db_20200316-140006.sql.gz
  11661400  2020-03-17T14:04:21Z  gs://mybackup/db_20200317-140003.sql.gz
  11668137  2020-03-18T14:05:16Z  gs://mybackup/db_20200318-140011.sql.gz
  11673629  2020-03-19T14:03:18Z  gs://mybackup/db_20200319-140007.sql.gz
  11678940  2020-03-20T14:04:19Z  gs://mybackup/db_20200320-140038.sql.gz
TOTAL: 5 objects, 58337326 bytes (55.63 MiB)

3 같이 보기[ | ]

4 참고[ | ]

https://stackoverflow.com/questions/47695209/gsutil-command-to-delete-old-files-from-last-day