Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clean MASVS repo #555

Open
sushi2k opened this issue Jul 22, 2021 · 0 comments
Open

Clean MASVS repo #555

sushi2k opened this issue Jul 22, 2021 · 0 comments

Comments

@sushi2k
Copy link
Collaborator

sushi2k commented Jul 22, 2021 •
edited

The current repo contains a lot of blobs that are not needed and we should remove them, to reduce unnecessary bandwidth /storage usage when checking out the repo.

This seems to be the way to go to remove the majority of the large files https://rtyley.github.io/bfg-repo-cleaner/

https://stackoverflow.com/a/17890278

$ git rev-list --objects --all \
| git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
| sed -n 's/^blob //p' \
| sort --numeric-sort --key=2 \
| cut -c 1-12,41- \
| $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest | grep "MiB"
bed78e1f0030  1.0MiB cover.jpg
13b742ea15b1  1.0MiB tools/cover.xcf
5009b0037ff9  1.0MiB tools/cover.xcf
c2da15abf619  1.0MiB cover.jpg
a0b71aff147a  1.5MiB asvs.md
42e27494cd0f  2.1MiB Document-ko/masvs-korean.pdf
aaa0defcf959  3.2MiB cover.jpg
431d9a53bcfa  3.2MiB Document/cover_full_page.jpg
501891a42b7f  3.4MiB cover.jpg
02ada71cfa1a  8.5MiB generated/MASVS-zhtw.docx
084324f0a05e  8.5MiB generated/MASVS.docx
866e9b417fc9  8.6MiB generated/MASVS-fr.docx
e02abfee96ea  8.6MiB generated/MASVS-ru.docx
c49ceb1580fc  8.6MiB generated/MASVS-ja.docx
0c6cfb8d6ecd  8.6MiB generated/MASVS-es.docx
f252841672eb  8.8MiB tools/reference.docx
5d8e84016b62   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
2a102144f42e   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
0c8a5213c9a4   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
0a69782875f6   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
2132f50b4a03   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
0df043410507   10MiB generated/MASVS_Document-ja.pdf
2c73e7ed6bce   10MiB generated/MASVS_Document-ja.pdf
933566e64729   10MiB generated/MASVS_Document-ja.pdf
df3f82fa82db   10MiB generated/MASVS_Document-ja.pdf
3a93a300d5af   10MiB generated/MASVS_Document-ja.pdf
588a0e8fa910   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
67cd9040e291   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
9ce318feb7b6   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
d15397a6d802   10MiB generated/MASVS_Document-ja.pdf
1e757d203f6f   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
6c6d8babb7c1   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
853b94d6ee73   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
9c69a4082908   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
c9e15b4a8ac9   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
cfd0ca49a655   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
eebca793154a   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
ff3f0be72479   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.1_Document-ja.pdf
7c211e7bc781   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
950b3639c196   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
398df781f701   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
746684363818   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
77983bebd1e5   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
18422a443c53   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
7ea720842884   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-zhtw.pdf
d9a95b29795f   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
956cc5620ad0   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
0af63352cc5e   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
271cf62c7f05   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
c23e8188d270   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
66cfde36ea1e   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.3_Document-zhtw.pdf
5345ce6f7b8f   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.3_Document-ja.pdf
f3f5904a3453   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.2_Document-ja.pdf
e9c45397da16   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.3_Document-zhtw.pdf
14c6b5de9479   10MiB generated/OWASP_Mobile_AppSec_Verification_Standard_1.1.3_Document-ja.pdf

The repo size is at the moment over 120MB:

$ du -hc
228K	./tools/docker
2.0M	./tools
560K	./Document-fa/images
728K	./Document-fa
1.5M	./Document-fr/images
1.6M	./Document-fr
816K	./Document-es/images
944K	./Document-es
1.4M	./Document-de/images
1.5M	./Document-de
560K	./Document-hi/images
776K	./Document-hi
696K	./Document-ptpt/images
820K	./Document-ptpt
964K	./Document-ko/images
1.1M	./Document-ko
588K	./Document-ptbr/images
720K	./Document-ptbr
800K	./Document/images
936K	./Document
976K	./Document-ru/images
1.1M	./Document-ru
4.0K	./.github/workflows/config
 24K	./.github/workflows
 32K	./.github
1.5M	./Document-ja/images
1.6M	./Document-ja
1.4M	./Document-zhtw/images
1.5M	./Document-zhtw
8.0K	./.git/.gitstatus.HOddY1
 20K	./.git/objects/61
 12K	./.git/objects/0d
 16K	./.git/objects/95
 16K	./.git/objects/59
 24K	./.git/objects/92
 16K	./.git/objects/0c
 24K	./.git/objects/66
 20K	./.git/objects/3e
3.7M	./.git/objects/50
 20K	./.git/objects/68
392K	./.git/objects/57
 32K	./.git/objects/3b
 20K	./.git/objects/6f
 12K	./.git/objects/03
136K	./.git/objects/9b
 36K	./.git/objects/9e
 20K	./.git/objects/04
 12K	./.git/objects/6a
 36K	./.git/objects/32
 32K	./.git/objects/35
 16K	./.git/objects/69
 40K	./.git/objects/3c
 16K	./.git/objects/51
 28K	./.git/objects/3d
4.0K	./.git/objects/58
 16K	./.git/objects/67
 56K	./.git/objects/0b
 24K	./.git/objects/93
 40K	./.git/objects/94
 40K	./.git/objects/0e
 20K	./.git/objects/60
8.0K	./.git/objects/34
 32K	./.git/objects/5a
 16K	./.git/objects/5f
 16K	./.git/objects/33
4.0K	./.git/objects/05
 32K	./.git/objects/9d
 36K	./.git/objects/9c
 20K	./.git/objects/02
8.0K	./.git/objects/a4
 24K	./.git/objects/a3
 28K	./.git/objects/b5
 12K	./.git/objects/b2
 32K	./.git/objects/d9
 16K	./.git/objects/ac
 24K	./.git/objects/ad
 32K	./.git/objects/bb
 20K	./.git/objects/d7
 12K	./.git/objects/d0
960K	./.git/objects/be
 20K	./.git/objects/b3
 16K	./.git/objects/df
 24K	./.git/objects/da
 20K	./.git/objects/b4
 36K	./.git/objects/a2
 20K	./.git/objects/a5
 32K	./.git/objects/bd
 48K	./.git/objects/d1
 28K	./.git/objects/d6
 12K	./.git/objects/bc
 20K	./.git/objects/ae
 48K	./.git/objects/d8
 24K	./.git/objects/ab
 24K	./.git/objects/e5
 32K	./.git/objects/e2
 20K	./.git/objects/f4
 12K	./.git/objects/f3
4.0K	./.git/objects/eb
 32K	./.git/objects/c7
 48K	./.git/objects/c0
 28K	./.git/objects/ee
 12K	./.git/objects/c9
 44K	./.git/objects/fc
 40K	./.git/objects/fd
 36K	./.git/objects/f2
 12K	./.git/objects/f5
 24K	./.git/objects/e3
 20K	./.git/objects/cf
 16K	./.git/objects/ca
4.0K	./.git/objects/e4
 40K	./.git/objects/fe
 16K	./.git/objects/c8
 12K	./.git/objects/fb
 32K	./.git/objects/ed
 28K	./.git/objects/c1
 16K	./.git/objects/c6
 12K	./.git/objects/ec
 52K	./.git/objects/4e
 48K	./.git/objects/20
 28K	./.git/objects/18
4.0K	./.git/objects/27
 32K	./.git/objects/4b
 96M	./.git/objects/pack
 20K	./.git/objects/11
 20K	./.git/objects/7d
 20K	./.git/objects/29
 12K	./.git/objects/7c
 52K	./.git/objects/16
 32K	./.git/objects/42
 20K	./.git/objects/89
 16K	./.git/objects/45
 32K	./.git/objects/1f
 20K	./.git/objects/73
 16K	./.git/objects/87
 24K	./.git/objects/80
 12K	./.git/objects/74
 12K	./.git/objects/1a
 24K	./.git/objects/28
 28K	./.git/objects/17
 28K	./.git/objects/7b
 36K	./.git/objects/8f
 24K	./.git/objects/8a
 16K	./.git/objects/7e
 16K	./.git/objects/10
 20K	./.git/objects/19
 20K	./.git/objects/4c
 36K	./.git/objects/26
 20K	./.git/objects/21
 28K	./.git/objects/4d
 28K	./.git/objects/75
 36K	./.git/objects/81
 12K	./.git/objects/86
 16K	./.git/objects/72
 40K	./.git/objects/44
 12K	./.git/objects/2a
 24K	./.git/objects/2f
8.0K	./.git/objects/43
 16K	./.git/objects/88
8.0K	./.git/objects/9f
 36K	./.git/objects/6b
 20K	./.git/objects/07
 28K	./.git/objects/38
 16K	./.git/objects/00
 16K	./.git/objects/6e
4.0K	./.git/objects/9a
 20K	./.git/objects/36
 28K	./.git/objects/5c
 28K	./.git/objects/09
 16K	./.git/objects/5d
 16K	./.git/objects/31
  0B	./.git/objects/info
 24K	./.git/objects/91
 24K	./.git/objects/65
 32K	./.git/objects/62
 32K	./.git/objects/96
 16K	./.git/objects/3a
 12K	./.git/objects/54
 20K	./.git/objects/98
 20K	./.git/objects/53
 44K	./.git/objects/3f
 20K	./.git/objects/30
968K	./.git/objects/5e
 20K	./.git/objects/5b
4.0K	./.git/objects/37
 32K	./.git/objects/08
 24K	./.git/objects/6d
 28K	./.git/objects/01
 40K	./.git/objects/06
8.0K	./.git/objects/6c
 24K	./.git/objects/39
 32K	./.git/objects/99
8.0K	./.git/objects/52
 32K	./.git/objects/55
 12K	./.git/objects/97
 12K	./.git/objects/63
 20K	./.git/objects/0f
 24K	./.git/objects/0a
 12K	./.git/objects/64
 44K	./.git/objects/90
 36K	./.git/objects/bf
 12K	./.git/objects/d3
 28K	./.git/objects/d4
 20K	./.git/objects/ba
 20K	./.git/objects/a0
 20K	./.git/objects/a7
 20K	./.git/objects/b8
 36K	./.git/objects/b1
 16K	./.git/objects/dd
 12K	./.git/objects/dc
 16K	./.git/objects/b6
 16K	./.git/objects/a9
 20K	./.git/objects/d5
 12K	./.git/objects/d2
 16K	./.git/objects/aa
120K	./.git/objects/af
 20K	./.git/objects/b7
8.0K	./.git/objects/db
 28K	./.git/objects/a8
 12K	./.git/objects/de
 12K	./.git/objects/b0
 20K	./.git/objects/a6
 12K	./.git/objects/b9
 12K	./.git/objects/a1
 20K	./.git/objects/ef
 12K	./.git/objects/c3
 40K	./.git/objects/c4
 24K	./.git/objects/ea
 20K	./.git/objects/e1
 24K	./.git/objects/cd
 20K	./.git/objects/cc
 28K	./.git/objects/e6
320K	./.git/objects/f9
 20K	./.git/objects/f0
 24K	./.git/objects/f7
 40K	./.git/objects/e8
 16K	./.git/objects/fa
 32K	./.git/objects/ff
8.0K	./.git/objects/c5
992K	./.git/objects/c2
 16K	./.git/objects/f6
 32K	./.git/objects/e9
 32K	./.git/objects/f1
8.0K	./.git/objects/e7
8.0K	./.git/objects/cb
 12K	./.git/objects/f8
 32K	./.git/objects/ce
 32K	./.git/objects/e0
 24K	./.git/objects/46
 28K	./.git/objects/2c
 20K	./.git/objects/79
8.0K	./.git/objects/2d
 24K	./.git/objects/41
 12K	./.git/objects/83
8.0K	./.git/objects/1b
 16K	./.git/objects/77
 40K	./.git/objects/48
 24K	./.git/objects/70
 24K	./.git/objects/1e
 40K	./.git/objects/84
 24K	./.git/objects/4a
 16K	./.git/objects/24
 28K	./.git/objects/23
 20K	./.git/objects/4f
 24K	./.git/objects/8d
8.0K	./.git/objects/15
 52K	./.git/objects/12
 24K	./.git/objects/8c
 24K	./.git/objects/85
 28K	./.git/objects/1d
 20K	./.git/objects/71
 44K	./.git/objects/76
 16K	./.git/objects/1c
 28K	./.git/objects/82
 24K	./.git/objects/49
 36K	./.git/objects/40
 24K	./.git/objects/2e
 16K	./.git/objects/2b
 36K	./.git/objects/47
 36K	./.git/objects/78
 16K	./.git/objects/8b
 12K	./.git/objects/13
 12K	./.git/objects/7f
 44K	./.git/objects/7a
 20K	./.git/objects/14
 16K	./.git/objects/8e
 28K	./.git/objects/22
 12K	./.git/objects/25
109M	./.git/objects
4.0K	./.git/info
 88K	./.git/logs/refs/heads
140K	./.git/logs/refs/remotes/origin
140K	./.git/logs/refs/remotes
232K	./.git/logs/refs
264K	./.git/logs
 48K	./.git/hooks
 80K	./.git/refs/heads
 20K	./.git/refs/tags
140K	./.git/refs/remotes/origin
140K	./.git/refs/remotes
252K	./.git/refs
8.0K	./.git/.gitstatus.inYqNU
110M	./.git
1.4M	./Document-zhcn/images
1.5M	./Document-zhcn
128M	.
128M	total
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sushi2k