Skip to content
This repository has been archived by the owner on Feb 12, 2019. It is now read-only.

[Mac OS] Old block DBs not being cleaned up #1558

Open
kklash opened this issue Apr 27, 2018 · 3 comments
Open

[Mac OS] Old block DBs not being cleaned up #1558

kklash opened this issue Apr 27, 2018 · 3 comments

Comments

@kklash
Copy link

kklash commented Apr 27, 2018

diskCacheBlocks.leveldb admin$ date
Fri 27 Apr 2018 18:36:24
diskCacheBlocks.leveldb admin$ pwd
/Users/admin/Library/Application Support/Keybase/kbfs_block_cache/v1/diskCacheBlocks.leveldb
diskCacheBlocks.leveldb admin$ ls -l
total 10540072
drwxr-xr-x  117 admin  staff   3.9K 27 Apr 15:36 .
drwxr-xr-x    5 admin  staff   170B 30 Sep  2017 ..
-rw-r--r--    1 admin  staff    50M 20 Feb 23:11 009400.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:11 009401.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:20 009459.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:20 009460.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:39 009608.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:39 009609.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:39 009610.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:39 009611.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:47 009668.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:47 009669.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:53 009726.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:55 009751.ldb
-rw-r--r--    1 admin  staff    50M 20 Feb 23:55 009752.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:00 009804.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:15 009981.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:15 009982.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:15 009983.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:15 009984.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:15 009985.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:20 010043.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:20 010044.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:25 010098.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:25 010099.ldb
-rw-r--r--    1 admin  staff    50M 21 Feb 00:25 010100.ldb
-rw-r--r--    1 admin  staff    17M 21 Feb 00:25 010101.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:27 010303.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010451.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010452.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010453.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010454.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010455.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010456.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010457.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:40 010458.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 10:51 010510.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010752.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010753.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010754.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010755.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010756.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010757.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010758.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010759.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010760.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:16 010761.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010991.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010992.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010993.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010994.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010995.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010996.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010997.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010998.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 010999.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:35 011000.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:44 011104.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:44 011105.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:44 011106.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:44 011107.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:47 011153.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:47 011154.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:47 011155.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:51 011213.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:51 011214.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:52 011251.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 11:55 011299.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011421.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011422.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011423.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011424.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011425.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011426.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:00 011427.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:04 011473.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:04 011474.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:04 011475.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:08 011533.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:08 011534.ldb
-rw-r--r--    1 admin  staff    50M 24 Feb 12:08 011535.ldb
-rw-r--r--    1 admin  staff    44M 24 Feb 12:08 011536.ldb
-rw-r--r--    1 admin  staff    50M 11 Mar 14:21 011632.ldb
-rw-r--r--    1 admin  staff    50M 11 Mar 14:21 011633.ldb
-rw-r--r--    1 admin  staff    50M 11 Mar 14:21 011634.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011852.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011853.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011854.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011855.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011856.ldb
-rw-r--r--    1 admin  staff    10M 26 Apr 10:29 011857.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011858.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011859.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011860.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:29 011861.ldb
-rw-r--r--    1 admin  staff    12M 26 Apr 10:29 011862.ldb
-rw-r--r--    1 admin  staff   7.5M 26 Apr 10:29 011863.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011864.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011865.ldb
-rw-r--r--    1 admin  staff    35M 26 Apr 10:30 011866.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011867.ldb
-rw-r--r--    1 admin  staff    45M 26 Apr 10:30 011868.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011869.ldb
-rw-r--r--    1 admin  staff    19M 26 Apr 10:30 011870.ldb
-rw-r--r--    1 admin  staff    47M 26 Apr 10:30 011871.ldb
-rw-r--r--    1 admin  staff    11M 26 Apr 10:30 011872.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011873.ldb
-rw-r--r--    1 admin  staff    50M 26 Apr 10:30 011874.ldb
-rw-r--r--    1 admin  staff    43M 26 Apr 10:30 011875.ldb
-rw-r--r--    1 admin  staff     0B 27 Apr 09:30 011887.log
-rw-r--r--    1 admin  staff    32M 27 Apr 15:36 011889.ldb
-rw-r--r--    1 admin  staff    46K 27 Apr 15:36 011890.ldb
-rw-r--r--    1 admin  staff    16B 27 Apr 09:30 CURRENT
-rw-r--r--    1 admin  staff     0B 30 Sep  2017 LOCK
-rw-r--r--    1 admin  staff   368K 27 Apr 15:37 LOG
-rw-r--r--    1 admin  staff   1.1M 21 Feb 00:28 LOG.old
-rw-r--r--    1 admin  staff    10K 27 Apr 15:36 MANIFEST-011888

Old .ldb's not being cleaned up. I made a backup, removed the old ones, relaunched keybase and everything still running normally.

As a sidenote, why are these DBs so big when I only have 17mb of files in my KBFS? (currently, had more before)

If there's any extra debugging information I can provide, just let me know!

@strib
Copy link
Contributor

strib commented Apr 27, 2018

@kklash: as explained here under the "Local disk usage policy" section, we keep up to 20 GB of blocks, depending on the free space of your drive. If you use more of the drive, we will shrink that cache automatically if KBFS is running.

We cache all blocks you've even seen, including from team and public folders, and past blocks from old versions of the files in some cases.

@adi-dev
Copy link

adi-dev commented Jan 24, 2019

I think we should re-consider the block storage. I have Kebyase on my laptop, I uploaded some files and deleted them. Now I have only like 400MB file but kbfs_block_cache eats almost 6GB on my hard drive. I'll try @strib solution to clean is up manually but sure it should remove blocks from the files what don't exists anymore. Btw. it's Windows 10 machine.

@strib
Copy link
Contributor

strib commented Jan 24, 2019

@adi-dev recently we did add something that cleans up the blocks from old files that have been deleted, and old versions of files. But I think it only works going forward I think, not for old blocks from before that version was released. So it might be the case that a single manual cleanup will fix things for you.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants