/
dirtop_example.txt
108 lines (82 loc) · 4.98 KB
/
dirtop_example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
Demonstrations of dirtop, the Linux eBPF/bcc version.
dirtop shows reads and writes by directory. For example:
# ./dirtop.py -d '/hdfs/uuid/*/yarn'
Tracing... Output every 1 secs. Hit Ctrl-C to end
14:28:12 loadavg: 25.00 22.85 21.22 31/2921 66450
READS WRITES R_Kb W_Kb PATH
1030 2852 8 147341 /hdfs/uuid/c11da291-28de-4a77-873e-44bb452d238b/yarn
3308 2459 10980 24893 /hdfs/uuid/bf829d08-1455-45b8-81fa-05c3303e8c45/yarn
2227 7165 6484 11157 /hdfs/uuid/76dc0b77-e2fd-4476-818f-2b5c3c452396/yarn
1985 9576 6431 6616 /hdfs/uuid/99c178d5-a209-4af2-8467-7382c7f03c1b/yarn
1986 398 6474 6486 /hdfs/uuid/7d512fe7-b20d-464c-a75a-dbf8b687ee1c/yarn
764 3685 5 7069 /hdfs/uuid/250b21c8-1714-45fe-8c08-d45d0271c6bd/yarn
432 1603 259 6402 /hdfs/uuid/4a833770-767e-43b3-b696-dc98901bce26/yarn
993 5856 320 129 /hdfs/uuid/b94cbf3f-76b1-4ced-9043-02d450b9887c/yarn
612 5645 4 249 /hdfs/uuid/8138a53b-b942-44d3-82df-51575f1a3901/yarn
818 21 6 166 /hdfs/uuid/fada8004-53ff-48df-9396-165d8e42925b/yarn
174 23 1 171 /hdfs/uuid/d04fccd8-bc72-4ed9-bda4-c5b6893f1405/yarn
376 6281 2 97 /hdfs/uuid/0cc3683f-4800-4c73-8075-8d77dc7cf116/yarn
370 4588 2 96 /hdfs/uuid/a78f846a-58c4-4d10-a9f5-42f16a6134a0/yarn
190 6420 1 86 /hdfs/uuid/2c6a7223-cb18-4916-a1b6-8cd02bda1d31/yarn
178 123 1 17 /hdfs/uuid/b3b2a2ed-f6c1-4641-86bf-2989dd932411/yarn
[...]
This shows various directories read and written when hadoop runs.
By default the output is sorted by the total read size in Kbytes (R_Kb).
Sorting order can be changed via -s option.
This is instrumenting at the VFS interface, so this is reads and writes that
may return entirely from the file system cache (page cache).
While not printed, the average read and write size can be calculated by
dividing R_Kb by READS, and the same for writes.
This script works by tracing the vfs_read() and vfs_write() functions using
kernel dynamic tracing, which instruments explicit read and write calls. If
files are read or written using another means (eg, via mmap()), then they
will not be visible using this tool.
This should be useful for file system workload characterization when analyzing
the performance of applications.
Note that tracing VFS level reads and writes can be a frequent activity, and
this tool can begin to cost measurable overhead at high I/O rates.
A -C option will stop clearing the screen, and -r with a number will restrict
the output to that many rows (20 by default). For example, not clearing
the screen and showing the top 5 only:
# ./dirtop -d '/hdfs/uuid/*/yarn' -Cr 5
Tracing... Output every 1 secs. Hit Ctrl-C to end
14:29:08 loadavg: 25.66 23.42 21.51 17/2850 67167
READS WRITES R_Kb W_Kb PATH
100 8429 0 48243 /hdfs/uuid/b94cbf3f-76b1-4ced-9043-02d450b9887c/yarn
2066 4091 8176 26457 /hdfs/uuid/d04fccd8-bc72-4ed9-bda4-c5b6893f1405/yarn
10 2043 0 8172 /hdfs/uuid/b3b2a2ed-f6c1-4641-86bf-2989dd932411/yarn
38 1368 0 2652 /hdfs/uuid/a78f846a-58c4-4d10-a9f5-42f16a6134a0/yarn
86 19 0 123 /hdfs/uuid/c11da291-28de-4a77-873e-44bb452d238b/yarn
14:29:09 loadavg: 25.66 23.42 21.51 15/2849 67170
READS WRITES R_Kb W_Kb PATH
1204 5619 4388 33767 /hdfs/uuid/b94cbf3f-76b1-4ced-9043-02d450b9887c/yarn
2208 3511 8744 22992 /hdfs/uuid/d04fccd8-bc72-4ed9-bda4-c5b6893f1405/yarn
62 4010 0 21181 /hdfs/uuid/8138a53b-b942-44d3-82df-51575f1a3901/yarn
22 2187 0 8748 /hdfs/uuid/b3b2a2ed-f6c1-4641-86bf-2989dd932411/yarn
74 1097 0 4388 /hdfs/uuid/4a833770-767e-43b3-b696-dc98901bce26/yarn
[..]
USAGE message:
# ./dirtop.py -h
usage: dirtop.py [-h] [-C] [-r MAXROWS] [-s {all,reads,writes,rbytes,wbytes}]
[-p PID] -d ROOTDIRS
[interval] [count]
File reads and writes by process
positional arguments:
interval output interval, in seconds
count number of outputs
optional arguments:
-h, --help show this help message and exit
-C, --noclear don't clear the screen
-r MAXROWS, --maxrows MAXROWS
maximum rows to print, default 20
-s {all,reads,writes,rbytes,wbytes}, --sort {all,reads,writes,rbytes,wbytes}
sort column, default all
-p PID, --pid PID trace this PID only
-d ROOTDIRS, --root-directories ROOTDIRS
select the directories to observe, separated by commas
examples:
./dirtop -d '/hdfs/uuid/*/yarn' # directory I/O top, 1 second refresh
./dirtop -d '/hdfs/uuid/*/yarn' -C # don't clear the screen
./dirtop -d '/hdfs/uuid/*/yarn' 5 # 5 second summaries
./dirtop -d '/hdfs/uuid/*/yarn' 5 10 # 5 second summaries, 10 times only
./dirtop -d '/hdfs/uuid/*/yarn,/hdfs/uuid/*/data' # Running dirtop on two set of directories