musctf19-fstl.ipynb

{"cells":[{"metadata":{},"id":"0bb5cf2c","cell_type":"markdown","source":"**This is just an introduction to the new ds4n6_lib, showing some of the analysis we can perform with just a few instructions. During the next weeks we will be publishing different blog posts explaining new techniques and commands to get more powerful analysis**"},{"metadata":{},"id":"db0e71bb","cell_type":"markdown","source":"## **FUNCTIONS, VARS, INIT**"},{"metadata":{},"id":"fd30dd7d","cell_type":"markdown","source":"**We create a couple of variables we will use later: home_path (home path of the created environment) and thisnb (name of the current notebook).**"},{"metadata":{"trusted":true},"id":"f450f0ce","cell_type":"code","source":"home_path=!pwd\nthisnb=home_path[0]+\"/musctf19-fstl.ipynb\"","execution_count":1,"outputs":[]},{"metadata":{},"id":"76545e7c","cell_type":"markdown","source":"## **IMPORTS & INIT**"},{"metadata":{},"id":"ac27db67","cell_type":"markdown","source":"**We load all the needed libraries defined in the imports.ipynb notebook and deactivate the warnings in order to have a clean result.**"},{"metadata":{"trusted":true},"id":"97042e7d","cell_type":"code","source":"%run imports.ipynb","execution_count":2,"outputs":[]},{"metadata":{},"id":"30c66901","cell_type":"markdown","source":"## **READ DATA**"},{"metadata":{},"id":"4c529031","cell_type":"markdown","source":"**We download the outputs we got from running fstl in the musctf-19 evidence.**"},{"metadata":{"trusted":true},"id":"a1e60e4a","cell_type":"code","source":"url = 'https://drive.google.com/uc?id=1wdMFUXmMI1w6g7mzC0kMvD4sd3NXYrqz'\nof = 'fstl.tgz'\ngdown.download(url, of, quiet=False)\ngdown.extractall(\"fstl.tgz\", to=\"fstl/\")","execution_count":3,"outputs":[{"output_type":"stream","text":"Downloading...\nFrom: https://drive.google.com/uc?id=1wdMFUXmMI1w6g7mzC0kMvD4sd3NXYrqz\nTo: /home/jovyan/fstl.tgz\n3.35MB [00:00, 93.5MB/s]\n","name":"stderr"},{"output_type":"execute_result","execution_count":3,"data":{"text/plain":"['fstl/fstimeline.full.csv']"},"metadata":{}}]},{"metadata":{},"id":"c78c87a2","cell_type":"markdown","source":"**Let's read the fstl outputs with the xread() function from ds4n6_lib. It will generate the .pkl file we will use with the dataframes and store the collection of dataframes in d4.out**"},{"metadata":{"trusted":true},"id":"157cc7c6","cell_type":"code","source":"%%time\n\nd4.debug = 0\n\n# Read raw data\nxread(tool=\"mactime\", rootpath=home_path[0]+\"/fstl\", notebook_file=thisnb, windows=True)","execution_count":4,"outputs":[{"output_type":"stream","text":"- Searching notebook for saved input file / folder (_mactimed)\n  + Found: /home/jovyan/fstl\n\n- Reading file(s):\n  + INFO: 1 files found to read.\n\n  + [1/1] fstimeline.full.csv                                          -> fstimeline.full                         [200000]\n\n- Saving Harmonized DataFrames dict (dfs) as pickle file:\n    /home/jovyan/fstl.ham.pkl\n- Done.\n\n- Reading Done.\n\n- Recover the object read by executing:  myvar = d4.out (e.g. mydf = d4.out)\n\nCPU times: user 8.05 s, sys: 1.68 s, total: 9.73 s\nWall time: 9.77 s\n","name":"stdout"}]},{"metadata":{},"id":"1eb446ab","cell_type":"markdown","source":"We create a dataframes dictionary named pldfs. It means we have many different dataframes (one per plaso artifact output) in this variable. You could display all these keys with pldfs.keys(), but you don't need to do it as we already have developed a function to display and select via gui."},{"metadata":{},"id":"d6b15449","cell_type":"markdown","source":"**We store the dataframes in a more friendly variable: fstldf.**"},{"metadata":{"trusted":true},"id":"2b555401","cell_type":"code","source":"fstldf = d4.out","execution_count":5,"outputs":[]},{"metadata":{"trusted":true},"id":"1a66685c","cell_type":"code","source":"# Automatically created - DO NOT EDIT OR REMOVE unless you want to change the file to read (in that case, remove this cell)\n# 2021-04-07 16:25:45.816934\n# _mactimed_f2read = \"/home/jovyan/fstl\"","execution_count":6,"outputs":[]},{"metadata":{},"id":"605daca7","cell_type":"markdown","source":"**This is an output generated by the framework that is used to let you know the input you are reading.**"},{"metadata":{},"id":"4c090d59","cell_type":"markdown","source":"## **ANALYSIS**"},{"metadata":{},"id":"2cc2dbc9","cell_type":"markdown","source":"**xdisplay is used to configure the settings of the default dataframe that will be displayed in the next cell.**"},{"metadata":{"trusted":true},"id":"c48c5359","cell_type":"code","source":"xdisplay()","execution_count":7,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Markdown object>","text/markdown":"#### Pandas options"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"IntText(value=60, description='display.max_rows: ', style=DescriptionStyle(description_width='initial'))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"485e16fcf0ab43f6b415ac960c32f129"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"IntText(value=10, description='display.min_rows: ', style=DescriptionStyle(description_width='initial'))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"c53b9ab436fb4136ae5daf5a47cee704"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"IntText(value=20, description='display.max_columns: ', style=DescriptionStyle(description_width='initial'))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f2c49031cdcc4f66bf0d984678c97ab2"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Dropdown(description='display.colheader_justify: ', options=('right', 'left'), style=DescriptionStyle(descript…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"e96bf0a0ae224d2c8f23ea2c9537d000"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Checkbox(value=True, description='display.expand_frame_repr', indent=False, style=DescriptionStyle(description…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"35051671014b4400aef5c24abf85d387"}},"metadata":{}}]},{"metadata":{},"id":"3626b349","cell_type":"markdown","source":"**We use the xmenu is to select and display the dataframe we want.** "},{"metadata":{"trusted":true},"id":"cc61873a","cell_type":"code","source":"xmenu(fstldf)","execution_count":8,"outputs":[{"output_type":"display_data","data":{"text/plain":"<IPython.core.display.Markdown object>","text/markdown":"**DataFrame visualization menu:**"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Box(children=(Dropdown(description='Select DataFrame', layout=Layout(width='550px'), options=('Select DataFram…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2c6fc36d16374ceab03681b99a9c3c85"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"HBox(children=(Label(value='Selected dataframe: '),))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"d2ff11739fb646328f9a1dc7bf1a54c2"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"HBox(children=(HTML(value=\"<div style='color: orange; font-weight: bold;'>Data Loading. Please Wait…</div>\"),)…","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"a7f088a5301544e9aa72d4f5fbaa886a"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Output(layout=Layout(overflow='auto', width='99%'))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"f90061e0db2f4f54a931bc90b0985fa8"}},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"Output(layout=Layout(overflow='auto', width='99%'))","application/vnd.jupyter.widget-view+json":{"version_major":2,"version_minor":0,"model_id":"2c1fe95cb4c2479a8f2333f836a26eb6"}},"metadata":{}},{"output_type":"execute_result","execution_count":8,"data":{"text/plain":"<ds4n6_lib.gui.dfs_explorer at 0x7f0c6c833990>"},"metadata":{}}]},{"metadata":{},"id":"3d06e6fd","cell_type":"markdown","source":"**This is just an introduction to the new ds4n6_lib, showing some of the analysis we can perform with just a few instructions. During the next weeks we will be publishing different blog posts explaining new techniques and commands to get more powerful analysis**"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.7.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":5}