Porting commands to the newshell

[caribpa]

The current state of the newshell commands, as mentioned by @ret2libc in 0.2.0 Priorities is:

keep moving commands to newshell and improve it as we go: this is unfortunately not as easily "quantifiable" because there are many many commands to convert. Also, we realize some things need improvements in RzCmd as we convert existing stuff.

Let's start with the output of ?:

| ![?]                  # Run given commands as in system(3)
| _                     # Print last output
| #![<interpreter-name> [<arg0> [<arg1> ...]]] # List all available interpreters / Run
                                                 interpreter
| $[?]                  # Alias commands and strings
| %[<varname>[=<varvalue>]] # Get/set environment variables
| &[jt=b-&?]            # Manage tasks
| ([?]                  # Manage scripting macros
| *<addr>[=<0xvalue>|<hexstring>] # Pointer read/write data/values
| -[?]                  # Open cfg.editor and run script
| .[ .-*!(?]            # Interpret commands
| /[?]                  # Search for bytes, regexps, patterns, ..
| =[?]                  # Connect with other instances of rizin
| ?*[j] [<search_cmd>]  # Search help
| ?[??]                 # Help or evaluate math expression
| < <characters>        # Push escaped string into the RzCons.readChar buffer
| 0[?]                  # Alias for `s 0x...`
| a[f]                  # Analysis commands
| b[?]                  # Display or change the block size
| c[?]                  # Compare block with given data
| C[?]                  # Code metadata (comments, format, hints, ..)
| d[c]                  # Debugger commands
| e[?]                  # List/get/set config evaluable vars
| f[?]                  # Manage flags
| g[?]                  # Generate shellcodes with rz_egg
| i[?]                  # Get info about opened binary file
| k[?]                  # Run sdb-query
| ls [<arg0> [<arg1> ...]] # List files and directories
| m[?]                  # Make directories and move files
| L[?]                  # List, unload, load rizin plugins
| o[?]                  # Open file at optional address
| p[?]                  # Print commands
| P<so?>                # Project management
| q[!?]                 # Quit rizin
| r[?]                  # Resize file
| s[?]                  # Seek commands
| t[?]                  # Types, noreturn, signatures, C parser and more
| uniq <filename>       # List uniq strings in file
| uname [-r]            # Provide system info
| V[?]                  # Enter visual mode
| v[?]                  # Enter visual panel mode
| w[?]                  # Write commands
| x[?]                  # Alias for 'px' (print hexadecimal)
| y[?]                  # Yank/paste bytes from/to memory
| z[?]                  # Zignatures management
| @[?]                  # '@' help, temporary modifiers
| @@[?]                 # '@@' help, iterators
| >[?]                  # Redirection help ('>')
| |[?]                  # Pipe help ('|')
| ~[?]                  # Internal grep help ('~')

Is there any subgroup (for example C[?]) with nothing moved to newshell as of now?

P.S. Is there any command that doesn't use newshell (or oldshell) and works using something else entirely? (so I can remove it from the list)

[ret2libc]

Status:

• !
• $
• (
• -
• /
• =!
• =g
• =h
• =H
• ?
• 0
• a
• afb
• afv
• afB
• afo
• afu
• b
• c
• C
• d
• dc
• ec
• f
• g
• i
• k
• m
• L
• o
• p
• q
• r
• s/
• t
• wh
• we
• wu
• wr
• wA
• wc
• wt
• wf
• ww
• wx
• wa
• wb
• wm
• wo
• wd
• ws
• x
• y

[ret2libc]

Let's say that most groups are not moved to newshell as of now :D

z is fully converted to newshell, w is partially converted, q is fully converted, e is fully converted, s is almost fully converted (just one command missing, but it requires a bit of thinking), . is fully converted, #! is fully converted, = is partially converted.

[caribpa]

Maybe it makes sense to start with one of the most used commands, from i or pd (as you said here), so that bugs (implementation related) could be detected early and ease the implementation of the other groups down the line.

[ret2libc]

If we want to catch bugs early on then it's probably pd, because i commands do not accept any argument in general, so the differences/advantages of converting them would be minimal.

[caribpa]

Two choices for 0.2.0:

• Most used from the i family (ie, iE, ih, ii, iI, im, ir, is, iS, and iSS, if I'm not forgetting anything)
• Base pd (which I guess should be really complex)

[caribpa]

I suggested some from the i family because they may be easier to implement (good for generating "implementation momentum") 🙂

[caribpa]

For the record, @XVilka is working on migrating types (t) in #728, and @officialcjunior is working on debug-step-until (dsu) in #709

@ret2libc, do you prefer pd or i?

[caribpa]

I'd like to start limiting the development for 0.2.0 somewhere 🙂

[ret2libc]

I am working on i.

[caribpa]

perfect

[ret2libc]

FYI, I will wait for some refactoring about RzBin that will be done by @thestr4ng3r before going ahead with i. In the meantime, I will focus on other commands and/or issues for 0.2.0.

[XVilka]

Types were finished, I will add autocompletion and other enhancements soon too.

[caribpa]

That was fast!

What command will you convert afterwards? (so that nobody else does it)
Unless @ret2libc especially wants to do it, the other priority is pd, as mentioned in another message above 🙂

[XVilka]

pd is huge to convert, since I am focusing on RzTypes I will not do this, probably C.

[caribpa]

Ok, it makes sense 👍

Still, converting pd (command, like pd 1, not the pd sub-group family, like pdf) should be done sooner rather than later, as it would allow further refactoring of the codebase, as well as catching bugs early 🙂

[XVilka]

To be honest, I think flags f should be converted before pd command, since they will benefit from the proper argument handling, especially for the names and all syntax extensions.

[XVilka]

I grepped Cutter for Rizin command calls, this list is also a priority one (i, f, a, d, and C commands):

[i] ℤ rg "cmdj\("                                                                                                                                                                                                                 13:17:50 
widgets/RizinGraphWidget.cpp
99:    QJsonDocument functionsDoc = Core()->cmdj(QString("%1j").arg(graphCommand));

widgets/FunctionsWidget.cpp
521:                mainAdress = (ut64)Core()->cmdj("iMj").object()["vaddr"].toInt();

widgets/Dashboard.cpp
79:    QJsonObject hashes = Core()->cmdj("itj").object();
127:    QJsonObject analinfo = Core()->cmdj("aaij").object();

widgets/DisassemblerGraphView.cpp
182:        QJsonDocument functionsDoc = Core()->cmdj("agJ " + RAddressString(fcn->addr));

widgets/CallGraph.cpp
80:    QJsonDocument functionsDoc = Core()->cmdj(global ? "agCj" : QString("agcj @ %1").arg(address));

python/cutter.py
13:def cmdj(command):

menus/DisassemblyContextMenu.cpp
324:    const QJsonArray array = Core()->cmdj("anj @ " + QString::number(offset)).array();
493:            Core()->cmdj("aoj @ " + QString::number(offset)).array().first().toObject();
729:    QJsonArray array = Core()->cmdj("pdj 1 @ " + RAddressString(offset)).array();

core/Cutter.cpp
383:    QJsonArray openFilesArray = cmdj("oj").array();
477:QJsonDocument CutterCore::cmdj(const char *str)
497:    res = cmdj(str);
725:    return cmdj("aoj @ " + RAddressString(addr))
734:    return cmdj("aoj @ " + RAddressString(addr))
970:            Core()->cmdj("pdj " + QString::number(count + 1) + "@" + QString::number(startAddr))
1260:    return cmdj("aeafj");
1266:    RVA value = cmdj("aoj @" + QString::number(addr))
1308:    return cmdj("ij");
1313:    return cmdj("iVj");
1318:    return cmdj("iCj");
1389:    QJsonObject registers = cmdj("drj").object();
1547:        return cmdj("dptj");
1549:        return cmdj("dptj " + QString::number(pid));
1557:        return cmdj("dpj");
1560:    return cmdj("dpj " + QString::number(pid));
1565:    return cmdj("drj");
1571:    QJsonObject varsObject = cmdj(QString("afvj @ %1").arg(at)).object();
1593:    QJsonArray registerRefArray = cmdj("drrj").array();
1883:        QJsonArray openFilesArray = cmdj("oj").array();
2151:    QJsonArray pluginArray = cmdj("dLj").array();
2429:    return cmdj("dbtj");
2435:    QJsonArray processArray = cmdj("dplj").array();
2456:    QJsonArray memoryMapArray = cmdj("dmj").array();
2543:    QJsonArray jsonArray = cmdj("sj").array();
2578:    QJsonObject jsonRoot = cmdj("iLj").object();
2606:    QJsonArray plugins = cmdj("oLj").object()["io_plugins"].toArray();
2630:    QJsonArray plugins = cmdj("Lcj").array();
2706:    QJsonArray importsArray = cmdj("iij").array();
2731:    QJsonArray exportsArray = cmdj("iEj").array();
2793:    QJsonArray headersArray = cmdj("ihj").array();
2816:    QJsonArray zignaturesArray = cmdj("zj").array();
2847:    QJsonArray commentsArray = cmdj("CCj").array();
2926:    QJsonArray flagspacesArray = cmdj("fsj").array();
2949:    QJsonArray flagsArray = cmdj("fj").array();
2970:    QJsonDocument sectionsDoc = cmdj("iSj entropy");
3000:    QJsonArray sectionsArray = cmdj("iSj").array();
3012:    QJsonArray segments = cmdj("iSSj").array();
3039:    QJsonArray entrypointsArray = cmdj("iej").array();
3062:    QJsonArray classesArray = cmdj("icj").array();
3108:    QJsonArray flagsArray = cmdj("fj@F:classes").array();
3318:    QJsonArray resourcesArray = cmdj("iRj").array();
3341:    QJsonArray vTablesArray = cmdj("avj").array();
3384:    QJsonArray typesArray = cmdj("tj").array();
3405:    QJsonArray typesArray = cmdj("tuj").array();
3425:    QJsonArray typesArray = cmdj("tsj").array();
3445:    QJsonObject typesObject = cmdj("tej").object();
3462:    QJsonObject typesObject = cmdj("ttj").object();
3535:        searchArray = cmdj(QString("%1 %2").arg(space, searchFor)).array();
3592:        statsObj = cmdj("p-j " + QString::number(blocksCount)).object();
3675:        xrefsArray = cmdj("axtj@" + QString::number(addr)).array();
3677:        xrefsArray = cmdj("axfj@" + QString::number(addr)).array();
3742:    auto r = cmdj(QString("fdj @") + QString::number(offset)).object();
3816:    QJsonArray array = cmdj(QString("pdJ ") + QString::number(lines) + QString(" @ ")
3944:    QJsonDocument files = cmdj("oj");
3951:    QJsonDocument themes = cmdj("ecoj");
4039:    QJsonArray ans = cmdj("oj").array();

core/Cutter.h
118:    QJsonDocument cmdj(const char *str);
119:    QJsonDocument cmdj(const QString &str) { return cmdj(str.toUtf8().constData()); }

core/MainWindow.cpp
1124:    auto hist = Core()->cmdj("sj");
1127:    for (auto item : Core()->cmdj("sj").array()) {

common/IOModesController.cpp
81:    QJsonArray changes = Core()->cmdj("wcj").array();

common/SettingsUpgrade.cpp
184:    const QStringList options = Core()->cmdj("ecj").object().keys()

common/ColorThemeWorker.cpp
135:        theme = Core()->cmdj("ecj").object().toVariantMap();
138:        theme = Core()->cmdj("ecj").object().toVariantMap();
279:    QString options = (Core()->cmdj("ecj").object().keys() << cutterSpecificOptions)
316:        rizinSpecificOptions = Core()->cmdj("ecj").object().keys();
cutter/src on  master [!] 
[i] ℤ rg "cmdRawAt\("                                                                                                                                                                                                             13:17:56 
widgets/HexdumpWidget.cpp
242:                selectedCommand != "" ? Core()->cmdRawAt(
248:                Core()->cmdRawAt(QString("ph md5 %1").arg(size), start_address).trimmed());
250:                Core()->cmdRawAt(QString("ph sha1 %1").arg(size), start_address).trimmed());
252:                Core()->cmdRawAt(QString("ph sha256 %1").arg(size), start_address).trimmed());
254:                Core()->cmdRawAt(QString("ph crc32 %1").arg(size), start_address).trimmed());
256:                Core()->cmdRawAt(QString("ph entropy %1").arg(size), start_address).trimmed());

widgets/HexWidget.cpp
669:                Core()->cmdRawAt(QString("psx %1").arg(selection.size()), selection.start())
672:        clipboard->setText(Core()->cmdRawAt(QString("p8 %1").arg(selection.size()),
707:        Core()->cmdRawAt(QString("w %1").arg(str), getLocationAddress());
723:    Core()->cmdRawAt(QString("w%1%2 %3")
747:        Core()->cmdRawAt(QString("w0 %1").arg(str), getLocationAddress());
775:    Core()->cmdRawAt(QString("w6%1 %2").arg(mode).arg(
796:        Core()->cmdRawAt(QString("wr %1").arg(nbytes), getLocationAddress());
813:    Core()->cmdRawAt(QString("wd %1 %2").arg(copyFrom).arg(nBytes), getLocationAddress());
828:        Core()->cmdRawAt(QString("ws %1").arg(str), getLocationAddress());
844:        Core()->cmdRawAt(QString("ww %1").arg(str), getLocationAddress());
860:        Core()->cmdRawAt(QString("wz %1").arg(str), getLocationAddress());

widgets/GraphWidget.cpp
87:    QString afcf = Core()->cmdRawAt("afcf", seekable->getOffset()).trimmed();

widgets/Dashboard.cpp
118:        QString entropy = Core()->cmdRawAt("ph entropy $s", 0).trimmed();

widgets/CutterGraphView.cpp
370:    qWarning() << Core()->cmdRawAt(QString("%0w \"%1\"").arg(graphCommand).arg(filePath), address);
381:    fileOut << Core()->cmdRawAt(QString("%0").arg(graphCommand), address);

menus/DecompilerContextMenu.cpp
157:        QString comment = Core()->cmdRawAt("CC.", this->firstOffsetInLine);

menus/DisassemblyContextMenu.cpp
543:    QString stringDefinition = Core()->cmdRawAt("Cs.", offset);
546:    QString comment = Core()->cmdRawAt("CC.", offset);

core/Cutter.h
108:    QString cmdRawAt(const char *cmd, RVA address);
111:     * @brief a wrapper around cmdRawAt(const char *cmd, RVA address).
113:    QString cmdRawAt(const QString &str, RVA address)
115:        return cmdRawAt(str.toUtf8().constData(), address);

dialogs/CommentsDialog.cpp
37:    QString oldComment = Core()->cmdRawAt("CC.", offset);

core/Cutter.cpp
446:QString CutterCore::cmdRawAt(const char *cmd, RVA address)
713:    cmdRawAt("f-", addr);
743:    cmdRawAt(QString("wa %1").arg(inst), addr);
749:    cmdRawAt("wao nop", addr);
755:    cmdRawAt("wao recj", addr);
761:    cmdRawAt(QString("wx %1").arg(bytes), addr);
767:    cmdRawAt(QString("wv %1").arg(bytes), addr);
773:    cmdRawAt("Cd-", addr);
804:    cmdRawAt(QString("%1 %2").arg(command).arg(size), addr);
810:    cmdRawAt("Cs-", addr);
816:    return cmdRawAt("ps", addr);
824:    cmdRawAt("Cd-", addr);
825:    cmdRawAt(QString("Cd %1 %2").arg(size).arg(repeat), addr);
832:    int size = cmdRawAt("Cd.", addr).toInt(&ok);
838:    cmdRawAt(QString("CCu base64:%1").arg(QString(cmt.toLocal8Bit().toBase64())), addr);
844:    cmdRawAt("CC-", addr);
865:    this->cmdRawAt(QString("ahi %1").arg(rzBaseName), offset);
875:    this->cmdRawAt(QString("ahb %1").arg(bits), offset);
885:    this->cmdRawAt("aht " + structureOffset, offset);
961:    RVA offset = cmdRawAt(QString("/O %1").arg(count), startAddr).toULongLong(&ok, 16);
1158:    return cmdRawAt("pi 1", addr).simplified();
1253:    QString ret = cmdRawAt(QString("af %1").arg(name), addr);
3523:    return !Core()->cmdRawAt(QString("om."), addr).isEmpty();
3723:    cmdRawAt(QString("f %1 %2").arg(name).arg(size), offset);
3863:    return cmdRawAt(QString("%1 %2").arg(command).arg(size), address);

dialogs/WriteCommandsDialogs.cpp
86:    QString bytes = Core()->cmdRawAt(QString("p8 %1").arg(QString::number(getNBytes())), offestFrom)

dialogs/EditVariablesDialog.cpp
17:    QString fcnName = Core()->cmdRawAt("afn.", offset).trimmed();

[XVilka]

Now with more progress, I created an issue to track unported commands here: #1342