Releases: k0retux/fuddly
v0.30
Fuddly 0.30
This release brings a lot of enhancements and new features as well as
fixing bugs. It is also time to remove python 2 support.
This version is not compatible with previous Fuddly versions (refer to "Fuddly Environment Change"
and "API Change").
Here under are listed what is the most noteworthy.
Highlighted New Features
-
New constraint-based backend for data description:
-
based on python-constraint to leverage
constraint programming in the description of data (Node-based). -
New operator
tWALKcsp
to walk through the
solutions of any CSP defined within input data (Node-based). -
New operator
tCONST
operates on a data model that leverages
the CSP backend. It negates the constraint one-by-one and output 1
or more samples for each negated constraint. -
Operators based on model walking infrastructure (like
tWALK
) get
constraint backend support for almost free -
New parameters in
Node.freeze
to support constraint backend:@restrict_csp
,@resolve_csp
-
Refer to the documentation
-
-
New tool plotty for visual analysis of the Fuddly Database (refer to the documentation)
-
Add a new Tasks infrastructure which can be leveraged by
scenario steps but also started when a target is launched
(refer to the documentation) -
The Logger interacts now with the terminal through another thread to
remove unnecessary latency while sending data. -
Add new backend infrastructure (SSHBackend, SerialBackend, ...) that
can be leveraged by Probes and Targets -
Add JSON data model:
-
create automatically Node-based models from any JSON
schema provided in <fuddly_data_path>/imported_data/json/,
allowing to generate data compliant to these JSON schema. -
or create Node-based models from any JSON data.
-
New Features and Enhancements
-
Fuddly database and
fmkdb.py
tool-
New
ASYNC_DATA
table added to record every data sent
asynchronously (e.g., periodic data sent from a Scenario). Such
data are sent throughTarget.send_data_sync()
or
Target.send_multiple_data_sync()
. -
Update
tools/fmkdb.py
to display async data information related to
any data ID. (option--with-async-data
) -
Add new option
--fbk-status-formula
tofmkdb.py
allowing to
restrict the data to be displayed to specific feedback
status. This option provides the formula to be used for feedback
status filtering -
Record by default in FmkDB some information from the session
(feedback timeout, project knowledge, etc.)
-
-
Project class
-
New parameters to enable/disable workspace and/or fmkDB at start
-
New parameters to allow changing the default behaviour regarding
fuddly workspace (where data are stored after being generated) -
It is now possible to provide default values for feedback timeout,
feedback mode, sending delay and burst value
-
-
Scenario related:
-
Add new method
Step.make_stutter()
-
Add
@sending_delay
parameter toStep()
-
Add new condition to cross transition:
dp_completed_guard
(refer to the documentation) -
Add the parameter
@user_args
toScenario
class -
Add a new
feedback_gate
that provide not only current feedback but a trail of 10 seconds -
A
Step()
can nowconnect_to()
an existingScenario()
. -
For scenarios that define
Step
content within callback function at runtime,
a specificStepStub()
class is now defined without the parameter@data_desc
.
-
-
Framework Plumbing and Shell:
-
Add new
FmkPlumbing.process_data_and_send()
-
FmkPlumbing.show_data_maker_types()
enhanced with DM info -
New shell commands:
collect_feedback
,enable_fbk_handlers
,
disable_feedback_handlers
-
-
Data Model new features and enhancements:
-
New keywords for data model description (refer to documentation):
-
namespace
andfrom_namespace
have been added
to make easier the naming of nodes in data model description -
always
has been added for separator description -
highlight
has been added for data model description. It is
currently leveraged bytTYPE
to highlight the altered nodes on
the console -
New customization
MH.Custo.NTerm.StickToDefault
for non-terminal nodes.
-
-
Helpers:
-
New generator node template
SELECT()
: returns a generator that
select a subnode from a non-terminal node and return it or a
copy of it. -
In specific condition, make a better choice for node modelling
when a regexp is provided in a node description. For instance
'\d{n,m}'
do not create multipleINT_str()
but only one
-
-
Parameter
@default
is now added to all typed nodes:String
,INT
, andBitField
have now a@default
parameter and a dedicated
method.set_default_value()
- new model description keyword added
default
- absorption operation is now changing the default value to what is absorbed
-
New description parameters have been added to have more
information when displaying Nodes (description parameters in
Node
,String
,Bitfield
,Int
, ...) -
Add
DataModel.customize_node_backend()
method in order to change
default node customization for all the descriptors/atoms
registered in the DataModel. -
Improve performance of
Node.get_reachable_nodes()
when the@path_regexp
criteria is used -
Non-terminal node evolution:
-
Non-terminal node can now handle a default quantity for their subnodes.
A new keyworddefault_qty
has been added. -
Revamp of non-terminal node algorithm to generate the different
possible shapes from the model description (considering the new
default_qty
). Simpler and better performance (about cpu and
memory consumption). -
The generation algorithm of the different shapes are now
customizable, through the parameterFullCombinatory
(either
full-combinatory or limited combinatory similar as before). -
A new customization parameter
CycleClone
allows to cycle among
the possible values of the subnodes which are duplicated several
times. -
A new parameter
@full_combinatory
has been added totWALK
and
tTYPE
in order to take benefit from this change. -
Add the method
Node.add()
for non-terminal nodes enabling to add a
node within an existing non-terminal node
-
-
Bitfield:
BitField.*_subfield()
methods can now reference subfields by description- Enhance
Bitfield Collapsing
operation
-
String:
String._populate_values()
gets called upon reset now when
determinist
is False in order to avoid getting same test case too
often- new
@case_sensitve
parameter with new related fuzzing cases - absorption operation leverages new
AbsCsts.SimilarContent
criteria
as a way to distinguish case-sensitive from case-insensitive String. RawCondition()
has now a new param @case_sensitive
-
Filename()
fuzzing cases revamped
-
-
Generators and Disruptors
-
tWALK
andtTYPE
leverage new framework features through new parameters -
Leverage the attribute Mutable of non-terminal node to customize
the behavior of theModelWalker
algorithm (used bytTYPE
and
tWALK
). If not mutable, then the algo will always stick to the
default form of the non-terminal. -
When a data model uses some existence conditions, walking through
the graph can be tricky. As a result, some data compliant with the
data model could never be generated bytWALK
. The walking
algorithm has been improved in order to take into account the
changes that may occur among the sibbling nodes of the node which
is currently walking through. And in the case there is some
change, then the algorithm will walk through the new nodes. This
improves the exploration capability oftWALK
. -
New parameters have been added to the Generators
automatically created from data models -
New Generator
GENP
to generate basic data based on a pattern and
different parameters.
-
-
Targets, Backends
-
TestTarget
provides several new features that allows to create
simulated environment composed of different interconnected
targets (simulated by different fuddly instance). -
Add new
SSHTarget
-
-
Monitoring:
- Add new
ProbeCmd
that enables you to execute shell commands and retrieve the output.
- Add new
-
Knowledge Infrastructure
-
FeedbackHandlers:
-
there are now triggered when a call to
Target.send_data_sync()
is performed asynchronously from outside the framework (e.g., from a Task) -
notify_data_sending()
is provided now with the relevant Target
in@target parameter
and not with the list of targets
-
-
the new subdirectory
user_info/
has been added in fuddly data folder
(automatically created if not existing). It is used to store user
specificInformation
(defined in the form of classes inheriting
theInformation
class) in order for instance to share information
between user projects and user data models.
-
Fuddly Environment Change
-
Add XDG Base Directory support. Note that if
~/fuddly_data
still exists,
XDG base directories won't be used. -
Due to some change in fmkDB structure, **previous ...
v0.27.2
Fuddly 0.27.2
This release brings some enhancements and fixes some bugs.
Here under what is the most noteworthy.
- The evolutionary fuzzing infrastructure is now callable from a project file through an updated API.
Refer to the documentation (evolutionary fuzzing chapter) to know how to use it. - Existence condition (for data modeling) enhancements.
IntCondition()
,BitFieldCondition()
andRawCondition()
provide new parameters to cover more use cases. - Add a new
Node.show()
parameter to ignore the display of generator nodes (while still displaying what is generated by them). - Documentation update.
v0.27.1
Fuddly 0.27.1
This release brings some enhancements, new features and fixes some bugs.
Here under what is the most noteworthy.
New Features
-
Add a new decoding feature to fmkdb.py to produce pretty printing
of sent data and/or feedback by leveraging newDataModel
methods. -
Add more support to automate data decoding
- raw data from
import_data/
directory are now automatically decoded based on the respective
data model (assuming the newDataModel.register_atom_for_absorption()
method is used) DataModel.decode()
leverages registered nodes of the data model to perform the decoding
- raw data from
-
Scenario Enhancements:
- Periodic class supports virtual targets
.graph()
method produces a more complete diagram of the scenario- new configurability feature at Scenario level (
user_context
parameter)
-
Feedback retrieval has been improved at plumbing level:
- in multi-targets setup even target not simulated are probed for feedback after
the sending step - fix feedback flushing behavior (especially with
NetworkTarget
) - better handling of target recovery
- in multi-targets setup even target not simulated are probed for feedback after
-
Enhance NetworkTarget regarding:
SOCK_RAW
interface support (Ethernet header can be automatically added)- Feedback retrieval and way to report its ready status
-
Generic disruptors change:
- Add a new dictionary parameter (
multi_mod
) to the disruptorMOD
in order
to enable multiple change on the data at once - Add new disruptor
CALL
to call a function on input data
- Add a new dictionary parameter (
-
To avoid polluting the printing flow of fuddly, the FeedbackHandler
class gain the ability to create a new terminal emulator window as a
new display for it.
API change
- New
FmkPlumbing.start()
method is required to be called first NetworkTarget._custom_data_handling_before_emission()
signature change
v0.27
Fuddly 0.27
This release brings several enhancements and new features as well as
fixing bugs and doing some cleanup. Here under what is the most
noteworthy.
Highlighted New Features
-
Add a Knowledge Infrastructure (refer to the documentation) which enables
to:- to dynamically collect data from Target/Probes, and extract
information from it through dedicated handlers to create knowledge - to add knowledge about the target under tests (like kind of OS,
used programming language, hardware, and so on) - to leverage this information in relevant fuddly subsystems (e.g.,
fuzzing a Filename typed-node with tTYPE will adapt the test cases
depending on the OS/Language/..., if this information is available)
- to dynamically collect data from Target/Probes, and extract
-
Multiple targets can now be used at the same time in all relevant
places of the framework.It makes possible to sequence different actions through various systems
or on the same system but through different kinds of interfaces
(represented by different targets); and to react on feedback retrieved
from all the stimulated targets in a centralized way.
Enhancements
- Some value types got enhanced to support more options and/or
provide more fuzzing cases:INT_str()
,String()
. - Scenarios can now be defined at project level through
Project.register_scenarios()
- Some disruptors got enhanced, especially
tTYPE
that now includes
tSEP
test cases and go beyond.C
andtSEP
have been slightly
improved. - New generic disruptor created:
OP
. - Feedback logs have been harmonized
- Add a configuration facility to Fuddly shell (through
config
command) - Add support in
fmkDB.py
for users to confirm or disprove automated
impact analysis. - Disruptors and scenario Steps get the ability to mark data as
altered or valid in order to enable different handling at Target
level - Add a parameter to describe values of an
INT()
object - NetworkTarget, LocalTarget and TestTarget got improved
- Data modeling got enhancements and bug fixes:
- Non-terminal nodes support now finite+random attributes combination
- Add support for absorption of non byte-aligned nodes
node_builder.RegexParser
useINT_str()
instead ofString()
in more cases- Fix
NT.unfreeze(recursive=True)
incorrect behavior in some condition - ...
- Helpers/Templates enhancements:
- XML helpers provide now way more flexibility.
- Add JSON helpers
- Add a new method
DataModel.validation_tests()
which is called automatically
by fuddly own non-regression tests - Add
send_loop
cosmetics for nicer display (enable it viaconfig shell send_loop aligned True
)
API change
- Due to a change in fmkDB structure, previous fmkDB versions are
not recognized anymore. - Generic and specific parameters that can be specified for
Generators, Disruptors and Operators are now merged into a single
kind of parameters in order to simplify the API. - The class
FeedbackHandler
becomesFeedbackGate
. ProbeStatus
get the new propertyvalue
that replace
set_status
/get_status
methods.- When iterating the
FeedbackGate
, the source parameter which was a string
is now aFeedbackSource
. FmkShell.do_send_loop_noseed
is now the defaultdo_send_loop
.FmkShell.do_send_loop
which was keeping a seed is now renameddo_send_loop_keepseed
.- Refactor
DataModel.absorb()
toDataModel.create_node_from_raw_data()
. - Align semantics of raw format within fmkdb.py and the
Logger()
class.
v0.26
Fuddly 0.26
This release brings several enhancements and new features as well as fixing bugs and doing some cleanup. Here under what is the most noteworthy:
-
Scenario infrastructure enhancements and fuzzing features (refer to the documentation available here):
- Add scenario fuzzing features to play around the protocol sequence, the timing constraints and the data emitted.
- Reorganize and add callbacks that triggers before sending data and assign them to a step (do_before_data_processing, do_before_sending).
- Add the ability for a step to trigger the sending of multiple data.
- Improve the graphical display (more information, current step identified, ...) and add the ability to open it automatically while
fuddly
run through a scenario. - New
FeedbackHandler
class to simplify feedback handling, especially with scenario callbacks. - Revamp the transition condition checking logic.
-
Add an Evolutionary fuzzing infrastructure (refer to the documentation available here).
-
New data model helpers/templates:
- Add helpers for describing XML data.
- Regroup all the data model helpers in the new
framework.dmhelpers
package.
-
Enhance and add new options to the
NetworkTarget
:- Add new parameters
target_address
andwait_for_client
to enable more behavior options for an interface configured in server mode (refer to the documentation). - Remove the artificial correlation between the
sending_delay
andfeedback_timeout
attributes.
- Add new parameters
-
Add the ability to change the
min/max
values of non-terminal node children through the methodset_subnode_minmax()
. -
Add an
HTTP
data model.
API modification
-
The parameters
mini
andmaxi
ofINT
-based classes (e.g.,UINT16_be
,SINT8
, ...) have been respectively renamedmin
andmax
. -
Redesign
Data()
andDataModel()
to make them content-agnostic; resulting in changing some usage patterns.
v0.25.2
Fuddly 0.25.2
In addition to bug-fixes and various cleanups, here under what is the most noteworthy:
- Add Scenario visualization through
graphviz
module. - Add support for more complex regular expression for data model description.
- Add new test cases to the disruptor
tSTRUCT
. - Enhance the FmkDB tool (
tools/fmkdb.py
): add new option--raw
and extend the use of--limit
.
v0.25.1
Fuddly 0.25.1
In addition to bug-fixes, enhancements and various cleanups, here under what is the most noteworthy:
- Add the new keyword
evolution_func
for tweaking data model description. - Target feedback retrieval is now guided by a specific mode (in addition to feedback timeout) that specify:
- either to wait for the full time slot allocated for feedback retrieval;
- or to wait until we received something from the target.
- Add a new
Backend()
to the monitoring infrastructure:Shell_Backend()
- Add new fuzzing cases for
INT_str()
-based typed nodes.
v0.25
Fuddly 0.25
This release fixes many bugs and counter-intuitive behaviors, as well as bringing several enhancements and new features. Here under what is the most noteworthy:
- Add support for specifying parts of a data model through regular expressions. Refer to the documentation available here.
- Increase the amount and diversity of fuzzed data produced by any disruptor leveraging the
ModelWalker
infrastructure (e.g.,tTYPE
,tSEP
,tWALK
) by better unwinding non-terminal nodes. - Improve data production quality of several disruptors (e.g.,
tTYPE
) by handling in more subtle ways data models (by preserving some constraints depending on what is being changed). - Enhance
tTYPE
production regardingString
-based nodes (enhance some fuzzing cases, add new ones and add new fuzzing magnitude parameter). - Add native support for python codecs (
utf8
,utf-16-be
, ...) toString
-based types (through the parametercodec
). - Add a generic probe to monitor the memory consumption of a process specified by name (
ProbeMem
). - Add a Serial backend and a SSH backend for generic probes, enabling to use them through a serial line or an SSH connection.
API modifications:
UTF8
/UTF16
/...Encoder
-based classes have been removed since theString
class supports codecs natively.- Rename respectively the parameters
val_list
andint_list
ofString
-based andINT
-based types tovalues
. - The generator templates
MH.LEN
,MH.QTY
,MH.CRC
,MH.WRAP
andMH.OFFSET
provide now generator nodes that are not freezable by default. These non-freezable generator nodes are still handled bytTYPE
which now deals with them gracefully. - Obsolete
tTERM
disruptor has been removed.
v0.24.2
Fuddly 0.24.2
More relevant additions, enhancements and bug-fixes are:
- Data description improvements with the high-level JSON-like API (refer to the documentation):
- Add the keywords
qty_from
,sync_size_with
andsync_enc_size_with
which are the counterparts ofMH.QTY
andMH.LEN
, but are compatible with generation and absorption. - Associate the keyword
specific_fuzzy_vals
to the existing feature for specifying a list of additional values to be leveraged by the disruptortTYPE
. - Modification of a Node clone is now possible (by specifying attributes in the descriptor).
- The
contents
keyword can now be provided with aNode
object. - Add a parameter
default
toINT
-based class andBitField
class.
- Add the keywords
- Node absorption enhancement regarding nodes which are set to be postponed for absorption. They can now precede any kind of nodes.
- Add support for size constraints alteration to the disruptor
tSTRUCT
. - Scenario infrastructure enhancements regarding
DataProcess
and bug fixes. - Revamp USB data model and describe it with the high-level API (JSON-like).
- Update PPPoE data model by adding a new scenario
SC_PADS
for fuzzingPADS
messages after correctly answeringPADI
messages. - Fix regressions with
NetworkTarget
and with monitoring integration. - Fix
Node.show()
limitation with multiple identical nodes with the same path.
v0.24.1
Fuddly 0.24.1
More relevant additions, enhancements and bug-fixes are:
- Addition of the PPPoE data model and a server scenario to send fuzzed
PADO
messages
in response toPADI
messages. - Enhancement to the Scenario Infrastructure enabling to:
- Make a step (
NoDataStep
) that won't trigger the sending of data. - Add a description to a step. (Auto-description is created when nothing is provided.)
- Make a step (
- Add new fuzzing cases for
INT
-based andBitField
-based nodes. - Revamp JPG data model and describe it with the high-level API (JSON-like).
- Update the
NetworkTarget
class to support SOCK_DGRAM server mode and SOCK_RAW.
(Fix also a bunch of bugs related to feedback retrieval.)