Fuddly 0.30
This release brings a lot of enhancements and new features as well as
fixing bugs. It is also time to remove python 2 support.
This version is not compatible with previous Fuddly versions (refer to "Fuddly Environment Change"
and "API Change").
Here under are listed what is the most noteworthy.
Highlighted New Features
-
New constraint-based backend for data description:
-
based on python-constraint to leverage
constraint programming in the description of data (Node-based). -
New operator
tWALKcsp
to walk through the
solutions of any CSP defined within input data (Node-based). -
New operator
tCONST
operates on a data model that leverages
the CSP backend. It negates the constraint one-by-one and output 1
or more samples for each negated constraint. -
Operators based on model walking infrastructure (like
tWALK
) get
constraint backend support for almost free -
New parameters in
Node.freeze
to support constraint backend:@restrict_csp
,@resolve_csp
-
Refer to the documentation
-
-
New tool plotty for visual analysis of the Fuddly Database (refer to the documentation)
-
Add a new Tasks infrastructure which can be leveraged by
scenario steps but also started when a target is launched
(refer to the documentation) -
The Logger interacts now with the terminal through another thread to
remove unnecessary latency while sending data. -
Add new backend infrastructure (SSHBackend, SerialBackend, ...) that
can be leveraged by Probes and Targets -
Add JSON data model:
-
create automatically Node-based models from any JSON
schema provided in <fuddly_data_path>/imported_data/json/,
allowing to generate data compliant to these JSON schema. -
or create Node-based models from any JSON data.
-
New Features and Enhancements
-
Fuddly database and
fmkdb.py
tool-
New
ASYNC_DATA
table added to record every data sent
asynchronously (e.g., periodic data sent from a Scenario). Such
data are sent throughTarget.send_data_sync()
or
Target.send_multiple_data_sync()
. -
Update
tools/fmkdb.py
to display async data information related to
any data ID. (option--with-async-data
) -
Add new option
--fbk-status-formula
tofmkdb.py
allowing to
restrict the data to be displayed to specific feedback
status. This option provides the formula to be used for feedback
status filtering -
Record by default in FmkDB some information from the session
(feedback timeout, project knowledge, etc.)
-
-
Project class
-
New parameters to enable/disable workspace and/or fmkDB at start
-
New parameters to allow changing the default behaviour regarding
fuddly workspace (where data are stored after being generated) -
It is now possible to provide default values for feedback timeout,
feedback mode, sending delay and burst value
-
-
Scenario related:
-
Add new method
Step.make_stutter()
-
Add
@sending_delay
parameter toStep()
-
Add new condition to cross transition:
dp_completed_guard
(refer to the documentation) -
Add the parameter
@user_args
toScenario
class -
Add a new
feedback_gate
that provide not only current feedback but a trail of 10 seconds -
A
Step()
can nowconnect_to()
an existingScenario()
. -
For scenarios that define
Step
content within callback function at runtime,
a specificStepStub()
class is now defined without the parameter@data_desc
.
-
-
Framework Plumbing and Shell:
-
Add new
FmkPlumbing.process_data_and_send()
-
FmkPlumbing.show_data_maker_types()
enhanced with DM info -
New shell commands:
collect_feedback
,enable_fbk_handlers
,
disable_feedback_handlers
-
-
Data Model new features and enhancements:
-
New keywords for data model description (refer to documentation):
-
namespace
andfrom_namespace
have been added
to make easier the naming of nodes in data model description -
always
has been added for separator description -
highlight
has been added for data model description. It is
currently leveraged bytTYPE
to highlight the altered nodes on
the console -
New customization
MH.Custo.NTerm.StickToDefault
for non-terminal nodes.
-
-
Helpers:
-
New generator node template
SELECT()
: returns a generator that
select a subnode from a non-terminal node and return it or a
copy of it. -
In specific condition, make a better choice for node modelling
when a regexp is provided in a node description. For instance
'\d{n,m}'
do not create multipleINT_str()
but only one
-
-
Parameter
@default
is now added to all typed nodes:String
,INT
, andBitField
have now a@default
parameter and a dedicated
method.set_default_value()
- new model description keyword added
default
- absorption operation is now changing the default value to what is absorbed
-
New description parameters have been added to have more
information when displaying Nodes (description parameters in
Node
,String
,Bitfield
,Int
, ...) -
Add
DataModel.customize_node_backend()
method in order to change
default node customization for all the descriptors/atoms
registered in the DataModel. -
Improve performance of
Node.get_reachable_nodes()
when the@path_regexp
criteria is used -
Non-terminal node evolution:
-
Non-terminal node can now handle a default quantity for their subnodes.
A new keyworddefault_qty
has been added. -
Revamp of non-terminal node algorithm to generate the different
possible shapes from the model description (considering the new
default_qty
). Simpler and better performance (about cpu and
memory consumption). -
The generation algorithm of the different shapes are now
customizable, through the parameterFullCombinatory
(either
full-combinatory or limited combinatory similar as before). -
A new customization parameter
CycleClone
allows to cycle among
the possible values of the subnodes which are duplicated several
times. -
A new parameter
@full_combinatory
has been added totWALK
and
tTYPE
in order to take benefit from this change. -
Add the method
Node.add()
for non-terminal nodes enabling to add a
node within an existing non-terminal node
-
-
Bitfield:
BitField.*_subfield()
methods can now reference subfields by description- Enhance
Bitfield Collapsing
operation
-
String:
String._populate_values()
gets called upon reset now when
determinist
is False in order to avoid getting same test case too
often- new
@case_sensitve
parameter with new related fuzzing cases - absorption operation leverages new
AbsCsts.SimilarContent
criteria
as a way to distinguish case-sensitive from case-insensitive String. RawCondition()
has now a new param @case_sensitive
-
Filename()
fuzzing cases revamped
-
-
Generators and Disruptors
-
tWALK
andtTYPE
leverage new framework features through new parameters -
Leverage the attribute Mutable of non-terminal node to customize
the behavior of theModelWalker
algorithm (used bytTYPE
and
tWALK
). If not mutable, then the algo will always stick to the
default form of the non-terminal. -
When a data model uses some existence conditions, walking through
the graph can be tricky. As a result, some data compliant with the
data model could never be generated bytWALK
. The walking
algorithm has been improved in order to take into account the
changes that may occur among the sibbling nodes of the node which
is currently walking through. And in the case there is some
change, then the algorithm will walk through the new nodes. This
improves the exploration capability oftWALK
. -
New parameters have been added to the Generators
automatically created from data models -
New Generator
GENP
to generate basic data based on a pattern and
different parameters.
-
-
Targets, Backends
-
TestTarget
provides several new features that allows to create
simulated environment composed of different interconnected
targets (simulated by different fuddly instance). -
Add new
SSHTarget
-
-
Monitoring:
- Add new
ProbeCmd
that enables you to execute shell commands and retrieve the output.
- Add new
-
Knowledge Infrastructure
-
FeedbackHandlers:
-
there are now triggered when a call to
Target.send_data_sync()
is performed asynchronously from outside the framework (e.g., from a Task) -
notify_data_sending()
is provided now with the relevant Target
in@target parameter
and not with the list of targets
-
-
the new subdirectory
user_info/
has been added in fuddly data folder
(automatically created if not existing). It is used to store user
specificInformation
(defined in the form of classes inheriting
theInformation
class) in order for instance to share information
between user projects and user data models.
-
Fuddly Environment Change
-
Add XDG Base Directory support. Note that if
~/fuddly_data
still exists,
XDG base directories won't be used. -
Due to some change in fmkDB structure, previous fmkDB versions are
not recognized anymore. Remove it from the Fuddly home so that it will be regenerated
when Fuddly will be started again. -
Fuddly configuration files have been updated. Fuddly won't start with previous ones.
Remove them and Fuddly will generate the new versions when started again.
API Change
-
Handle feedback timeout at Target level
-
FmkPlumbing:
- rename
set_fuzz_delay()
toset_sending_delay()
- rename
set_fuzz_burst()
toset_sending_burst_counter()
- dissociate the states "target_ready" and "feedback_received" in
FmkPlumbing.wait_for_target_readiness
(previously.check_target_readiness
) - feedback timeout is now computed in a more relevant way
Logger()
@console_display_limit
parameter is renamed@term_display_limit
- rename
-
Target:
- rename
collect_pending_feedback()
tocollect_unsolicited_feedback()
- rename
-
Data Model:
-
rename
DataModel.register_atom_for_absorption()
to.register_atom_for_decoding()
and change@decoding_scope
usage -
add new parameter
@scope
toDataModel.decode()
-
rename
AbsCsts
parameter@contents
to@content
-
Node
class:-
Node
dictionary interface update (refer to the documentation) -
Node
searching primitives could have side effects, namely freezing part of the node graph in the process.
This was caused by the resolution of Generator-nodes taking nodes parameter (if present in the graph), as
in the process of resolving/expanding the Generator-nodes the parameters are also resolved thus frozen.
To have control on this side effect a new parameter has been added to the node searching primitives:resolve_generator
.
It defaults to False in order to avoid side effect, but it will also affect searches as nothing will be
retrieved by default from the generator nodes. -
get_node_by_path()
is removed and replaced by:get_first_node_by_path()
anditer_nodes_by_path()
.
-
-
-
Generators and Disruptors
- Some parameters have been renamed as well as default values in
tWALK
,tTYPE
,tALT
,tSTRUCT
,tSEP
.
- Some parameters have been renamed as well as default values in
-
Evolutionary fuzzing (refer to the documentation)
- Remove disruptors dependency to perform crossover algorithm
- parameter
@init_process
ofDefaultPopulation
expects aDataProcess()
now