phpMussel v0.5-r0.5.1

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version 0.5 / Release 0.5.1 (Previous Release: 0.5; Next: 0.5a) ===
SUMMARY:
Non-ClamAV signatures update only (55+19429+5.1).
No script changes (high or medium priority).

NEW CHANGES:

• For reasons that I do not know, it appears that the mirror for the ClamAV
  database from which I usually obtain the daily signature updates is currently
  hosting an outdated copy of the daily signatures from a few months back,
  which is older than the signatures already implemented to phpMussel.
  Hopefully this problem will be resolved prior to my intended date for the
  next release after this one, but for this release, at least, I'm not going to
  make any changes to phpMussel's ClamAV signature files. That said, although
  there hasn't been any changes to the core code for phpMussel nor to the
  ClamAV signature files since the previous release, I have written some new
  signatures based on my own research that may be of extreme interest to a
  select few of phpMussel's users, due to what these signatures are intended to
  protect against, which I think warrants the existence of this current
  release. These new signatures primarily concern PHP-specific shell hacks,
  backdoors and several website defacement tools and related indicators.

This release should be considered HIGH PRIORITY if:

• You are allowing the upload of script-based files such as PHP and HTML files,
  and/or if your phpmussel.ini does not explicitly deny them (this includes if
  you have a website that makes use of some sort of web-based self-update
  functionality or similar).
• You do not have any form of file upload sanitization in place or if phpMussel
  is the only defence against dangerous file uploads that you are using.
• Any chameleon attack detection directives are disabled at all.

Otherwise, this release should be considered MEDIUM PRIORITY.

This release is available both as a FULL version (containing all phpMussel
files, as per the norm) and as a CHANGED FILES version (containing only those
files modified since the previous release).

Maikuolan,
5th October 2014.

phpMussel v0.4d

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.4d (Previous Release: 0.4c; Next: 0.5) ===
SUMMARY:
Updated to the latest signatures set (55+19354+4d).
Sub-minor release (medium priority).

NEW CHANGES:

• Modified code for checking against graphics signatures as so that phpMussel
  now checks signatures against both normalised and verbatim content (whereas
  previously, it only checked against verbatim content) in order to harden
  defences against a few particularly nasty known threats.
• Added the ability for phpMussel to detect the presence of and decode Base64,
  GZ, ROT13 and Hex2Bin decode commands within file uploads, thus allowing
  phpMussel to understand and scan such encoded data (when detected and
  decoded correctly). This has allowed me to write a few additional signatures
  for improved defences against known threats.
• Changed the way in which phpMussel reads the actual raw content of files in
  the hopes of improving performance and speed (this has resulted in the
  addition of a new function within the Core Script). Comparing the before and
  the after of this change, I have noticed a slight improvement, but it isn't
  significant and doesn't entirely satisfy me yet. Will likely try to develop
  this further throughout some consequent releases.
• Added the ability for phpMussel to optionally check against custom signatures
  based upon the values of variables available to the scope of the core
  scanning function of the Core Script. This ability functions similarly to how
  other conditional signature checks function.
• Added the ability to whitelist specific files via MD5 hash and filesize, in a
  similar manner to how the MD5 and PE Sectional scanning functions. With this,
  the whitelist entries included with the ClamAV signatures are now supported.
  However, as most of the entries included with the ClamAV signatures are very
  vaguely described and it appeared difficult to determine what most of those
  whitelist entries were actually for, only a very small handful of them have
  been included with phpMussel (those that I could adequately determine what
  they were most likely for and for which I was adequately confident would not
  provide any potential weaknesses to the defences provided by phpMussel).
  Refer to the README documentation for more details on how this file
  whitelisting ability works.
• Numerous other minor changes to the Core Script (phpmussel.inc).
• A few additional directives added to the phpMussel configuration.
• decode_threshold: Optional limitation to the length of raw data within which
  decode commands should be detected (in case there are any noticeable
  performance issues whilst scanning).
• scannable_threshold: Optional limitation to the length of raw data to which
  phpMussel is permitted to read and scan (in case there are any noticeable
  performance issues whilst scanning).
• Directives pertaining to the new whitelisting ability.
• Shifted a small handful of phpMussel signatures from the General signatures
  files to the ASCII signatures files for improved detection.
• All documentation updated to reflect the changes brought about as of this
  version of phpMussel.

PROBLEMS/BUGS FIXED:

• Corrected a bug whereby scanning for a specific file via CLI mode would
  trigger the false-positive, "phpMussel-FN.Illegal.Character-5C", due to the
  inherent presence of forward-slashes in the full-path.
• A minor translation correction done to the Language Data; PT (lang.inc).

Maikuolan,
11th September 2014.

phpMussel v0.4c

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.4c (Previous Release: 0.4b; Next: 0.4d) ===
SUMMARY:
Updated to the latest signatures set (55+19312+4c).
Sub-minor release (medium priority).

NEW CHANGES:

• Partially rewrote the code for checking against archive metadata signatures,
  with which, as opposed to the previous release of that code whereby only
  the formats GZ and ZIP were supported, now, it additionally supports the
  formats BZ and LZF (although, certain PECL extensions of php will be required
  in order for that additional supported to be functional; absence of those
  PECL extensions will prevent that additional support from being functional,
  though, such absences will have no adverse effect otherwise and phpMussel
  will continue to function normally regardlessly).
• Minor non-significant code optimisations of the Loader (phpmussel.php).
• Although the case is much the same as is with the implementation of support
  for the normalised ASCII signatures introduced in the previous release, for
  this release, I've implemented partial support for the normalised HTML
  signatures of ClamAV. No changes have been made for this release to the
  actual normalisation method I coded for the previous release, though, owing
  to some minor rethinking and improving of my actual method of preparing
  signatures from the ClamAV database for inclusion with phpMussel, the actual
  total number of signatures between the previous release and this release has
  increased significantly, from the previous ~150 signatures to the current
  ~1,300 for normalised ASCII signatures and to ~1,900 for the normalised HTML
  signatures.

Maikuolan,
28th August 2014.

phpMussel v0.4b

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.4b (Previous Release: 0.4a; Next: 0.4c) ===
SUMMARY:
Updated to the latest signatures set (55+19279+4b).
Sub-minor release (medium priority).

NEW CHANGES:

• Many thanks to BlueEyed Zebra, who has successfully translated to German the
  entirety of the phpMussel documentation and the phpMussel internal language
  data, with which, phpMussel is now fully supported in the German language.
  With the addition of German, phpMussel now -fully- supports five languages
  (English, French, Indonesian, Italian, German) and -partially- supports an
  additional three (Spanish, Portuguese, Dutch; All three, internal language
  data has been fully translated but translation of the documentation is very
  incomplete).
  Refer spambotsecurity.com/forum/viewtopic.php?f=57&t=2929 (2014.08.05).
• I've -began- working on support for the normalised ASCII signatures, and what
  I've coded thus far is working correctly, though I wouldn't call it anywhere
  near complete yet. Currently, the normalisation method that I've written
  isn't entirely identical to that which is employed by ClamAV (and thus that
  which the normalised ASCII signatures of ClamAV are written in mind with),
  and ideally, at the least, what I write should eventually either mimic or
  improve on that which is employed by ClamAV. That said, I haven't been
  working on this part of phpMussel for a particularly long time yet and it's
  certain to improve with time, and as is, still does a reasonable job. Not
  all of the relevant signatures match in the same way between ClamAV and
  phpMussel yet, but those that do (and thus, which should correctly match
  during scans by phpMussel without false positives) have been included
  with this release (which works out to roughly ~150 signatures of the total
  ~3K±K or so normalised ASCII signatures available from ClamAV).
  Part of the reason that I'm including it with this release, as opposed to
  waiting until I've got it 100% implemented in the way that I'm aiming for,
  is that there are some critical common CMS vulnerabilities that can be
  protected against by inclusion of a number of signatures that, at the least,
  require the level of implementation that I've currently achieved, but which
  I don't think I should hold back on for some unknown future date, thus,
  releasing where it's currently at. In any case, it shouldn't cause problems.
• Minor non-significant code optimisations of the Core Script (phpmussel.inc).
  No changes to the Language Data (lang.inc) or Update Script (update.inc)
  exist between this and the previous version.

Maikuolan,
13th August 2014.

phpMussel v0.4a

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.4a (Previous Release: 0.4; Next: 0.4b) ===
SUMMARY:
Updated to the latest signatures set (55+19260+4a).
Sub-minor release (medium priority).

NEW CHANGES:

• General commands CSV tweaked slightly (hex_general_commands.csv).
• As usual, improved signature set.

PROBLEMS/BUGS FIXED:

• Missing break statement from end of looped section in update script would
  cause forced update via CLI to infinitely loop (update.inc); Fixed.
• Corrected some minor spelling errors in the language file (lang.inc).

Maikuolan,
4th August 2014.

phpMussel v0.3g.1

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.3g.1 (Previous Release: 0.3g; Next: 0.4) ===
SUMMARY:
Updated to the latest signatures set (55+19131+3g.1).
Files modified, but no changes to the actual script or code (medium priority).

NEW CHANGES:

• Updated -all- documentation - phpMussel is now -fully- supported (I define
  a language as being "fully supported" by phpMussel when translations for
  that language have been completed in the language file, "lang.inc", and when
  there is a complete, thorough translation of the README documentation in that
  language available) in four languages: English, French, Indonesian, Italian;
  Partially supported by an additional four languages (German, Spanish,
  Portuguese, Dutch), which should come to be fully supported as time passes,
  with additional languages to eventually come where time, energy, capacity,
  need and interest from the userbase permits it to be so.

PROBLEMS/BUGS FIXED:

• Found some incorrect peripheral information in the documentation; Corrected.
• Corrected some minor spelling errors in the language file (lang.inc).
• Removed some problematic false positives from the signature set, most
  notably, "phpMussel-FN.Illegal.Character-3F", which was originally introduced
  with the previously most recent release (v0.3g), potentially falsing where
  due to a discrepancy between the actual default character encoding
  ("charset") of the upload as per assigned by the sending browser and the
  actual charset of the system where PHP is installed, the filename would be
  mangled, with question marks rendered in place of unrecognised characters.
  Refer spambotsecurity.com/forum/viewtopic.php?f=57&t=2824

Maikuolan,
26th June 2014.

phpMussel v0.3g

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version/Release 0.3g (Previous Release: 0.3f.2; Next: 0.3g.1) ===
SUMMARY:
Updated to the latest signatures set (55+19079+3g).
Sub-minor release (medium priority).

NEW CHANGES:

• phpMussel now displayed as process title when in CLI mode (PHP 5 >= 5.5.0).
• Big changes across the language file, the core script and the update script.
• As usual, improved signature set (a few potential false positives removed,
  better protection against filename manipulation, new set of MD5s, new round
  of phishes, a number of others; also see changes noted under BUGS); Slightly
  improved signature mapping algorithm.
• Rewrote code for data normalisation and rewrote how encoding+decoding is
  handled by phpMussel (should now be a bit more thorough and bit more
  consistent throughout the different parts of the script).
• As compensation for my no longer including signatures with zero-or-more
  quantifiers in the default signature set (the zero-or-more quantifier -is-
  still supported by phpMussel, so, you can still use it, if you want, although
  I don't recommend it unless absolutely necessary in-lieu of the PCRE bug),
  I've introduced support for an equivalent into the code for handling the
  standard, non-regex based signatures. Now, if you include an ">" within a
  standard, non-regex signature, phpMussel will first match against everything
  before the immediate ">", and then, if a match is found, skip ahead in the
  data being checked to the point where the match was found and reinterpret
  the signature from the point after the immediate ">" (thus, repeat);
  Functionally equivalent to a zero-or-more match (in most, but not all, cases)
  but without the inherent problem of potentially backticking into oblivion.
• phpMussel now includes CRC32 checksums for the names and contents of scanned
  files (regardless of whether anything was detected or not) in the scan_log
  file. This could be useful for tracking down files that may be difficult to
  locate after having been uploaded (such as, for example, when trying to
  locate files containing something undesirable that may have been missed by
  phpMussel when scanned).
• A few default directives in the phpMussel configuration altered slightly
  to reflect recent changes made to the core script. Nothing significant.

PROBLEMS/BUGS FIXED:

• Not really a -bug-, per se, but, found a few areas in the core script where
  there was some level of duplication and redundancy. Fixed.
• Took me a while, but I finally managed to get to the bottom of the PCRE bug
  (the one that I'd mentioned earlier in the change_log whereby certain
  signatures would occasionally cause PHP to crash), work out exactly what was
  triggering it and find a proper work-around for it. This release and all
  future releases, assuming I've done it correctly, should no longer contain
  ".risky" files, as the risk of PHP crashing in this particular manner should
  no longer be a problem for these future releases. The trigger was phpMussel
  trying to match signatures containing zero-or-more quantifiers () against
  non-matching files above certain system dependent filesize values leading to
  rampant runaway backticking. In short, via a lot of testing and debugging..
  Although I already knew it was -possible- to do this, I never realised quite
  just how easy it was before, but, I've discovered just how easy it is, in
  fact, when dealing with PCRE functions, to cause PCRE to make PHP backtick
  itself into oblivion by way of throwing sufficiently large non-matching data
  at those functions. As a result of this, 12 (of the some 40,000+) of the
  currently active signatures in use by phpMussel as per supplied by ClamAV
  have been redacted, and 245 signatures have been modified, replacing the
  zero-or-more quantifiers () with specific numeric quantifiers ({n,n}),
  and changes have been made to the core script in an attempt to prevent the
  situation from arising again.
  Refer spambotsecurity.com/forum/viewtopic.php?f=58&t=2794
• There was a minor bug in CLI mode whereby specifying directories to scan
  without including a trailing slash in the directory path would result in
  phpMussel stating that the directory didn't exist, regardless of whether it
  actually existed or not. Fixed; Directories are now interpret the same,
  regardless of whether there is an included trailing slash or not.

Maikuolan,
11th June 2014.

phpMussel v0.3f

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!

=== Version 0.3f (Previous: 0.3e; Next: 0.3f.1) ===
SUMMARY:
Updated to the latest signatures set (55+19012+3f).
Sub-minor release (medium priority).

NEW CHANGES:

• Added better error detection for file uploads.
• Continued work and improvement on multilingual support; Core script/program
  now supports eight languages (available languages listed in configuration);
  No new language support for documentation since previous version. Take in
  mind that, because my native language is English and I am not fluent with any
  other languages, there'll probably be the odd error here and there. I make no
  apologies for this, but endeavour to fix them whenever they become apparent.
• Undid some of the *_standard.cvd to *_regex.cvd swaps from the previous
  update for improved performance, manually removing any falsables I could
  find.
• Improved the default Upload Denied message template file slightly.
• Minor code tweaks for improved performance.

PROBLEMS/BUGS FIXED:

• A compatibility issue between phpMussel and several CMS and forum systems
  that handle $_FILES in unexpected ways having been brought to my attention,
  I've added a new compatibility section to the phpMussel configuration. As it
  stands at this time, that section only has one directive, which concerns the
  specific issue brought to attention, but will probably grow as time goes by.
  The issue brought to attention was that these CMS and forum systems would
  behave in ways that resulted in $_FILES array elements being parsed to
  phpMussel with the elements of these array elements, which should normally
  correspond to the details of specific files being uploaded, containing no
  data, causing phpMussel to attempt to scan uploaded files when no files
  were actually being uploaded, resulting in an "Unauthorised file manipulation
  detected!" message being returned to the client and the subsequent pageload
  being prevented. To work around this problem, the compatibility directive
  "ignore_upload_errors" has been implemented (by default, deactivated), which
  when activated in configuration should hopefully prevent this issue from
  arising.
  Refer spambotsecurity.com/forum/viewtopic.php?f=58&t=2786
• Failed condition matching whereby signature file instance should be
  terminated found to not always be triggering correctly. Fixed.

Maikuolan,
22nd May 2014.

phpMussel v0.3e

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!
Originally released "19th February 2014".
Originally released as "Version 0.3e".
Refer to "change_log.txt" for further information.

phpMussel v0.3d

HISTORICAL COMMIT! OUTDATED CONTENT; DO NOT DEPLOY!
Originally released "16th February 2014".
Originally released as "Version 0.3d".
Refer to "change_log.txt" for further information.