0.83.0

0.83.0, September 28th 2018

Core:

• Security

  • Updated the default and allowed TLS configurations (#254).

    Use TLS v1.2 (the latest supported version in Golang now) and limited cipher suite recommended by Internet Assigned Numbers Authority and OWASP for most transport connections.

    Use TLS v1.0 and extended cipher suit for MySQL TLS connections due to the limited support of MySQL database and drivers.

  • Improved security of transport connection between Acra's services by validating the clientId length. This decreases the chance of misusing the clientId (#253).

• Key management – key rotation

  • Added AcraRotate utility for rotation of Zone keys and re-encryption of AcraStructs. AcraRotate generates a new Zone keypair (zoneid_zone.pub and zoneid_zone) for a particular ZoneId and re-encrypts the corresponding AcraStructs with new keys. ZoneId stays the same (#256, #239).

    AcraRotate doesn't affect the ACRA_MASTER_KEY or storage keypair used without Zones (clientid_storage.pub / clientid_storage keys).

    AcraRotate rotates only the Zone storage keys and affects only the AcraStructs encrypted with Zones.

    AcraRotate works with AcraStructs stored both in database cells (MySQL or PostgreSQL) or files.

    Read the full documentation on AcraRotate on the Documentation Server.

• AcraCensor – SQL filter and firewall

  • Improved SQL filtering through more complex pattern matching (#264, #263, #262, #238).

    • %%VALUE%% pattern represents literal value (string, binary, number, boolean) and is supported in the following expressions: WHERE, IN, ORDER BY, GROUP BY, BETWEEN.
    • %%LIST_OF_VALUES%% pattern represents several values one by one, used with IN expressions.
    • %%SUBQUERY%% pattern represents a subquery expression inside the main query.
    • %%WHERE%% pattern represents one or more expressions after a WHERE statement. This pattern works for SELECT/UPDATE/DELETE queries.
    • %%COLUMN%% pattern represents a column expression used after SELECT and ORDER BY expressions.
    • %%SELECT%% pattern represents a whole SELECT expression.

  Read the detailed description and usage examples on the AcraCensor page on DocServer.

• AcraWriter

  • Added Java/Android AcraWriter library, added examples and tests (#252).

    Read the usage guide and examples in examples/android_java folder.

  • Added SQLAlchemy type wrappers for the Python AcraWriter (#257).

  • Improved and refactored the Python AcraWriter example of encrypting data and reading it from the database (#258).

• Prometheus Metrics

  • Added functionality for exporting the basic metrics of AcraServer, AcraConnector, and AcraTranslator to Prometheus: if incoming_connection_prometheus_metrics_string is set, the service will generate specific metrics (time of connection life, time of processing requests, AcraStruct decryption counters) and push them to Prometheus (#260, #251, #234).

• Other

  • Improved AcraConnector's compatibility with PostgreSQL: AcraConnector now correctly handles the database's denial to use TLS connection (#259).

  • Added export of CLI parameters for AcraServer, AcraConnector, and AcraTranslator to markdown (#261).

  • Improved readability of CEF-formatted logs by sorting extension fields in alphabetical order (#255).

  • Improved quality of our codebase — cleaned up the old unnecessary code (#250).

Infrastructure:

• Added AcraRotate as a ready-to-use tool inside AcraTranslator and AcraServer Docker containers (#236).

Documentation:

• Made the Documentation Server the primary and the only regularly updated source of documentation for Acra. The most recent version of the documentation, tutorials, and demos for Acra can be found there. The GitHub Wiki documentation for Acra is still available, but is no longer updated starting with the version 0.82.0 (with the exception of Changelog and README files with every new version release).

• AcraCensor: updated the details on how the "patterns" filter works.

• AcraRotate: added a tutorial for using AcraRotate to rotate Zone keys and re-encrypt the data.

• Tons of small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;).

0.82.0

Core:

• AcraTranslator

  AcraTranslator is a lightweight server that receives AcraStructs and returns the decrypted data. AcraTranslator doesn’t care about the source of the data, it accepts AcraStructs via HTTP or gRPC API. An application can store AcraStructs anywhere it is convenient: as cells in the database, as files in the file storage (local or cloud storage, like AWS S3). An application sends AcraStructs as binary data and receives plaintext (or decryption error) from AcraTranslator.

  However, sending plaintext data over a non-secure channel is a bad idea, so AcraTranslator requires the use of Themis Secure Session encryption channel (which is basically an encrypted TCP/unix sockets). To establish a Secure Session connection, an application doesn’t need to contain the crypto-code itself, only to direct the traffic through AcraConnector instead.

  AcraTranslator supports AcraStructs via HTTP and gRPC API, uses in-memory LRU cache to store encryption keys, and detects poison records. AcraTranslator is shipped as a pre-built binary and as a Docker container.

  Read the detailed guide on how to install, configure, and run AcraTranslator.

(#213, #212, #207, #205, #204, #203, #200, #199, #198, #197)

• Updated AcraConnector to support connections with AcraTranslator (#206).

• Logging

  • Improved startup logs for AcraServer, AcraTranslator, and AcraConnector: now it's easier to understand that the service is up and running (#242).

  • Added clientId to AcraServer logs: now it's easier to understand which client was sending the request that led to a failed or successful AcraStruct decryption (#214).

  • Improved logging by masking query parameters: neither AcraServer nor AcraCensor won't leak sensitive query while logging the content (#216, #211).

• Poison records

  • Poison record detection for AcraServer and AcraTranslator can now be turned on and off. Poison records detection is ON by default: AcraServer/AcraTranslator will try to detect poison record and log to stderr if a poison record is detected (#240, #230, #215).

  • Increased performance of AcraServer/AcraTranslator if poison records are enabled (#232).

• Key management

  • Improved processing of decryption keys for AcraServer, AcraTranslator, and AcraConnector: now it is possible to store private keys encrypted in files and in memory, decrypt them before using, and purge after usage (#202).

  • Added configurable LRU cache for the in-memory keys: this will increase the performance if you operate with hundreds of storage keys on AcraServer and AcraTranslator (#219).

• AcraCensor / SQL filter and firewall

  • Improved SQL filtering by adding pattern matching: now you can blacklist or whitelist queries that match particular patterns, like SELECT %%COLUMN%% FROM company %%WHERE%%.

  Currently supported patterns: %%VALUE%%, %%COLUMN%%, %%WHERE%% and %%SELECT%%.

  Read the detailed description and usage examples on the AcraCensor page.

  (#248, #247, #246, #245, #243, #238, #231, #226, #217)

  • Improved AcraCensor performance for queries matching (#208).

• AcraWriter

  • Added iOS/Objective-C AcraWriter library as CocoaPod, added examples and tests.

  Read the usage guide and examples in examples/objc folder.

  (#241, #235, #233).

  • Improved security of AcraWriter libs for Go, Ruby, Nodejs, and iOS through zeroing secret keys where it was missing (#244).

• AcraRollback

  • Improved handling of insert query parameter to simplify using AcraRollback from bash (#210).

• Other

  • Improved AcraStruct decryption by multiple validations of AcraStruct format before decrypting. This fix improves AcraServer/AcraTranslator error messages in case of a failed decryption (#201).

  • Improved stability of integration test suite, trying to avoid 'timed out' errors from CircleCI (#200).

  • Improved code quality, fixing gofmt and golint issues (#229, #228, #227, #224, #223, #221).

• WIP

  • Adding a way to export decryption metrics (decryption time, number of connections) from AcraServer to Prometeus (#234).

  • Prototyping AcraRotate tool for rotating the Zone keys easily (#239).

Infrastructure:

• Drop testing and supporting go version below 1.8. This will only affect the users who build Acra manually from sources.
  You can install pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder
  (#209).

• Added AcraTranslator as pre-built binary and docker container (#222).

• Added AcraTranslator and AcraConnector docker-compose files: now it's easy to set up a demo stand just by running one command (#225).

• Added AcraRollback and AcraPoisonRecordMaker as ready-to-use tools inside AcraTranslator and AcraServer Docker containers (#236).

Documentation:

• Key management: clarified key names and default locations, illustrated public key exchange in details.

• AcraServer: improved examples of how to run AcraServer.

• AcraTranslator: added description of the service, installation and launching guide, added ready-to-use examples for HTTP and gRPC API.

• AcraConnector: added examples of how to run AcraConnector with AcraTranslator.

• AcraCensor: added examples of the configuration file, described in details how the "patterns" filter works.

• AcraRollback: added examples of running AcraRollback in local mode, which allows storing the decrypted data locally, without pushing it back to the database.

• This is the last version of Acra for which the main documentation will be actively updated in the GitHub Wiki. From now on, the most recent version of the documentation, tutorials, and demos for Acra will be available in the official Cossack Labs Documentation Server.

0.81.0

Core:

• Prepared Statements

  • Added support of prepared statements for PostgreSQL/MySQL. Both binary and text response formats are supported (#192).

• SQL requests filtering in AcraCensor

  AcraCensor got smarter in preventing SQL Injections.

  • Improved flexibility for parsing queries. If AcraCensor can't parse an SQL query, it is considered as potentially too dangerous to send it to the database, so AcraCensor blocks such "unparseable" queries by default.
    However, setting the configuration flag ignore_parse_error to true will make AcraCensor ignore the "unparseable" quality of queries and send them to the database anyway. Check out the configuration example in configs/acra-censor.example.yaml (#194).

  • Added support of complex JOIN queries (#191).

  • Improved reading/writing QueryCapture log file. Now AcraCensor uses bufferisation before writing queries into a log file.
    Changed format of QueryCapture log to JSON Line (each query sits in a separate line in a log file instead of having an array of JSON objects) (#193).

  • Introduced a few fixes here and there, made integration tests for AcraCensor more stable (#184).

• Improving MySQL support

  We introduced MySQL support just a few Acra releases ago and we continue polishing it. Now we've updated the example projects so you can jump right into the code!

  Take a look at how to use Acra for both PostreSQL and MySQL databases in these examples:

  • Go: see the examples/golang folder (#190).

  • Ruby: see the examples/ruby folder (#189).

  • Python: see the examples/python folder (#188).

• Other

  • Updated handling of message formats for PostgreSQL and MySQL protocols (#186).

  • Improved logging in CEF and JSON formats for high load systems (#195).

  • Added comprehensive Readme to every project in /examples folder (#196).

  • Added pre-generated configuration file for AcraAuthmanager. Now it's easier to configure AcraServer using AcraWebconfig (#187).

Documentation:

• Updated Acra Architecture and Data flow graphic schemes to better illustrate Acra's components, connections between them, and typical use-cases.
• Updated AcraCensor's description to explain how unparseable queries are handled.
• Described typical Public Key Infrastructure with some advice on where to put Acra in the general scheme of things.
• Described Acra's Security Model, possible threats, and possible consequences of compromisation.
• Added a page describing the ways Acra can help you better comply with GDPR.

0.80.0

0.80.0, May 31st 2018

Core:

• Renaming

  • Global renaming of Acra components and their configuration parameters.
    We believe that the updated naming will decrease confusion about the components' functions and will make Acra's setup and usage process easier.

  Main services:

  Old name	New name	Function
  AcraServer	AcraServer	decrypts data from the database
  AcraWriter	AcraWriter	encrypts data on the client side
  AcraProxy	AcraConnector	encrypts traffic between the client and the server using Themis Secure Session
  AcraCensor	AcraCensor	firewall, part of AcraServer, blocks suspicious SQL requests to the database
  AcraConfigUI	AcraWebConfig	lightweight HTTP web server for managing AcraServer's certain configuration options

  Utilities:

  Old name	New name	Function
  acra_rollback	AcraRollback	decrypts the whole database
  acra_genkeys	AcraKeymaker	generates encryption keys for storage and transport of the Acra components
  acra_genauth	AcraAuthmanager	generates user accounts for AcraWebConfig
  acra_genpoisonrecord	AcraPoisonRecordMaker	generates poision records for databases
  acra_addzone	AcraAddzone	generates Zones' header for AcraWriter

  Check the configurations of components inside /configs folder and read Migration Guide for more details (#175, #174, #173, #170, #169, #168).

• SSL/TLS

  • Improved SSL/TLS connections between AcraServer<->AcraConnector and AcraServer<->database. Added TLS authentication mode (tls_auth) argument to the AcraServer/AcraConnector configuration files:
    • for AcraConnector it indicates how to authenticate AcraServer during a TLS connection;
    • for AcraServer it indicates how to authenticate database during a TLS connection.
  • Updated TLS configuration to provide other less strict authentication methods (do not authenticate client from server, ask for any certificate, ask and check) (#171).

• SQL requests filtering

  • Added support of filtering SQL requests for PostgreSQL databases. Now you can setup AcraCensor rules for both MySQL and PostgreSQL databases (#177).

  • Improved QueryCapture: AcraCensor writes allowed/blocked queries into a separate log file without blocking the main process (#176, #172).

  See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

• AcraWriter in Ruby

  • Updated AcraWriter Ruby wrapper for ActiveRecord tutorial and pushed a new gem (#166).

• Key Handling

  • Added make keys target in the Makefile: one command now generates keys and places them into correct folders for all Acra components (#182, #181).
  • Improved handling of master key length longer than 32 bytes (#183).

• Other

  • Updated notification when AcraConnector is launched in an environment without netstat (#167).
  • Updated error handling for AcraServer working with Zones and fix some corner-cases in using PostgreSQL protocol (#186, #179).

Infrastructure:

• Even better Docker support

  • Added more ready-to-use Docker Containers: acra-keymaker, acra-authmanager. As a result, each Acra component is wrapped into a Docker container, allowing you to try Acra into your infrastructures easily.

  • Added easy-to-use docker-compose files for setting up the whole Acra-based environment connected to MySQL database. Possible configurations include setup with/without SSL, with/without AcraConnector, with/without Zones (#180).
    Check out the instructions and examples in the /docker folder: we have examples for both MySQL and PostgreSQL databases.

  • Updated descriptions for official Cossack Labs packages on Docker Hub.

  • Updated Getting started with Docker guide to make starting out with Acra even easier.

• OS

  • Added support of Ubuntu Xenial, Ubuntu Bionic (added precompiled binaries and tests to make sure that Acra is compiling/building/working well on 16.04/18.04).

Documentation:

• Updated tutorials about protecting a Ruby on Rails app and a Django app.
• Every single document, code line, and image are updated using the new naming.

0.77.0

Core:

• MySQL databases

  • Added support for MySQL: now you can connect Acra to MySQL databases. Works with any SSL mode: require, allow, disable.
  • Tested and supported on: MySQL (#155, #140).

  Note: Prepared statements are not supported yet, but this feature is coming soon!

  Read about the new configurations on the AcraServer documentation page.

• Keeping keys in secret

  • Added encryption for the keys' folder: private keys are now symmetrically encrypted by master_key (#143) for storage.
  • Added ability to generate public/private keys in the separate folders (#148, #142).

  Read more about the current changes in key management here.

• Filtering requests for MySQL

  • Added firewall component named AcraCensor to handle MySQL queries.
    
    You can provide a list of restricted or allowed tables, columns, and exact queries to handle. AcraCensor will pass the allowed queries and return error on forbidden ones. Rules are configured and stored in yaml file. Each request is logged in real time. Moreover, all the queries and their states are logged into a separate log file. (#151, #138, #136, #132, #125, #108).
    

  See a detailed description of AcraCensor on the corresponding AcraCensor documentation page.

• Web Config UI

  • Added lightweight HTTP web server for managing AcraServer's certain configuration options.
    
    You can update the proxy address and port, database address and port, handling of Zone mode and poison records. On saving new configuration, acraserver will gracefully restart and use these settings automatically. The access to thiw web page is restricted using basic auth. (#153, #141, #123, #111).
    

  See the interface screenshot and detailed instructions at Acra Config UI page.

• Logging

  • Added support of new logging formats: plaintext, CEF, and json.
    
    Logging mode and verbosity level is configured for AcraServer, AcraProxy, and AcraConfigUI in the corresponding yaml files. Log messages were slightly improved, custom error codes were added (which we believe will help to understand and debug any issues) (#135, #126, #110).

  Read more about the log analysis at Logging page.

• Tests

  • Added many new integartion tests, fixed stability and handling of more complicated use-cases (#150, #147, #137, #117, #116, #115).

Infrastructure:

• Docker support

  • Added Docker Container for every main component: AcraServer, AcraProxy, AcraConfigUI, and key generators (AcraGenKeys and AcraGenAuth). You can find the containers in /docker folder or on the Docker Hub (#139).

  • Updated Getting started with Docker guide to make starting out with Acra even easier.

  • Added easy-to-use docker-compose files to launch Acra in different environments, including key distribution. Possible configurations are:

    • acraserver + acra_configui ;
    • connecting to PostreSQL or MySQL databases;
    • using Secure Session or SSL as transport encryption;
    • with or without acraproxy;
    • with or without zones.
      

    This is huge! We encourage you to try it! Check out the instructions and examples in the /docker folder. (#154, #146, #134, #133, #102).

• Go versions

  • Updated the list of supported versions of Go. Every Acra component can now be built using Go >1.7, except acra_rollback that requires Go >1.8. No worries, you can still download Acra as a binary package anyway :)

• OS

  • Dropped support of Debian Wheezy (no autotests, no precompiled binaries now).

Documentation:

• Updated QuickStart documentation about launching and building Acra components.
• Described how to setup AcraCensor and AcraConfigUI.
• Added more details and described new options (like using TLS and connecting to MySQL databases) for AcraServer and AcraProxy.
• Described new logging formats.
• Updated description of Key management approach we encourage you to use.
• Described Docker components and ready-to-use Docker Compose configurations based on the Docker Readme.
• Updated Getting started with Docker guide.
• Distributed the information about master key across the docs.
• Many small improvements.

0.76

Core:

• SSL / TLS support

Now you can use PostgeSQL with SSL/TLS settings enabled. Acra supports two modes of connection between AcraServer and the database: using SSL/TLS or using Secure Session (#113, #119).

• Unix sockets

Acra now supports usage of both TCP and Unix Sockets as a connection layer between AcraWriter <-> AcraProxy <-> AcraServer.

• Tests

  • Updated integration test suit to support multiple connection modes between the Acra components and the database (#115, #117, #118, #120).
  • Added Docker image to make testing easier (#104).

Infrastructure:

• Added support of Go 1.10, removed support of older Go versions (<1.6).
• Added support of Ubuntu 17.10, Ubuntu 16.04, Ubuntu 14.04, Debian Stretch.
• Updated dependency libraries (libthemis and libcrypto) to use the latest ones.

Documentation:

• Updated the documentation and tutorials to reflect the latest changes.

0.75

This is the initial public release of Acra, a database protection suite.

This version of Acra:

• works on Ubuntu, CentOS, Debian linuxes
• supports PostgreSQL 9.4+
• has AcraWriter packages for Python, PHP, Go and NodeJS