Skip to content

What's new in 0.35.0 for Metabase driver authors

Jump to bottom
Simon Belak edited this page Mar 27, 2020 · 8 revisions

A few things have changed in Metabase 0.35.0 for Metabase drivers. For JDBC-based drivers, you probably won't need to change anything. For other drivers, you'll need to implement a new method.

The easiest way to make sure your driver is compatible with Metabase 1.35.0 is to try it! If you're an Emacs user, load the source up in CIDER and see what gets highlighted red because it is deprecated and change as needed.

Changes are listed below:

ToC

Potentially Breaking Changes

execute-query replaced by execute-query-reducible

Metabase 0.35.0 features a new Query Processor implementation that streaming results directly from databases to the client, rather than keeping the entire result set in memory. In older versions of Metabase, drivers implemented execute-query to run queries and return results:

(defmethod driver/execute-query :my-driver
  [driver query]
  {:columns ["ID" "Name"]
   :rows    [[1 "Cam"]
             [2 "Lucky Bird"]]}

In Metabase 0.35.0, drivers instead must implement execute-query-reducible to return metadata and reducible result rows. An example implementation might look something like

(require '[metabase.query-processor.context :as context])

(defmethod reducible-query :my-driver
  [_ query context respond]
  (respond
   {:cols [{:name "ID", :base_type :type/Integer}
           {:name "Name", :base_type :type/Text}]}
   (get-reducible-rows ...)))

Your driver should return rows in a reducible format, i.e. something that implements IReduceInit. You can use the helper function reducible-rows:

(require '[metabase.query-processor.reducible :as qp.reducible])

(defmethod reducible-query :my-driver
  [_ query context respond]
  (letfn [(get-row-thunk []
            ...)]
    (respond
     {:cols [{:name \"my_col\"}]}
     (qp.reducible/reducible-rows get-row-thunk (context/canceled-chan context)))))

The respond function will fully reduce the result rows. Thus, you implementation should hold connections, cursors, ResultSets, or the like open for until all rows are reduced, and no longer:

(defmethod reducible-query :my-driver
  [_ query context respond]
  (with-open [results (run-query! query)]
    (letfn [(get-row-thunk []
              ...)]
      (respond
       {:cols [{:name \"my_col\"}]}
       (qp.reducible/reducible-rows get-row-thunk (context/canceled-chan context))))))
Note

The :sql-jdbc driver has a default implementation for execute-query-reducbile, so JDBC-based drivers won't need to make any changes unless they had a custom implementation of execute-query. :sql-jdbc-based drivers can also implement a handful of new methods, such as metabase.driver.sql-jdbc.execute/connection-with-timezone, to customize the :sql-jdbc driver implementation. See the sql-jdc.execute namespace for more details.

metabase.driver/process-query-in-context removed

Because drivers can now hold connections or cursors open as needed in execute-query-reducible, process-query-in-context is not longer needed and has been removed. Few drivers implemented this method, and those that did should migrate the logic to their implementation of execute-query-reducible

MBQL aggregation :stddev is now implemented as population standard deviation

MBQL aggregations and filters now take arbitrary expressions as arguments

For instance the following is now valid MBQL:

[:count-where [:between [:- [:round [:power [:field-id (data/id :venues :price)] 2]] 1] 1 5]]

Deprecated/renamed methods

metabase.driver.date-add is deprecated in favor of metabase.driver.sql.query-processor/add-interval-honeysql-form

The original method was moved and renamed in Metabase 0.34.2 to better clarify its intent and purpose. For the time being, Metabase will fall back to using the old method if an implementation is present, but the old method will be removed entirely in a future release. If you have implemented date-add, please change the name of the method implemented to add-interval-honeysql-form.

metabase.driver.sql.query-processor/current-datetime-fn is deprecated in favor of metabase.driver.sql.query-processor/current-datetime-honeysql-form

The original method was renamed in Metabase 0.34.2 to better clarify its intent. For the time being, Metabase will fall back to using the old method if an implementation is present, but the old method will be removed entirely in a future release. If you have implemented current-datetime-fn, please change the name of the method implemented to current-datetime-honeysql-form.

metabase.driver.sql-jdbc.execute/set-timezone-sql is now considered deprecated; new method metabase.driver.sql-jdbc.execute/connection-with-time-zone replaces it

connection-with-time-zone is now used to fetch a java.sql.Connection in the given time zone. The default implementation continues to use set-timezone-sql to se the time zone if this method is implemented. In the future, we plan to ask all drivers to return a Connection in the desired time zone in whatever way is easiest or most efficient for that driver, rather than the current one-size-fits-all approach.

Core Metabase drivers continue to use set-timezone-sql for the time being, so you need not rush out and change your implementation just yet. But for new implementations please implement connection-with-time-zone instead.

New features

New MBQL functions

General utilities

We added :case and :coalesce expressions. They both follow standard SQL definitions and semantics.

:coalesce needs only be implemented if the driver supports :expressions flag.

String extraction functions

We added the following string functions to custom expressions: :substring, :trim, :rtrim, :ltrim, :upper, :lower, :replace, :concat, :length, and :regex-match-first. They all follow standard SQL definitions and semantics. :trim, :rtrim, :ltrim only come in 1-airity.

These only need to be implemented if the driver supports :expressions flag. If the driver does not support regular expressions, you can set the new flag :regex to false.

Math functions

We added the following math functions to custom expressions: :round, :ceil, :floor, :abs, :power, :sqrt, :log, and :exp. They all follow standard SQL definitions and semantics. :log is base-10 logarithm.

These only need to be implemented if the driver supports :expressions flag. :power, :sqrt, :log, and :exp are further governed by :advanced-math-expressions flag.

Aggregations

We added the following aggregations: :variance, :median, :percentile. :variance is implemented as population variance. Note that :median is just sugar for 0.5 percentile.

:variance falls under :standard-deviation-aggregations flag. Support for :median and :percentile is declared by the new :percentile-aggregations flag.

Metabase Logo

Clone this wiki locally