Bokeh 0.12.4 Released

by Bryan Van de Ven on Friday, January 6, 2017

We're pleased to announce the release of Bokeh 0.12.4! This update has the following highlights:

  • Efficient binary array protocol for column data sources.
  • Generalized mechanism to add CustomJS callbacks to any property.
  • Several fixes to log plots, as well as custom extensions.
  • Support for read-only (client-side) properties, such as a plot's "inner" screen dimensions.
  • Guidance and examples for embedding a Bokeh server directly in standalone scripts, Jupyter notebooks, and Tornado and Flask applications.
  • Improved documentation and Sphinx extensions for the bokeh.palettes module.
  • Major refactor of bokeh.core including tests and expanded documentation relevant to extension authors.

As well as many other small bugfixes and docs additions. For full details, see the CHANGELOG and Release Notes.

As a reminder, we now package and upload examples for each Bokeh release to our CDN. Click here to download.


This release can most easily be installed from the Bokeh channel on Anaconda.org by executing the command conda install -c bokeh bokeh if you are using Anaconda or pip install bokeh otherwise.


Community

Before getting to the technical details of this release, I wanted to take a moment to recognize some stats about the community:

  • The public mailing list has more than 700 members
  • @BokehPlots has nearly 4000 followers
  • The number of GitHub stars just hit 5200 (!)
  • Before the holidays, combined conda+pip downloads exceeded 120k/month

Any one of these facts alone is humbling, and a testament to the work of so many contributors over the last few years. Which brings me to the most impressive number yet. Several people rose to the End of 2016 Contributor Challenge I posted on the mailing list, and I am happy to report that the tally is now:

Without fail, the number of issues grows with the number of users, so growing a base of developers to help with those issues is crucial to the long term health of the project. I'd like in particular to thank new contributors: bgbg, ivan-krukov, jfinkels, and souravsingh. Several small but important features and fixes were made by them, quickly, and Bokeh is all the better for it. Some of them are already working on their second PRs!

Binary Array Protocol

Since the project's start, Bokeh has taken a fairly straightforward approach to serializing data to and from BokehJS, namely "JSON all the things". With Bokeh applications, this also happens to occur over a websocket, but it's still the same JSON mechanism, with the same associated overhead. For many common use cases this is fine, but when the data starts to grow larger (for example when dealing with data for images where the size grows as N2), this overhead becomes noticeable.

For a long time we have wanted to investigate the use of a binary encoding for arrays, to help improve the performance of serialization. I am happy to say that this task has been accomplished, thanks to the great work of Philipp Rudiger, who is also a core contributor to the Holoviews project. As a result of Philipp's effort, integer and floating point column data source arrays are now automatically serialized using a binary format. Encoding this way is already more efficient by itself, but it also means that BokehJS can now ingest data directly into JavaScript typed arrays, another efficiency improvement.

One quick and easy benchmark compares the "before" and "after" time to render images of different sizes:

As can be seen, things could get really bad for image sizes above 500x500. The new binary encoding reduces the time to display drastically.

Finally, during the course of implementing this, Philipp also discovered that the internal logic that prevents Bokeh app events from "boomeranging" between the client and server could be greatly simplified. This opened up even more opportunity for performance improvement.

Another way of visualizing the collective effect of these changes is shown below: A before and after execution log of a running Bokeh app with a periodic callback updating a plot every 100ms. In the before picture, individual events can be seen to be stacking up:

After Philipp's changes, the execution handling each event is clearly separated:

Guidance for Embedding Bokeh Servers

As nice and useful as it is to run Bokeh applications with bokeh serve app.py there are also situations where it might be preferable to be able to embed a Bokeh server inside another script or application.

In particular an important missing piece has been the ability to define and interact with Bokeh applications directly in Jupyter notebooks. Although there is still some work to do to refine this capability, an example notebook was added that demonstrates a technique that can be used immediately. You can see the result below:

In addition to the notebook, examples were added to show how to embed a Bokeh server in a standalone script, a Flask application, and a Tornado application. You can see all the examples in th GitHub repository:

General JS Callbacks

Early on, when we were still sorting out the vision and kinks of the Bokeh server, the capability to add little snippets of JavaScript was added to afford people more possibilities for rich interactions. Until now, the places where CustomJS callbacks could be used were added in an ad-hoc fashion, often as a callback property on some Bokeh model. For instance, column data sources have a callback property that can trigger a JavaScript callback whenever a selection is made. This mechanism proved popular, with users wanting to put CustomJS callbacks... basically everywhere. It's time to make a more solid, generalized means for adding these callbacks. Now CustomJS callbacks can be added to any property of any Bokeh model, using the js_on_change method:

callback = CustomJS(args=dict(source=source), code="""
    x0 = cb_obj.start;
    x1 = cb_obj.end;
    eps = (x1-x0) / 100
    for (i=0; i<101; i++) {
        source.data.x[i] = x0 + i*eps
        source.data.y[i] = Math.sin(source.data.x[i])
    }
    source.trigger('change')
""")

p.x_range.js_on_change('start', callback)
p.x_range.js_on_change('end', callback)

With this code, we can make a simple plot with a CustomJS callback that updates the value of the curve any time any of the x-range attributes changes:

Of course, you might have more interesting real-world use-cases, such as time-series or other windowed data!

The original motivation for this was actually to be able to have JS callbacks for ColumnDataSource streaming events. That's also possible with an expression like this:

source.js_on_change('stream', stream_callback)

One last thing to mention: with a general way to add CustomJS callbacks anywhere, the small set of ad-hoc callback locations that existed up until now will probably be deprecated in the future.

Documentation Improvements

The documentation for palettes was improved by adding full docstrings throughout the module, as well as by creating a new :bokeh-palette: Sphinx extension for displaying visual palettes in the docs. The result is nice sections like this one:

You can check out the full changes in the bokeh.palettes section of the Reference Guide.

Speaking of the Reference Guide, some parts of the library that were previously "internal" are now more interesting to general users, especially those who might be writing custom extensions. Accordingly, we've made a big push to document those parts much more thoroughly. Check out the bokeh.core docs section.

Client-side Read-only Properties

Another long-requested feature (by the Holoviews and Datashader project and others) is the ability to determine and respond to the "inner" dimensions on a plot.

The main obstacle to implementing this was the lack of a mechanism for "read-only" properties, which is to say, properties that are only able to be set from BokehJS, in the browser. The inner dimensions of a plot, determined on the fly by the BokehJS layout machinery, are just like this. Mateusz Paprocki implemented a mechanism for client-only properties in general, that can be used with with on_change callbacks in Bokeh applications, just like any other properties. The first two such properties added were inner_width and inner_height, on plots:

p.on_change("inner_width", some_callback)
p.on_change("inner_height", some_callback)

As a reminder, on_change style callbacks require a Bokeh server. However, these properties work in any situation with the new js_on_change feature introduced above, as well.

Improvements to Log Plotting

Basic log plot capability was added on early in Bokeh's history, but has not seen much attention since then. Recently, more users have been asking for improvements to log plotting, so recent contributor Claire Tang tore through a whole pile of issues related to log plots:

#2789—Range padding possibly discards the log axis properties

#3834—Plot is empty when log scale is used

#5389—Creating a line plot with x_axis_type='log' fails when x_max < 1

#5549—Correctly handle data values <= 0 on a log scale

#5576—Initial range calculation for log plots can cause empty plots

Even more fixes and improvements to log plotting are on the way.

What's Next?

A lot of the older (or "bigger vision") tasks are getting put to rest. There is still plenty to do, but it's time to focus on a few remaining features, polishing, and stabilizing. With that in mind, I expect the 0.12.5 release notes to look something like:

  • Fix lingering issues with layouts, changing Document children, and custom extensions.
  • Add a new interactive legend capability for muting or hiding glyphs.
  • Remove JQuery, which will shrink BokehJS by another 10 percent.
  • Expose the UI event bus to make it accessible to JS and python callbacks.
  • Refactor categoricals to be more efficient, and support nested coordinate systems.
  • Continue strong pushes on documentation and testing.
  • Fix many other small (but annoying) bugs.

For more details, see the current 0.12.5 Milestone on GitHub or the larger Roadmap Document. These are not set in stone, and may certainly change. We do welcome feedback and discussion to help shape and set priorities.

Of course, with more help, we can get more done, more quickly. So please consider getting involved.


As we're getting closer to a Bokeh 1.0 release, I'd like to thank the 200+ total contributors who have helped make Bokeh such an amazing project. As always, for questions, technical assistance or if you're interested in contributing, please ask the Bokeh mailing list or join the chat on Gitter.

Thanks,

Bryan Van de Ven