Adding build figures to presentations in IPython notebooks
CommentsAvailable as a github gist.
Last week when I was up to give lab meeting presentation, I decided that I would use my new favorite programming environment, IPython notebook to make my slides. I learned a lot of tips from searching around, like how to hide the input cells (on @damian_avila's excellent blog).
Making slides was incredibly easy (see this tutorial). You can do things like easily display equations using latex, and not have to go generate new pdfs every time you realize you have a typo in one of your figure legends. Want to tweak a figure? No problem, just edit the python code above that cell and rerun what you need to in order to regenerate the plot, all in one place.
One thing I was missing, which I used extensively in PowerPoint, was the ability to create "build" figures. Often I would use a figure to tell a story, building up the different components with each click. For instance, I might show a plot with only the results of the negative controls of an experiment, and click to reveal the actual results on the same plot. Specifically, here I am doing an eQTL analysis, where we perform thousands of tests of association between the genotype of a locus and expression of a nearby gene. I also have a set of negative controls, where we randomly permuted our data, which should follow the null hypothesis. A good way to visualize the distributions of the test statistics is a QQ plot. In my power point presentation, I would first show the negative controls, then the test of the real data, and then draw a line showing where we drew the threshold for significance. With the IPython slideshows, the best I could do was to build up the figure by redisplaying it on multiple consecutive "subslides".
I asked @damian_avila about this on Twitter, who suggested I look at @jakevdp's "A Javascript View for Matplotlib Animations". @jakevdp also suggested his ipywidgets. The ipywidgets seemed close to what I was looking for, so I forked the repository and got to work.
The goal was to make a widget that would take in a list of functions to apply to a matplotlib Axes object, and create a static "build figure", where on each click the next function is applied, just like you would get using the animation features of PowerPoint. For instance, following on the example above, I might want to apply the three functions:
import math
import numpy as np
observed_pvals = map(lambda x: float(x.strip()), open("/home/mgymrek/pvals.txt", "r").readlines())
control_pvals = map(lambda x: float(x.strip()), open("/home/mgymrek/ctrlpvals.txt", "r").readlines())
expected_pvals = np.random.uniform(low=0, high=1, size=len(observed_pvals))
observed_pvals.sort()
expected_pvals.sort()
control_pvals.sort()
# Draw the negative controls
def f1(ax):
ax.scatter([-1*math.log10(item) for item in expected_pvals],
[-1*math.log10(item) for item in control_pvals], color="darkgray")
ax.text(2, 1, "Permutation controls", size=15, color="black")
# Draw the observed points
def f2(ax):
ax.scatter([-1*math.log10(item) for item in expected_pvals],
[-1*math.log10(item) for item in observed_pvals], color="red")
ax.text(2.2,8, "Observed", size=15, color="red")
# Draw a line giving the significance threshold
def f3(ax):
ax.axhline(y=3, linestyle="dashed", lw=3, color="blue")
ax.text(0.1, 3.2, "FDR 10%", size=15, color="blue")
I also want to do several things no matter what functions have been applied, like draw the axis labels and set the x and y limits. I can define a function for this that will get called every time the figure is drawn:
def init(ax):
fig = ax.get_figure()
fig.set_size_inches((8,8))
maxX = int(math.ceil(-1*math.log10(min(expected_pvals))))
maxY = int(math.ceil(-1*math.log10(min(observed_pvals))))
ax.plot([0,maxX],[0,maxX], color="black")
ax.set_xlabel("-log10 Expected p-value", size=20)
ax.set_ylabel("-log10 Observed p-value", size=20)
ax.set_xticks(np.arange(0,maxX+1,1))
ax.set_xticklabels(np.arange(0,maxX+1,1), size=15)
ax.set_yticks(np.arange(0,maxY+1,2))
ax.set_yticklabels(np.arange(0,maxY+1,2), size=15)
ax.set_xlim(left=0, right=maxX)
ax.set_ylim(bottom=0, top=maxY)
Following on the ipywidgets example, we will precompute the result of apply each function in succession. So if we define 3 functions, we will produce four separate figures: one with none of the functions applied, one with only the first function applied, one with the first two applied, etc. For each of these versions we will also apply the init function. We can just import ipywidgets and do:
from ipywidgets import StaticBuildFigure
StaticBuildFigure([f1, f2, f3], apply_to_all=init)
You can click on the above animation to progress through each of the images. You can use the "a" (advance) and "r" (reverse) keys to go backwards and forwards (in case someone asks you to go back to something when you are presenting).
Making it work
Similar to the StaticInteract class in ipywidgets, we define Javascript functions that will update which figure is displayed. It keeps track of a counter, and each time we click (or press "r" to go backwards through the animation), we update the counter and display the appropriate item:
JS_FUNCTION="""
<script type="text/javascript">
function ProgressForward(div){{
var control = div.getElementsByTagName("input")[0];
var outputs = div.getElementsByTagName("div");
control.value = parseInt(control.value) + 1;
if (control.value >= outputs.length - 1) {{
control.value = outputs.length - 2;
}}
for(i=0; i<outputs.length; i++){{
var name = outputs[i].getAttribute("name");
if(name == "name" + control.value){{
outputs[i].style.display = 'block';
}} else if (name != "control"){{
outputs[i].style.display = 'none'
}}
}}
}}
function ProgressBackward(div){{
var control = div.getElementsByTagName("input")[0];
var outputs = div.getElementsByTagName("div");
control.value = parseInt(control.value) - 1;
if (control.value <= 0) {{
control.value=0;
}}
for(i=0; i<outputs.length; i++){{
var name = outputs[i].getAttribute("name");
if(name == "name" + control.value){{
outputs[i].style.display = 'block';
}} else if (name != "control"){{
outputs[i].style.display = 'none'
}}
}}
}}
// Use "a" to go forward, "r" to go back. Or click to progress
function HandleKey(div){{
var key = window.event.keyCode;
if (key == 65) {{
ProgressForward(div);
}}
if (key == 82) {{
ProgressBackward(div);
}}
}}
</script>
"""
We precompute all the figures we will need using this function inside the StaticBuildFigure class:
def GenerateFigure(self, i):
"""
Generate figure after applying the first i functions
"""
fig = plt.figure()
ax = fig.add_subplot(111)
for f in self.function_list[0:i]:
f(ax)
if self.apply_to_all is not None: self.apply_to_all(ax)
return fig
which we call for i in range(len(self.function_list)+1, and put each figure in a div named "name0", "name1", etc. Then we wrap all the divs and call the above functions on a click or key event:
BUILD_FIGURE_JS = """
<div name="control" onclick="ProgressForward(this.parentNode);" onkeyup="HandleKey(this.parentNode);" tabindex="0">
<div name=name0 style="display:{display}">
HTML content of figure 0...
</div>
<div name=name1 style="display:{display}">
HTML content of figure 1...
</div>
... more divs ...
<input type="none" value="0" style="display:none;">
</div>
"""
The full code for the class is below:
class StaticBuildFigure(object):
"""
Make build figures, useful for presentations.
Example:
def f1(ax):
ax.axhline(y=2)
def f2(ax):
ax.axhline(y=3)
def init(ax):
ax.set_xlabel("This is the X axis")
ax.set_ylabel("This is the Y axis")
ax.set_xlim(left=0, right=1)
ax.set_ylim(bottom=0, top=5)
StaticBuildFigure([f1, f2], apply_to_all=init)
"""
template = """
<style>
*:focus {{
outline:none;
}}
</style>
<script type="text/javascript">
function ProgressForward(div){{
var control = div.getElementsByTagName("input")[0];
var outputs = div.getElementsByTagName("div");
control.value = parseInt(control.value) + 1;
if (control.value >= outputs.length - 1) {{
control.value = outputs.length - 2;
}}
for(i=0; i<outputs.length; i++){{
var name = outputs[i].getAttribute("name");
if(name == "name" + control.value){{
outputs[i].style.display = 'block';
}} else if (name != "control"){{
outputs[i].style.display = 'none'
}}
}}
}}
function ProgressBackward(div){{
var control = div.getElementsByTagName("input")[0];
var outputs = div.getElementsByTagName("div");
control.value = parseInt(control.value) - 1;
if (control.value <= 0) {{
control.value=0;
}}
for(i=0; i<outputs.length; i++){{
var name = outputs[i].getAttribute("name");
if(name == "name" + control.value){{
outputs[i].style.display = 'block';
}} else if (name != "control"){{
outputs[i].style.display = 'none'
}}
}}
}}
// Use "a" to go forward, "r" to go back. Or click to progress
function HandleKey(div){{
var key = window.event.keyCode;
if (key == 65) {{
ProgressForward(div);
}}
if (key == 82) {{
ProgressBackward(div);
}}
}}
</script>
<div name="control" onclick="ProgressForward(this.parentNode);" onkeyup="HandleKey(this.parentNode);" tabindex="0">
{outputs}
<input type="none" value="0" style="display:none;">
</div>
"""
subdiv_template = """
<div name="{name}" style="display:{display}">
{content}
</div>
"""
def __init__(self, function_list, apply_to_all=None):
"""
StaticBuildFigure(function_list, apply_to_all=None)
function_list: list of functions. Each function must take in
a pyplot.Axes instance and modify that axis
apply_to_all: apply this function to all plots
Generate "build" figures, progressively applying functions in
the list. Each time you click on the figure, apply the next
function. Press "a" (advance) or "r" (reverse) to
move forward and backward through the animation
"""
self.function_list = function_list
self.apply_to_all = apply_to_all
def GenerateFigure(self, i):
"""
Generate figure after applying the first i functions
"""
fig = plt.figure()
ax = fig.add_subplot(111)
for f in self.function_list[0:i]:
f(ax)
if self.apply_to_all is not None: self.apply_to_all(ax)
return fig
def _output_html(self):
# get results
results = []
for i in range(len(self.function_list)+1):
results.append(self.GenerateFigure(i))
# Get divnames (name<i> is after applying first i functions)
divnames = ['name%s'%i for i in range(len(self.function_list)+1)]
display = [True] + [False]*(len(self.function_list))
tmplt = self.subdiv_template
return "".join(tmplt.format(name=divname,
display="block" if disp else "none",
content=_get_html(result))
for divname, result, disp in zip(divnames,
results,
display))
def html(self):
return self.template.format(outputs=self._output_html())
def _repr_html_(self):
return self.html()
And that's all there is to it!
Summary and wish list
I plan to use these clickable build figures in future IPython presentations. There are a couple of things I'd like to improve:
- Integrate the build slide with the "next" and "previous" function of pressing the left and right arrows during a presentation made with reveal.js, instead of having to click on the figure for the animation.
- Add the ability to "undo" functions somewhere in the squence. For instance, pass a function list that is something like this: [f1, f2, f3, undo f3, f4]