Plotting with Vega-Altair

Objectives

• Be able to create simple plots with Vega-Altair and tweak them

• Know how to look for help

• Reading data with Pandas from disk or a web resource

• Know how to tweak example plots from a gallery for your own purpose

• We will build up this notebook (spoiler alert!)

Instructor note

• 10 min: Introduction

• 10 min: Type-along (creating a first plot)

• 20 min: Exercise (using visual channels)

• 20 min: Exercise (adapting a gallery example and customizing)

• 10 min: Key points, discussion, and Q&A

Repeatability/reproducibility

From Claus O. Wilke: “Fundamentals of Data Visualization”:

One thing I have learned over the years is that automation is your friend. I think figures should be autogenerated as part of the data analysis pipeline (which should also be automated), and they should come out of the pipeline ready to be sent to the printer, no manual post-processing needed.

• Try to minimize manual post-processing. This could bite you when you need to regenerate 50 figures one day before submission deadline or regenerate a set of figures after the person who created them left the group.

• There is not the one perfect language and not the one perfect library for everything.

• Within Python, many libraries exist:

  • Vega-Altair: declarative visualization, statistics built in

  • Matplotlib: probably the most standard and most widely used

  • Seaborn: high-level interface to Matplotlib, statistical functions built in

  • Plotly: interactive graphs

  • Bokeh: also here good for interactivity

  • plotnine: implementation of a grammar of graphics in Python, it is based on ggplot2

  • ggplot: R users will be more at home

  • PyNGL: used in the weather forecast community

  • K3D: Jupyter Notebook extension for 3D visualization

  • Mayavi: 3D scientific data visualization and plotting in Python

  • …

• Two main families of libraries: procedural (e.g. Matplotlib) and declarative (e.g. Vega-Altair).

Why are we starting with Vega-Altair?

• Concise and powerful

• “Simple, friendly and consistent API” allows us to focus on the data visualization part and get started without too much Python knowledge

• The way it combines visual channels with data columns can feel intuitive

• Interfaces very nicely with Pandas (earlier episode)

• Easy to change figures

• Good documentation

• Open source

• Makes it easy to save figures in a number of formats (svg, png, html)

• Easy to save interactive visualizations to be used in websites

Example data: Weather data from two Norwegian cities

We will experiment with some example weather data obtained from Norsk KlimaServiceSenter, Meteorologisk institutt (MET) (CC BY 4.0). The data is in CSV format (comma-separated values) and contains daily and monthly weather data for two cities in Norway: Oslo and Tromsø. You can browse the data here in the lesson repository.

We will use the Pandas library to read the data into a dataframe. We have learned about Pandas in an earlier episode.

Pandas can read from and write to a large set of formats (overview of input/output functions and formats). We will load a CSV file directly from the web. Instead of using a web URL we could use a local file name instead.

Pandas dataframes are a great data structure for tabular data and tabular data turns out to be a great input format for data visualization libraries. Vega-Altair understands Pandas dataframes and can plot them directly.

Reading data into a dataframe

We can try this together in a notebook: Using Pandas we can merge, join, concatenate, and compare dataframes, see https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html.

Let us try to concatenate two dataframes: one for Tromsø weather data (we will now load monthly values) and one for Oslo:

import pandas as pd

url_prefix = "https://raw.githubusercontent.com/AaltoSciComp/python-for-scicomp/master/resources/data/plotting/"

data_tromso = pd.read_csv(url_prefix + "tromso-monthly.csv")
data_oslo = pd.read_csv(url_prefix + "oslo-monthly.csv")

data_monthly = pd.concat([data_tromso, data_oslo], axis=0)

# let us print the combined result
data_monthly

Before plotting the data, there is a problem which we may not see yet: Dates are not in a standard date format (YYYY-MM-DD). We can fix this:

# replace mm.yyyy to date format
data_monthly["date"] = pd.to_datetime(list(data_monthly["date"]), format="%m.%Y")

With Pandas it is possible to do a lot more (adjusting missing values, fixing inconsistencies, changing format).

Plotting the data

Now let’s plot the data. We will start with a plot that is not optimal and then we will explore and improve a bit as we go:

import altair as alt

alt.Chart(data_monthly).mark_bar().encode(
    x="date",
    y="precipitation",
    color="name",
)

Monthly precipitation for the cities Oslo and Tromsø over the course of a year.

Let us pause and explain the code

• alt is a short-hand for altair which we imported on top of the notebook

• Chart() is a function defined inside altair which takes the data as argument

• mark_bar() is a function that produces bar charts

• encode() is a function which encodes data columns to visual channels

Observe how we connect (encode) visual channels to data columns:

• x-coordinate with “date”

• y-coordinate with “precipitation”

• color with “name” (name of weather station; city)

We can improve the plot by giving Vega-Altair a bit more information that the x-axis is temporal (T) and that we would like to see the year and month (yearmonth):

alt.Chart(data_monthly).mark_bar().encode(
    x="yearmonth(date):T",
    y="precipitation",
    color="name",
)

Apart from T (temporal), there are other encoding data types:

• Q (quantitative)

• O (ordinal)

• N (nominal)

• T (temporal)

• G (geojson)

Monthly precipitation for the cities Oslo and Tromsø over the course of a year.

Let us improve the plot with another one-line change:

alt.Chart(data_monthly).mark_bar().encode(
    x="yearmonth(date):T",
    y="precipitation",
    color="name",
    column="name",
)

Monthly precipitation for the cities Oslo and Tromsø over the course of a year with with both cities plotted side by side.

With another one-line change we can make the bar chart stacked:

alt.Chart(data_monthly).mark_bar().encode(
    x="yearmonth(date):T",
    y="precipitation",
    color="name",
    xOffset="name",
)

Monthly precipitation for the cities Oslo and Tromsø over the course of a year plotted as stacked bar chart.

This is not publication-quality yet but a really good start!

Let us try one more example where we can nicely see how Vega-Altair is able to map visual channels to data columns:

alt.Chart(data_monthly).mark_area(opacity=0.5).encode(
    x="yearmonth(date):T",
    y="max temperature",
    y2="min temperature",
    color="name",
)

Monthly temperature ranges for two cities in Norway.

What other marks and other visual channels exist?

• Overview of available marks

• Overview of available visual channels

• Gallery of examples

Exercise: Using visual channels to re-arrange plots

Plotting-1: Using visual channels to re-arrange plots

1. Try to reproduce the above plots if they are not already in your notebook.

2. Above we have plotted the monthly precipitation for two cities side by side using a stacked plot. Try to arrive at the following plot where months are along the y-axis and the precipitation amount is along the x-axis:

   

3. Ask the Internet or AI how to change the axis title from “precipitation” to “Precipitation (mm)”.

4. Modify the temperature range plot to show the temperature ranges for the two cities side by side like this:

   

Solution

1. Copy-paste code blocks from above.

2. Basically we switched x and y:

   alt.Chart(data_monthly).mark_bar().encode(
       y="yearmonth(date):T",
       x="precipitation",
       color="name",
       yOffset="name",
   )
   

3. This can be done with the following modification:

   alt.Chart(data_monthly).mark_bar().encode(
       y="yearmonth(date):T",
       x=alt.X("precipitation").title("Precipitation (mm)"),
       color="name",
       yOffset="name",
   )
   

4. We added one line:

   alt.Chart(data_monthly).mark_area(opacity=0.5).encode(
       x="yearmonth(date):T",
       y="max temperature",
       y2="min temperature",
       color="name",
       column="name",
   )
   

More fun with visual channels

Now we will try to plot the daily data and look at snow depths. We first read and concatenate two datasets:

url_prefix = "https://raw.githubusercontent.com/AaltoSciComp/python-for-scicomp/master/resources/data/plotting/"

data_tromso = pd.read_csv(url_prefix + "tromso-daily.csv")
data_oslo = pd.read_csv(url_prefix + "oslo-daily.csv")

data_daily = pd.concat([data_tromso, data_oslo], axis=0)

We adjust the data a bit:

# replace dd.mm.yyyy to date format
data_daily["date"] = pd.to_datetime(list(data_daily["date"]), format="%d.%m.%Y")

# we are here only interested in the range december to may
data_daily = data_daily[
    (data_daily["date"] > "2022-12-01") & (data_daily["date"] < "2023-05-01")
]

Now we can plot the snow depths for the months December to May for the two cities:

alt.Chart(data_daily).mark_bar().encode(
    x="date",
    y="snow depth",
    column="name",
)

Snow depth (in cm) for the months December 2022 to May 2023 for two cities in Norway.

What happens if we try to color the plot by the “max temperature” values?

alt.Chart(data_daily).mark_bar().encode(
    x="date",
    y="snow depth",
    color="max temperature",
    column="name",
)

The result looks neat:

Snow depth (in cm) for the months December 2022 to May 2023 for two cities in Norway. Colored by daily max temperature.

We can change the color scheme (available color schemes):

alt.Chart(data_daily).mark_bar().encode(
    x="date",
    y="snow depth",
    color=alt.Color("max temperature").scale(scheme="plasma"),
    column="name",
)

With the following result:

Snow depth (in cm) for the months December 2022 to May 2023 for two cities in Norway. Colored by daily max temperature. Warmer days are often followed by reduced snow depth.

Let’s try one more change to show that we can experiment with different plot types by changing mark_bar() to something else, in this case mark_circle():

alt.Chart(data_daily).mark_circle().encode(
    x="date",
    y="snow depth",
    color=alt.Color("max temperature").scale(scheme="plasma"),
    column="name",
)

Snow depth (in cm) for the months December 2022 to May 2023 for two cities in Norway. Colored by daily max temperature. Warmer days are often followed by reduced snow depth.

Themes

In Vega-Altair you can change the theme and select from a long list of themes. On top of your notebook try to add:

alt.themes.enable('dark')

Then re-run all cells. Later you can try some other themes such as:

• fivethirtyeight

• latimes

• urbaninstitute

You can even define your own themes!

Exercise: Adapting a gallery example

In this exercise we can try to adapt existing scripts to either tweak how the plot looks or to modify the input data. This is very close to real life: there are so many options and possibilities and it is almost impossible to remember everything so this strategy is useful to practice:

• Select an example that is close to what you have in mind

• Being able to adapt it to your needs

• Being able to search for help

Plotting-2: Adapting a gallery example

This is a great exercise which is very close to real life.

• Browse the Vega-Altair example gallery.

• Select one example that is close to your current/recent visualization project or simply interests you.

• First try to reproduce this example, as-is, in the Jupyter Notebook.

• If you get the error “ModuleNotFoundError: No module named ‘vega_datasets’”, then try one of these examples: (they do not need the “vega_datasets” module)

  • Slider cutoff (below you can find a walk-through for this example)

  • Multi-Line tooltip

  • Heatmap

  • Layered histogram

• Then try to print out the data that is used in this example just before the call of the plotting function to learn about its structure. Consider writing the data to file before changing it.

• Then try to modify the data a bit.

• If you have time, try to feed it different, simplified data. This will be key for adapting the examples to your projects.

Example walk-through for the slider cutoff example

In this walk-through I imagine browsing: https://altair-viz.github.io/gallery/index.html

Then this example caught my eye: https://altair-viz.github.io/gallery/slider_cutoff.html

I then copy-paste the example code into a notebook and try to run it and I get the same result.

If you get stuck below, you can also browse all the steps in a notebook using nbviewer.

Next, there is a lot of code that I don’t (need to) understand yet but my eyes are trying to find alt.Chart which tells me that the data must be the “df” in alt.Chart(df):

import altair as alt
import pandas as pd
import numpy as np

rand = np.random.RandomState(42)

df = pd.DataFrame({
    'xval': range(100),
    'yval': rand.randn(100).cumsum()
})

slider = alt.binding_range(min=0, max=100, step=1)
cutoff = alt.param(bind=slider, value=50)

alt.Chart(df).mark_point().encode(
    x='xval',
    y='yval',
    color=alt.condition(
        alt.datum.xval < cutoff,
        alt.value('red'), alt.value('blue')
    )
).add_params(
    cutoff
)

My next step will be to print out the data df just before the call to alt.Chart:

import altair as alt
import pandas as pd
import numpy as np

rand = np.random.RandomState(42)

df = pd.DataFrame({
    'xval': range(100),
    'yval': rand.randn(100).cumsum()
})

slider = alt.binding_range(min=0, max=100, step=1)
cutoff = alt.param(bind=slider, value=50)

print(df)

alt.Chart(df).mark_point().encode(
    x='xval',
    y='yval',
    color=alt.condition(
        alt.datum.xval < cutoff,
        alt.value('red'), alt.value('blue')
    )
).add_params(
    cutoff
    )

The print reveals that df is a dataframe which contains x and y values:

    xval       yval
0      0   0.496714
1      1   0.358450
2      2   1.006138
3      3   2.529168
4      4   2.295015
..   ...        ...
95    95 -10.712354
96    96 -10.416233
97    97 -10.155178
98    98 -10.150065
99    99 -10.384652

[100 rows x 2 columns]

The next thing that often helps me is to save the data to a comma-separated values (CSV) file:

import pandas as pd

df.to_csv("data.csv", index=False)

I then open the file in an editor and see that it contains 100 rows:

xval,yval
0,0.4967141530112327
1,0.358449851840048
2,1.0061383899407406
3,2.5291682463487657
4,2.2950148716254297
5,2.060877914676249
6,3.6400907301836405
7,4.407525459336549
8,3.938051073401597
9,4.4806111169875615
...

Saving the data to file often helps me to see the structure of the data and now I am in a position to replace this with my own data. I create a file called “mydata.csv” and there I use the maximum temperatures for months 1-10 from the Tromso monthly data which we used further up:

xval,yval
01,7.7
02,6.6
03,4.5
04,9.8
05,17.7
06,25.4
07,26.7
08,25.1
09,19.3
10,9.8

In the notebook I then verify that the reading of the data works:

mydata = pd.read_csv("mydata.csv")

mydata

Now I can replace the example with my own data (note how I now can comment out some code that I don’t need any longer):

import altair as alt
import pandas as pd
# import numpy as np

# rand = np.random.RandomState(42)

# df = pd.DataFrame({
#     'xval': range(100),
#     'yval': rand.randn(100).cumsum()
# })

slider = alt.binding_range(min=0, max=100, step=1)
cutoff = alt.param(bind=slider, value=50)

# print(df)
df = pd.read_csv("mydata.csv")

alt.Chart(df).mark_point().encode(
    x='xval',
    y='yval',
    color=alt.condition(
        alt.datum.xval < cutoff,
        alt.value('red'), alt.value('blue')
    )
).add_params(
    cutoff
    )

Seems to work! I then make few more adjustments (I want the slider to work on the y-axis and have a more reasonable default):

import altair as alt
import pandas as pd

slider = alt.binding_range(min=0, max=30, step=1)
cutoff = alt.param(bind=slider, value=15)

df = pd.read_csv("mydata.csv")

alt.Chart(df).mark_point().encode(
    x='xval',
    y='yval',
    color=alt.condition(
        alt.datum.yval < cutoff,
        alt.value('red'), alt.value('blue')
    )
).add_params(
    cutoff
    )

My next steps would then be to change axis titles, display the month names, add a legend, and refine from here.

You can also browse all the steps in a notebook using nbviewer.

---

Keypoints

• Browse a number of example galleries to help you choose the library that fits best your work/style.

• Minimize manual post-processing and try to script all steps.

• CSV (comma-separated values) files are often a good format to store the data that we wish to plot.

• Read the data into a Pandas dataframe and then plot it with Vega-Altair where you connect data columns to visual channels.