Data Mining

Did you know the German parliament publishes protocols for all of its proceedings in PDF format? It is relatively straightforward to download and parse them, so we can easily collect a dataset of transcripts of what seems to be every speech in the Bundestag since the Second World War. My original idea was to mine the speeches for word associations. Some words will be associated with other words based on the intended connotation, and this association might change over time as the connotations …

I like to read tagesschau.de, so I wrote a script to scrape it in regular intervals. My original goal was to determine which articles stay on the front page the longest, which ones allow commenting (a feature that seems to have been disabled almost entirely since March 2020), and if articles are modified after the initial release (without mentioning this), because I sometimes feel that headlines change. Dataset Creation § Tagesschau provides a JSON API, so fetching all of the articles is …

During the last weeks, I worked with some colleagues on a website that aims to improve access to social work literature. We described the results in out paper Social Work Research Map – ein niederschwelliger Zugang zu internationalen Publikationen der Sozialen Arbeit, which has been published in the journal Soziale Passagen. While the paper is written in german, there is also a technical report in english. Abstract § Internationalization is a central topic in higher education policy in Germany. …

Inspired by David Kriesel’s talk “Spiegel-Mining”, a friend of mine and a professor from the Hochschule Magdeburg scraped a German website that regularly publishes reviews of social work literature and mined the resulting 18.000 articles, hoping for interesting insights. In an attempt to visualize the discourse, we created several topic maps, like the one below, which you can find on the accompanying (German) website. The colors represent the gender of the authors of the …